Accessibility and web practitioners: A study of factors influencing web development for the public sector

(1)

BACHELOR’S THESIS IN INFORMATION ARCHITECTURE, SPECIALISATION WEB CONTENT MANAGER AND DESIGNER, FACULTY OF LIBRARIANSHIP, INFORMATION, EDUCATION AND IT

2020

Accessibility and web practitioners

A study of factors influencing web development for the public sector

KAJSA EKLÖF REBECKA NÄSSTRÖM

© Kajsa Eklöf & Rebecka Näsström

Partial or full copying and distribution of the material in this thesis without permission is forbidden.

(2)

English title: Accessibility and web practitioners: A study of factors influencing web development for the public sector

Author(s): Kajsa Eklöf & Rebecka Näsström

Completed: 2020

Abstract: For the past few years, Sweden has worked towards digitalisation of the public sector, however without any legislated requirements for accessibility. In order to make public sector body websites and applications accessible to all users a new Swedish law, based on the Web Accessibility Directive, has been implemented. This new law dictates that public sector websites and applications need to comply with the Web Content Accessibility Guidelines (WCAG) in September of 2020. This study explores factors influencing web practitioners when

developing accessible websites, applications and services for the public sector. This is done through a survey and semi-structured interviews, using both quantitative and qualitative methods. The new law does not provide guidance on methods or

evaluations to develop products and services accessible for all users. Instead it has a main focus on compliance with WCAG 2.1.

The result show that the close connection between the law and the guidelines removes the focus from the users in evaluations. Instead the focus is on measurable compliance with the law through the use of static checklists or automatic testing tools. As long as the method of using a checklist or automatic tool is presented as equally sufficient as user testing in terms of reaching compliance with the law, clients requesting accessible solutions will not change their priorities or allocate resources towards user testing.

The findings are presented in a model representing the factors influencing web development and accessibility evaluation and how these factors are connected and dependent on each other.

Keywords: Web Accessibility, Accessibility, WCAG, Web

Practitioners, Web Accessibility Directive,

Evaluation, Web Development

(3)

1 Introduction

In a society where everyday tasks, such as making an appointment with a doctor or paying your bills, demands that citizens are able to access and manage new digital services, replacing the previous ways of carrying out these tasks, accessibility for everyone has become absolutely vital. The real value of new digital innovations in society will only emerge when they are accessible to everyone and do not cause people to feel excluded.

Accessibility is a human right and new legislation has been incorporated globally in order to make digital services in the public sector more accessible. However, the web practitioners who are designing and developing these digital services are often a step behind, in both their experience and knowledge, due to the fact that their speciality is mainly in design or writing code and not accessibility (Gay et al., 2017b, p. 1). It is therefore problematic that these people are those left to interpret the laws, guidelines and accessibility evaluations of the project or services they are working on. A new accessibility law implemented in Sweden presents requirements for accessibility but does not provide any guidance on methods or processes for web practitioners to follow. Therefore, there is a risk the legislation does not achieve the level of accessibility intended and people are still being excluded from society as they are not able to access or manage the new digital services in order to participate in society to their full capacity.

In this study we research the knowledge, experience and attitudes among Swedish web practitioners developing accessible digital products and services for the public sector. We present our findings and analysis on the current situation among web practitioners in Sweden as the new legislation is due to take effect in full later this year.

1.1 Background

To introduce the problem and focus of this study, it is essential to understand the role of digitalisation, authorities in charge and international standards connected to the new accessibility law in Sweden. In this section we present the background and the context for the problem concerning this study. We also explain relevant terminology and how the terms have been used throughout the study.

1.1.1 A More Digital Society

The government in Sweden has, for the past few years, been working towards the digitalisation of the public sector. Between 2015 and 2018 a program called Digital First (Digitalt Först) was introduced to help aid digital innovation (Regeringskansliet, 2017). According to the government the program should promote that “digital services, where possible and relevant, shall be the first choice in the public sector’s contact with private individuals and businesses”

[translated from Swedish] (Regeringskansliet, 2017).

In order to drive the digitalisation of the public sector forward, a new

government authority was created in September 2018 called Agency for Digital

Government (Myndigheten för digital förvaltning), shortened to DIGG. The new

authority’s main purpose is to “serve as a catalyst to enable more people to

(6)

2 accomplish more and to promote the development of simpler, more open and more efficient administration” (DIGG, n.d.-c). DIGG has also been identified as the authority to implement and regulate the new accessibility law in Sweden. In addition to running the implementation and regulation of the new law, DIGG will “serve as a hub for digitalisation of the public sector” (DIGG, n.d.-a) and they are responsible for numerous digital services used by the public sector (DIGG, n.d.-e). As the monitoring authority, DIGG is also responsible for collecting accessibility complaints filed by users of public sector websites and applications (DIGG, n.d.-b). If DIGG finds the complaint valid, they will contact the site owner and inform them of the accessibility issues. If issues regarding accessibility are not corrected in due time, more formal measures will be taken which could eventually lead to fines.

1.1.2 Web Content Accessibility Guidelines (WCAG) 2.1

In 1999 the first Web Content Accessibility Guidelines (WCAG) 1.0 was published. The guidelines is a technical document developed by the Accessibility Guidelines Working Group (AG WG) which is part of the World Wide Web Consortium (W3C) Web Accessibility Initiative (WAI) (W3C Web Accessibility Initiative (WAI), 2018). WCAG is for developers and “others who want or need a standard for web accessibility, including for mobile accessibility”

(W3C Web Accessibility Initiative (WAI), 2018). There have been two updated versions since the first set of guidelines were published, WCAG 2.0 was published on 11 December 2008 and the most recent version, WCAG 2.1, was published on 5 June 2018.

The guidelines are recognised globally and approved as an ISO standard. The ISO standard is the same version as WCAG 2.0 (International Organization for Standardization, 2012). To understand the guidelines and techniques to adhere and meet the requirements when developing websites and digital services, the W3C Web Accessibility Initiative host many documents and guides on their website (W3C Web Accessibility Initiative (WAI), 2018).

1.1.3 New Laws on Accessibility

In 2016 the European Commission published a new directive on accessibility of the websites and mobile applications of public sector bodies (Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the Accessibility of the Websites and Mobile Applications of Public Sector Bodies, OJ L 327, 2.12.2016, p. 1-15 (“Web Accessibility Directive”)). The accessibility requirements in the Web Accessibility Directive states that each EU country “shall ensure that public sector bodies take the necessary measures to make their websites and mobile applications more accessible by making them perceivable, operable, understandable and robust” (Web Accessibility Directive, Article 4). In Sweden, in order to implement the Web Accessibility Directive, a new law was created and came in to force during 2019. The law is called “Lagen om tillgänglighet till digital offentlig service” (Lag om tillgänglighet till digital offentlig service, SFS 2018:1937) and will be referred to as the Law on Accessibility to Digital Public Service in this study.

Both the Web Accessibility Directive and the new Swedish Law on Accessibility

to Digital Public Service are based on the harmonised European standard

Accessibility requirements for ICT products and services. Clause 9 in the

(7)

3 harmonised European standard concerns web accessibility and requires conformance with WCAG 2.1 (European Telecommunications Standards Institute, 2018).

Since the Web Accessibility Directive, the European Commission has published another directive on accessibility. In 2019 a new directive known as the European Accessibility Act (Directive (EU) 2019/882 of the European Parliamnet and of the Council of 17 April 2019 on the Accessibility Requirements for Products and Services, 2019, OJ L 151, 07.06.2019, p. 70-115 ("European Accessibility Act")) was published. This new directive will cover a wider range of products and services, including for example e-commerce and online banking services (European Accessibility Act, Article 2) and the legislation will apply from 28 June 2025. According to the directive all EU countries have to implement the directive as a law by 28 June 2022 (European Accessibility Act, Article 31). The European Accessibility Act will affect products, services and companies in the private sector. Although the European Accessibility Act is not the focus of this study, it is important to mention in order to understand that accessibility is getting more attention on a legislative level.

1.1.4 Terminology

The definitions below explain how we interpret and use the following terminology throughout our study.

Accessibility: “Extent to which products, systems, services, environments and facilities can be used by people from a population with the widest range of user needs, characteristics and capabilities to achieve identified goals in identified contexts of use” (International Organization for Standardization, 2018).

Digitalisation: The process in which the public sector is reforming to offer their products and services digitally, rather than in person, spanning over several areas in society. These areas could be, but are not limited to: ID verification, tax services, healthcare and general contact with the official authorities. This definition does not include digitisation of analogue media to modern formats.

Disabilities: The limitations that occur for people with impairments when they carry out activities. The World Health Organization (WHO) describes disabilities as “an umbrella term, covering impairments, activity limitations, and participation restrictions.” (World Health Organization, n.d.).

Impairment: The loss or variation of physiological or psychological function for example visual impairment or cognitive impairment. WHO describes impairment as “a problem in body function or structure” (World Health Organization, n.d.).

Web practitioners: A general term used to describe the specialists involved in the creation and management of websites, web applications, digital services or digital content. This could for example refer to developers, designers (graphic design or UI/UX), content creators or project managers.

1.1.5 Problem Domain

The Law on Accessibility to Digital Public Service requires websites and

applications, in accordance with the Web Accessibility Directive and WCAG, to

(8)

4 be “perceivable, operable, understandable and robust” (Web Accessibility Directive, Article 4). In Sweden the authority responsible for monitoring that websites and digital services meet the requirements for accessibility is DIGG. In addition to providing accessible websites and applications, the new law states that the public sector bodies affected by the law need to provide an accessibility statement, published on their website or mobile application, as well as accessible versions of non-accessible material and documents. DIGG describes the work on the accessibility adaptation with the words “by following a certain European standard (EN 301 549 V2.1.2), which in turn is based on WCAG 2.1, websites and mobile applications can comply with the requirements” [translated from Swedish] (DIGG, n.d.-d). The literal interpretation of the wording can be perceived to signify that work that complies with the guidelines automatically means that a website or mobile application is accessible to all people. However, numerous studies on using WCAG to achieve accessibility have proven that this is not the case (Brajnik et al., 2012, p. 26; Rømen & Svanæs, 2012, p. 376).

In order to understand the difficulties experienced by users with disabilities while using digital information services, it is first necessary to understand the differences between different types of disabilities (Begripsam, 2018, p. 4). It is not likely that all users with an impairment will experience difficulties using the internet. For example it is not likely that a person with reduced mobility in their legs will automatically have a hard time using the internet (Begripsam, 2018, p.

4). This is why it is problematic to assume “people with disabilities” is one large group, when in fact the user needs are widespread and multi-varied. Johansson (2019, p. 106) argues that from a design perspective, the user's impairment or diagnosis is not the important part. Instead, Johansson recommends looking at the actual difficulties or functions of users, such as “attention, memory and keeping focus on a given task” (Johansson, 2019, p. 107).

Johansson (2019, p. 35) highlights the fact that there is often only one or a few people in an entire organisation responsible for evaluating accessibility, and that these people might not have adequate knowledge to evaluate whether the websites and applications are complying with the guidelines, let alone if it is accessible to all users. Other research seems to support this opinion. For example, Brajnik et al. (2012, p. 20) noted that when websites were tested in their study, the evaluators and designers with less experience took three times longer to evaluate and the results showed poor validity. Their study also showed that sufficient validity could not be reached even when the results of two experienced evaluators were matched (Brajnik et al., 2012, p. 26).

It is particularity problematic for web practitioners to test their websites and applications for cognitive accessibility using a set of guidelines such as WCAG.

First of all there are many accessibility problems with no direct correlation to a

particular guideline or criteria (Rømen & Svanæs, 2012, p. 380) so there is a

high risk that difficulties experienced by users will be missed. Secondly the

reason why there are few guidelines related to cognitive accessibility in WCAG

is due to the fact that requirements need to be testable by web practitioners to

provide a true or false result. The problem is not that experts disagree on the

importance of such guidelines but that there are still no outlined methods for

testing and evaluating requirements related to cognitive accessibility (Johansson,

2019, p. 37). W3C has acknowledged the lack of testable criteria concerning

cognitive accessibility and they state that it is an area within the guidelines they

(9)

5 are working on improving (W3C Web Accessibility Initiative (WAI), 2019).

However, for the time being, evaluating cognitive accessibility is still a highly subjective process for which web practitioners are responsible.

Implementing a new law on accessibility for websites and digital services and basing it on the globally acknowledged guidelines WCAG does not automatically make websites and applications accessible (Rømen & Svanæs, 2012, p. 375). In order to achieve accessibility, the web practitioners who develop and design websites and digital services need to possess the knowledge to do so. A study in Brazil surveying the knowledge and experience of developers in relation to accessibility showed that most developers have never developed an accessible website and that it is not something they are consciously working with in their projects (Antonelli et al., 2018, p. 76). Another concern is that developers rarely have both the desired developer skills and deep knowledge about the difficulties users with disabilities experience when using websites and digital services (Gay et al., 2017b, p. 1). This, Gay et al. state, is due to the fact that accessibility as a subject and awareness about disabilities are not usually included in computer science programs and leaves the developers to learn about the subject on their own.

The Law on Accessibility to Digital Public Service requires websites and mobile applications to be accessible for all users. However, the law only provides technical guidelines, in the shape of WCAG 2.0, and no guidance at all on how to achieve accessibility through a practical process.

1.2 Problem Statement

Sweden’s society is changing to become more and more digital and the physical contact between individuals and public sector bodies are being replaced with digital services (Regeringskansliet, 2017). Areas where the new services are most noticeable are digital ID verification, tax and health services. The digitalisation across the public sector demands that citizens are able to access and handle the new digital services in order to take part in society and carry out necessary daily activities (Digitaliseringsrådet, 2019, p. 5).

The new Law on Accessibility to Digital Public Service states that, in all European Union (EU) member states, public sector websites and mobile applications published before 23 September 2018 need to conform to the accessibility requirements by 23 September 2020. All public sector websites and mobile applications published after 23 September 2018 should already be conforming to the new law and should have done so since 23 September 2019 (DIGG, n.d.-d). The new law is based on the harmonised European standard (European Telecommunications Standards Institute, 2018) which outlines that it is WCAG 2.1 that websites and applications should conform to in order to reach accessibility and become perceivable, operable, understandable and robust.

Johansson (2019) highlights the problems with basing accessibility on legislation, meaning that even after complying with rules and laws “even after careful and meticulous compliance to the rules; some users with some impairments will still claim that they are excluded due to inaccessibility”

(Johansson, 2019, p. 31). Even though Sweden is considered to be at the

forefront of digitalisation and the internet habits of Swedes have been studied

(10)

6 and analysed, users with disabilities are repeatedly being excluded from this type of research (Johansson, 2019, p. 5). It does not only depend on the lack of knowledge in how to access these groups, but also that “people with disabilities”

are continually seen as one large homogenous group when in reality they are made up of many smaller groups with widespread difficulties (Johansson, 2019, p. 56).

Within the WCAG guidelines there are three different levels of accessibility that can be achieved. The lowest level being A, the middle being AA and the highest level being AAA. The middle level, AA, is the one that the public sector has to comply with in order to adhere to the new law and the Web Accessibility Directive. This means that organisations and companies only have to ensure that this level of accessibility is met to comply with the law. Member states within the EU are free to choose whether they want to do the bare minimum to comply or to do more extensive work. The risk with following only what is in the WCAG guidelines could lead to insufficient web accessibility, and more extensive work could be necessary in order to achieve accessibility for all groups in society (Brajnik et al., 2012, p. 26; Rømen & Svanæs, 2012, p. 376).

With the Swedish government’s direction on digital innovation, it is clear that digitalisation will play an important and essential role in the development of the modern society. For example, public sector bodies are encouraged to engage with private individuals and businesses through digital services. The new law establishes the fact that digital products and services offered by the public sector should be accessible. However, it does not state any methods or evaluations that web practitioners should apply to their processes in order for digital products and services to become accessible. Neither does it set any standards for the level of knowledge required in web practitioner teams in order to comply with the law.

Previous research has shown that web practitioners today lack sufficient knowledge and experience when it comes to designing for and maintaining digital accessibility (Antonelli et al., 2018, p. 75; Steen-Hansen & Fagernes, 2016, p. 440; Swallow et al., 2016, p. 483). There are several aspects contributing to web practitioners struggling with implementing web accessibility, not only on the technical level but also on a societal, economical, educational and individual level (Steen-Hansen & Fagernes, 2016, pp. 440–441).

Steen-Hansen and Fagernes argues that accessibility is a large, complicated and multifaceted subject which requires web practitioners to have competence within several different areas. For example, making colleagues and clients understand why accessibility is important and getting the resources in the form of time and money to work with accessibility, having knowledge about what tools and methods to use as well as understanding the output of tools used. Swallow et al.

(2016, p. 484) also argues that web practitioners, even those with a passion for accessibility, might struggle due to insufficient knowledge and lack of outlined methods for development of accessible websites and applications. When Swallow et at. analysed why this is they concluded that web practitioners need tools that are easy to understand, use and integrate into their work process. In the addition to these areas Oswal (2019, p. 5) argues there is also a lack of knowledge among web practitioners about disabilities and impairments.

Furthermore, Oswal (2019, p. 1) states that when learning about user-centred

design, the users with disabilities are often overlooked since the industry tends

to view able-bodied users as the standard. Steen-Hansen and Fagernes (2016, p.

(11)

7 446) argue that these are not isolated factors but rather depend on each other and need to be integrated in the entire development process. This means that decisions regarding accessibility will have to be made not only once but throughout the process and from the very beginning.

Insufficient knowledge regarding developing for accessibility presents a risk for future digital products and services. If knowledge is insufficient and only a limited number of people have the experience required to evaluate and implement accessibility, there is a risk that the websites and applications of public sector bodies will not comply with the law. Or that they comply with the law, but only by the minimum requirements, and will still be inaccessible to users with some impairments leaving people excluded and leading to a future where the digital society is for a select group only.

1.3 Purpose and Research Question

To be able to successfully develop for accessibility, web practitioners today need a broad understanding of several different areas and accessibility needs to be considered throughout the process. This also means there are multiple points in the development process where decisions will have to be made regarding accessibility and the priority it should have in the project. The problem that this study will address is that depending on the decisions made at these points a website or application could be more or less accessible when catering to the needs of users with disabilities. Therefore, the purpose of this study is to explore and identify factors that influence the decisions made by web practitioners regarding web accessibility when developing and creating accessible websites, applications and services for the public sector. In addition, we will also study the implications of decisions made on the basis of these factors.

By using an analytical framework that was adapted for this study, we analyse the process to achieve web accessibility, how the web practitioners perceive developing for accessibility and whether it is a part of the development process that gets prioritised. The framework also considers the new legal requirements and with it we can analyse if the law has impacted development for accessibility and in what way.

By studying the factors influencing the work of web practitioners when developing accessible websites and applications for the public sector, our knowledge contribution is:

• An account of the factors influencing the development of accessible websites and applications.

• A model representing the complexity of the integration and dependencies of these influential factors.

We will base our study on the following research question: Which factors

influence the work of web practitioners when developing accessible websites

and applications for the public sector?

(12)

8 1.4 Previous Research

In this section we present previous research related to our problem domain. We describe core concepts related to accessibility, information architecture, WCAG and web practitioners. The research covers the difficulties users with disabilities might experience when using a website or an application, the components contributing to meaningful information architecture and the impact of WCAG when used as a tool for evaluating accessibility. We also present research on proposed strategies for involving users with disabilities in the development process as well as research on the knowledge of accessibility among web practitioners.

1.4.1 User Difficulties and the Web

Many users experience and express difficulties when using the internet and digital services (Begripsam, 2018, p. 15). The difficulties stretch over the borders and definitions of many different types of impairments. Examples of difficulties users might experience can be to “search for information, navigate, understand content, use e-services, handle passwords, design and layout or incompatibility of technical tools and aids” [translated from Swedish] (Begripsam, 2018, p. 17).

All of these difficulties experienced by users can be related to areas within information architecture such as usability, findability and understandability (Rosenfeld et al., 2015, p. 24).

Findability is all about finding, or re-finding, the right information within reasonable time and without frustration (Rosenfeld et al., 2015, p. 25).

Understandability deals with the act of understanding the things we find, which often happens when what we find is put in relation to other things (Rosenfeld et al., 2015, p. 53). Lastly, usability is used to describe the extent to which products and services help the user complete tasks in a desirable way (International Organization for Standardization, 2018). Usability in itself requires a human- centred approach and, as Benyon (2019, p. 108) argues, evaluation is a central part, often done through user testing since expert evaluations only covers a small part of usability problems (Benyon, 2019, p. 250).

Information architecture as described by Rosenfeld et al. (2015, p. 56), which findability, usability and understandability is a part of, could be understood by comparing it to physical architecture. Buildings are designed for specific purposes where each added element serves a function to the intended users of the building. If, in the physical world, it is forgotten that motor impaired people might not be able to use the stairs and visually impaired people cannot read the signs without special adaption, the built environment cannot be accessed by this group. In the same way in digital information architecture users with disabilities might have problems finding the right information, understanding complex language, navigate unorganised menus or memorising complex information such as passwords or usernames and so the digital environment cannot be used by this group.

Closely connected to information architecture is user experience (UX),

concerned with the emotions of the user interacting with a product or service

(Benyon, 2019, p. 2). Thus, UX is about designing for the interaction between

humans, design, technology and context (Benyon, 2019, p. 9). It is therefore an

(13)

9 important part of accessibility, since accessibility is about users interacting with products.

Inaccessible websites and applications not only affect people with visible impairments but also people with invisible impairments. One example of such impairments are those that affect the cognition. Illnesses such as dementia or impairments connected to loss of vision, hearing and mobility are common in the older generation of users. Statistics show that Sweden has a large number of people that belong to the older generation (Statistiska Centralbyrån, 2019b) as shown in Figure 10 (Appendix A: Charts). Statistics also show that the average life expectancy is on the rise (Statistiska Centralbyrån, 2019a) as shown in Figure 11 (Appendix A: Charts). This growing demographic together with the users with disabilities makes up a large group of users in society that would be affected if websites and applications remain inaccessible. According to a recent report on Swedes’ internet habits, those over 75 years do not utilise digital services from the public sector as much as other age groups (Andersson, 2019, p. 30). The report describes how this older group are the ones feeling the least helped in their daily life, when it comes to digital products. This, the report suggests, might be due to this group’s long habit of doing errands without using digital methods at all and that therefore, acceptance of digitalisation is harder and the time to learn is longer (Andersson, 2019, p. 33).

1.4.2 WCAG Not as Accessible as Promised

Numerous studies that set out to evaluate the efficiency of the WCAG guidelines found them lacking in several areas. A key finding being that the guidelines are not developed well enough to cover the problem areas for accessibility conformance, particularly when it comes to groups of users with cognitive disabilities or difficulties.

Brajnik (2009, p. 131) points out that the WCAG guidelines are being presented as automatically testable, but argues there is no scientific proof this is true.

Rather, they require human evaluation carried out by experienced evaluators. In Brajniks’ study, aimed at presenting the connection between testability, validity and reliability of the WCAG guidelines, the author concludes that there are large differences in the efficiency of the guideline criteria and the possibility to find problems related to accessibility. There are large differences in the effectiveness of the guideline sets (Brajnik, 2009, p. 137) and neither WCAG 1.0 nor WCAG 2.0 scored high on reliability and validity with reliability well under the desired 80% mark. Brajnik also argues the low reliability was not due to some criteria specifically requiring subjective interpretation, but rather for the whole set of guidelines in general.

In a study conducted by Rømen and Svanæs (2012), the different sets of WCAG

guidelines (1.0 and 2.0) were tested and compared with the help of users with

visual impairments, dyslexia and motor impairments. The study showed that

several of the problems found on a website could not be attributed to any of the

guideline criteria. When WCAG 2.0 was tested, only 15 of the 47 identified

accessibility problems were found to be due to the lack of adherence to the

guidelines (Rømen & Svanæs, 2012, p. 380). Furthermore, the authors

emphasise the problem with the guidelines often being used for policy making

(Rømen & Svanæs, 2012, p. 375). This could mean they are the only criteria

used for accessibility evaluation of websites and products, especially when

(14)

10 presented as a complete and satisfactory solution for sufficient accessibility (Vollenwyder et al., 2019, p. 353).

For people working to achieve and maintain accessibility on a daily basis, as well as for people educated in professions where digital accessibility plays an important role, we see a potential risk of WCAG being perceived and taught to be the only method for testing websites and products for accessibility. Bai et al.

(2019, p. 1) points out that the criteria within WCAG 2.0 is not enough to provide sufficient accessibility and emphasises the problem with the guidelines being the industry standard. This, the authors mean, could be an indicator that developers are not aware of any other methods of testing for accessibility. In their study, Bai et al. tested different methods for user testing for accessibility with the help of 53 people working in the industry (developers, designers, project managers).

Evaluation of accessibility by using the WCAG 2.0 guidelines scored lowest and was perceived by the evaluators as complex and hard, even by experts. The other methods (SiteImprove, Cambridge Glasses, screen readers, dyslexia simulators and personas) scored much higher (Bai et al., 2019, p. 11). In conclusion, Bai et al. (2019, p. 13) notes that the combined use of numerous accessibility evaluation methods would improve the chance to find problems with accessibility and is preferred instead of using one method alone. The evaluators also reported an increase in empathy for their users. Bai et al. also note that teams routinely should seek out other methods that are not the WCAG guidelines when working with accessibility.

Vollenwyder et al. (2019) presents the same type of criticism we noted in other studies about WCAG – that the guidelines themselves are not enough to provide a satisfactory user experience, that they are difficult to interpret and thus to test and evaluate, and that there is no real support for users with cognitive disabilities or difficulties. Vollenwyder et al. (2019, p. 353) also emphasises that web developers have to accept there is no “one size fits all”-solution to web accessibility and that working with accessibility is a process that requires dedicated effort from the beginning. They believe that if users with disabilities are involved early on, the status of developing for accessibility could be raised, while increased accessibility leads to an improved user experience for all user groups – with or without disabilities (Vollenwyder et al., 2019, p. 358).

Similarly to what Vollenwyder et al. (2019, p. 358) presents, other authors have also raised questions about involving users with disabilities late in the process.

For example, Oswal (2019, p. 2) argues that work on accessibility often occurs late in the development process, whereby developers and designers miss out on the opportunity to involve the opinions and requirements of the people most in need of the adaptation. Thus, any solutions to problems will be retrofitted into the design rather than being incorporated as conscious choices in the conceptual stages of the development process. Oswal (2019, p. 6) proposes a strategy where involvement of users with disabilities happens early on and urges the developers to realise that these groups possess expert knowledge of their needs of adaptation and should be seen as equals and co-designers. Furthermore, Oswal (2019, p. 5) believes web practitioners need to be educated on issues related to disabilities as well as web accessibility in itself.

Based on the studies and articles we have reviewed in our research, it seems

WCAG 2.0 is commonly used as the lowest benchmark when developing for

(15)

11 accessibility. Adherence to the guidelines is presented as a one size fits all solution that will ensure accessibility for all users. However as previous research shows a one size fits all solution will not work (Vollenwyder et al., 2019, p. 353).

Instead, the focus should be on highlighting the connection between accessibility, usability and user experience and how these qualities interact to develop improved products for all users (Aizpurua et al., 2016, p. 14).

1.4.3 Research Presents New Strategies for Developing for Accessibility

Studies have shown strong connections between values traditionally associated with user experience and those associated with accessibility. This is true especially in the way pragmatic and hedonic qualities are perceived by users, according to a study by Aizpurua et al. (2016, p. 14). Hedonic qualities such as

“professional”, “presentable”, “integrating”, “creative” and “exciting” as well as pragmatic qualities such as “practical”, “manageable”, “simple” and “clear” had strong connections with perceived web accessibility (Aizpurua et al., 2016, p.

17). Furthermore, the authors highlight the importance of understanding that for people with disabilities, the internet is not only a tool for information gathering and essential everyday tasks, but also a space for communication, leisure and fun (Aizpurua et al., 2016, p. 14). Based on this it should not come as a surprise that aesthetics, innovation and understandable structure is key to the user experience for groups with disabilities as well.

When websites and applications are retrofitted to be accessible, they become something they were never intended to be from the beginning and problems arise. In light of this, new strategies for working with accessibility is proposed by several authors. Oswal (2019, p. 6) suggests a method involving users with disabilities early and Steen-Hansen and Fagernes (2016, p. 442) proposes a process-oriented approach. Johansson (2019) also suggests new strategies, presented in his thesis on the topic, focusing on what he calls participatory design, using this to develop a strategy for further work in the accessibility field in order for all users to be able to access digital information and services.

The current industry standard seems to be based on just sometimes including users with disabilities through user testing, workshops or surveys. This way of working supports the backwards idea that these people are merely convenient data containers rather than valued participants in the design and development process (Johansson, 2019, p. 103). Johansson (2019, p. 6) also explains that inclusion of users with disabilities are often not representative of the users with the most severe difficulties, due to the lack of knowledge in how to reach and include these groups.

Together with Begripsam, an organisation working for accessibility in the digital

society through for example workshops and research, Johansson has developed

new guidelines based on participatory design. Participatory design means that

people with disabilities should be included as valuable experts in the

development process. Including the expertise of these groups could shed new

light on inequality and unequal power relationships and inspire new ways of

solving problems (Johansson, 2019, p. 103). Our estimate, is that this way of

working would also minimise normative thinking in web practitioner teams,

lower the stigma of disabled people as well as minimise poorly designed retrofits

of digital products and services.

(16)

12 1.4.4 Current Knowledge and Experience

In a survey study on web developers’ knowledge and experience on web accessibility Antonelli et al. (2018, p. 75) found that few respondents reported having any expertise in the subject. Over half of the participants stated they have never developed an accessible website (Antonelli et al., 2018, p. 76), however the participants’ mean age was 29.51 years old (Antonelli et al., 2018, p. 75) so there is a possibility they have not been actively working as developers for a long time. There is no data in the study on the length of their experience only the experience level the participants believe they are at. The study also presents data on the priority of addressing accessibility in projects the developers have worked on, and it shows the majority express that there is a low priority (Antonelli et al., 2018, p. 77). It is clear that the knowledge and experience among the participants is low, however a majority still agreed developers have a shared responsibility for accessibility in projects.

A reason why developers might lack knowledge about developing for accessibility can be the complications of including this subject in a computer science or developer program (Gay et al., 2017, p. 1). Gay et al. further states that including accessibility studies in these types of programs will take time as it requires experts with deep understanding of web technologies as well as disabilities and that these experts are rare in the industry. Steen-Hansen and Fagernes (2016, p. 440) states that since accessibility is a large and complex area it might feel intimidating and developers can find it difficult to understand where to start. Since understanding how to apply accessibility in a development process takes time there might also be a view that it is “difficult, expensive and in the way” (Steen-Hansen & Fagernes, 2016, p. 440).

Although there have been many guidelines on how to achieve web accessibility web practitioners are still finding it difficult and web accessibility is not improving as it should (Swallow et al., 2016, p. 483). Research shows that web practitioners often find the outcome of accessibility tools difficult to understand and interpret and that the tools do not increase their understanding of accessibility issues (Swallow et al., 2016, p. 483). Swallow et al. (2016, p. 484) presents three key areas; “language”, “organisation” and “volume”, within accessibility tools and guidelines that hinders the web practitioners in their work with accessibility. Swallow et al. remarks that the language in guidelines is often ambiguous and too domain-specific for web developers which leads to the language being unsupportive. Furthermore, the organisation of the information in the guidelines and tools tends to be by the domain, “such as ‘perceivable’ or

‘operable’” (Swallow et al., 2016, p. 484) rather than how you would approach the structure or creation of a website or application. Finally, the vast amount of information and items to test in guidelines and tools often creates information overload. The large amount of information can also lead to lower confidence and the web practitioner using the guideline might turn to an automated accessibility tool instead.

1.5 Delimitations

In this study we do not research the Law on Accessibility to Digital Public

Service, the Web Accessibility Directive or the European Accessibility Act in

themselves. Neither do we study specific cases, websites, applications or digital

(17)

13 services, since our focus of the study is web practitioners and the factors influencing the work they do.

This study focuses on the web practitioners that mainly carry out work for the public sector, and particularly are involved in projects that are required to adhere to the Law on Accessibility to Digital Public Service, since this is the focus of our study. In order to gather relevant data, we need to include web practitioners that work or has worked in or with the public sector and have experience developing for accessibility.

The European Accessibility Act is not considered when discussing laws on

accessibility in this study as it is a law affecting the private sector. In addition, it

is still only a directive and not an official law at this point in time.

(18)

14

2 Analytical Framework

In this section we present the framework used when analysing our results, as well as putting them into context. The Prerequisites for Accessible Digital Development, in section 2.1, is our own framework developed specifically for this study. It is inspired by the work of Johansson (2019, p. 76), and will be used as a means of understanding, comparing and analysing the effects of human interaction and human evaluation in the process of digitalisation and work for accessibility.

We chose to incorporate the four themes of the WCAG guidelines,

“perceivable”, “understandable”, “operable” and “robust”, into our framework as a means of understanding and analysing how the work of web practitioners has been affected from a legal standpoint.

2.1 Prerequisites for Digital Development of Accessible Products and Services

Johansson (2019, p. 76) developed a framework for understanding the different factors involved when a person with a disability is interacting with accessible digital products and services. In his thesis, he calls it the “14 Prerequisites for Participation in the Digital Society”. What Johansson’s framework describes is not only what happens when a person interacts with a product or device, but rather the societal, economic, political and individual factors that will ultimately contribute to the extent to which a person can participate in the digital society and thus benefit from digital accessibility.

Johansson’s framework is well suited when trying to describe the user side of the digital society and the factors that are part of digitalisation when it is happening at a user level. However, it is not suited to describe what happens on a web practitioner level. What we found in previous research was how the use of WCAG or legislative processes alone did not automatically achieve accessibility (Rømen & Svanæs, 2012, p. 385). Accessibility is not only the result of legislation and guidelines but the development process relies heavily on other contributing factors on the societal, educational, economical and individual level (Steen-Hansen & Fagernes, 2016, p. 440).

In the early stages of data collection, we tried coding and categorising the data

using Johansson’s (2019, p. 76) framework as a tool for analysis, but soon

realised they were not entirely compatible. The codes derived from Johansson’s

framework could not be applied properly to the data in this study, since the codes

had a clear focus on the individual user. However, because we are researching

web practitioners, a larger professional group working in the same field, it makes

Johansson’s framework, which is focused on the individual user, incompatible

with our data. We saw a connection between Johansson’s way of thinking and

our own but realised we needed different kinds of categories suitable for digital

development specifically. It is still true that contributing factors of accessibility

are societal, educational, economical and individual, but the scope is no longer

the individual user interacting with a product or service but rather the web

practitioners developing that product or service. This led us to adapt the existing

framework to better fit this study as a tool for analysis, inspired by the one

(19)

15 Johansson presents. We call this framework Prerequisites for Digital Development of Accessible Products and Services, as presented in Figure 1.

Figure 1

Prerequisites for Digital Development of Accessible Products and Services

We decided to discard the categories that were too user specific, such as “access to the internet”, “support”, “applications” and “devices” as well as “practice and praxis”. We also chose to discard the category “accessible design” as this act as a perspective from which we view the categories, rather than a category or factor in itself. In place of the categories we discarded, we decided to add categories that are specific to web practitioners and the web development process.

Research suggests that having access to and an understanding of the right tools for accessibility evaluation is essential in the development process, as well as having someone with expertise in accessibility to ask for advice (Steen-Hansen

& Fagernes, 2016, p. 442). Furthermore, Steen-Hansen and Fagernes (2016, p.

444) argues that involving users with disabilities will bring web practitioners closer to the people they are developing the product for by increasing understanding of difficulties that may arise when a user interacts with a website or application. Therefore, in the “access” category, we added “access to users”,

“tools” and “expertise”.

Bai et al. (2019, p. 2) mentions the importance of selecting the appropriate

methods for the context that will complement each other in the work process. In

addition to applying the appropriate methods and strategies, web practitioner

teams need to collaborate and communicate to work with and understand

accessibility (Steen-Hansen & Fagernes, 2016, p. 443). Furthermore, Steen-

Hansen and Fagernes argues that accessibility is not the responsibility of one

individual but rather the team as a whole. In order to analyse the work process,

a code to identify methods and strategies when working with accessibility was

(20)

16 required, which is why we added a “work process” category consisting of

“practice and process” and “team and collaboration”.

Research suggests that factors outside the control of web practitioners, on the client side, might have an impact on accessibility. Swallow et al. (2016, p. 483) mentions the attitude of clients being one such factor. In addition, Steen-Hansen and Fagernes (2016, p. 447) argue that resources such as time and money if allocated to accessibility might have a positive impact. Therefore, the “client impact” section was added consisting of “client knowledge”, “client attitude”

and “affordability”, meaning resources in terms of time and money. We argue attitudes are based on previous knowledge, but we needed a way to separate what clients know from their opinions.

The framework is, after the changes made, appropriate for describing the prerequisites for work on accessibility carried out by web practitioners. This study focuses on the development side, for which user specific categories are redundant. Table 1 shows the differences between Johansson’s original framework and our adaptation. Further figures and explanations of categories in our framework can be found in Appendix D: Coding Manual.

Table 1

Differences in Categories: The Original Framework and our Adaptation

Johanssons’ framework Our framework

Access to the internet Access to expertise

Access to devices Access to tools

Access to applications Access to users

Access to support Team and collaboration

Education Education

Expectations Expectations

Experience Experience

Competence Competence

Attitude Attitude

Self-efficacy Self-efficacy

Policy, law & regulations Law

Practice and praxis Practice and process

Affordability Affordability

Accessible design WCAG

- Client Attitude

- Client Knowledge

(21)

17 We argue that in the same way a user has certain base prerequisites that have to be fulfilled before being able to access the accessible design itself, the web practitioners in our study face a similar situation where certain prerequisites will have an impact on the perceived accessibility of the final design. All of the prerequisites are tightly interconnected and dependent on each other, but for ease of understanding we divided them into six categories.

2.1.1 Access

The first three prerequisites are regarding access. Access to expertise, access to tools and access to users. This section covers necessary resources the web practitioner needs in order to evaluate the websites and applications they are working on. Access to expertise deals with the fact that a web practitioner might need someone with special knowledge about design, functions, disabilities or technology to help them or support them in their decision making. Access to tools covers the fact that web practitioners need to know which tools, digital or analogue, web or software, to use in order to evaluate accessibility. Access to users deals with to what extent web practitioners are able to test their websites and applications with real users.

2.1.2 Knowledge: Your Own and Others

Education, experience, expectation and competence are all connected to knowledge. Being a web practitioner in the modern society calls for constant updates of one’s knowledge - no matter who you are, it is a lifelong learning. If you gain constant knowledge and get the skills and chances to practice them, chances for success is greater. In this context, other people’s help and knowledge contributions could also be critical to one’s own understanding and participation in the development process. The expectations set within projects could also have an effect on productivity and sense of contribution.

2.1.3 Law & Guidelines

The new law, based on the Web Accessibility Directive, is a step in the right direction towards a more inclusive society where accessibility is a fundamental part of every digital product and service. This law also provides the web practitioners with a new type of legal requirement in their projects. Thus, the law section covers the parts of development that is directly connected to legislation and the influence it might have on the process.

The Law on Accessibility to Digital Public Service is based on WCAG, serving as a technical set of guidelines intended to make sure the websites and applications fulfil certain criteria. We chose to include these guidelines in the framework since they are central to the process of adhering to the new law. The criteria in WCAG are split into four categories with subsections, these categories are:

• Perceivable: defined as “users must be able to perceive the information

being presented (it can't be invisible to all of their senses)” (W3C Web

Accessibility Initiative, n.d.). The subsections in this part are text

alternatives, time-based media, adaptable and distinguishable.

(22)

18 • Operable: deals with the guidelines on how users should be able to operate the interface (W3C Web Accessibility Initiative, n.d.). It covers the subsections keyboard accessible, enough time, seizures and physical reactions, navigable and input modalities.

• Understandable: defined as “users must be able to understand the information as well as the operation of the user interface (the content or operation cannot be beyond their understanding)” (W3C Web Accessibility Initiative, n.d.). The subsections in this part are readable, predictable and input assistance.

• Robust: deals with the fact that the content needs to be compatible with assistive technologies and interpreted by user agents such as text or screen readers. It is important that the content is compatible with the latest versions of user agents (W3C Web Accessibility Initiative, n.d.).

This theme only contains the subsection compatible.

2.1.4 Opinion

The opinion section covers attitude and self-efficacy. Attitudes, our own and others, towards accessible design, visual or technical solutions, management and priorities to mention a few can heavily influence the decisions made in the development process as well as people’s perception of each other.

Self-efficacy, or our own perceptions of how the skills we have can influence our level of ambition, feeling of inclusion in projects and participation. The measurement of one’s own skill set against others is a constant inner struggle and can be a factor when working in a team.

2.1.5 Work Process

The work process section covers both the actual work practices and processes applied within an organisation as well as team and collaboration. Practice and process deals with the methods, processes, standards and strategies used by web practitioners, a team or an organisation in order to secure a higher level of accessibility. It describes how people work.

Team and collaboration are closely connected to process. Developing an accessible digital product or service requires several different work roles and skill sets to come together and collaborate. Having a strategy for working together as a team is critical for success.

2.1.6 Client Impact

Client impact deals with the customer side of digital development of accessible products and services. The decisions and priorities within web practitioner teams are greatly influenced by the client side, in a variety of ways.

Affordability deals with resources allocated to a project, whether it is time or

money. Client knowledge deals with the extent to which the customer knows

what they are asking for, how to ask for it and what to expect when briefing or

sending a requirement specification to an organisation working in digital

development. Client attitude is connected to client knowledge and covers what

(23)

19 might not be obvious gaps in client knowledge, but rather conscious decisions

or priorities made in connection to accessibility.

(24)

20

3 Method

In order to answer our research question and study the factors that impact the decisions made by web practitioners when developing accessible websites, applications and services for the public sector, we chose a mixed method approach. This section outlines the details of the methods applied in our study, describes the process of recruiting the study participants and provides a detailed description of how the data in the study was collected and processed as well as with which analytical tools the data was analysed. Lastly, we present a reflection of the research ethics related to this study.

3.1 Research Strategy

In order to gather data to answer our research question, we applied both quantitative and qualitative methods by using a mixed method approach. By using a quantitative method, in the form of a survey, we could get a broader understanding of the decisions made by web practitioners as this method allowed us to gather a larger amount of data from many web practitioners. In contrast to the survey, a qualitative method in form of semi-structured interviews allowed us to expand our knowledge about what was discovered in the survey data to gain a deeper understanding of the influencing factors. While the survey was used to gain knowledge on a surface level the interviews allowed us to explore more complex questions related to work process, attitudes and experience by having insightful conversations with only a few web practitioners.

Denscombe (2018, p. 220) suggests that a combination of methods can generate data from different perspectives that will provide a more comprehensive and nuanced understanding of the problem studied. Further Denscombe (2018, p.

225) argues that the purpose of combining methods can be that one component of the study acts as a foundation for another component. In addition, Denscombe states that part of the data can be used to control the direction of subsequent research, as in this study, where a survey acted as a foundation for interviews.

As Bryman (2012, p. 645) suggests, the combination of a quantitative and a qualitative method allows for the qualitative data to provide context for understanding the studied phenomena.

In this study we applied the mixed method approach by starting our data collection with the survey and followed up with the semi-structured interviews.

This explanatory sequential design let us create a structure where the data from the survey acted as a foundation for the data from the interviews. Denscombe (2018, p. 224) emphasises that studies with both quantitative and qualitative methods do not have to assign both methods the same amount of significance within the study. Instead it is more likely that one method is considered to be more prominent than the other and relied upon as the main method. In our study, the qualitative method is the primary method and the quantitative method takes on a supportive role.

The framework Prerequisites for Digital Development of Accessible Products

and Services as presented in section 2.1 was used as a foundation for coding,

categorising, and creating themes of the data gathered during our study. This is

further explained in section 3.4.