Validity and reliability

Validity, reliability and generalisability are important aspects to discuss in relation to the choice of method and research design. In this section I will primarily focus on the measures I used in the papers to increase the validity and reliability. Validity is about whether the findings represent “the truth”, which is a complex term in relation to qualitative research. Instead, I tried to address validity in all parts of the research process, for example, by carefully selecting categories for the analysis and data triangulation. Reliability is often regarded as something that ensures that another researcher would be able to replicate a study and draw the same conclusions.

However, strictly speaking, this is not possible for a qualitative interview study.

Instead, reliability in a qualitative study can be about ensuring that other researchers can connect the empirical material to the conclusions drawn by the researcher by carefully describing the studied case, designing the questions in the interview guide, as well as consistent analysis in which coding is confirmed with other researchers (Kvale and Brinkman, 2009). Below, I will describe the measures taken in the papers to ensure the validity and reliability of the research. The measures include triangulating information through data sources and interviewees, accounting for cases and material, adopting systematic analysis procedures, as well as discussions between the involved researchers. The possibility of generalising my findings will be further discussed in Chapter 6.

One of the main ways of increasing the validity in the papers was through data triangulation. One of the challenges of interviews is whether the interviewees remember information correctly. Thus, in papers I, II and IV, when the interviews were used as the main method, factual information was triangulated with information from documents to ensure accuracy. However, since my main interest was with the stakeholders’ view of what was challenging and what worked well in the introduction, their version, based on their position as agents, was my primary interest. Thus, the position of the interviewees and their role in the process became very important. In papers I and IV, the information in the interviews was also validated by interviewing several stakeholders with different perspectives chosen through purposive sampling. This was not the case in paper II, in which only one stakeholder was interviewed in each region. However, this approach was motivated by the aim to verify the information from paper I by covering several regions.

Another example of the triangulation of findings can be seen in paper III in which the results of the document study was complemented by a workshop in order to achieve a better understanding of what had led up to the information included in the documents.

Careful descriptions of the studied cases in each paper was an important way of improving the reliability of the studies. This thesis also includes an extensive background section to further show the context in which the research was conducted. In paper III, which used content analysis, it was important to conduct

the analysis in an objective and systematic way in order to achieve the most reliable results possible. Objectivity includes providing a clear motivation for the selection of documents and categories and ensuring the inclusion of the same type of information; meaning that if another researcher used the same categories, the analysis should result in the same outcome (Kassarjian, 1977). It was therefore important to set clear boundaries when the documents were chosen. This was achieved by including the procurement of buses from all Swedish transport regions over a ten-year period. We managed to include 95% of tenders, excluding school bus services.

Another important way to ensure reliability and validity was to adopt a systematic analysis procedure. When interviews were used as the main method, as well as for the content analysis, the choice of categories for the analysis was important. The categories were based on previous research (for example, GPP literature) or official documents (sector guidelines for environmental technical specifications). However, the entire process was important. Analysing interviews is not a process that only takes place in the final stage. It is an ongoing part of the entire research process.

Thus, validity and reliability must be ensured throughout the entire research process when choosing interviewees, conducting interviews, adopting a systematic analysis procedure and validating the findings (Kvale and Brinkman, 2009). One challenge during the process of interviewing was, for example, to avoid guiding the interviewees’ responses while being influenced by my preconceptions and prior knowledge. This was addressed by designing an interview guide based on topics that started with more open questions.

Lastly, discussions between the involved researchers played an important role in ensuring the validity of the results. In the content analysis in paper III, the documents were partially cross analysed by two researchers and the findings were discussed throughout the data collection process and analysis. The method section in paper III contains a further description of how this was done. Also, in paper I, the analysis was conducted by two researchers, and themes, coding and findings were discussed continuously throughout the process. In paper II, I was the sole author, meaning I conducted the interviews and analysis largely on my own, with supervisors and colleagues functioning mainly as discussion partners and providing feedback.

Finally, all the papers also collectively offer an additional triangulation of the findings. For example, papers I, II and III cover very similar themes but have different research designs. The qualitative interviews used in papers I, II and IV also strongly focused on context-specific issues and I have been careful about making generalisations from the studies. However, Halkier (2011) claims that it is possible to generalise from qualitative studies but that it must be more specific and context bound than quantitative studies and can be seen as producing context bound topicalities. I also believe that, collectively, the papers and their scope will make a good basis for more general discussions about the challenges of introducing

renewable fuel in the public transport sector. The quantitative document analysis in paper III was also a way to help confirm some of the patterns identified in the interviews.

This section has focused on how interviews and document studies have been used in the papers and the possibilities and limitations of my choices of research approach, methods and materials. In the next chapter, I will summarise the aims and results of the four papers included in the thesis.