UPPSALA UNIVERSITY
MASTER’S PROGRAMME IN GLOBAL HEALTH
THESIS
“Recruitment of research participants into randomized controlled trials of internet-based cognitive behavioural therapy (iCBT) for depression:
a systematic review and meta-analysis”
Authors: Ida Englund
Date: 2020-05-17
Word Count: 10 046
2
Abstract
Introduction
Research about health has become a very important part of the world today. Theres a constant need for new treatment methods and evidence. Recruitment is one of the most challenging parts of conducting a trial, especially in trials regarding mental health. The result of this is often a waste of money and resources in research.
Method
This study is a systematic review and meta-analysis investigating the recruitment in randomized controlled trials on internet-based CBT interventions for depression.
Results
The recruitment rates were calculated as number of participants screened divided by number of participants randomized into the trial. The overall recruitment rates of all the trials was 54.3%. The analysis of the recruitment moderators shown that a clinical recruitment setting together with referral as recruitment personnel.
List of tables and figures:
Table 1: Study characteristics Table 2: Participant characteristics Table 3: Recruitment rates
Table 4: Recruitment numbers and recruitment rates Figure 1: PRISMA flow diagram
List of abbreviations
CBT = Cognitive behavioural therapy
iCBT = Internet-based cognitive behavioural therapy WLC = Waitlist control
TAU = Treatment as usual CI = Confidence interval
EBM = Evidence based medicine RTC = Randomized controlled trial SD = Standard deviation
Table of contents 1. INTRODUCTION
1.1 Research on research……..…….………..……….1
1.1.1 Evidence based medicine………..………..1
1.1.2 Study designs for evidence based medicine………...………2
1.1.3 Reporting issues……….……….3
1.1.4 Potential biases………...………3
1.2 Recruitment……….4
1.2.1 The importance of recruitment………4
1.2.2 Clinical recruitment…...………..5
1.2.3 Community recruitment………..……….5
1.2.4 Recruitment into mental health trials………...………5
1.2.5 Optimising recruitment………...……….6
1.3 Internet-based CBT for depression………...…………..7
1.3.1 Depression and Global health………..7
1.3.2 Screening tests for depression………..………8
1.3.3 Internet-based CBT………..………9
1.3.4 Supported vs. unsupported iCBT………...………10
1.4 Aims and outcomes……….………..………12
1.4.1 Aims………...……….12
1.4.2 Research questions……….……….12
1.4.3 Primary outcome………..………..….12
1.4.4 Secondary outcome………..………..………….12
2. METHOD………..………….12
2.1 Design …….……….12
2.1.1 Protocol writing and registration……….………..………13
2.1.2 Search strategy…..……….13
2.1.3 Inclusion and exclusion criteria………...………..13
2.1.4 The screening process ………..……….14
2.2 Data extraction ……….15
2.3 Data synthesis………...15
2.4 Ethical considerations………...16
3. RESULTS ………..………17
3.1 Included studies ………...………17
3.2 Study characteristics……….………18
3.3 Categorization of recruitment moderators………19
3.4 Participants characteristics ………...………23
3.5 Reporting issues ………...…………25
3.6 Recruitment rates………..26
3.7 Moderator analysis………28
4. DISCUSSION ………30
4.1 Summary of key findings………..…30
4.1.1 What is a successful recruitment? ………..30
4.1.2 Level of delivered support………..31
4.1.3 The consequences of bad reporting………31
4.1.4 Potential bias of the included studies…..………32
4.2 Comparison with existing literature…………..………32
4.5 Importance for the Global Health field……….………33
4.6 Implications for further studies………...……….……….34
2
4.7 Strengths and limitations………...… ………34
4.7.1 Strengths …………..………..34
4.7.2 Limitations……….………....……….34
5. CONCLUSION………...………35
6. ACKNOWLEDGEMENTS………...……….35
1
1. Introduction
1.1 Research on research
Research can be defined as a systematic investigation developed to gain knowledge and understand concepts in a specific area of interest. In other words, the primary objective of research is to produce new knowledge. This includes the generation of ideas and information and testing and evaluation that leads to new improved insights that have a relevance to society (1-9). All research requires detailed planning of the design, implementation, data collection, and reporting of findings to be successful. Even if science is one of the key drivers of human and societal progress there is little evidence on how to best conduct research (10). If research fails to be conducted or reported properly it doesn’t have any real value to the society. It is estimated that as much as up to 85% of all research might be wasteful because of poorly designed studies, inaccessible results, and biased reporting (11). Meta-research can be easily explained as “research about research” and is a new discipline with the primary objective of understanding and improving how to perform, communicate and evaluate research (12).
1.1.1 Evidence-based medicine
Medical research has a high value to society and provides knowledge and practice that saves lives daily by providing important information about disease trends, risk factors, treatments, and much more. However, medical research comes with several issues, one of the most common one is the division between clinical research and clinical practice (13).
In the past, medicine was based on the authority of the master, for example a cultural leader of the ancient societies. Back then, experiments were not usual or acceptable and it was mainly literature that provided the medical expertise. The practitioner was just an executor and there was hardly any progress in the medical field. Slowly, free thinking and human autonomy developed within societies. This resulted in more decision making for the practitioner and medicine became more experience based. Still, medicine was by no means standardized and the outcomes could vary depending on the skills of the practitioner and the circumstances.
The birth of statistics and the scientific method in medicine gradually shifted the focus to
Evidence-Based Medicine (14). Evidence-based medicine (EBM) can be defined as “the
process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions” (15). EBM was developed as a reaction to the disparity2
between research and practice, it is an interdisciplinary approach to medicine and research that helps to optimize the decision making and improves outcomes for the patient (16, 17).
This is done by integrating the practitioner’s expertise with the strongest and best evidence from research (17). Three major tools in conducting evidence-based medicine is systematic reviews, randomized control trials (RTCs) and meta-analysis (16).
1.1.2 Study designs for evidence-based medicine
Randomized controlled trials (RCTs) are trials in which the subjects are randomly assigned to one of two groups and are regarded as the most effective and reliable method for evaluating interventions (18, 19). Other study designs can also assess the associations between an intervention and an outcome but it's hard to rule out the possibility that it can be caused by a third factor. The randomization process in RTCs makes sure that there are no systematic differences between the intervention and control groups. The two groups in the trial are often the experimental group that receives the treatment or intervention and the other group is the comparison group or control group that is receiving an alternative treatment or no treatment at all. The results in these trials contain the follow-up comparisons of the two groups (20, 21).
Randomized control trials are a very effective research design but even this design has its limitations. Most of the limitations are due to ethical and practical concerns. In trials with one intervention group and control group there is an issue concerning the fact that the trialists are giving participants an intervention believed to be inferior to current treatment. This can be perceived as unethical in many cases because everyone should always receive the best treatment (22). A randomized controlled trial may also be ethical but not feasible because of the difficulties with recruitment and randomization. There is a risk that the new intervention becomes widespread and that creates an unwillingness for participants to experiment with alternative treatments or interventions (23). Preferences from the participants can also limit the recruitment and create a biased study (24). RCTs are also generally more costly and time- consuming than other study designs maybe therefore it is important to time and use these trials correctly. Questions you must ask yourself when conducting an RTC is if the intervention is developed well enough to permit an evaluation and is there preliminary evidence of the intervention is likely to be beneficial (25).
Systematic reviews collect all the empirical evidence on one specific topic or according to one or more research questions that fit the pre-specified eligibility criteria. The systematic
measures methods of these kinds of reviews are used to minimize bias. (26,27). This method
3
was pioneered and developed by Cochrane and was developed for the need to ensure that decision making was based on evidence (28). By systematically assessing current published research systematic reviews aims to present a summary research knowledge of intervention, treatment, prognostic factor, diagnostic test, or other results of research in the healthcare field (29). During the systematic review process it is common to use a statistical meta-analysis of the studies collected (30). A meta-analysis is a set of valid and objective techniques used to combine the results of different reports (31). This method is mainly conducted on RTCs to obtain more reliable results (30). The primary aim of this type of analysis is to increase the statistical power to answer new questions arisen from the included studies (32). The reporting of systematic reviews and/or meta-analysis should be described in detail, first in a flowchart of the literature search, the inclusion and exclusion criteria, and a table of the characteristics of the included studies (33). If systematic reviews and meta-analysis are properly designed and conducted it can increase the power and accuracy that one single RCT have difficulties to perform (33).
1.1.3 Reporting issues
As stated above RCTs provide the most reliable evidence when it comes to the efficacy of healthcare interventions, this however is only if the trial is properly designed and conducted.
If there is a lot of information missing in the reporting of RCTs or if it's not accurately described it is hard to draw any conclusion from it (34). Many reviews have investigated the insufficient reporting of clinical trials, only around 21% of 519 reports in Pubmed by the year 2000 described the methods of randomization and in 2006 only 34% of them reported this (36). The CONSORT statement (36) was created early in the 1990s as a set of
recommendations on the reporting of trials (37). CONSORT consist of a checklist of items that should be reported in RTCs and a diagram for the flow of participants in the trial (36).
The primary objective for CONSORT is to give guidance to authors about how to improve the reporting of trials by making them clear, complete, and transparent (38, 39). Year 2010 the new updated version of CONSORT came, and this is the version most authors are using today to sufficiently report their trials (40).
1.1.4 Potential biases
In every trial there are a degree of bias and this can result in a misleading assessment of an
intervention (41). Bias can be described as the deviation of the results from the truth, bias
occurs in two main forms. First the selection bias, this occurs when the two groups in a trial
4
differs systematically. The second is the observer or information bias that occurs when there is a systematic difference in how the information from the two groups are being collected or studied (20). In RTCs the use of random allocation and blinding is so far the inly method that can eliminate these biases (41, 42). However, RTCs are not completely free from biases. The healthy user bias or also called the healthy user effect is the theory about that the participants who receive different kind of interventions and choose to partake in trials are already those who are healthy. People that already exercise, has a healthy diet, avoid tobacco and
overconsumption of alcohol may be those who is responding to invitations to health trials.
This can create problems with validation of the treatment. If a study is evaluating an effect of a treatment on a related outcome, in this case iCBT on depression, without adjusting for related preventive behaviours the effect for the treatment can become overstated (43). Another aspect of this is the healthy adherer effect which means that already healthy participants are more likely to adhere to treatment and stay in the trial (44).
1.2 Recruitment
Recruitment is an important part of a trial and a good estimate of the target sample is important. This does it possible to set a timeframe and a budget for the trial but also other resources such as number of study centres and number of people working with the research project (45).
1.2.1 The importance of recruitment
Research that is not conducted correctly and efficiently is wasteful for both researchers and participants. There are many cases where well-designed studies addressing important questions fail to meet recruitment and retention standards (46). It is unclear why some trials manage to recruit participants well and others fail but published data shows that it is a minority of trials that recruit successfully (47, 48, 49). Recruitment and retention of participants is often the most challenging part of a trial, this is much due to the under-
estimation of the recruitment challenges while planning a trial (50). Poor recruitment in a trial can lead to consequences, one of these is that the study becomes underpowered which may report clinically relevant effects as statistically non-significant. Another problem with underpowered trials is that it raises an ethical problem: the researchers have exposed
participants to an intervention that has uncertain benefits. With poor recruitment sometimes
the trial must be extended, this increases costs. Despite all this, the importance of recruitment
in trials has received relatively little attention in research (51).
5
1.2.2 Clinical recruitment
There are many ways of recruiting participants to a study, from older and traditional methods in a clinical setting to more recently developed ones such as online recruitment (52). It is common that clinical trials recruit within academic medical centres, hospital systems or other institutions focused on clinical care (53). In these recruitment settings is common to use healthcare staff to conduct the recruitment. When the researcher established the planning of the study someone from the healthcare staff, for example a GP, nurse, or caregiver, carries out the recruitment, screening, and data collection in the clinical setting of the study (53, 54).
These settings are beneficial because they can offer access to patients, clinical data, and research teams and recruiter may influence the patients to partake in the study (53, 54, 55).
However, in some cases the health care providers are not that acquainted with trials and might not have enough details about a specific study and therefor has trouble recruiting participants.
Also, healthcare staff has a big workload in many cases and might not see recruitment into trials that important, just additional work (56).
1.2.3Community recruitment
Another approach to patient recruitment is a community-based recruitment strategy.
Community engagement and recruitment in research allows members of the community to understand health issues and inform researchers about the community priorities (57). There are several ways to recruit participants from the general population. Community activities such as setting up booths at community meetings, health fairs or similar environments has been widely used within community recruitment. Other strategies can be to use ads, printed materials such as flyers or different media platforms. Internet has become a promising way of recruiting participants (58). Social media is one way to recruit participants online. In
recruitment on social media adverts are often posted and promoted to specific target audiences by demographics such as region, age, or gender (59). The widespread access to the Internet nowadays offers alternative recruitment strategies into trials. The technical advantages in the recruitment stage but also in the data collection can, if done right, reduce the administrative procedures and also improve the quality of the data (60). Recruitment online can be done by placing ads on websites or by sending automatic messages over the internet (61, 62). Some studies have compared online and offline recruit methods, these studies have shown that the online recruitment is simpler, has higher recruitment rates and lower costs (63, 64, 65, 66).
1.2.4 Recruitment into mental health trials
There have been difficulties identified in the development and testing of new eHealth
6
programs during the grant timelines (67, 68). Recruitment is by no means easy in any trial, but mental health trials seems to have even bigger challenges in recruiting participants (69, 70). It is difficult to determine the causes of poor recruitment into mental health trials from existing literature since it is rather limited. One systematic review by Briel et al. (71) identified barriers to recruitment from both participants and the recruiter’s perspective. They found that the most common recruitment failure reasons were: different opinions from recruiters and participants on the intervention effectiveness, an overestimation of eligible participants and the burden of trial enrolment for participants and recruiters. Two other reviews on the same topic from confirms these study results on explanations for recruitment failure (72, 73).
Another review looked at participation in mental health research and the barriers that was established in this review was; Stigma of mental illness, acceptance of illness, help- seeking/negative attitude to therapy, underutilization of mental health services, severity of illness, fear of relapse, substance abuse, distrust and/or fear of research (74).
1.2.3 Optimising recruitment
There are some core factors can have a big influence on the recruitment into trials regardless of the population of interest. Suitability of the recruiter and existing relationships with
participants or hospital staff and community leaders can act as either facilitators or barriers to successful recruitment (75). Another very important aspect to consider regardless is how the participants are invited. The recruitment method can influence both recruitment rates and the response rates (63, 76, 77).
In a systematic review on how to improve recruitment from 2018 they wanted to quantify the effects of strategies for improving the recruitment of participants to randomized trials (78). In this review 68 studies were found eligible for inclusion and the results were grouped
according to the type of intervention. This resulted in seven categories; design, pre-trial planning, trial conduct changes, modification to the consent process, interventions aimed at the recruiter of recruitment site and incentives (78). There were three main findings of this review. The first was an open design that compared to a blinded, placebo-controlled design increases the recruitment. This is unlikely to be attractive though because open trial designs often struggle with internal validity problems. Secondly, to use a telephone reminder to contact participants that didn’t respond to a postal invitation increases the recruitment as well.
This finding however is more likely to be found in trials with low (<10%) underlying
recruitment. Thirdly, optimizing the participant leaflet trough bespoke development plus
formal user-testing makes little or no difference to the recruitment (78). The author concludes
7
that the research world needs a process that priorities what recruitment interventions are most in need of an evaluation. Recruitment interventions in pediatric trials should be a priority and there is a bigger need for replication then innovations in the recruitment field. More
knowledge is needed on how to improve recruitment and trialists should apply an evidence- based strategy in the recruitment process (78). To identify unexpected issues in the
recruitment process it can be beneficial to use an internal pilot. In a study on how to optimize recruitment from 2019 they used a 6-month internal pilot to refine the recruitment procedure (47).This pilot showed that recruiting the sample size of 380 participants would not be done according to the time plan. They also added additional recruiting sites to find more
participants and a projection equation was used to estimate the overall trial recruitment. The conclusion after the internal pilot was conducted was that this method allows researchers to set realistic recruitment targets but also it helps to identify the recruitment issues the trial may face (47).
1.3 Internet-based CBT to depression
1.3.1 Depression and Global Health
People all over the world are affected by serious or less serious disorders such as dementia, drug, and alcohol dependence, psychosis and different types of depression diagnosis.
Depression is a common and costly societal problem and is affecting least 5% of any
population is affected and at least 300 to 400 million people worldwide (79, 80, 81). The high prevalence and the often-long-term effect of depression makes it a major public health
problem. Depression also generates large costs for the individual with the condition and society, both direct and indirect (82). Depression is characterized by sadness, lack of interest of please and can also disturb sleep and appetite resulting in tiredness and poor concentration.
The effects of depression can be long-lasting and affect a person’s ability to live a rewarding life. If not treated, depression has a negative impact on employment status and social
functioning (81). Mental disorders are killing people in indirect and direct ways, suicide is a
direct effect of mental disorders and is amongst the leading cause of death for adults in all
countries (83). One of the problems is the lack of effective treatments in low-income
countries. Many of these countries are already facing challenges to overcome and treat
infectious diseases and therefore mental disorders don't get as much attention (84).
8
What should be done to improve the mental health around the world? First of all, the burden of mental disorders has to be recognized as a burden of disease when it comes to disability, lost life and medical care costs. Second, better and more effective treatments for several psychiatric disorders are needed in countries with limited medical infrastructures (85). There is also a need for international partnerships and campaigns in the mental health field to establish interventions and plans but also to remove the stigma around psychiatric disorders (86).
1.3.2 Screening tests for depression
When conducting research on depression it is common to measure participants level of depression at baseline and posttreatment (see annex 1). This is often done by using patient- administered screening tools (87).
• Beck Depression Inventory. One of the most popular measures of depressive
symptoms is the 21-item Beck Depression Inventory (BDI) first proposed by Beck et al. The BDI uses a cut-off at 10 for depression. (88).
• Beck Depression Inventory II. The BDI has was revised in 1996 that resulted in Beck Depression Inventory-II. 21-item format with four options under each item, ranging from not present at 0 to severe 3 (89).
• Beck Depression Inventory – IA. The BDI also revised in 1978 as BDI-IA. It contains 21 items and is a multiple-choice, self-report questionnaire. Higher total scores
indicate more severe depressive symptoms (90).
• Chinese-bilingual Beck Depression Inventory. CB BDI has same cut-off score and scale as original BDI but in Chinese.
• Center for Epidemiological Studies Depression Scale. The CES-D is a 20-item self- administered questionnaire measuring frequency of symptoms during the past week.
The standard cut-off score for moderate depression according to this questionnaire is 16 (91).
• Hospital Anxiety and Depression Scale. The HADS is a 14-item measure that is used to screen depression and anxiety. The cut-off score for moderate depression is 11-14 (92)
• Inventory of Depressive Symptomatology Self-Report. The IDS-SR is a 28-item self-
report scale for depression with total score ranging from 0 to 84 (93).
9
• Montgomery and Asberg Depression Rating Scale. The MADRS consists of 10 items evaluating the core symptoms of depression with a cut-off score on 20 for moderate depression (94).
• Patient Health Questionnaire-9. The Patient Health Questionnaire-9 (PHQ-) is a scale contains a nine-item questionnaire assessing symptoms experienced in the last two weeks. The standard cut-off score for major depression is 10 or above according to this test (95).
• Patient Health Questionnaire-8. PHQ-8 is a shorter version of the PHQ-9 but is measured in the same way and with identical cut-off scores (95).
• Quick Inventory of Depressive Symptomatology. QIDS is a 16-item scale derived from the previous mentioned IDS (96).
• Quick Inventory of Depressive Symptomatology Self-Report. QIDS-SR is the self- report version of QIDS (96).
1.3.3 Internet-based CBT
Cognitive Behavioural Therapy, or CBT, is a frequently used form of therapy that has its roots in a belief that our cognitions, emotions, and behaviours are related (97, 98). Traditional CBT is delivered by a clinician for treating a range of psychological disorders such as acute
depression (99). CBT is often preferred as it has the largest evidence base of current treatment methods (100). In CBT different ways of encouraging clients to explore whether their
negative thinking and emotions are real and challenge the thought process. The CBT process can be explained as an intervention that is helping people to examine the evidence base for their thoughts and emotions (101). Although traditional CBT is almost as effective as antidepressant medication for acute depression it is not accessible for everyone (102, 100).
The interest for alternative cost-effective interventions has risen from the burden both on healthcare but also on the economic burden for the individual (103). Previous studies have shown that only between 10% and 50% of people suffering from depression receive the appropriate treatment within health care services (81, 104). Geographic distance, cost of insurance and care, unavailability of clinicians, and perceived stigma around mental illness are all barriers to access traditional treatments for depression (105, 106).
In 1997 the first online standardized cognitive-behavioural treatment was developed by
researchers at the University of Amsterdam and this became the start of the rise of internet-
based programs. The program was measuring posttraumatic stress symptoms on a website and
10
used this data to treat a small group of students and the results were very positive despite the lack of face-to-face contact (107). Numerous studies have analysed the effects of iCBT on depression and shown that even though dropout rates were high the remaining participants have experienced reduced depressive symptoms (108, 109). Internet-delivered CBT has many potential benefits for clinicians, health care organizations, and the general population (110).
Many internet-delivered CBT programs have been delivered across Australia and the United Kingdom and they were shown successful particularly in adherence, cost-effectiveness, and efficacy in primary care (111, 112, 113, 114). For the patients, CBT programs delivered through the internet is providing an option for those unable or not interested in face-to-face traditional CBT. It can also be beneficial for the clinicians because it offers a treatment option for those who geographically can’t reach or can’t go to appointments because of severe symptoms or financial constraints (115). Numerous reviews have measured the clinical efficacy of internet-based treatments against waitlist or TAU (116, 117). In one randomized controlled trial on clinical efficacy on iCBT for depression in primary care the withdrawal rate from the intervention was only 35%. This is similar to the rates of withdrawal reported in traditional CBT (112, 118). When patients in this trial reported their satisfaction with the treatment it was significantly higher in the ICBT group compared to the TAU group (112).
Internet-delivered CBT doesn’t only satisfy patients to a larger extent it also provides the patients an opportunity to participate in the design of the program due to rapid, iterative testing while developing the program (119).
Many studies are also showing that internet-delivered CBT programs are more cost-effective compared to traditional CBT. Despite this, it is problematic to estimate the costs because of the upfront expenses of selecting the right product and integrating this into the health system (120, 121). An RCT from 2010 measuring the cost-effectiveness of these interventions also suggests that ICBT is more cost-effective and highlights the fact that using technology to deliver therapy is a good value for money (122, 103).
1.3.4 Supported vs unsupported iCBT
The delivery of iCBT can be supported or unsupported. Supported iCBT programs can also be referred to as guided and unsupported programs can also be called unguided. The supported iCBT interventions usually involves support from a clinician or a technician whereas
unsupported interventions are pure self-help with no support (123). Whilst unsupported iCBT
11
can suffer from big dropout rates the supported interventions has shown to have higher
retention rates and better outcomes (124). Many reviews encourage the use of supported iCBT over unsupported for treating depression and anxiety (125, 126). There are different levels of support delivered in internet-based CBT, from guided interventions to completely self- administered. To be able to categorize different levels of support interventions are using the Taxanomy of Support were developed (127, 128, 129).
Table 1: Taxonomy of support, informed by Glasgow & Rosen (1978), Farrand & Woodford (2013, 2015)
Self-Administered
Following an initial session that may be offered introducing the self-help intervention, patient uses the self-help intervention exclusively on their own without any form of support, with the exception of potential on-going contact for data collection purposes only.
Guided
Regular or planned scheduled support sessions with a practitioner with knowledge of specific factors to interact directly with the patient to identify the patients understanding regarding the theoretical rationale for techniques included within the self-help intervention, support personal reflection on techniques adopted, aid problem solving in the event of difficulties using techniques and uses common factors to encourage progress through the self-help intervention.
Minimal Contact
Patient relies exclusively upon the self-help materials but has scheduled regular brief support sessions with a practitioner who monitors use, adopts common factors to encourage engagement and directs progress through the intervention but does not use specific factors to engage the patient in reflection on techniques adopted or appreciate understanding. In the event of difficulties understanding or adopting any techniques, the practitioner will use common factors but direct the patient back to the relevant technique or part of the self-help intervention.
Standardised
Support in the form of standardized prompts and reminders sent directly by an online or app based self-help intervention or through standardized emails sent by a practitioner to prompt usage, and offer additional support upon request. The standardized prompts and reminders may additionally adopt common factor skills to provide encouragement or empathy in the event the patient is struggling to engage with the self-help intervention or outcome measures taken indicate no or little improvement.
12
1.4 Aims and outcomes
1.4.1 Aims
This review aims to find out the number of participants included in different steps of the recruitment process and to be calculate the recruitment rates into internet-based randomized controlled trials on cognitive behavioural therapy for depression and to identify possible moderators.
1.4.2 Research question
RQ 1: “What are the proportions of participants included at different stages of recruitment in randomized controlled trials of CBT for depression?”
RQ 2: “What are the potential methodological and clinical moderators associated with recruitment rates?”
1.4.3 Primary outcome
The primary outcome for this review is the recruitment rate in the included studies.
Recruitment will be calculated as a proportion of the number of participants randomized into the RCT out of the total number eligible for the study.
1.4.4 Secondary outcome
The secondary outcome will be the moderator analysis on participants randomized into the RCT out of those screened for eligibility. The moderators has been chosen based on what can have an impact on the recruitment rates. The following moderators will be examined:
a) Recruitment setting (e.g. clinical, community, mixed) b) Recruitment mode (e.g. invitation, referral, online) c) Recruitment personnel (e.g. researchers, healthcare staff)
2. Method
2.1 Design
The design of this thesis is a quantitative systematic review and meta-analysis with data based
on numerical data. The purpose of the review of literature is to summarise the knowledge
about recruitment into the trials. The meta-analysis estimated the recruitment rates and the moderators effect.13
2.1.1 Protocol writing and registration
A protocol was developed with including information about the search strategy, the data to be extracted and the planned analysis. The Preferred Reporting Items for Systematic reviews and Meta-analyses Protocols (PRISMA-P) statement has been used to guide the design of the review protocol, and results will be recorded in accordance with the PRISMA Statement (130). Protocol was submitted in PROSPERO International Prospective Register of
Systematic Reviews 2020-04-08. Registration number has not yet been issued due to a high demand for registration.
2.1.2 Search strategy
The search strategy (see annex 1) that was used was based on the PICOS criteria. Searches was carried out in relevant databases; CINAHL, Pubmed and PsyINFO. The databases were chosen based on the specific topic and question for the review. CINAHL is an authoritative index especially for nursing and health journals. Pubmed is also widely used in the health field and frequently appear in existing reviews in the field. PsyINFO covers a lot of literature in the psychology field. Only studies in English were accepted in this review because of limited resources for translating. Forward citation and reference were not checked for the included studies because of time limitations (131, 132, 133).
2.1.3 Inclusion and exclusion criteria
Inclusion and exclusion criteria for this review was developed by using the PICOS:
Population, Intervention, Comparator, Outcome, and Study design (29).
• Population: Inclusion criteria for the participants in the study are adults aged 16 years old and above with onset depression. Depression will be diagnosed by using one of the traditional scales to measure depression-like the Beck's Depression Inventory (88), Patient Health Questionnaire (95) amongst other depression scales (see appendix 1) In this review other mood disorders or mental health disorders such as bipolar disease and schizophrenia will be excluded. Participants with other chronic diseases and women with postnatal or postpartum depression will also be excluded, so is caregivers and participants with a depression level below mild.
• Interventions: This review will be exclusively looking at CBT interventions targeting
depression. This also includes other terms like behavioural activation and problem-
solving. Only those interventions that are delivered through the internet will be
included. Following interventions will be excluded; those targeting depression
14
preventions or mixed depression and anxiety, non-traditional CBT based interventions, transdiagnostic CBT or blended CBT, interventions targeting other mental health difficulties than depression.
• Comparator: The control conditions that will be included are non-active control conditions such as; 1) no-treatment controls; 2) wait-list controls; 3) treatment-as- usual; 4) Attention controls. All studies with active control conditions will be excluded.
• Outcomes: This review will only involve quantitative outcomes. Outcomes looked at will be the recruitment process. The outcomes will be the number of participants in every step of the recruitment process into iCBTs for depression. Moderators that will be used are recruitment setting, recruitment strategies and recruitment personnel.
• Study design: The only study design that will be included in randomized controlled trials. All reviews will be excluded as the primary focus is on recruitment methods used in research studies. Non-peer reviewed reports like conference papers and dissertations will be excluded for this review due to the higher risk of bias in those kinds of publications. Non-English publications will be excluded because of the limited resources to translate them into English.
2.1.4 The screening process
Decisions about what studies to select was based on the eligibility criteria in the title and abstract, this is to remove duplicates and to minimize the chance of having non-relevant articles. The screening was done using Rayyan (134). by two reviewers and duplicates of studies were identified and removed in EndNote (135). Titles and abstracts were screened first against the inclusion/exclusion criteria to exclude studies that are not relevant to the research questions. Inclusion discrepancies was discussed between the two reviewers and in cases whereby an agreement cannot be met, a third reviewer was consulted
Full paper checks were conducted and the studies that do not meet the PICOS criteria for this
review were excluded and reasons for exclusion will be documented. Inclusion discrepancies
will be initially discussed between the two reviewers and in cases whereby an agreement
cannot be met, a third reviewer will be consulted. The rest of the papers was included in our
review. These steps and the proportion of studies included and excluded in every step is to be
presented in a PRISMA flow diagram (136). Reference list of included studies was checked
15
and forward citation searches were done to make sure no papers been missed in our electronic search.
2.2. Data extraction
Data extraction was done by two independent reviewers. All data extraction was conducted by one reviewer and then checked for accuracy by another.
The data extracted will be presented in a tables (see table 1 & 2).
Data that will be extracted in this study is:
(i) Study characteristics: country of origin, type of intervention, type of control condition, duration of intervention (weeks), recruitment mode, recruitment setting, recruiting personnel.
(ii) Participant characteristic: gender (%males), age (mean, SD), depression scale, mean depression score (SD), depression label.
(iii) Participant flow: number of participants invited, number of participants interested, number of participants screened, number of participants eligible, number of participants randomized.
2.3 Data synthesis
The data synthesis in this review was done on the data retrieved in the extraction sheet and performed in the Comprehensive Meta-Analysis (CMA) software version 3.0 according to the Cochrane Guidance (136). The inconsistency between the included studies was done by measured by calculate the heterogeneity of the recruitment proportions. An heterogeneity on 0-40 % can be considered not important, 30-60% might represent heterogeneity, 50-90%
might represent substantial heterogeneity and 75-100% a considerable heterogeneity. This was done by the non-parametric Cochrane Q test (29). Recruitment rates was calculated as number of participants screened divided by number of participants screened with 95%
confidence intervals. Moderator analyses and meta-regressions were conducted using logit
units to examine the moderating effect of various factors on the primary outcome. These
factors are the moderators, as mentioned above: (a) recruitment setting, (b) recruitment setting
(c) recruitment personnel.
16
2.4 Ethical considerations
Systematic reviews consist of second-hand data and do not request any deep, personal, or confidential information from participants. Therefore, Institutional Review Boards for the ethical conduct of research don’t have any specific guidelines for systematic reviews.
However, interests of stakeholders can be presented in a review like ours (137). There are six
identified phases where ethical considerations are applicable in a systematic review. Some of
these felt relevant to consider in working on this review. Systematic reviewers must ensure
the review is a cost-benefit analysis and has potential benefits with critical considerations to
various groups of stakeholders (137). This review focusing on an online treatment option for
depression and its recruitment can be considered quite important for trialists in the field. To
be able to test more ICBT interventions there is a need for an answer to the question on how
to recruit to these trials. A synthesis of recruitment and drop out rates from already existing
trials will give a useful overview. Systematic reviews do not have direct access to participant
and neither did I in my work. The reviewers must therefore ethically consider the quality and
relevance of the findings reported before including them in the review (138).
17
3. Results
3.1 Included studies
Fig. 1 PRISMA Flow Diagram
The electronic search resulted in 2820 results from the chosen three databases; Pubmed,
CINAHL, and PsycINFO. There were 771 number of duplicates and after they were removed
a total of 2049 papers were screened for inclusion at the title-and-abstract stage. In the title-
abstract screen studies that didn’t meet the PICOS inclusion criteria were excluded. Studies
were excluded in this stage because of not published in English. not targeting depression, not
have an RTC design, using iCBT through a mobile application, or because the intervention
was targeting caregivers or participants with physical or mental conditions. After excluding
1,959 studies in the title/abstracts screen 91 full text were retrieved. In the full-text screen,
another 61 studies were excluded because of not being iCBT interventions, measuring the
wrong outcome, or because the study had active control conditions. A total of 30 studies were
eligible for inclusion after the full-text screening. The 30 included studies were used in the
data extraction, data synthesis, and analyzing.
18
3.2 Study characteristics
Most of the studies were described as iCBT interventions, some of them put focus on different parts of CBT such as behavioural activation or problem solving and some of them added additional modules or extracted certain parts (139-168). Some of the interventions were focusing on problem-solving which is one part of a CBT intervention, others had more focus on behavioral activation. Culturally adapted interventions were used in two of the trials. “The
Brighten Your Mood Program” was a culturally modified version of an already tested iCBTsadness program to avoid negative connotations related to depression in Chinese culture (142). The “AOC-TR” intervention was also culturally adapted but to the Turkish population in the Netherlands, still focusing on depression but discussing migrant and culture-specific problems and cases (166). Three of the included studies were using and comparing two different iCBT interventions in their trials together with a waitlist control conditions for example one study had a trial on “Beating the Blues” which is an interactive multimedia computer-based CBT and “MoodGYM” which is a free web-based CBT program for depression (145). Another trial only having two comparison groups were using an internet- based CBT program and combined it with evidence-based information about depression (151). The majority of studies only used two comparison groups, one intervention group and one control group, and the most common was interventions described as iCBT for depression.
The majority of the studies were conducted in Europe with some exceptions; Germany (n=7), Australia (n=5), USA (n=4) Netherlands (n=4), Sweden (n=3), UK (n=2), Colombia (n=1), China (n=1), Spain (n=1), Norway (n=1) (139-168). Only non-active controls were included in this review and the control conditions that were used were waitlist control (n=24),
treatment as usual (n=5), and attention control (n=1). Duration of the time of the intervention varied widely over the studies with a range from 4 (152, 153) to 14 weeks (145). The majority of the studies had an intervention period of 8 weeks (139, 142, 144, 147, 150, 151, 156, 159, 162, 164,165). One of the studies did not have any decided amount of weeks that the
participants had to partake in, participants were advised to complete two sessions a week but a minimum of at least four (143). The most common was to have one session per week but some intervention had as many as 3 sessions/modules a week.
All of the included studies except for one (141) reported if support was provided and in that
case on what levels and how it was delivered. In those studies support was provided (n= )
19
email and/or phone was the most frequently used way to deliver it (n= ) but some were also using face-to-face support with a clinician. The level of support was compared by the Taxonomy of Support (127, 128, 129) and divided into; self-administered, guided, minimal contact and standardized. All of the included studies except for one (141) reported the level of the support; minimal contact (n=17), standardized (n=5), self-administered (n=4), guided (n=2) (139-168).
3.3 Categorization of recruitment moderators
The recruitment details (see table 1) are the moderators that were used in the moderator analysis (table 3). As mentioned earlier three of the included studies were not reporting most of the recruitment information. The recruitment moderators were; recruitment setting, recruitment mode and recruitment personnel. The recruitment setting are refers to where participants were recruited. The two most common settings are community and clinical setting. The recruitment settings in this review were categorized into four different settings;
clinical (n=17), community (n=7), university (n=2), and mixed (n=1). The recruitment mode is the way participants were recruited, in other words the method or strategy. The categorize that were identified in this moderators was; ad mixed (n=12), referral (n=4), ad online (n=3), ad website (n=3), database (n=1), invitation (n=3) and invitation or referral (n=1). The category “ad mixed” were those trials that used a combination of ads, online and offline, on several different places. This could include for example flyers, ads in newspapers or on television and social media. “Ad website” was studies that were using the website for the trial to recruit the participants and ad online referred to studies that used different online platforms and social media such as Facebook, Twitter, Google Ads or other platforms. Referral can be explained as the typical strategy to use in a clinical recruitment setting, referral is often done by some kind of healthcare staff such as GPs, nurses, caregivers or other healthcare staff.
Invitations were mostly email invitations but did also include invitations done by regular post.
One study did use both referrals and invitations which and was therefore put together as their own category.
The personnel moderator was information on who recruited the participants and were
categorized into researchers (n=8), online (n=7), healthcare staff (n=6), students (n=1). In
those trials were researchers were used this simply meant that the research staff conducting
the study also were responsible to recruit participants. Online recruitment was those trials who
20
didn’t have any particular person being responsible of recruiting participants and this often meant that participants interested in the study responded on ads posted online and by self- registration. The healthcare staff category was, as stated above, referrals from different professionals in the healthcare setting. One study was using students as responsible for the recruitment (139-168).
Table 1: Study characteristics
Study Author
(year)
Intervention Country Control Duration (weeks)
Support Setting Mode Personnel
1.Beewers et al. (2017)
Internet-based CBT interventions
designed for adults with depression.
USA Waitlist control
8 Minimal Community Ad
online
Online
2.
Beiwinkel et al. (2017)
Internet-based CBT, awareness
training and systemic counselling.
Germany Waitlist control
12 Standardi zed
Clinical Letter Researchers
3. Berger et al. (2011)
Internet-based self-help program covering a variety of therapeutic
content.
Germany Switzerlan
d
Waitlist control
10 NA Community Ad
mixed
Online
4. Choi et al. (2012)
Culturally modified Sadness
iCBT Program.
Australia Waitlist control
8
Standardi
zed
Community Ad
mixed
Not stated
5. Ebert et al.(2018)
Internet-based behavioural therapy and problem solving
program.
Germany Waitlist control
6 – 12 Minimal Community Ad mixed
Not stated
6. Forand et al. (2018)
Internet-based CBT program for
depression.
USA Waitlist control
8 Guided Community Ad
mixed
Not stated
7. Gilbody et al. (2015)
Intervention 1:
Beating the Blues (is an interactive,
multimedia, computer based
CBT) Intervention 2:
MoodGYM (s a
UK Australia
Treatment as usual
14 Minimal Clinical Referral or letter
Health care staff
21
free to use web based CBT programme for
depression)
8, Hoifodt et al. (2013)
Internet-based CBT program.
Norway Waitlist control
6
_
Guided Clinical Referral Healthcare staff
9.
Johansson et al. (2019)
Internet-based CBT and psychoeducation
Sweden Waitlist control
8 Minimal Clinical Referral Healthcare staff
10 Kivi et al.
(2014)
Internet-based CBT program with focus on behavioural
activation.
Sweden Treatment as usual
12 Minimal Clinical Referral Healthcare staff
11. Klein et al. (2016- 2017)
Internet-based CBT program.
Germany Treatment as usual
12 Minimal Community Ad
mixed
Researchers
12. Lambert et al.
(2018)
Internet-based CBT treatment based on BA with
added PA promotion.
UK Waitlist control
8 Minimal Community Ad
mixed
Researchers
13. Lintvedt et al. (2013)
Internet-based self-help programme based
on CBT, IPT and relaxation techniques + evidence-based information about
depression.
Norway Waitlist control
8 Self-
administe red
University Invitat- ion
Researchers
14. Lokman et al. (2017)
Internet-based Complaint- Directed Mini- Interventions for
Depressive Complaints
Netherlan ds
Waitlist control
At least 4 Self- administe
red
Community Ad
online
Online
15. Ludtke et al. (2018)
Internet-based psychoeducation
information and behavioural
Germany Waitlist control
4 Self-
administe red
Clinical Database Online
22
activation.
16. Meyer et al. (2009)
Internet-based BA and mindfulness.
Germany Treatment as usual
9 Self-
administe red
Community Ad
online
Online
17.
Montero- Marin et al.
(2019)
Internet-based self-help program
for depression.
Spain Treatment as usual
12 Minimal Clinical Referral Healthcare staff
18. Moritz et al. (2019)
Internet-based program with
cognitive behavioural
techniques.
Germany Waitlist control
8 Standardi
zed
Community Ad
online
Online
19. Nygren et al. (2019)
Culturally adapted internet-based
CBT for depression.
Sweden Waitlist control
8 Minimal Community Ad
mixed
Students
20. Perini et al. (2009)
Internet-based CBT with storytelling.
Australia Waitlist control
8 Minimal Community Ad
website
Researchers
21.Richards et al. (2015)
Internet-based CBT program
Ireland Waitlist control
8 Minimal Community Ad
website
Online
22. Rosso et al. (2017)
Internet-based CBT program
USA Attention control
10 Minimal Community Ad
mixed
Healthcare staff
23.
Ruwaard et al. (2009)
Internet-based CBT program with an interactive
workbook
Netherlan ds
Waitlist control
11 Guided Community Ad
mixed
-
24. Schure et al. (2019)
Internet-based individually tailored CBT intervention for
depression
USA Waitlist control
8 Standardi
zed
Community Ad
mixed
Researchers
25.
Salamanca- Sanabria (2020)
Internet-based CBT including self-monitoring,
behavioral activation, cognitive
Colombia Waitlist control
7 Minimal University Invite Online
23
restructuring
26. Titov et al. (2010)
Internet-based behavioural
activation, cognitive restructuring, problem solving, and assertiveness
skills
Australia
Waitlist
control 8 Minimal Community
Ad
website Researchers
27. Titov et al. (2015)
Internet-based CBT program
Australia Waitlist control
8 Minimal Community Ad
website
Researchers
28. Ünlü et al. (2013)
Internet-based culturally adapted
problem-solving
Netherlan ds
Waitlist control
5 Minimal Community Ad
mixed
Researchers
29.
Warmderda m et al.
(2008)
Intervention 1:
Internet-based problem solving
therapy.
Intervention 2:
Internet-based CBT.
Netherlan ds
Waitlist
control 12
Minimal Community Ad mixed
-
30. Yeung et al. (2018)
Internet-based cognitive behavioral treatment program
depression.
China Waitlist control
5 Standardi zed
Clinical
Referral Healthcare staff
3.4 Participants characteristics
The baseline participant characteristics that were extracted were the overall; gender (%
female), age (mean and SD), depression scores (mean and SD), depression scale, and level of depression. Twenty-eight of the included studies reported the mean age of the participants (139-168) all studies together had an age mean of 39,42 with a range from 22,15 to 65,28. All of the studies except one (157) had a majority of women with a range of 57% (147) to 88%
(142). All of the studies used some kind of depression scale at pre-treatment and post-
treatment. The depression scales that were used were PHQ-9 (n=7), followed by BDI- II
(n=5), CES-D (n=4), BDI (n= 4), MADRS (n=1), QIDS (n=1), PHQ-8 (n=1), BDA-IA (n=1),
HRSD, (n=1), Many studies did, however, combine two or more depression scores, the score
measuring the primary outcome that is reported in the table or the score used the most is
24
extracted in the table. but half (n=15) of the studies did not report the mean depression score or the standard deviation for the overall. All of the studies except for two (n=28) reported the level of depression the participant had at baseline or gave a mean and SD. Moderate was the most common level of depression for participants (=22) followed by Severe (n=3),
Moderately-severe (n=) and Mild (n=1). All of the studies were reporting additional
information about their participants for example educational level, income and ethnicity but they were not considered relevant for this review and were therefore not extracted (139-168).
Study Author
(year)
Gender (%
female)
Age, mean (SD)
Depression scale Mean depression score (SD)
Depression level
1. Beewers et al.
(2017)
74,4 % 31,91
(SD 11,2)
QIDS-SR NS Moderate
2. Beiwinkel et al.
(2017)
68,3 % 47,74
(SD 10,92)
PHQ-9 11,10
(4,45)
Moderate
3. Berger et al.
(2011)
69,7% 38,8
(SD 14,0)
BDI NS Moderate
4. Choi et al.
(2012)
88 % 39,0
(SD 11,7)
CB BDI NS Moderate
5. Ebert et al.
(2018)
80,4 % 44,20
(SD 11,73)
QIDS 8,14
(SD 3,75)
Moderate
6. Forand et al.
(2018)
NS NS PHQ-9 17.30
(4,66)
Moderately-severe
7. Gilbody et al.
(2015)
67 % 40,6
(SD 13,8)
PHQ-9 16,6
(4,0)
Moderately-severe
8. Hoifodt et al.
(2013)
72,6 % 36,1
(SD 11,3)
BDI-II NS Moderate
9. Johansson et al.
(2019)
57 % 39
(-)
MADRS NS Moderate
10. Kivi et al.
(2014)
NS NS BDI NS NS
11. Klein et al.
(2016-2017)
68,86 % 42,85 PHQ-9 NS Moderate
12. Lambert et al.
(2018)
84 % 38,1 PHQ8 14,6
(3,2)
Moderate
13. Lintvedt et al.(2013)
76 % 28,15 CES-D NS Moderate
75,7 % 43 IDS-SR NS Mild
Table 2: Participants characteristics
