The impact of telephone versus e-mail therapist
guidance on treatment outcomes, therapeutic
alliance and treatment engagement in
Internet-delivered CBT for depression: A randomised
pilot trial
P. Lindner, E.L. Olsson, A. Johnsson, M. Dahlin, Gerhard Andersson and P. Carlbring
Linköping University Post Print
N.B.: When citing this work, cite the original article.
Original Publication:
P. Lindner, E.L. Olsson, A. Johnsson, M. Dahlin, Gerhard Andersson and P. Carlbring, The
impact of telephone versus e-mail therapist guidance on treatment outcomes, therapeutic
alliance and treatment engagement in Internet-delivered CBT for depression: A randomised
pilot trial, 2014, Internet Interventions, (1), 4, 182-187.
http://dx.doi.org/10.1016/j.invent.2014.09.001
Copyright: Elsevier
http://www.elsevier.com/
Postprint available at: Linköping University Electronic Press
http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-116371
The impact of telephone versus e-mail therapist guidance on treatment
outcomes, therapeutic alliance and treatment engagement in
Internet-delivered CBT for depression: A randomised pilot trial
Philip Lindner
a,⁎
, Elinor Linderot Olsson
b, Amanda Johnsson
b, Mats Dahlin
c,
Gerhard Andersson
a,d, Per Carlbring
ea
Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden
b
Department of Psychology, Umeå University, Umeå, Sweden
c
Psykologpartners, Linköping, Sweden
dDepartment of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden e
Department of Psychology, Stockholm University, Stockholm, Sweden
a b s t r a c t
a r t i c l e i n f o
Article history: Received 25 March 2014
Received in revised form 28 August 2014 Accepted 1 September 2014
Available online 16 September 2014 Keywords:
Depression Therapist guided
Cognitive behavioral therapy Internet intervention Telephone E-mail
Background: Internet-administered cognitive behavioural therapy (iCBT) is an effective treatment of depression, yet much remains to be learned about the specific mechanisms influencing symptom reduction. Although previ-ous research has consistently shown that therapist-guided iCBT is more effective than unguided iCBT, it is un-known whether the medium used for therapist-client communication has an impact on results.
Methods: Thirty-eight subjects with major depression were recruited from the waiting list of another iCBT study and randomised to a guided iCBT program with therapist guidance either by telephone calls (n = 19) or e-mail correspondence (n = 19). Outcome measures were self-rated measures of depression, anxiety and quality of life. Results: At post-treatment, both groups showed significant and large symptom reductions yet did not differ from each other. Neither was there any betwegroup difference in client-rated therapeutic alliance or treatment en-gagement. Symptom reductions were maintained at a three-month follow-up.
Conclusion: Therapist guidance by telephone does not appear to differ from therapist guidance by e-mail in iCBT for depression, although further research featuring larger samples is necessary to draw more definite conclusions.
Trial registration: None
© 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).
1. Introduction
Accumulating evidence shows that Internet-delivered self-help based on cognitive behavioural therapy (iCBT) (Andersson, 2009) is an effective treatment of depression (Richards and Richardson, 2012; Titov, 2011), somatic disorders (Cuijpers et al., 2008) and a range of anxiety disorders (Andersson et al., 2013a). The addition of a therapist to guide the client through the self-help program has been shown to yield greater effects (Johansson and Andersson, 2012; Newman et al., 2011; Palmqvist et al., 2007; Spek et al., 2007), equal in size to tradition-al face-to-face CBT (Andersson et al., 2013b, in press; Cuijpers et al., 2010). The mechanisms behind these effects have not been the subject of much research and remain largely unknown (Andersson et al., 2009). In particular, much remains to be learned regarding what specific
aspects of the therapist-client interaction that influence treatment out-comes in iCBT. Recent iCBT research has investigated the effect of the amount, intensity and content of the therapist support (Bendelin et al., 2011; Paxling et al., 2012; Titov, 2011; Titov et al., 2010), the po-tential added benefit of additional telephone support (Andersson et al., 2003) and the development of a therapeutic alliance (TA) (Andersson et al., 2012; Bergman Nordgren et al., 2013; Jasper et al., 2014). To our knowledge, no previous study on depression has directly investigated the potential impact of the medium by which the therapist-client com-munication is provided in iCBT, e.g. via telephone or e-mail. Nor has there been any meta-analysis published directly comparing these two ways of delivering therapist guidance. Hence, it is currently unknown whether the therapist-client communication medium influences treat-ment outcomes.
Although there is scarce prior research to motivate a hypothesis of superiority of one specific type of communication medium, equality cannot be assumed. Communication via telephone is more similar to traditional, face-to-face CBT and may be experienced by some as more
⁎ Corresponding author at: Psychiatry building R5:00, Karolinska University Hospital Solna, 171 76 Stockholm, Sweden.
E-mail address:philip.lindner@ki.se(P. Lindner).
http://dx.doi.org/10.1016/j.invent.2014.09.001
2214-7829/© 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).
Contents lists available atScienceDirect
Internet Interventions
j o u r n a l h o m e p a g e : w w w . i n v e n t - j o u r n a l . c o m /personal, thereby possibly aiding the formation of a therapeutic alliance and indirectly influencing symptom reduction. Communicating via e-mail, on the other hand, allows for more time to reflect upon content and to formulate thoughts and feelings into comprehensible words, which are some of the goals and assumed therapeutic mechanisms of all psychotherapy. Thus, both communication formats have inherent benefits of importance for symptom reduction. We hypothesised that real-time feedback and therapist contact over the telephone would be more beneficial than asynchronous e-mail contact in part due to the telephone calls requiring scheduling and that they would thus be less easy to avoid and postpone compared to reading the e-mail feedback. Indeed, there is some evidence that scheduled support boosts treatment effects (Kenwright et al., 2005). If guidance by telephone would be found superior to guidance by e-mail, this would have great implica-tions for the design, implementation and effectiveness of future iCBT in-terventions. Here, we report a pilot randomised trial designed to investigate this question.
2. Methods 2.1. Procedure
Participants were recruited from the waiting list control group in-cluded in a previous study on iCBT for depression (Carlbring et al., 2013a). Hence, the pre-treatment measures reported in this second-stage study correspond to the control group post-measurements in
Carlbring et al. (2013a). The study was approved by the Regional Ethical Review Board and all participants provided written informed consent. Out of the 40 participants originally randomised to the waiting list con-trol group in theCarlbring et al. (2013a)study, 38 participants completed the post-treatment measurement that served as the pre-measurement in the current study. These 38 participants were randomised to either tele-phone or e-mail therapist guidance using an online, true random gener-ator. Two MSc students in clinical psychology, supervised by an experienced psychotherapist, served as therapists. To minimise the po-tential risk of a therapist effect, both therapists provided both types of therapist guidance via random allocation of patients. See theflowchart (Fig. 1) for group-wise dropout numbers during the study.
2.2. Treatment
After randomisation, both groups were given access to a self-help program consisting of seven modules (one per week) focused on behav-ioural activation (cf.Carlbring et al., 2013b) with some influences from acceptance and commitment therapy (Hayes et al., 2003), namely, defusion (through acceptance and mindfulness) and values work. See
Carlbring et al. (2013a)for more details on the treatment program. In both treatment groups, therapist-client communication focused on a module summary along with personal reflections and question, written and submitted by secure e-mail each week by all participants. In order to distinguish the pure effect of therapist support type, attempts were made to match the two groups on other factors of hypothesised impor-tance: Prior to treatment commencing, the therapists were instructed to devote the same amount of time (approx. 15 minutes per participant and week) and the same type of individualised, non-standardised sup-port (primarily validating the participants' experiences, providing feed-back on performed tasks and reinforcing progress and plans of future work), regardless of communication type. Treatmentfidelity was mon-itored during supervision sessions, but was not systematically recorded. The e-mail support group received a reply from their therapist within 24 hours after submitting a module summary, while the telephone sup-port group had scheduled, regular telephone sessions regardless of whether the weekly module summary had been submitted or not. These telephone sessions lasted approximately 10 minutes, with an ad-ditionalfive minutes spent on preparation. In the e-mail support group,
participants who failed to submit a weekly module summary received reminders and encouragements by e-mail.
2.3. Participants
Participants included were at least 18 years old and had a score range of 15–30 on the Montgomery–Åsberg Depression Rating Scale Self-Rated (Svanborg and Åsberg, 1994). A diagnostic screening inter-view was conducted via telephone (Rohde et al., 1997), using the Struc-tured Clinical Interview for DSM-IV—Axis I disorders (SCID-I) (First et al., 2002). All participants fulfilled DSM-IV criteria for major depres-sive disorder (as their primary diagnosis), with or without comorbid dysthymia, and reported neither participating in other current psycho-logical treatments nor any change in psychoactive medication (if any) during the last three months. When commencing the current study, all but two participants had a Beck Depression Inventory II (Beck et al., 1996) score ofN10 and were hence within the clinical range (see
Table 1for full demographics and clinical characteristics). Importantly, there were no between-group differences in pre-treatment clinical var-iables. Statistical analyses did however reveal a significant and large (Cohen's d = 0.8) between-group difference in age. However, there was no correlation between age and any clinical variable except for pre-treatment Quality of Life Inventory (QOLI) (Frisch et al., 1992) scores. To ensure no confounding effect of age, the QOLI repeated mea-sures analyses were repeated using age as a covariate. This did not change the between-groupfindings.
2.4. Measures
Outcome measures consisted of the Beck Depression Inventory (BDI-II) (Beck et al., 1996), the self-rated Montgomery–Åsberg Depression Rating Scale (MADRS-S) (Svanborg and Åsberg, 1994), the Beck Anxiety Inventory (BAI) (Beck et al., 1988) and the Quality of Life Inventory (QOLI) (Frisch, 1998; Frisch et al., 1992; Paunović and Öst, 2004). These self-report scales have been independently psychometrically val-idated for Internet-administration (Carlbring et al., 2007a; Holländare et al., 2010; Lindner et al., 2013; Thorndike et al., 2009). The BDI-II served as primary outcome measure. All items were mandatory. Partic-ipants answered the forms before commencing treatment (pre), imme-diately afterwards (post) and at a three months follow-up, always via a secure, dedicated online platform. To investigate whether the therapist guidance communication medium had any influence on the formation of a therapeutic alliance, possibly as mediator of symptom reduction, participants also answered the client version of the short (12-item) Working Alliance Inventory (WAI-S) (Tracey and Kokotovic, 1989) after the second treatment week, post-treatment and at the follow-up. The overall alliance score was used as a metric. As in previous iCBT re-search using the WAI-S (Bergman Nordgren et al., 2013), a wording-revised version of the WAI-S suitable for Internet therapy was used. Fi-nally, self-rated treatment engagement was measured post-treatment by asking participants to estimate both how much time they dedicated to their treatment each week and to rate their effort on afive-point scale corresponding to the range Very large to None at all (ranked 4–0). Ther-apists recorded the amount of time spent each week on each client. 2.5. Statistical analyses
Statistical analyses were carried out using IBM SPSS Statistics 22 and the R statistical environment (version 2.15.3). In this pilot study, our primary objective was to investigate whether iCBT with therapist guid-ance via telephone was superior to therapist guidguid-ance via e-mail. Pro-spective power analyses revealed a 67% probability of detecting a large (Cohen's d = 0.8) effect in an independent samples t-test (with p = 0.05), which was deemed acceptable for this pilot trial. Longitudi-nal outcome data was aLongitudi-nalysed using repeated-measures ANOVA. Since F tests revealed that participants with missing post-treatment
Table 1
Demographics, clinical and treatment details.
Telephone support (n = 19) E-mail support (n = 19) Test of group difference
Sex: M/F 1/18 5/14 χ2
(1, n = 38) = 3.167, p = 0.180 Mean age (SD) 49.95 (11.89) 40.47 (11.91) F(1, 36) = 6.024, p = 0.019, Cohen's d = 0.8
Occupation χ2 (6, n = 38) = 9.400, p = 0.110 Employed (full-time) 11 9 Retired 4 0 Student 2 3 Unemployed 0 3 Sick-leave (full-time) 0 1 Employed (part-time) 2 2 Maternal/paternal leave 0 1 Marital status χ2 (5, n = 38) = 6.006, p = 0.322
Married with children 8 6
Married without children 2 7
LAT with children 0 1
Single with children 4 1
Single without children 4 3
Widowed with children 1 1
Education χ2 (1, n = 38) = 2.320, p = 0.561 Finished primary 2 1 Finished secondary 4 3 Ongoing tertiary 1 4 Finished tertiary 12 11
Parallel psychotropic medication: No/Yes 17/2 14/5 χ2
(1, n = 38) = 1.576, p = 0.405
Prior psychological treatment: No/Yes 7/12 7/12 χ2
(1, n = 38) = 0, p = 1
Comorbid dysthymia: No/Yes 17/2 15/4 χ2
(1, n = 38) = 0.792, p = 0.660 Total appropriated therapist time (in minutes)† 109.88 (30.85) 95.33 (39.15) F(1, 32) = 1.421, p = 0.242 Self-reported weekly average time spent on treatment (in hours)† 3.10 (2.70) 3.92 (4.78) F(1, 32) = 0.383, p = 0.540
Self-rated effort in therapy‡ χ2
(4, n = 35) = 6.468, p = 0.173 None 2 0 Small 0 4 Moderate 5 6 Large 8 7 Very large 2 1
†, Post-treatment data available for n = 34. ‡, Post-treatment data available for n = 35. Abbreviations used: M, male; F, female; SD, standard deviation; LAT, living apart together. Fig. 1. Studyflowchart.
data had significantly higher pre-treatment scores on the BDI-II, MADRS and BAI, we concluded that outcome data were not missing at random, making missing-data approaches such as mixed effects models and mul-tiple imputations inappropriate. We therefore replaced missing values using the last-observation-carried-forward approach for the outcome measures. Missing WAI data were not replaced. All data were analysed on an intention-to-treat basis, departing from the randomisations into the current study's two arms. Single time-point between-group differ-ences were tested using F tests and Pearson (exact) Chi-square tests. Bonferroni correction was applied whenever applicable. Cohen's d effect sizes (Cohen, 1988) were calculated within- and between-groups using the compute.es R package (http://cran.r-project.org/web/packages/ compute.es/index.html). Relevant clinical change was defined as having a BDI-II score ofN10 pre-treatment, and ≤10 post-treatment and at follow-up (two participants excluded due to low pre-treatment BDI-II scores).
3. Results
Seven participants dropped out during the course of the trial: 6 in the telephone support group and 1 in the e-mail support group. This dif-ference was not significant yet at trend-level (χ2
[1, n = 38] = 4.38, p = 0.09). Most drop-outs occurred after the material was made avail-able but beforefirst contact with the therapist (n = 4, including the 1 in the e-mail group) and three occurred during treatment.
3.1. Outcomes
Both groups showed significant improvement from pre-treatment to post-treatment on all measures, with effects maintained at follow-up. As shown inTable 2, groups did not differ on any measure, nor were there any group × time interaction effects. Adding estimated appropri-ated therapist time, mid-treatment WAI-S scores, or self-rappropri-ated treat-ment engagetreat-ment as separate covariates did not result in any effect of group. To test whether the drop-out rates impacted results, we also conducted non-intention-to-treat analyses including only the partici-pants who did not drop-out and completed all measurements (full completers; n = 26). The results did not change across any significance level, with the exception of BAI outcomes where there was a significant time × group effect when using Huynh–Feldt correction for non-sphericity (p = 0.047), albeit not when using Greenhouse–Geisser cor-rection (p = 0.052).
3.2. Clinically relevant improvement
Groups did not differ in regards to proportions of subjects scoring≤ 10 10 on the BDI-II post-treatment (χ2[1, n = 36] = 0.11, p = 1.00) or at follow-up (χ2[1, n = 36] = 0.358, p = 0.730). Group-wise pre- and post-treatment scores are plotted inFig. 2.
3.3. Alliance
Twenty-seven participants provided WAI scores at all three timepoints. At post-treatment, both groups rated higher (p = 0.007) WAI scores (M = 66.63, SD = 12.58) compared to mid-treatment (M = 58.37, SD = 10.55), which was followed by a significant decrease at follow-up (M = 62.67, SD = 11.85; p = 0.042; overall effect of time F [2, 15] = 8.145, p = 0.002). There was no effect of group (F[1, 15] = 0.282, p = 0.6), or any group × time interaction effect (F[1, 15] = 1.50, p = 0.234). Group-wise average WAI scores over time are displayed in
Fig. 3(all available data at each timepoint plotted). 4. Discussion
Internet-delivered, therapist-guided self-help programs based on cognitive behavioural therapy have repeatedly been found to be effec-tive, yet little research has investigated the mechanisms influencing these effects. In contrast with our hypothesis, we found no indications that therapist guidance via scheduled telephone sessions was superior to therapist guidance via e-mail correspondence with regards to symp-tom reduction, therapeutic alliance or patient-perceived treatment en-gagement. Although preliminary and thus to be interpreted with caution, these results indicate that future guided iCBT interventions for depression can be designed according to local logistic prerequisites to feature therapist guidance either via telephone calls or e-mail corre-spondence, without loss of effect.
UnlikeKenwright et al. (2005), we found no indication that the scheduling aspect (featured only in the telephone guidance interven-tion) produced superior results. However, it should be noted that our study featured a different clinical population and that our study was not designed to specifically test the importance of scheduled versus un-scheduled guidance. Several other details of this study deserve con-sideration. In this pilot trial, participants were randomised to their condition; whether or not there is an added benefit of allowing patients to choose the communication medium themselves remains to be
Table 2
Means, standard deviations (SD) and Cohen's d for outcome measures.
Telephone group (n = 19) E-mail group (n = 19) Statistics
Mean SD d within Mean SD d within
Beck Depression Inventory II
Pre 23.32 6.41 - 23.16 9.17 – T: F(2, 36)* = 42.996, pb 0.001
Post 13.42 12.02 1.03 11.32 9.21 1.29 G: F(1, 36) = 0.147, p = 0.703
Follow-up 11.63 12.10 1.21 10.74 9.14 1.36 T × G: F(2, 36)* = 0.235, p = 0.710 Montgomery–Åsberg Depression Rating Scale Self-Rated
Pre 17.95 5.56 – 15.37 6.57 - T: F(2, 36)* = 21.095, pb 0.001
Post 10.89 8.84 0.96 10.53 7.26 0.70 G: F(1, 36) = 0.569, p = 0.456
Follow-up 10.95 8.48 0.98 9.26 7.53 0.86 T × G: F(2, 36)* = 0.497, p = 0.538 Beck Anxiety Inventory
Pre 13.11 6.50 - 14.05 7.02 - T: F(2, 36)* = 20.676, pb 0.001
Post 8.95 7.18 0.61 8.74 7.33 0.74 G: F(1, 36) = 0.007, p = 0.933
Follow-up 8.68 6.81 0.67 8.47 7.55 0.77 T × G: F(2, 36)* = 0.292, p = 0.660 Quality of Life Inventory
Pre 0.91 1.85 - 0.45 1.74 - T: F(2, 36)* = 15.700, pb 0.001
Post 1.52 2.38 0.29 1.71 1.90 0.69 G: F(1, 36) = 0.001, p = 0.972
Follow-up 1.90 2.50 0.45 2.11 1.94 0.90 T × G: F(2, 36)* = 1.203, p = 0.295 *Huynh–Feldt correction for non-sphericity applied but not displayed. All within-group effect sizes are vis-à-vis pre-treatment values. Missing values carried forward. QOLI within-group effect sizes are inverted for the sake of clarity (higher QOLI scores equal higher quality of life). Abbreviations: SD, standard deviation; T, time effect; G, group effect; T × G, time × group effect.
investigated (c.f.Andersson et al., 2011; Johansson et al., 2013). It should be noted that all participants submitted written assignments each week regardless of group allocation. The between-group difference was thus limited to how the therapist responded to this written material and the patient's response. This procedure eliminated the potential con-founding effects of different accumulated therapist time and client prep-aration time, and other possible consequences of having participants in
the telephone group also presenting their assignment over the telephone.
It is unclear to what extent the results of this trial are applicable to Internet interventions for other psychiatric disorders. In the case of so-cial anxiety disorder, for example, difficulty speaking on the telephone with an authorityfigure (such as therapist) is a common symptom (Carlbring et al., 2007b). While this impairment may impose an initial barrier to developing a therapeutic alliance and conducting therapeutic work (resulting in symptom reduction) in an iCBT program with thera-pist support via the telephone, such a component could also be consid-ered additional exposure therapy and may alternatively result in additional symptom reduction. Hence, it is important that future re-search investigates whether therapist guidance communication medi-um has an impact in iCBT interventions for other psychiatric disorders. It should also be noted that although there was no significant difference in drop-out rates between groups, there was a slight trend towards it (p = 0.09). There were also some indications of a time × group effect on anxiety ratings when only including full completers, yet the statistics here were not conclusive and thisfinding should thus be interpreted with caution. Future studies with larger samples should address wheth-er scheduled thwheth-erapist guidance via telephone poses a greatwheth-er barriwheth-er to treatment completion compared to written correspondence, especially when treating depression. Such studies may also benefit from systemat-ically recording therapist experiences with the two communication me-diums, an aspect important for clinical implementations not covered by the present study.
In light of the guidelines suggested byProudfoot et al. (2011), this pilot study has some limitations in need of recognition. First, our sample size was small, limiting the statistical power to detect between-group differences. Future research featuring larger sample sizes are required to draw more definite conclusions on the equivalence of delivering therapist guidance through different communication mediums. Larger samples will also be required to test whether demographic characteris-tics (e.g. sex or age) influence results. The significant group difference in age is also a limitation, although no statistics indicated a confounding effect of age on results. Second, with regards to causal mechanisms, the ethical protocol did not allow for measuring and comparing thera-pist and patient behaviours in written correspondence or during tele-phone sessions. While there are some results showing no effect of specific therapist behaviours (e.g. clinical advice) on treatment out-comes (Titov et al., 2010), another study suggests such a correlation (Paxling et al., 2012). Hence, we cannot exclude the possibility that a true effect of communication medium was masked by an effect of medium-specific therapist behaviours. In addition, our study was not immune to problems commonly found in randomised trials, e.g. missing data (Christensen et al., 2009). Standard procedures were employed to counteract these issues. Finally, negative effects were not measured as suggested byRozental et al.(2014). These limitations notwithstanding, the pilot randomised trial herein described extends the extant literature on factors influencing treatment outcomes in guided iCBT.
5. Conclusions
The results of this pilot study suggest that in Internet-delivered, therapist-guided cognitive behavioural therapy for depression, thera-pist guidance by telephone appears to be equal to therathera-pist guidance by e-mail with regards to symptom reduction, therapeutic alliance and patient-rated treatment engagement. Further research using larger samples and additional clinical populations is required to conclude what aspects of therapist guidance influence treatment outcomes. Acknowledgments
This study was made possible by a generous grant from the Swedish Research Council for Health, Working Life and Welfare (FORTE) and Swedish Research Council (Vetenskapsrådet).
Fig. 3. Group-wise patient alliance ratings (WAI scores) over time. Error bars are 95% con-fidence intervals. No missing data carried forward (all available data at each timepoint plotted, n = 32, 35 and 27 at respective timepoint).
Fig. 2. Group-wise pre- and post-treatment BDI-II scores. No missing post-treatment values carried forward displayed infigure.
References
Andersson, G., 2009.Using the Internet to provide cognitive behaviour therapy. Behav. Res. Ther. 47, 175–180.
Andersson, G., Lundström, P., Ström, L., 2003.Internet-based treatment of headache: does telephone contact add anything? Headache 43, 353–361.
Andersson, G., Carlbring, P., Berger, T., Almlov, J., Cuijpers, P., 2009.What makes Internet therapy work? Cogn. Behav. Ther. 38, 55–60.
Andersson, G., Estling, F., Jakobsson, E., Cuijpers, P., Carlbring, P., 2011.Can the patient de-cide which modules to endorse? An open trial of tailored Internet treatment of anx-iety disorders. Cogn. Behav. Ther. 40, 57–64.
Andersson, G., Paxling, B., Wiwe, M., Vernmark, K., Felix, C.B., Lundborg, L., Furmark, T., Cuijpers, P., Carlbring, P., 2012.Therapeutic alliance in guided Internet-delivered cog-nitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder. Behav. Res. Ther. 50, 544–550.
Andersson, G., Carlbring, P., Ljótsson, B., Hedman, E., 2013a.Guided Internet-based CBT for common mental disorders. J. Contemp. Psychother. 43, 223–233.
Andersson, G., Hesser, H., Veilord, A., Svedling, L., Andersson, F., Sleman, O., Mauritzson, L., Sarkohi, A., Claesson, E., Zetterqvist, V., Lamminen, M., Eriksson, T., Carlbring, P., 2013b.Randomised controlled non-inferiority trial with 3-year follow-up of internet-delivered versus face-to-face group cognitive behavioural therapy for de-pression. J. Affect. Disord. 151, 986–994.
Andersson, G., Cuijpers, P., Carlbring, P., Riper, H., Hedman, E., 2014.Internet-based vs. face-to-face cognitive behaviour therapy for psychiatric and somatic disorders: a sys-tematic review and meta-analysis. World Psychiatry (in press).
Beck, A.T., Epstein, N., Brown, G., Steer, R.A., 1988.An inventory for measuring clinical anxiety: psychometric properties. J. Consult. Clin. Psychol. 56, 893–897.
Beck, A.T., Steer, R.A., Brown, G.K., 1996.Beck Depression Inventory-II, BDIII San Antonio TX The Psychological Corporation. The Psychological Corporation.
Bendelin, N., Hesser, H., Dahl, J., Carlbring, P., Nelson, K.Z., Andersson, G., 2011. Experi-ences of guided Internet-based cognitive-behavioural treatment for depression: a qualitative study. BMC Psychiatry 11, 107.
Bergman Nordgren, L., Carlbring, P., Linna, E., Andersson, G., 2013.Role of the working al-liance on treatment outcome in tailored internet-based cognitive behavioural thera-py for anxiety disorders: randomized controlled pilot trial. JMIR Res. Protoc. 2, e4.
Carlbring, P., Brunt, S., Bohman, S., Austin, D., Richards, J., Öst, L.-G., Andersson, G., 2007a.
Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Comput. Hum. Behav. 23, 1421–1434.
Carlbring, P., Gunnarsdóttir, M., Hedensjö, L., Andersson, G., Ekselius, L., Furmark, T., 2007b.Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br. J. Psychiatry 190, 123–128.
Carlbring, P., Hägglund, M., Luthström, A., Dahlin, M., Kadowaki, A., Vernmark, K., Andersson, G., 2013a.Internet-based behavioral activation and acceptance-based treatment for depression: a randomized controlled trial. J. Affect. Disord. 148, 331–337.
Carlbring, P., Lindner, P., Martell, C., Hassmén, P., Forsberg, L., Ström, L., Andersson, G., 2013b.The effects on depression of Internet-administered behavioural activation and physical exercise with treatment rationale and relapse prevention: study proto-col for a randomised controlled trial. Trials 14, 35.
Christensen, H., Griffiths, K.M., Farrer, L., 2009.Adherence in internet interventions for anxiety and depression. J. Med. Internet Res. 11, e13.
Cohen, J., 1988.Statistical power analysis for the behavioral sciences, Statistical Power Analysis for the Behavioral Sciences2nd ed. Lawrence Erlbaum Associates, Hillsdale, NJ.
Cuijpers, P., van Straten, A., Andersson, G., 2008.Internet-administered cognitive behavior therapy for health problems: a systematic review. J. Behav. Med. 31, 169–177.
Cuijpers, P., Donker, T., van Straten, a, Li, J., Andersson, G., 2010.Is guided self-help as ef-fective as face-to-face psychotherapy for depression and anxiety disorders? A sys-tematic review and meta-analysis of comparative outcome studies. Psychol. Med. 40, 1943–1957.
First, M.B., Spitzer, R.L., Miriam, G., Williams, J.B.W., 2002.Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition (SCID-I/P). Biometrics ResearchNew York State Psychiatric Institute, New York.
Frisch, M.B., 1998.Quality of life therapy and assessment in health care. Clin. Psychol. Sci. Pract. 5, 19–40.
Frisch, M.B., Cornell, J., Villanueva, M., Retzlaff, P.J., 1992.Clinical validation of the Quality of Life Inventory. A measure of life satisfaction for use in treatment planning and out-come assessment. Psychol. Assess. 4, 92–101.
Hayes, S.C., Strosahl, K.D., Wilson, K.G., 2003.Acceptance and Commitment Therapy. An Experiential Approach to Behavior Change. Guilford Press, New York.
Holländare, F., Andersson, G., Engström, I., 2010.A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. J. Med. Internet Res. 12, e49.
Jasper, K., Weise, C., Conrad, I., Andersson, G., Hiller, W., Kleinstäuber, M., 2014.The Work-ing Alliance in a randomized controlled trial comparWork-ing Internet-based self-help and face-to-face cognitive behavior therapy for chronic tinnitus. Internet Interv. 1, 49–57.
Johansson, R., Andersson, G., 2012.Internet-based psychological treatments for depres-sion. Expert. Rev. Neurother. 12.
Johansson, R., Nyblom, A., Carlbring, P., Cuijpers, P., Andersson, G., 2013.Choosing be-tween Internet-based psychodynamic versus cognitive behavioral therapy for depression: a pilot preference study. BMC Psychiatry 13, 268.
Kenwright, M., Marks, I., Graham, C., Franses, A., Mataix-Cols, D., 2005.Brief scheduled phone support from a clinician to enhance computer-aided self-help for obsessive-compulsive disorder: randomized controlled trial. J. Clin. Psychol. 61, 1499–1508.
Lindner, P., Andersson, G., Ost, L.-G., Carlbring, P., 2013.Validation of the Internet-administered Quality of Life Inventory (QOLI) in different psychiatric conditions. Cogn. Behav. Ther. 42, 315–327.
Newman, M.G., Szkodny, L.E., Llera, S.J., Przeworski, A., 2011.A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy? Clin. Psychol. Rev. 31, 89–103.
Palmqvist, B., Carlbring, P., Andersson, G., 2007.Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? Expert Rev. Pharmacoecon. Outcomes Res. 7, 291–297.
Paunović, N., Öst, L.-G., 2004.Clinical validation of the Swedish version of the Quality of Life Inventory in crime victims with posttraumatic stress disorder and a nonclinical sample. J. Psychopathol. Behav. Assess. 26, 15–21.
Paxling, B., Lundgren, S., Norman, A., Almlöv, J., Carlbring, P., Cuijpers, P., Andersson, G., 2012.Therapist behaviours in Internet-delivered cognitive behaviour therapy: analy-ses of e-mail correspondence in the treatment of generalized anxiety disorder. Behav. Cogn. Psychother. 1–10.
Proudfoot, J., Klein, B., Barak, A., Carlbring, P., Cuijpers, P., Lange, A., Ritterband, L., Andersson, G., 2011.Establishing guidelines for executing and reporting Internet intervention re-search. Cogn. Behav. Ther. 40, 82–97.
Richards, D., Richardson, T., 2012.Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 32, 329–342.
Rohde, P., Ph, D., Lewinsohn, P.M., Seeley, J.R., 1997.Comparability of telephone and face-to-face interviews in assessing Axis I and II disorders. Am. J. Psychiatry 154, 1593–1598.
Rozental, A., Andersson, G., Boettcher, J., Ebert, D., Cuijpers, P., Knaevelsrud, C., Ljótsson, B., Kaldo, V., Titov, N., Carlbring, P., 2014.Consensus statement on defining and measur-ing negative effects of Internet interventions. Internet Interv. 1, 12–19.
Spek, V., Cuijpers, P., Nyklícek, I., Riper, H., Keyzer, J., Pop, V., 2007.Internet-based cogni-tive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol. Med. 37, 319–328.
Svanborg, P., Åsberg, M., 1994.A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr. Scand. 89, 21–28.
Thorndike, F.P., Carlbring, P., Smyth, F.L., Magee, J.C., Gonder-Frederick, L., Ost, L.-G., Ritterband, L.M., 2009.Web-based measurement: Effect of completing single or mul-tiple items per webpage. Comput. Hum. Behav. 25, 393–401.
Titov, N., 2011.Internet-delivered psychotherapy for depression in adults. Curr. Opin. Psy-chiatry 24, 18–23.
Titov, N., Andrews, G., Davies, M., McIntyre, K., Robinson, E., Solley, K., 2010.Internet treatment for depression: a randomized controlled trial comparing clinician vs. tech-nician assistance. PLoS One 5, e10939.
Tracey, T.J., Kokotovic, A.M., 1989.Factor structure of the Working Alliance Inventory. Psychol. Assess. J. Consult. Clin. Psychol. 1, 207–210.
