Couns Psychother Res. 2020;20:435–441. wileyonlinelibrary.com/journal/capr | 435
1 | INTRODUCTION
The past decade is marked with an increasing focus on mecha-nisms of change in psychotherapy. Mechamecha-nisms of change are the theoretically postulated underlying targets that facilitate
therapeutic change. For example, cognitive therapy for depres-sion postulates that it is key to target distorted cognitions about the self, the world and the future in order to reduce depressive symptoms. Psychodynamic therapy assumes that patients need to gain insight into problematic relationship patterns, to become Received: 4 October 2019
|
Revised: 30 December 2019|
Accepted: 6 January 2020DOI: 10.1002/capr.12293
S P E C I A L S E C T I O N P A P E R
Using time-lagged panel data analysis to study mechanisms
of change in psychotherapy research: Methodological
recommendations
Fredrik Falkenström
1| Nili Solomonov
2| Julian Rubel
3This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
© 2020 Linköping University. Counselling and Psychotherapy Research published by John Wiley & Sons Ltd on behalf of British Association for Counselling and Psychotherapy.
Contributing authors: Nili Solomonov (nis2051@med.cornell.edu), Julian Rubel (julian.rubel@psychol.uni-giessen.de)
1Linköping University, Linköping, Sweden 2Weill Cornell Medical College, New York, NY, USA
3University of Giessen, Giessen, Germany
Correspondence
Fredrik Falkenström, Linköping University, Linköping, 581 83, Sweden.
Email: fredrik.falkenstrom@liu.se Funding information
Nili Solomonov funded by NIMH grant T32 MH019132 (PI: George Alexopoulos).
Abstract
The introduction of novel methodologies in the past decade has advanced research on mechanisms of change in observational studies. Time-lagged panel models allow us to track session-by-session changes and focus on within-patient associations be-tween predictors and outcomes. This shift is crucial as change in mechanisms in-herently takes place at a within-patient level. These models also enable preliminary casual inferences, which can guide the development of effective personalised inter-ventions that target mechanisms of change, used at specific treatment phases for optimal effect. Given their complexity, panel models need to be implemented with caution, as different modelling choices can significantly affect results and reduce replicability. We outline three central methodological recommendations for use of time-lagged panel analysis to study mechanisms of change: (a) taking patient-specific effects into account, separating out stable between-person differences from within-person fluctuations over time; (b) properly controlling for autoregressive effects; and (c) considering long-term time trends. We demonstrate these recommendations in an applied example examining the session-by-session alliance–outcome association in a naturalistic psychotherapy study. We present limitations of time-lagged panel analysis and future directions.
K E Y W O R D S
cross-lagged panel model, mechanisms of change, process–outcome research, psychotherapy research
aware of avoided emotions and/or to have corrective emotional experiences with their therapist. Mechanism of change research empirically tests clinical targets that can lead to the improvement of patients' psychological distress.
Ideally, mechanisms will be studied in dismantling trials, where the effect of the candidate mechanism can be isolated, and a treat-ment that targets this mechanism is compared with an equivalent treatment that contains all components, except for the candidate mechanism. Although such a design is optimised for detection of causal effects, single components are unlikely to have a large con-tribution to the overall treatment effect. Because of this, sufficient statistical power to detect a true population effect would require a very large sample (Cuijpers, Cristea, Karyotaki, Reijnders, & Hollon, 2019). The effect of nonspecific or common factors com-ponents is strong, which reduces the chance of identifying a sig-nificant difference between a mechanism versus a no-mechanism condition (e.g. Wampold & Imel, 2015). Furthermore, some mech-anisms cannot be manipulated or removed due to ethical and/or conceptual reasons.
The most common method used in clinical trials to assess the effects of mechanisms on outcome is mediation analysis. It assumes that the treatment effect on final outcome (e.g. symptoms) is caused by an improvement in a candidate mechanism over the course of treatment. However, given that even in randomised controlled trials (RCTs), the treatment, but not the mediator, is randomised, there is a risk of confounding, and casual interpretation is limited (VanderWeele, 2015). Additionally, some mechanisms may not be treatment-specific (e.g. working alliance), and most cannot be ob-served in no-treatment conditions. However, in order to investigate mediated treatment effects, the mediator needs to be measured in both experimental and control conditions.
In naturalistic studies, or in RCTs when the candidate mecha-nism is either common across treatments or only measured in one experimental condition, we believe that the optimal analytic ap-proach is a class of models based on time-series (usually needing at least 30 repeated measurements) or panel data (20 repeated measures or less) analysis. In this paper, we focus on methods for panel data analysis, especially variants of the cross-lagged panel model (CLPM; Allison, Williams, & Moral-Benito, 2017; Hamaker, Kuiper, & Grasman, 2015; Zyphur et al., 2019). Below, we describe the classic CLPM and its advantages of testing temporal relation-ships between variables.
2 | THE CROSS-L AGGED PANEL MODEL
The basic CLPM includes two variables, X and Y, measured at two time points (Figure 1). As an example, X is insight and Y is depression sever-ity, both measured at baseline and Week 3. We are interested in the causal relationship—whether greater insight predicts lower depression severity, or the other way around (i.e. the bidirectional relationship over time). We can hypothesise that patients who begin treatment with greater insight may benefit more, and thus show greater reduction indepression. We could also predict that patients who enter treatment with lower depression severity may be better able to engage in treat-ment, reflect on their experiences with the help of their therapist and thus show greater depression reduction. A cross-lagged effect is the effect of one variable on another variable at a later time point. It allows us to investigate temporal precedence, which is crucial in inferring cau-sality between a candidate mechanism and outcome. The cross-lagged effects are adjusted for the effect of each variable at one time on the same variable at a later time, that is the so-called autoregressive effect which represents the stability of each variable over time. In our ex-ample, there are two cross-lagged effects: (a) depression severity at baseline predicting change in insight by Week 3, while adjusting for baseline level of insight; and (b) insight at baseline predicting change in depression severity by Week 3, while adjusting for baseline depression severity. The CLPM can be estimated using any structural equation modelling software, such as Mplus, Lisrel, Amos or Lavaan (in R).
3 | EX AMINING CASUAL REL ATIONSHIPS
IN CLPM
Cross-lagged panel model takes into account temporality, one of the three classical criteria for causal inference,1 which is its main
advan-tage compared to a simple cross-sectional correlational analysis. Nevertheless, effects tested may still be affected by external or ‘third’ variables (i.e. the nonspuriousness criterion). This criterion can only be fully met through randomisation of patients to variables studied. A common solution is the inclusion of observed covariates (e.g. age, gen-der, employment status) in multiple regression analyses to test whether the effect remains significant. However, the effect of confounders that are unknown or unmeasured remains. Still, with panel data it is possi-ble to rule out some classes of unknown/unmeasured confounders by separation of within- and between-level variances.
3.1 | Separation of within- and
between-patient variances
Figure 2 presents a session-by-session plot of self-rated depres-sion scores (Patient Health Questionnaire-9) for four patients over the course of 16 sessions of psychotherapy for depression. One
F I G U R E 1 Basic cross-lagged panel model (CLPM) between
patient shows high depression severity throughout treatment (red/ solid line). Another patient shows low depression severity through-out treatment (blue/long-dashed line), while the other two patients are in-between, but with fairly strong fluctuations over time. To test whether change in a candidate mechanism causes these session-by-session changes in depression, we are only interested in the
fluctua-tions over time within each patient, not in the variafluctua-tions among patients on group average depression level.
Psychotherapy research historically focused on between-per-son differences, either by comparing groups or by studying correlations between individual characteristics (Molenaar & Campbell, 2009). However, this approach may not always repre-sent reality. For example, while data show that people who tend to get up earlier in the morning are on average more successful, if
one decides to get up earlier every morning, success in life may not change as a result (example taken from a recent newspaper article implying that the key to success lies in getting up very early in the mornings). However, a true causal relationship may be more likely if we find that on days in which one gets up early, one also works more productively.
This is an illustration of between- and within-person relationships. In this example, the between- and within-person relationships between the variables ‘getting up early’ and ‘success’ may be in the same direc-tion—that is, it would be unlikely that if people who get up early, on average, are more successful than people who sleep late, we would find that within the same persons, getting up early would be associated with less success during the day (although it is certainly possible). There are, however, other examples in which between- and within-person
F I G U R E 2 Line graph of Patient Health
Questionnaire-9 scores for 40 patients undergoing 16-session treatment for depression [Colour figure can be viewed at wileyonlinelibrary.com]
F I G U R E 3 Person-mean-centred
Patient Health Questionnaire-9 scores for four patients undergoing 16-session treatment for depression [Colour figure can be viewed at wileyonlinelibrary.com]
relationships are in opposite directions. Hamaker (2012) offers as an example the relationship between typing speed and number of typing errors. On a within-person level, fast typing results in more errors on average. However, on a between-person level, people who type faster, on average, make fewer typing errors. The reason is that typing is influ-enced by speed and reduced number of errors at the between-person level.
In addition to improved interpretability of results, the separation of within- and between-person levels has the advantage that the within-person level is more likely to reflect causal effects than be-tween-person associations. This is because any observed or unob-served variable that is stable over time cannot, by design, confound within-level variables. This is because a variable that is stable over time has zero variation within persons over time, so their correlation with within-level variables always remains zero.2
3.2 | Methods for separating within- and
between-person variability in psychotherapy research
Within-person, rather than between-person, relationships in psy-chotherapy studies are much more likely to reflect change mecha-nisms. Thus, our first recommendation is to separate these two levels
of variation. The simplest method of isolating the within- from
be-tween-person differences is person-mean centring—subtracting each person's mean from all of their repeated observations and creating a new variable with time-specific deviations from each per-son's average. Figure 3 shows person-mean-centred scores for the same four patients presented in Figure 2. Each person's mean across all sessions is now zero, but the deviations from zero are captured in the time-specific scores.
To analyse within-person relationships, one tred variable can simply be regressed on another person-mean-cen-tred variable.3 Temporal ordering of processes is achieved if one of
the variables is time-lagged. The results of such a model indicate whether increasing one unit of the postulated mechanism in
rela-tion to the average of a particular individual is related to a subsequent
increase/decrease in outcome, in relation to the average of that individual.
3.3 | Combining within–between separation with
estimation of autoregression
The simple regression of one person-mecentred variable on an-other ignores the fact that repeatedly measured variables tend to show stability over time, which is crucial when predicting change be-tween one time point and the next. Adding a lagged version of the dependent variable as a covariate in a multiple regression model to account for autoregression is problematic because it violates the as-sumption of independence between predictors and the model error term (e.g. Falkenström, Finkel, Sandell, Rubel, & Holmqvist, 2017; Nickell, 1981) and tends to yield biased results. This bias is especially
pronounced in shorter time series of about 15 or fewer repeated measurements.
Thus, our second recommendation is to properly adjust for
au-toregression. In econometrics, there are regression-based models
that theoretically can be used to correct the above-mentioned bias (e.g. Arellano & Bond, 1991), but these models are difficult to specify correctly and results may be biased unless sample size is very large (Kiviet, Pleus, & Poldermans, 2015). A more straightfor-ward way to separate within- and between-person levels is through latent variable modelling. There are several structural equation models designed for this purpose (Allison et al., 2017; Bollen & Curran, 2004; Curran, Howard, Bainter, Lane, & McGinley, 2014; Hamaker et al., 2015; McArdle, 2009; Zyphur et al., 2019), and dis-cussing their differences is beyond the scope of the present paper (but see Usami, Murayama, & Hamaker, 2019). We focus on the random intercept CLPM (RI-CLPM; see Figure 4; Hamaker et al., 2015), which could be extended to additional waves. Y and X are two variables measured at three time points, creating six observed variables Y1–Y3 and X1–X3. The variables YB and XB are random intercepts, which extract variance that is constant across all time points (factor loadings set to 1). Thus, they represent stable pa-tient-specific (‘trait-like’; Zilcha-Mano, 2017) aspects of the vari-able of interest and are approximately equal to the person-specific average across time. The variables yw1–yw3 and xw1–xw3 are la-tent within-person variables, constructed from the residuals of the random intercept equation (i.e. the part of the investigated
F I G U R E 4 Three-wave random intercept cross-lagged panel
variable that is unstable and fluctuates over time). Thus, yw1–yw3 and xw1–xw3 represent time-specific deviations from the person's mean, which corresponds with person-mean centring. Finally, a classic CLPM is estimated using these within-person deviation variables (yw1–yw3 and xw1–xw3).
3.4 | Handling long-term trends in panel
data modelling
The two left-hand plots of Figure 3 demonstrate clear trends in depression scores over time. If two random variables that are un-associated with each other both change linearly over time (e.g. IQ and shoe size), they will, despite their independence at a specific time point, show substantial correlations when analysed longitudi-nally. Importantly, this correlation will not be affected by lagging; thus, long-term time trends are a potential source of spuriousness when analysing CLPMs. A common solution is ‘detrending’, that is removing any time trends in the data, before conducting analy-ses. This method may address the risk for spurious findings due to cross-lagged relationships that are by-products of correlated time trends. However, correlated time trends may not always be due to external variables acting as confounders of the cross-lagged ef-fects, but may themselves be by-products of the effects of inter-est, and in that case, removing them before analysis will bias our findings.
Currently, there is no consensus within the CLPM literature on how to handle time trends. Curran and Bauer (2011) advocate for removing time trends, while others (e.g. Wang & Maxwell, 2015; Zyphur et al., 2019) argue that detrending is sometimes unnecessary and may bias results. When there is a theoretical basis to assume that all variables studied change naturally over time by common processes of maturation, detrending is a sensible option. However,
often the intervention studied (e.g. treatment) is likely to produce change in the variables studied. In those instances, detrending may be contraindicated. Given the complexity of this issue, we would recommend presenting findings for both detrended and nondetrended
analyses (see also Falkenström et al., 2017). When findings remain
significant and comparable in direction (although not necessarily in effect size) after detrending, causal inference is strengthened.
There are several detrending methods. One option is to detrend against the sample's average time trend. This is common in economy and sociology, where all subjects (or organisations) are measured in real time (e.g. years, months). In economics, adjusting for abrupt and sharp changes at certain times in history is vital. However, in psychotherapy research, such changes are unlikely to happen at the same time for all participants. The alternative is to detrend against the person's own trend over time (i.e. person-specific detrending; Figure 5). When comparing Figures 3 and 5, the only difference is that the graphs are ‘tilted’ so that the average trend over time be-comes flat in Figure 5, as the mean for each person as well as the linear trend across all sessions equals zero.
4 | EMPIRICAL EX AMPLE
We demonstrate our recommendations in an applied example from a study of primary care psychotherapy in Sweden, conducted between November 2009 and April 2010 (Holmqvist, Sröm, & Foldemo, 2014). The original data analysis was reported by Falkenström, Granström, and Holmqvist (2013). All participants completed the Clinical Outcomes in Routine Evaluation—Outcome Measure (CORE-OM; Evans et al., 2002) before each session, and the Working Alliance Inventory—Short Form-Revised (WAI-SR; Hatcher & Gillaspy, 2006) after each session. In total, 1,090 patients filled out the CORE-OM at least once. We tested whether the working alliance at a given session predicts change
F I G U R E 5 Detrended Patient Health
Questionnaire-9 scores for four patients undergoing 16-session treatment for depression [Colour figure can be viewed at wileyonlinelibrary.com]
in symptom distress by the next session. We regressed CORE-OM on WAI-SR at the previous session using raw scores and found that on average, for each increase in 1 point on the WAI-SR at a given session, the CORE-OM was expected to drop by 1.94 points by the next ses-sion (SE = .10, z = −18.61, p < .001, 95% CI: −2.14, −1.73).
4.1 | Separation of within- and
between-patient variances
The above analysis ignores the within–between-person variability distinction. Therefore, we estimated the within-person regression, that is using person-mean-centred scores instead of raw scores. The alliance effect dropped from −1.94 in the previous analysis to −1.45 (SE = .13, z = −11.17, p < .001, 95% CI: −1.70, −1.19). Thus, the effect size for the pure within-person effect was about 25% smaller than the effect of aggregated within- and between-person variances.
4.2 | Combining within–between separation with
estimation of autoregression
We used lagged WAI-SR to predict change in CORE-OM from one session to the next, that is adjusting for autoregression. We used the RI-CLPM on the same data, again with within- and between-person levels separated using latent variables and modelling au-toregression for both variables. The effect of alliance on change in symptom distress by the next session was −0.67 (SE = .14,
z = −4.76, p < .001, 95% CI: −0.95, −0.40), that is a further
re-duction in effect size but still significant and in the theoretically expected direction.
4.3 | Handling long-term trends
Finally, we estimated a detrended model following the Curran et al. (2014) latent variable modelling approach. The model is similar to RI-CLPM, but adds a latent slope factor used for detrending. The result for this model was −0.36 (SE = .16, z = −2.209, p = .03, 95% CI: −0.68, −0.04), that is a further reduction in effect size but still interpreted as previously.
4.4 | Discussion of empirical example
In this example, the effect size was reduced when each of our rec-ommendations was followed. It is possible that in other cases, the effect may increase after separating within- and between-person variances, adjusting for autoregression, and/or detrending. In ad-dition, the issue of effect size in autoregressive and cross-lagged models is complex, since in addition to the direct effects, there are also multiple indirect effects affecting outcome at longer time lags. However, this topic is beyond the scope of the present paper.
5 | CONCLUSIONS AND FUTURE
DIRECTIONS
When working with observational data, third-variable confound-ing is always a potential problem. However, some designs allow for greater confidence in causal inference. In this paper, we dem-onstrated ways to increase rigour of causal analysis when mecha-nisms of change are studied using observational data. The main limitation of the methods we outlined is the remaining risk of po-tential time-varying confounders, that is variables that covary over time on a within-person level in both the candidate mechanism and outcome. Future work could expand on this issue by develop-ing methods to address these risks.
ORCID
Fredrik Falkenström https://orcid.org/0000-0002-2486-6859
Nili Solomonov https://orcid.org/0000-0003-1573-5715
Julian Rubel https://orcid.org/0000-0002-9625-6611
NOTES
1 The other two are nonrandom empirical relationship and
nonspurious-ness (e.g. Antonakis, Bendahan, Jacquart, & Lalive, 2010).
2 Unless the between-level variable is correlated with the within-level
variable at any particular time point, or has an effect that itself varies over time. The first possibility is unlikely in the context of psychother-apy research unless the between-level variable is related to some par-ticular phase of treatment. Between-level variables with time-varying effects are a theoretical possibility, but it is difficult to imagine what such a variable would be.
3 This is called ‘fixed-effects regression’ in econometrics (e.g. Allison,
2009). It actually also involves an adjustment to the degrees of free-dom of the coefficient tests, but in principle, it is just simple regression analysis with person-mean-centred variables.
REFERENCES
Allison, P. D. (2009). Fixed effects regression models. Thousand Oaks, CA: Sage Publications.
Allison, P. D., Williams, R., & Moral-Benito, E. (2017). Maximum likelihood for cross-lagged panel models with fixed effects. Socius, 3, 1–17. https ://doi.org/10.1177/23780 23117 710578
Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21, 1086–1120. https ://doi.org/10.1016/j. leaqua.2010.10.010
Arellano, M., & Bond, S. (1991). Some tests of specification for panel data: Monte Carlo evidence and an application to employ-ment equations. Review of Economic Studies, 58, 277. https ://doi. org/10.2307/2297968
Bollen, K. A., & Curran, P. J. (2004). Autoregressive latent trajectory (ALT) models: A synthesis of two traditions. Sociological Methods and Research, 32, 336–383. https ://doi.org/10.1177/00491 24103 260222
Cuijpers, P., Cristea, I. A., Karyotaki, E., Reijnders, M., & Hollon, S. D. (2019). Component studies of psychological treatments of adult depression: A systematic review and meta-analysis. Psychotherapy Research, 29(1), 15–29. https ://doi.org/10.1080/10503 307.2017.1395922 Curran, P. J., & Bauer, D. J. (2011). The disaggregation of
Annual Review of Psychology, 62, 583–619. https ://doi.org/10.1146/ annur ev.psych.093008.100356
Curran, P. J., Howard, A. L., Bainter, S. A., Lane, S. T., & McGinley, J. S. (2014). The separation of between-person and within-person com-ponents of individual change over time: A latent curve model with structured residuals. Journal of Consulting and Clinical Psychology, 82(5), 879–894. https ://doi.org/10.1037/a0035297
Evans, C., Connell, J., Barkham, M., Margison, F., McGrath, G., Mellor-Clark, J., & Audin, K. (2002). Towards a standardised brief outcome measure: Psychometric properties and utility of the CORE–OM. British Journal of Psychiatry, 180, 51–60. https ://doi.org/10.1192/ bjp.180.1.51
Falkenström, F., Finkel, S., Sandell, R., Rubel, J. A., & Holmqvist, R. (2017). Dynamic models of individual change in psychotherapy process re-search. Journal of Consulting and Clinical Psychology, 85(6), 537–549. https ://doi.org/10.1037/ccp00 00203
Falkenström, F., Granström, F., & Holmqvist, R. (2013). Therapeutic alli-ance predicts symptomatic improvement session by session. Journal of Counseling Psychology, 60, 317–328. https ://doi.org/10.1037/ a0032258
Hamaker, E. L. (2012). Why researchers should think 'within-person': A paradigmatic rationale. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 43–61). New York, NY: Guilford Publications.
Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. P. P. (2015). A critique of the cross-lagged panel Model. Psychological Methods, 20, 102–116. https ://doi.org/10.1037/a0038889
Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a re-vised short version of the Working Alliance Inventory. Psychotherapy Research, 16, 12–25. https ://doi.org/10.1080/10503 30050 0352500 Holmqvist, R., Ström, T., & Foldemo, A. (2014). The effects of psycho-logical treatment in primary care in Sweden – a practice-based
study. Nordic Journal of Psychiatry, 68, 204–212. https ://doi.
org/10.3109/08039 488.2013.79702
Kiviet, J. F., Pleus, M., & Poldermans, R. (2015). Accuracy and efficiency of various GMM inference techniques in dynamic micro panel data mod-els (January 31, 2015). Retrieved from SSRN http://ssrn.com/abstr act=2560749
McArdle, J. J. (2009). Latent variable modeling of differences and changes with longitudinal data. Annual Review of Psychology, 60, 577– 605. https ://doi.org/10.1146/annur ev.psych.60.110707.163612 Molenaar, P. C. M., & Campbell, C. G. (2009). The new person-specific
paradigm in psychology. Current Directions in Psychological Science, 18(2), 112–117. https ://doi.org/10.1111/j.1467-8721.2009.01619.x
Nickell, S. (1981). Biases in dynamic models with fixed effects. Econometrica, 49(6), 1417–1426. https ://doi.org/10.2307/1911408 Usami, S., Murayama, K., & Hamaker, E. L. (2019). A unified framework
of longitudinal models to examine reciprocal relations. Psychological Methods, 24(5), 637–657. https ://doi.org/10.1037/met00 00210 VanderWeele, T. J. (2015). Explanation in causal inference: Methods for
me-diation and interaction. New York, NY: Oxford University Press. Wampold, B., & Imel, Z. E. (2015). The great psychotherapy debate (2nd
ed.). New York, NY: Routledge.
Wang, L. P., & Maxwell, S. E. (2015). On disaggregating between-per-son and within-perbetween-per-son effects with longitudinal data using Multilevel Models. Psychological Methods, 20, 63–83. https ://doi.org/10.1037/ met00 00030
Zilcha-Mano, S. (2017). Is the alliance really therapeutic? Revisiting this question in light of recent methodological advances. American Psychologist, 72(4), 311–325. https ://doi.org/10.1037/a0040435 Zyphur, M. J., Allison, P. D., Tay, L., Voelkle, M. C., Preacher, K. J., Zhang,
Z., … Diener, E. D. (2019). From data to causes I: Building A general cross-lagged panel model (GCLM). Organizational Research Methods. https ://doi.org/10.1177/10944 28119 847278
AUTHOR BIOGR APHIES
Fredrik Falkenström is associate professor in clinical
psychol-ogy at the Department of Behavioral Sciences and Learning, Linköping, Sweden.
Nili Solomonov is Postdoctoral Associate of Psychology in
Psychiatry at Weill Cornell Medical College, New York, USA.
Julian Rubel is assistant professor at the Department of
Psychology, Justus Liebig University Giessen, Giessen, Germany.
How to cite this article: Falkenstrom F, Solomonov N, Rubel J.
Using time-lagged panel data analysis to study mechanisms of change in psychotherapy research: Methodological
recommendations. Couns Psychother Res. 2020;20:435–441.
