Test-retest reliability of the short version of the everyday technology use questionnaire (S-ETUQ).

(1)

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=iocc20

Scandinavian Journal of Occupational Therapy

ISSN: 1103-8128 (Print) 1651-2014 (Online) Journal homepage: https://www.tandfonline.com/loi/iocc20

Test-retest reliability of the short version of the

everyday technology use questionnaire (S-ETUQ)

Camilla Malinowsky, Louise Nygård, Monica Pantzar & Anders Kottorp

To cite this article: Camilla Malinowsky, Louise Nygård, Monica Pantzar & Anders Kottorp (2020): Test-retest reliability of the short version of the everyday technology use questionnaire (S-ETUQ), Scandinavian Journal of Occupational Therapy, DOI: 10.1080/11038128.2020.1744715

To link to this article: https://doi.org/10.1080/11038128.2020.1744715

Published online: 03 Apr 2020.

Submit your article to this journal

Article views: 137

View related articles

(2)

ORIGINAL ARTICLE

Test-retest reliability of the short version of the everyday technology use

questionnaire (S-ETUQ)

Camilla Malinowskya , Louise Nygårda , Monica Pantzara and Anders Kottorpa,b

a

Department of Neurobiology, Care Sciences and Society, Division of Occupational Therapy, Karolinska Institutet, Huddinge, Sweden;

b

Faculty of Health and Society, Malm€o University, Malm€o, Sweden ABSTRACT

Background: Everyday technologies (ET) such as smart phones, and internet banking are increasingly incorporated into daily activities. Therefore, valid assessments are needed to identify ability in ET use and to design and evaluate interventions.

Aims: To evaluate and compare the stability over time (test-retest reliability) of measures gener-ated with the short version of the Everyday Technology Use Questionnaire (S-ETUQ) in older adults with cognitive impairment or mild dementia.

Materials and Methods: Data was collected with S-ETUQ at two occasions (m ¼ 20.9 days in between) in a sample of (n ¼ 73) older adults with cognitive impairment of different origin (n ¼ 38) or mild dementia (n ¼ 35). Stability of each participant’s S-ETUQ measure was examined using standardised differencez-comparisons. The test-retest reliability coefficient of the S-ETUQ measures was determined by Intraclass Correlation Coefficients. Comparisons were performed using Mann-Whitney u-tests.

Results: The S-ETUQ measures were statistically stable between the two occasions. Hence, the group of persons with cognitive impairment demonstrated slightly higher stability and fewer dif-ferences compared to the group with mild dementia. The ICCs (0.82–0.90) indicated good to excellent agreement.

Conclusions: S- ETUQ can be used with older people with cognitive impairments of varying degree to gather reliable and precise information regarding their use of ET.

ARTICLE HISTORY Received 28 June 2019 Revised 4 February 2020 Accepted 15 March 2020 KEYWORDS Cognitive impairment; dementia; IADL; instrument development; older adults

Introduction

Everyday technology (ET) such as smart phones, auto-mated ticket machines and internet banking play an increasingly central role in our everyday lives. Use of ET is incorporated into many daily activities and is sometimes even a prerequisite for the performance of the activity.

Hence, valid assessments are needed to identify individuals’ as well as target groups’ ability in ET use, for example to design and evaluate interventions. To meet this need, the Everyday Technology Use Questionnaire, ETUQ [1], was developed to evaluate the relevance of ETs and peoples’ perceived ability in ET use. The ETUQ has been shown to measure a uni-dimensional construct and has demonstrated good psychometric qualities in several populations: older adults with mild cognitive impairment (MCI), mild dementia or without cognitive impairment [2,3] and

chronic obstructive pulmonary disease (COPD [4], adults with acquired brain injury [5] and adults with mental retardation [6]. The ETUQ has been translated into several languages (e.g. English, Japanese, Danish, Portuguese) and these versions have also been shown to demonstrate good psychometric qualities [4,7,8]. Moreover, in a sample with chronic obstructive pul-monary disease, COPD [4] the Danish version of the ETUQ has been confirmed to have an excellent inter-rater and test-retest reliability. Yet, test-retest reliabil-ity of the ETUQ has so far only been evaluated in a sample with COPD and is not yet known in other populations.

Nevertheless, studies with the ETUQ have showed that the outcome measures of perceived ability to use ET are sensitive enough to separate (a) older adults with MCI from controls and as well as those with MCI from people with early stage [2,3], (b) people of working age with different levels of impairment after CONTACT Camilla Malinowsky Camilla.Malinowsky@ki.se Department of Neurobiology, Care Sciences and Society, Division of Occupational Therapy, Karolinska Institutet, Fack 23200, S-141 83 Huddinge, Sweden

ß 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

SCANDINAVIAN JOURNAL OF OCCUPATIONAL THERAPY

(3)

acquired brain injury (ABI) from controls [5] as well as (c) adults with different levels of mental retardation [6]. The ETUQ has also been used in longitudinal studies, showing associations between decreasing abil-ity to use ETs and progression from MCI to demen-tia, as well as between decreasing ability to use ETs and decreasing involvement in daily activities such as shopping or leisure. The ability to use ETs was also demonstrated to add significant information to the decreasing activity involvement of participants with MCI and dementia also when associations with the parallel processes of time, diagnostic change, and aging were taken into account [9].

In order to be useful in both clinical work and research, instruments need to be easily administered and time-effective and, therefore, short versions of instruments are often developed, for instance, the Geriatric Depression Scale, GDS15 [10] was devel-oped from the original version of GDS [11] and SF-12 [12] from SF-36 [13]. For the same reasons, a short version ETUQ; the S-ETUQ [14] was devel-oped. The psychometric properties of the S-ETUQ have been evaluated in a sample of older adults with and without cognitive impairment and found to be acceptable [14] and to generate valid person ability measures of ET use. The S-ETUQ has also been shown to be sensitive to detect subtle changes in cognitive decline [15,16]. Still, more knowledge about the S-ETUQ measures regarding precision, sta-bility over time and reliasta-bility is needed [17). The psychometric properties of the S-ETUQ related to repeated measurement over time and therefore need to be evaluated. Research on the short-versions of the instruments mentioned above, GDS15 and SF-12, has demonstrated that these versions of the instru-ments are just as reliable as the longer versions in test-retest [18,19]. Hence, a short version of an instrument can be as reliable as a longer one. Yet, even if the full version of the ETUQ has shown to have an excellent test-retest reliability [4], this too needs to be evaluated for the S-ETUQ.

Adequate reliability is a requirement of an assess-ment that is intended to be used within research or in clinical practice as outcome measure for rehabilita-tion or intervenrehabilita-tions. With a good test-retest reliabil-ity, it is possible to detect that changes in the outcome measures over time are changes in person ability and therefore not due to measurement errors [20]. Test-retest reliability is evaluated by assessing each participant twice with a period of time in between. The time between the assessments should be short enough to minimize changes in personal

capacities and/or abilities that could impact on the outcome and long enough to avoid learning from the first assessment [20]. However, decrease and fluctu-ation in cognitive abilities may impact on test-retest reliability evaluations that are based on self-reports as in the case of the S-ETUQ. Test-retest reliability stud-ies of self-report assessments evaluating quality of life in people with dementia demonstrate that persons with cognitive decline such as dementia can be con-sidered good informants regarding their own condi-tions [21,22]. However, measuring overall quality of life through self-report may not be possible in persons with more severe cognitive decline [23].

Nevertheless, another study showed that test-retest reliability of a quality of life assessment used by people with more severe dementia (moderate to severe stage) ranged from poor for daily and social activities, to moderate for feelings and pain, and satisfactory for physical functioning [24]. Consequently, there seem to be differences in the reliability of outcome measures based on the focus of the assessment and the severity of dementia. Test-retest studies of self-reports on ability to man-age daily activities among people with dementia seem rare, with a few exceptions [25]. Studies addressing precision in instruments when used with people with dementia mostly focus on comparing responses from people with dementia with responses from their caregivers (as a golden standard), or are only targeting caregivers as informants, for example [26], and are ignoring the voices of people with dementia as potentially reliable sources of informa-tion regarding everyday funcinforma-tioning. Lack of appro-priate, valid and reliable methods to evaluate the perceived ability to use ET in persons with cogni-tive impairment or dementia can, therefore, create a gap. On the one side what ETs these persons con-sider relevant to use and how they perceive their ability to use them, and on the other what health care professionals (and caregivers) may focus on in evaluations, interventions as well as daily support. To prevent such a gap, the aim with this study was to evaluate the stability (test-retest reliability) over time of self-rated outcome measures generated with the S-ETUQ (relevance of and perceived ability to use ET) in a sample of older adults with cognitive impairment of different origin e.g. MCI, ABI, mul-tiple sclerosis or mild dementia. Additionally, the test-retest reliability was compared between individ-uals with cognitive impairment and those with mild dementia.

(4)

Materials and methods

This is a test-retest study of the Swedish S-ETUQ. Data was collected with the S-ETUQ in a Swedish sample of older adults with dementia or cognitive impairment of different origins, e.g. MCI, Parkinson’s disease, multiple sclerosis. The sample size was deter-mined to be at least 33 participants in each of the groups in order to detect mean differences between groups (power ¼ 0.90, p ˂ 0.05) based on empirical studies with the S-ETUQ in similar populations.

Everyday technology use questionnaire

The standardized the S-ETUQ was used to evaluate the participants’ perceptions of relevance of different ETs, and perceived ability in using them [14]. The S-ETUQ comprises 33 items, i.e. everyday technological artefacts and services, ETs (e.g. coffee machine, radio, and mobile phone) covering areas such as household, economy and purchasing and accessibility. In the ear-lier development of the S-ETUQ [14], the items had first been derived by item reduction out of the 90þ items in the ETUQ guided by assertions of a Rasch model [27]. Thereafter, the items were checked to ensure they included a variety of items, relevant to a majority of people. The S-ETUQ items, therefore, range from very difficult to very easy and cover both newly developed and well-known ETs [14]. As for ETUQ, a structured manual-based training course is conducted to ensure raters achieve adequate consist-ency by providing detailed instructions for adminis-tration, definitions of items, scoring criteria [28]. The S-ETUQ is administered in a 20–30 min face-to-face interview. As in the ETUQ, the level of perceived dif-ficulty in the use of those ETs that are relevant for each person is registered on a six-step category scale [28]. An ET is regarded as relevant if the person has access to it and (a) uses it, (b) has been using it, or (c) intends to start using it. When needed, a signifi-cant other can be a support to the person during the interview [1].

Participants and procedures

Data were collected in a sample of older adults div-ided into two groups, one group with persons with dementia and one group of persons with cognitive impairment of different origin. General inclusion cri-teria for all participants were (a) aged 55 years or older, (b) a need to use ET in everyday life, (c) moti-vated to participate in the study, (d) visual and hear-ing impairments should be compensated for by

appropriate devices so the S-ETUQ could be carried out, and (e) ability to participate in the interview in Swedish or English.

Persons with mild-stage dementia (n ¼ 37) were recruited, in collaboration with staff, through units for early investigation of memory disorders in the Stockholm area. The dementia diagnosis was set by a physician and based on the standardized DSM-IV cri-teria [29] or identified as major neurocognitive dis-order at the mild stage according to the DSM-V [30]. Since the group with cognitive impairment (n ¼ 41) was planned to include persons with cognitive impair-ment of different origins, e.g. MCI, ABI, multiple sclerosis, they were recruited, in collaboration with staff, from units with such patients. The criteria for persons with cognitive impairment were either diag-nosed with MCI [31,32], memory impairment (object-ive or subjective), or memory impairment in combination with other diagnoses such as Parkinson’s disease or stroke. Persons with cognitive impairment related to an episode of depression or temporary con-fusion were excluded. All diagnoses were given by expert physicians in each unit respectively. The vari-ation in origin for cognitive impairment was sought based on a clinical perspective, acknowledging that cognitive impairment can give similar consequences in everyday life irrespective of origin and the S-ETUQ needs to function regardless of the origin of the per-son’s impairment.

Data were collected by four trained and experi-enced occupational therapists. All of them had partici-pated in a one-day course on how to use and score interviews with the S-ETUQ. Data collection was, after written consent from the participant, undertaken in each participant’s home. To broaden the knowledge about the daily lives of persons with dementia or cog-nitive impairment it is important to include them in research. A consent process for research with older persons with dementia, similar to how it was per-formed in the present study, is described by Dewing [33]. The participants were when possible; interviewed with the S-ETUQ twice within 2–4 weeks, however, there was a range from 2 to 17 weeks(m ¼ 20.9 days, median 15 days, range 12–120 days) to provide data allowing analyses of stability for the S-ETUQ person ability measures [20]. For seven of the participants (cognitive impairment, n ¼ 4, dementia, n ¼ 3), it was, for various reasons (e.g. travels, illness), not possible to conduct the second interview within four weeks; however, their perceptions of ETs’ relevance and their ability to use ET was not expected to change during this relatively short period of time (six of these seven SCANDINAVIAN JOURNAL OF OCCUPATIONAL THERAPY 3

(5)

persons were interviewed within almost two months n ¼ 36–69 days. For one person with dementia there was a period of 120 days between the interviews). Information about the participants’ sex, age, living situation, and years of education was gathered in an interview to describe the sample. Furthermore, data on general cognitive functioning was measured with the Mini Mental State Examination (MMSE) [34] and data of abilities to perform everyday activities was col-lected using the Frenchay Activity Index (FAI) [35].

Based on all available information gathered in con-tact with the participant, including the S-ETUQ inter-view, a judgment was made of overall functional level and the need for assistance to live in the community. This was done by the interviewer against a set of given criteria for overall functional level and inde-pendence related to the requirements for living in the community using a four-grade scale; (1) independent, (2) minimal assistance or supervision, (3) moderate assistance, (4) maximum assistance. This overall func-tional level scale is an adapted version of the original three-grade scale used to estimate a person’s overall functional level developed in association with the Assessments of Motor and Process Skills (AMPS), a thoroughly evaluated and validated assessment of occupational performance [36]. The level of overall functioning was here used to describe the sample.

Data analysis Preparatory analyses

A computer application of a Rasch measurement model, Winstep’s version 3.92. [37] was initially used to convert the ordinal raw scores from the S-ETUQ interviews into abstract linear interval individual per-son measures through logistic transformation proce-dures [27]. These outcome measures are defined as person ability measures of perceived ability to use ET expressed in log odds probability units; logits. The Rasch analysis makes it possible to compare the per-son ability measures among the participants even though not all participants did use all or the same ETs out of the 33 ETs listed in the S-ETUQ. The per-son ability measures with associated individual stand-ard errors (SE) were generated for all participants for each of the two interviews with the S-ETUQ (here-after called S-ETUQ1 and S-ETUQ2). The number of ETs reported as relevant was obtained by counting all items registered as relevant for each participant for the two interview occasions respectively.

The Statistical Package for Social Sciences (SPSS) version 24 [38] was used for descriptive statistics of

the demographic data, MMSE- and FAI-scores, the S-ETUQ person ability measures, number of ETs reported as relevant, and the analysis of test-retest reliability. The level of significance was set at p ˂ 0.05 in all analyses. Before analyses, data were tested for normality and outliers. The variables of years of edu-cation, overall functional ability, MMSE- and FAI-scores, S-ETUQ person ability measures, and number of ETs reported as relevant were all found not to be normally distributed (in one of the groups/both groups or on one or two occasions) and non-para-metric tests were, therefore, used in the analyses for these variables. Outliers were detected with the outlier labelling rule and a factor of 2.2 [39] as well as the Grubbs outlier test [40]. In these tests, three outliers (dementia, n ¼ 1; cognitive impairment, n ¼ 2) were found, and these were removed from further analyses. After that, an analysis of the S-ETUQ person abil-ity measures and their associated SEs revealed that some individuals’ measures differed from the rest of the sample by showing much higher individual SEs on their measures of perceived ability to use ET (5.77–9.72 logits) compared with the mean of the rest of the sample (mean, 1.93 logits for S-ETUQ1, 1.87 logits for S-ETUQ2). Artificially high and imprecise measures indicate that these could be questioned from a validity perspective. Hence, we decided to explore the test-retest reliability of the S-ETUQ by excluding these potentially invalid responses with SEs over 5.0 logits (dementia, n ¼ 1; cognitive impairment, n ¼ 1) from further analysis. Sociodemographic characteristics and descriptive data for MMSE- and FAI-scoring and overall functional level for the final sample of participants (n ¼ 73, dementia, n ¼ 38; cog-nitive impairment, n ¼ 35) are presented inTable 1.

Primary analyses

The stability of each participant’s S-ETUQ person ability measures from the two occasions was first examined using standardized difference z-compari-sons. The z-scores were calculated using (S-ETUQ1-S-ETUQ2 measures)/(SE1þ SE2). The interpretation of the z-scores was based on the criterion that the dif-ferences between the score from the two occasions should be less than z ± 1.96, corresponding to a 95% confidence interval [41]. Then, the presence of sys-tematic differences between the two occasions for the S-ETUQ interviews was evaluated by calculating the difference between the mean person ability measures (second occasion minus first occasion). A difference of five logits was used as a criterion for a noticeable difference [42].

(6)

Finally, the test-retest reliability coefficient of the S-ETUQ person ability measures and the number of ETs reported as relevant were determined by Intraclass Correlation Coefficients (ICCs) [43], using a one-way random effect for single measures of vari-ance model (i.e. ICC 1.1) at the level of individual sub-dimensions (single measures ICC for agreement). An association between the ability measures from the two interview occasions of 0.75–1.0 is considered excellent, an association of 0.6–0.75 good, 0.4–0.59 moderate and an association of less than 0.4 indicates poor agreement [44]. Comparisons between the two groups of participants were performed using Mann–Whitney U-tests.

Results

The S-ETUQ raw data of person ability measures and number of ETs reported as relevant for total sample (n ¼ 73) and the two groups on the two occasions respectively, as well as comparisons between groups and occasions, are presented in Table 2. The group of persons with cognitive impairment demonstrated a significantly higher ability to use ET in both S-ETUQ1 and S-ETUQ2 (Table 2). The differences in the S-ETUQ person ability measure between the two occasions were small in both groups (median 0.12

logits), but it was significantly smaller in the group of persons with cognitive impairment (dementia group median 0.46 logits, cognitive impairment group median 0.16 logits). A larger number of ETs were reported as relevant in the group of persons with cog-nitive impairment than in the group of persons with dementia in both S-ETUQ1 and S-ETUQ2 (dementia group median 17/16 cognitive impairment group median 20.5/21). The differences in the number of ETs reported as relevant in S-ETUQ1 and S-ETUQ2 were, however, relatively equal between the groups (a median difference of 1 ET in both groups).

Stability of each participant’s S-ETUQ person ability measures between the two occasions

It was demonstrated that all the S-ETUQ person abil-ity measures were statistically stable for the two occa-sions, S-ETUQ1 and S-ETUQ2, all z-values were less than ±1.96 (seeTable 2).

The presence of systematic differences between the S-ETUQ ability measures from the

two occasions

The median difference in person ability measures between S-ETUQ1 and S-ETUQ2 was 0.12 (IQR -1.19 to 1.38) logits. Close to 96% (n ¼ 70) of all

Table 1. Sociodemographic characteristics of the participants (n ¼ 73) and descriptive data for MMSE and FAI-scoring. Totaln ¼ 73 Cognitive impairmentn ¼ 38 Dementian ¼ 35

Comparisons between groups Sex,n (%) Women 42 (57.5) 24 (63.2) 18 (51.4) Pearsonv2 Men 31 (42.5) 14 (36.8) 17 (48.6) Ns Age, years Mean (SD) 78.37 (7.06) 77.16 (6.94) 79.69 (7.04) t-test Min-Max 64–99 65–99 64–94 ns Living conditions,n (%) Single 36 (50.7) 21 (56.8) 15 (44.1) Pearsonv2 Cohabiting 35 (49.3) 16 (43.2) 29 (55.9) Ns Missing 2 1 1 Years of educationn (%)

Median 12 12 11 Mann–Whitney

IQR 9–14 10–15.25 8–13 Ns

Min-Max 6–21 6–21 6–18

Significant other present during interviewn (%)

Yes 26 (36) 10 (26) 16 (46) Pearsonv2

No 47 (64) 28 (74) 19 (54) Ns

Overall functional abilityn (%)

Independent 20 (27.4) 15 (39.5) 5 (14.3) Pearsonv2

Minimal 37 (50.7) 19 (50.0) 18 (51.4) p ¼ 0.05

Moderate 14 (19.2) 4 (10.5) 10 (28.6)

Maximum 2 (2.8) 0 2 (5.8)

MMSE, score (max 30)

IQR 23–28 26–28.25 21–27 p ˂ 0.001

Min–Max 11–30 20–30 11–30

FAI, score (max 45)

IQR 20–33 24–36 18–31 p ˂ 0.05

Min-Max 11–40 14–40 11–36

(7)

participants demonstrating a difference of less than five logits indicating stable person ability measures between the two occasions. Yet, three participants showed a difference over 5 logits (5.08–5.59 logits), two with dementia and one with cognitive impair-ment, indicating a minor difference between S-ETUQ1 and S-ETUQ2. In the raw data, it was found out that one of these three participants (with cogni-tive impairment) had had an accident between the two occasions and could, therefore, presumably not use technology in the same way as on the first occa-sion. For the other two participants (2.8%), no clear explanations were found for the differences.

The test-retest reliability coefficient of the S-ETUQ measures and number of ETs reported as relevant

The ICC for the S-ETUQ measures ranged from 0.82 to 0.86 and the ICC for number of ETs reported as relevant ETs ranged from 0.86 to 0.90, all indicating good to excellent agreement when taking confidence intervals into account. The ICC were slightly higher

in the group of persons with cognitive impairment, indicating a somewhat higher level of agreement in that group (seeTable 3).

Discussion

This study demonstrated a good test-retest of the ETUQ regarding the two outcome measures: the S-ETUQ person ability measure and the number of ETs reported as relevant in a sample of older adults with dementia or cognitive impairment of different origins. At a group level, the differences between the first and the second occasions were small, a median of less than one logit difference in the S-ETUQ person abil-ity measures and less than one item (ET) difference in the number of ETs reported as relevant in the total sample as well as in each of the groups. Since neither the participants’ perception of ETs’ relevance nor their own ability to use ET were presumed to have changed between the two occasions, the findings, therefore, support the stability of the S-ETUQ when used for interviews with these groups of older adults

Table 2. Presentation of S-ETUQ person ability measures and z-test of each participant’s S-ETUQ person ability measures as well as S-ETUQ relevance at the two occasions.

Totaln ¼ 73 Cognitive impairmentn ¼ 38 Dementian ¼ 35

Comparisons between groups S-ETUQ1

Person ability measure in logits

Median 52.91 54.58 51.04

IQR 50.68–55.42 52.43–57.26 48.71–53.84 Mann–Whitney

Min-Max 42.19–66.08 47.90–66.08 42.19–59.89 p ¼ 0.014

S-ETUQ2

Person ability measure in logits

Median 53.44 54.14 51.58 Mann–Whitney

IQR 50.93–56.28 52.82–56.73 49.71–54.43 p ¼ 0.014

Min-Max 42.57–66.21 47.60–66.21 42.57–59.28

Differences between occasions S-ETUQ1 and S-ETUQ2

Median 0.12 0.16 0.46 Mann-Whitney

IQR 1.19 to 1.38 1.84–1.12 0.48–0.46 p ¼ 0.026

Min-Max 5.08 to 5.59 4.57–5.53 5.08–5.59

Significance ns ns ns

z-test, person ability measures

Median 02 0.01 0.25 Mann–Whitney

IQR 0.58 to 0.44 0.53 to 0.61 0.89 to 0.16 p ¼ 0.046

Min-Max 1.61 to 1.94 1.49 to 1.31 1.61 to 1.94

S–ETUQ1

Number of relevant ETs, max¼ 33

Median 18 20.5 17 Mann–Whitney

IQR 14.50–23 16–25 14–19 p ¼ 0.037

Min–Max 8–28 13–28 8–24

S-ETUQ2

Number of relevant ETs, max¼ 33

IQR 15–23 17.75–25 14–20 p ˂ 0.01

Min-Max 9–28 14–28 9–25

Differences between occasions S-ETUQ1 and S-ETUQ2

IQR 0–2 0–2 0–2 ns

(8)

both with and without cognitive decline. This was confirmed in the ICC analysis comparing S-ETUQ1 and S-ETUQ2 for both the S-ETUQ person ability measures and number of ETs reported as relevant, indicating a generally high correlation and good to excellent agreement of the S-ETUQ measures. Interestingly, one of the participants who demon-strated a larger than expected change in the ETUQ ability measures between occasions, had an accident that impacted their ability to use ET. This change in ability was also detected by the ETUQ evaluation, and, therefore, also contribute to the evidence of pre-cision of the ETUQ measures.

According to Ettema et al. [24], test-retest reliabil-ity for measures based self-reports among people with cognitive impairment or dementia appears to differ, based on the focus of the assessment as well as on the severity of the person’s cognitive impairment. However, while the study by Ettema et al. [24] dem-onstrated a poor test-retest reliability for experienced functional status regarding daily and social activities, our study suggests the contrary when it comes to self-reports on ability to use ET – which is also part of daily life and social activities. One potential explan-ation for this difference is that, with the S-ETUQ, the person is not asked to specifically judge their ability to use ET, but rather to tell, in their own words, about their ET use here and now. This approach, grounded in the person’s everyday life and environ-ments, and asking concrete questions about everyday technology use, may explain the somewhat surprising findings of consistency in responses from older people with cognitive impairment.

Also, compared to the sample in the study by Ettema et al. [24], the participants in the present study had less severe cognitive decline which could well have had an impact on their responses. Moreover, earlier research has shown that an inter-view with clients, including persons with dementia or cognitive impairment, about ET use is oftentimes also perceived as less threatening than questioning the ability to manage daily life activities, thus allowing the person to be more open about challenges [45].

Challenges in ET use are common to all and it may, therefore, be easier to acknowledge than those that occur within daily activities. Irrespective of explana-tions, our study demonstrates that the ability to use ET seems to be an aspect of everyday functioning that can reliably be captured in a precise manner with the S-ETUQ in a sample of older adults with cognitive impairment or dementia. Nevertheless, a slightly lower ICC was shown in the dementia group in both the S-ETUQ person ability measure and number of ETs reported as relevant, indicating a higher agree-ment in the group of persons with cognitive impair-ment, which may be explained by a higher degree of cognitive decline, resulting in higher variations in responses at different time points. To evaluate test-retest reliability on item (ET) level may provide more in-depth knowledge and explanation about these variations.

The focus of this study is to monitor the test-retest reliability of self-perceived ability to use tech-nology among older adults with cognitive decline. The findings support that the outcome measures generated from the S-ETUQ provide good evidence of stability over time. Notably, this finding should, however, not be interpreted as telling us how accur-ate or valid this information is in relation to other proxy- or observation-based evaluations of ability to use technology. Nevertheless, a recent study demon-strates that self-perceived ability to use ET using the S-ETUQ in a sample of persons with mild dementia or cognitive impairment does not differ from the proxies’ estimations of their relatives/friend’s ability to use ET [46]. Also, it has earlier been shown that the perceived use of ET in a sample of persons with cognitive impairment after a stroke or a traumatic brain injury had a medium- strength correlation to observations made by occupational therapists [47]. This knowledge, in combination with the findings from the current study could give evidence that sta-ble, as well as accurate measures of ET use, could be generated from older adults with cognitive impair-ment or deimpair-mentia. In future studies, the S-ETUQ would be preferable to use in studies within this

Table 3. The test-retest reliability coefficient of S-ETUQ measures and S-ETUQ relevance.

Totaln ¼ 73 Cognitive impairmentn ¼ 38 Dementian ¼ 35 S-ETUQ measures

ICC (CI) 0.86 (CI 0.79 0.91) 0.86 (CI 0.74 0.92) 0.82 (CI 0.67 0.90)

p p ˂ 0.001 p ˂ 0.001 p ˂ 0.001

Cronbach’s a 0.93 0.92 0.91

S-ETUQ relevance

ICC (CI) 0.90 (CI 0.84 0.93) 0.89 (CI 0.79 0.94) 0.86 (CI 0.73 0.92)

p p ˂ 0.001 p ˂ 0.001 p ˂ 0.001

Cronbach’s a 0.95 0.94 0.92

(9)

sample to evaluate interventions aiming to support ET use.

Additionally, a comparison of the correspondence between observations of ET use made by occupational therapists and the self-reports of participants with dementia, cognitive impairment and older adults without known cognitive impairment showed that self-reported ability in all groups to a great extent (over 70% in all groups) corresponded to the raters’ assessment of the observed ability [48]. Earlier studies have also shown that self-rated measures of quality of life among persons with dementia are stable over time [21,22] indicating that they, in general, are stable in such self-ratings. Notably, since it in the present study were three participants with a slightly larger differ-ence in logits than was acceptable between the two occasions, some caution for changed conditions and unexpected responses may be needed when re-evaluating ability to use ET. For two of these three participants, no clear explanations for the differences were found. However, for a majority of the partici-pants the measures were stable and for the person for whom we could assume a difference in ability due to an accident, the difference was shown in the S-ETUQ measures.

Our findings of a good to excellent test-retest reli-ability of the S-ETUQ are comparable to the results of a study with the original longer version of the ETUQ used in a Danish sample of people with COPD [4]. However, in that study, the test-retest reliability was evaluated on an item level and the participants also perceived overall higher ability in the use of ET than in the participants in the current study. Hence, larger variation in ability and a more heterogeneous sample were recommended in order to obtain more detailed information on the reliability of ETUQ [4]. Based on the raw score responses, almost 40% of the participants perceived some difficulties in their ability to use ET in the current study which may have increased the variation in the sample with regard to the person ability measures

Methodological considerations

The relatively small sample (n ¼ 73) in this study could be seen as a limitation, yet the sample was rela-tively heterogeneous and offered an overall variation in the S-ETUQ ability measures for the reliability ana-lysis. The time between the two S-ETUQ interviews varied and, for some of the participants, there was more than a month between the interviews and thereby a longer interval than in the prior test retest

study of the ETUQ [4]. The time should be short enough for person ability to be stable [20] and there is a risk that the ability to use ET did change between the occasions. However, even though dementia is a progressive disease, it is not likely that any major dif-ferences occurred during this period. Also, the fact that we do not ask for self-ratings contributes to limit-ing the well-known effect of fluctuation. The psycho-metric properties of the ETUQ/S-ETUQ item and rating scale have earlier been validated in several stud-ies [1,3,6,14] and this has resulted in a revised and clarified manual [28] about scale steps and the proce-dures for scoring of the ETUQ/S-ETUQ. Raters also participate in a one-day course in the use of ETUQ before using the ETUQ/S-ETUQ in research or prac-tice. These aspects may have supported the raters being sound in the interview and scoring procedures.

Conclusions

The S-ETUQ can be used with older people with cog-nitive impairments of varying degree to gather reliable and precise information regarding their use of ET. The good to excellent reliability over time in responses from older people with cognitive impair-ment may be explained by the fact that the S-ETUQ is based on an approach grounded in the person’s everyday life and the environments the person encounter, asking concrete questions about use of specific ETs. Reliable responses over time are an important departure point for co-developing interven-tions and support to older people with cognitive impairments to manage everyday activities in home and society involving ET use.

Acknowledgement

First, the authors would like to thank the participants who generously shared their use of everyday technology with us. We also want to thank the professionals who helped us with the recruitment of participants. Furthermore, thanks goes to Cecilia Bråkenhielm and Annika Persson Vasiliou for data collection and management.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Funding

The authors wish to acknowledge Swedish Council for Health, Working Life and Welfare (FORTE) [10.13039/501100006636]

(10)

and the Swedish Research Council [2013-02104] for support with funding.

ORCID

Camilla Malinowsky http://orcid.org/0000-0002-6938-5282

Louise Nygård http://orcid.org/0000-0003-1813-7390

References

[1] Rosenberg L, Nygård L, Kottorp A. Everyday Technology Use Questionnaire (ETUQ) – psycho-metric evaluation of a new assessment of compe-tence in technology use. OTJR. 2009;29(2):52–62. [2] Rosenberg L, Kottorp A, Winblad B, et al. Perceived

difficulty in everyday technology use among older adults with or without cognitive deficits. Scand J Occup Ther. 2009;16(4):216–226.

[3] Nygård L, Pantzar M, Uppgard B, et al. Detection of disability in older adults with MCI or Alzheimer’s disease through assessment of perceived difficulty in using everyday technology. Aging Ment Health. 2012;16(3):361–371.

[4] Kaptain RJ, Kottorp A, Patomella A-H, et al. Inter-rater and test-retest reliability of the Danish version of the everyday technology use questionnaire. Scand J Occup Ther. 2017;26:1–9.

[5] Fallahpour M, Kottorp A, Nyg~A¥Rd L, et al. Percived diffculites in everday technology use in people with aquired brain injury: Comparision to controls. J Rehabil Med. 2014;46(7):635–641. [6] H€allgren M, Nygård L, Kottorp A. Everyday

technol-ogy use among people with mental retardation – relevance, perceived difficulty, and influencing fac-tors. Scand J OccupTher. 2014;21(3):210–218. [7] Patomella A-H, Kottorp A, Ferreira M, et al.

Everyday technology use among older adults in Sweden and Portugal. Scand J Occup Ther. 2018; 25(6):436–445.

[8] Malinowsky C, Kottorp A, Tanemura R, et al. Everyday technology use among older adults in Sweden and Japan: a comparative study. Scand J Occup Ther. 2018;25(6):446–456.

[9] Hedman A, Nygård L, Kottorp A. Everyday technol-ogy use related to activity involvement among peo-ple in cognitive decline. Am J Occup Ther. 2017; 71(5):7105190040p1–7105190040p8.

[10] Sheikh JI, Yesavage JA. Geriatric Depression Scale (GDS) recent evidence and development of a shorter version. Clin Gerontol. 1986;5(1/2):165–173.

[11] Yesavage JA, Brink TL, Rose TL, et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res. 1982–1983;17(1):37–49.

[12] Ware JE, Kosinski M, Keller S. 12-item short-form health survey. Construction of scales and prelimin-ary tests of reliability and validity. Med Care. 1996; 34(3):220–233.

[13] Ware JE, Snow K, Kosinski M, et al. SF-36 health survey manual and interpretation guide. Boston (MA): The Health Institute, New England Medical Center; 1993.

[14] Kottorp A, Nygård L. Development of a short-form assessment for detection of subtle activity limita-tions: can use of everyday technology distinguish between MCI and Alzheimer’s disease? Expert Rev Neurother. 2011;11(5):647–655.

[15] Malinowsky C, Kottorp A, Rosenberg L, et al. Differences in the use of everyday technology among persons with MCI, SCI and older adults without known cognitive impairment. Int Psychogeriatr. 2017;29(7):1193–1200.

[16] Ryd C, Nygård L, Malinowsky C, et al. Associations between activities of daily living and everyday tech-nology. Scand J Occup Ther. 2015;22(1):33–42. [17] American Educational Research Association,

American Psychological Association and National Council of Measurement in Education. The stand-ards for educational and psychological testing. Washington DC (MA): American Psychological Association; 2004.

[18] Nyunt MS, Fones C, Niti M, et al. Criterion-based validity and reliability of the Geriatric Depression Screening Scale (GDS-15) in a large validation sam-ple of community-living Asian older adults. Aging Ment Health. 2009;13(3):376–382.

[19] Bohannon RW, Maljanian R, Landes M. Test-retest reliability of short form (SF)-12 component scores of patients with stroke. Int J Rehabil Res. 2004;27(2): 149–150.

[20] Streiner DL, Norman GR, Cairney J. Health meas-urement scales: a practical guide to their develop-ment and use. 5th ed. Oxford: Oxford University Press; 2015.

[21] Adler E, Resnick B. Reliability and validity of the dementia quality of life measure in nursing home residents. West J Nurs Res. 2010;32(5):686–704. [22] Brod M, Stewart AL, Sands L, et al.

Conceptualization and measurement of quality of life in dementia: The Dementia Quality of Life Instrument (DQoL). Gerontologist. 1999;39(1):25–36. [23] Gerritsen DL, Steverink N, Ooms M, et al.

Measurement of overall quality of life in nursing homes through self-report: the role of cognitive impairment. Qual Life Res. 2007;16(6):1029–1037. [24] Ettema TP, Hensen E, De Lange J, et al. Self report

on quality of life in dementia with modified COOP/ WONCA charts. Aging Ment Health. 2007;11(6): 734–742.

[25] Tabali M, Jeschke E, Dassen T, et al. The Nottingham health profile: a feasible questionnaire for nursing home residents? Int Psychogeriatr. 2012; 24(3):416–424.

[26] Hubbarb G, Downs MG, Tester S. Including older people with dementia in research: challenges and strategies. Aging Ment Health. 2003;7(5):351–362. [27] Bond TG, Fox CM. Applying the Rasch model:

fun-damental measurement in the human sciences. Mahwah (NJ): Lawrence Erlbaum; 2007.

(11)

[28] Nygård L. Manual to the questionnaire about everyday technology in home and society: Everyday Technology Use Questionnaire (ETUQ). Stockholm: Karolinska Institutet, Department of Neurobiology, Care Sciences and Society, Division of Occupational Therapy; 2016. [29] American Psychiatric Association. Diagnostic and

statistical manual of mental disorders. Washington DC (MA); 2000.

[30] American Psychiatric Association. Diagnostic and stat-istical manual of mental disorders. 5th ed. Arlington (VA): American Psychiatric Association; 2013.

[31] Petersen RC. Mild cognitive impairment as a diag-nostic entity. J Intern Med. 2004;256(3):183–194. [32] Winblad B, Palmer K, Kivipelto M, et al. Mild

cog-nitive impairment– beyond controversies, toward a consensus: report of the International Working Group on mild cognitive impairment. J Intern Med. 2004;256(3):240–246.

[33] Dewing J. Process consent and research with older persons living with dementia. Res Ethics Rev. 2008; 4(2):59–64.

[34] Folstein MF, Folstein SE, McHugh PR.”Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J. Psychiat. Res. 1975;12(3):189–198.

[35] Holbrook M, Skilbeck CE. An activity index for use with stroke patients. Age Ageing. 1983;12(2):166–170. [36] Fisher AG, Jones KB. Assessment of motor and

pro-cess skills. Vol. 2. Fort Collins (CO): Three Star Press, Inc; 2012.

[37] Linacre JM. Winsteps– Rasch model computer pro-gram (Version 3.92.1). Beaverton (OR): Winsteps.com; 2016.

[38] IBM Corp. IBM SPSS statistics for windows, Version 24.0. Armonk (NY): IBM Corp; 2016.

[39] Hoaglin DC, Iglewicz B, Tukey JW. Performance of some resistant rules for outlier labeling. J Am Stat Assoc. 1986;81(396):991–999.

[40] Grubbs FE, Beck G. Extension of sample sizes and percentage points for significance tests of outlying observations. Technometrics. 1972;14(4):847–854. [41] Cohen J. Statistical power analysis for the behavioural

sciences. 2nd ed. Hillsdale (NJ): Erlbaum; 1988. [42] Linacre JM. Sample size and item calibration

stabil-ity. Rasch Meas Trans. 1994;7(4):328.

[43] Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2): 420–428.

[44] Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. Hoboken (NJ): Wiley; 2004. [45] Nygård L, Rosenberg L. How attention to everyday

technology could contribute to modern occupational therapy: A focus group study. Br J Occup Ther. 2016;79(8):467–474.

[46] Jakobsson E, Nygård L, Kottorp A, et al. Comparing the ability to use everyday technology and the amount of relevant everyday technologies as per-ceived by older adults with cognitive impairment and as estimated by proxies. (in review).

[47] Malinowsky C, Larsson Lund M. The association between perceived and observed ability to use every-day technology in working age people with ABI. Scand J Occup Ther. 2014;21(6):465–472.

[48] Malinowsky C. Managing technology in everyday activities: a study of older adults with dementia, MCI, and no cognitive impairment [dissertation]. Stockholm: Karolinska Institutet, Department of Neurobiology, Care Sciences and Society, Division of Occupational Therapy; 2011.