Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=core20
Oxford Review of Education
ISSN: 0305-4985 (Print) 1465-3915 (Online) Journal homepage: https://www.tandfonline.com/loi/core20
Exploring the Letterbox Club programme’s impact on foster children’s literacy: potent intervention or general support?
Hilma Forsman
To cite this article: Hilma Forsman (2019): Exploring the Letterbox Club programme’s impact on foster children’s literacy: potent intervention or general support?, Oxford Review of Education, DOI:
10.1080/03054985.2019.1595559
To link to this article: https://doi.org/10.1080/03054985.2019.1595559
© 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
Published online: 09 Apr 2019.
Submit your article to this journal
View Crossmark data
Exploring the Letterbox Club programme ’s impact on foster children ’s literacy: potent intervention or general support?
Hilma Forsman
Stockholm University, Sweden
ABSTRACT
The book-gifting programme, the Letterbox Club, was developed as a response to the increased interest in ways of improving the educational outcomes of children in out-of-home care. By report- ing quantitative and qualitativefindings from a Swedish trial, and compilingfindings from previous British evaluations, the purpose of this paper is to further our understanding of the programme’s potential impact. Pre/post measurements of the reading age of 72 foster children showed an average improvement of 2.5 months in comparison to the national average. With some exceptions, inter- views with children and carers showed that the programme was well received and indicated that it could increase reading engage- ment and carer involvement. The current empirical base knowl- edge suggests that the Letterbox Club has a small impact on foster children’s literacy. The results do not allow for causal interpreta- tions. Long-term outcomes are unknown. The programme lacks a theoretical foundation, and the implementation is dependent on individual and contextual factors. However, the programme is simple to administer, low-cost, and can reach a large number of children. The article therefore suggests that the Letterbox Club could be seen as a general supportive measure, and promotion of carer involvement is proposed as a way of improving its potential impact.
KEYWORDS
Foster children; literacy;
intervention; general support; evaluation; impact
Introduction
It is well known that children with experience of out-of-home care tend to be low achievers in school. Studies have shown that the educational gap between foster children and their peers tends to begin at a young age, to increase as they get older, and to persist into midlife (Brännström, Vinnerljung, Forsman, & Almquist,2017; Sebba et al., 2015; Trout, Hagaman, Casey, Reid, & Epstein, 2008; Vinnerljung & Hjern, 2011).
Furthermore, foster children’s poor school performance has been identified as a major risk factor for future adverse outcomes, e.g. economic hardship, illicit drug use, crimin- ality, and mental health problems (Berlin, Vinnerljung, & Hjern, 2011; Forsman, Brännström, Vinnerljung, & Hjern,2016). Addressing foster children’s educational under- achievement may thus not only help them catch up in school, but also improve their overall life chances.
CONTACTHilma Forsman hilma.forsman@socarb.su.se https://doi.org/10.1080/03054985.2019.1595559
© 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
In light of the above, researchers and policymakers have become increasingly inter- ested in ways of improving foster children’s academic skills and providing educational support through tailored interventions. However, intervention studies within thisfield are scarce and characterised by variable design and quality. Furthermore, the programmes tend to use an ad-hoc approach, and theoretical considerations are lacking (Evans, Brown, Rees, & Smith,2017; Forsman & Vinnerljung,2012). An exception is tutoring programmes, which have good empirical support, and stronger theoretical foundations (Flynn, Marquis, Paquet, Peeke, & Aubry,2012; Harper & Schmidt,2016; Hickey, 2018). Examples of other evaluated initiatives that have demonstrated promising results include, e.g. paired reading projects (Osborne, Alfano, & Winn,2010; Vinnerljung, Tideman, Sallnäs, & Forsman,2014), individualised educational support models (Durbeej & Hellner,2017; Tideman, Vinnerljung, Hintze, & Aldenius,2011; Tordön, Vinnerljung, & Axelsson, 2014), and book-gifting pro- grammes (Griffiths,2012; Wolfendale & Bryans,2004). One intervention that has received particular interest, and has been spread to thousands of children across the UK, is a book- gifting programme called the Letterbox Club.
The Letterbox Club
The Letterbox Club is an intervention that seeks to encourage children in out-of-home care to engage in reading and mathematics by providing them with learning materials through the post. Children enrolled in the programme become members of a club through which they receive personalised monthly parcels to their foster home for 6 months, usually including the summer holiday. Each parcel typically contains a letter to the child, two books (a mixture of fiction and non-fiction, carefully selected by a panel), a number game, and stationery items (Griffiths,2012).
The Letterbox programme shares a common ground with book-gifting interventions in general, which have shown varying levels of effectiveness (Burnett, Daniels, & Bailey,2014;
Jakobsen & Andersen,2013; Slavin, Lake, Chambers, Cheung, & Davis 2009). Book-gifting interventions rest on the assumptions that book ownership, enthusiasm for books, and shared book reading can lead to improved literacy (Burnett,2017). However, the Letterbox Club is not manual-based, and lacks an explicit theory of change (Mooney, Winter, &
Connolly,2016). The devising of the programme has rather been described as shaped by constraints, i.e. it had to be low-cost and administered without action through schools or foster carers. The intervention is thereby directed at the child. Although there is a hope that carers will get involved, this is not expected, and there is no guidance on how carers can offer support (Griffiths,2012).
Previous British studies report that the programme has been well received and appre- ciated by both children and carers. Furthermore, pre/post measurements have shown small but significant gains in the participating children’s reading ability in comparison with the national average. That is, the improvements recorded from these groups of foster children were larger than the expected progress among same-aged peers (Dymoke & Griffiths,2010;
Griffiths, 2012; Griffiths & Comber, 2011; Griffiths, Comber, & Dymoke, 2010; Winter, Connolly, Bell, & Ferguson, 2011). Recently, the programme was evaluated in a randomised controlled trial (RCT) in Northern Ireland. The trial found no significant effects in terms of improvements to the children’s reading ability. However, according to the authors, the trial was underpowered to detect anticipated effect sizes (Mooney et al.,
2016). Thus, it is still unclear in what ways the Letterbox Club could be useful in targeting foster children’s poor school performance.
The overall purpose of this article is to further our understanding about the Letterbox programme’s potential impact on foster children’s reading attainment.
The article adds to the knowledge base in the following ways. First, it reports the quantitative findings from a Swedish trial of the Letterbox Club with foster children aged 8–11 by means of pre/post measures of the participating children’s reading ability, making it the first evaluation of the programme outside the UK. Second, it explores the children’s and carers’ views and experiences of the programme, thus providing more insights into how the programme is received in a new context. Third, it compiles the findings from the Swedish trial and previous British evaluations, thereby providing an overview of the current empirical base knowledge. Fourth, the article discusses whether the Letterbox Club could and should be seen as a basic, yet useful supportive measure rather than a potent academic intervention for children in out-of-home care.
Methods
The Swedish trial of the Letterbox Club was part of a reading promotion initiative targeting vulnerable children in two national counties—Stockholm and Jönköping (both include urban and rural areas). The project managers were in close contact with the Letterbox founder in order to secure programme compliance. Hence, the same principles of e.g. choosing and delivering materials. Yet, in the Stockholm region, the intervention period of 6 months did not include the summer holiday. Additionally, children in Stockholm were only sent reading materials, since appropriate number games in Swedish had not yet been developed.
The trial included a pre/post evaluation of the children’s reading age development, and an interview study with some of the participating children and their carers. The trial was approved by the regional ethics committee in Stockholm (dnr 2014/929–31; dnr 2015/859–31).
Pre/post evaluation
Criteria for inclusion in the pre/post evaluation were: children aged 8–11 (attending second to fifth grade in compulsory school) placed in foster family care.
Children were excluded if they were part of another intervention programme specifically targeting foster children’s academic skills (n = 5), if they had poor Swedish language skills and thus were not able to perform the literacy test (n = 0), or if they had severe special needs and thereby were not likely to benefit (n = 8). In addition, it is estimated that 13 children did not participate since consent was not received from their birth parents, carers, and/or child welfare caseworkers.
In total, 81 children were eligible to participate in the study. Out of these, one dropped out, two did not want to do the post-intervention test, and test results were incomplete or not returned for another six. Consequently, the participant group consists of 72 children (attrition 11.1%).Table 1summarises the demographic characteristics at
baseline (there were no noticeable differences between the participant group and children who were lost to follow-up).
As can be seen, the participant group was gender balanced. Age ranged from 8 to 11 (mean: 9.54 years). Most children came from the Jönköping region (61.1%), but children from urban and rural areas were represented. The majority of children (90.3%) and foster homes (95.8%) had Swedish as their native language. The average child had spent more than 4 years in the present foster home.
Tests
Each child’s reading ability was assessed pre and post intervention (6–8 months later) with a national age-standardised literacy test (LäSt: Elwér, Fridolfsson, Samuelsson, &
Wiklund,2016). Two word reading subtests were chosen:‘Decoding of non-words’ (LäSt Nonord) and‘Decoding of words’ (LäSt Ord). In each test, the child is expected to read and decode a set of made up non-words and a number of common words, respectively.
The former mainly measures the child’s alphabetical reading skills, which is a step towards developing orthographical reading skills, constitutes a basis for the develop- ment of writing skills, and continues to be an important strategy when children are exposed to new words when reading. With development of orthographic reading skills, which is measured by the latter, reading becomes an automatic process. Results are expressed as‘reading age’ in years and months.
The tests were administered by local teachers. Most children performed the tests at their school, and some in the foster home.
Statistical analysis
Test results were analysed with paired samples t-tests of average levels of reading age pre/post intervention, after adjusting the level at T2 for estimated time effects (i.e. the T2
Table 1.Descriptive statistics,n = 72.
Variable Frequency (Percent)
Gender
Boys 36 (50.0)
Girls 36 (50.0)
Age
8 9 (12.5)
9 26 (36.1)
10 26 (36.1)
11 11 (15.3)
Region
Stockholm 28 (38.9)
Jönköping 44 (61.1)
Swedish as native language
Yes 65 (90.3)
No 7 (9.7)
Swedish as native language in the foster home
Yes 69 (95.8)
No 3 (4.2)
Placement duration in the present foster home (months)
(Range: 3-129) 50.31 (mean)
48.00 (median) Number of children in the foster home
(Range: 1-7) 2.42 (mean)
2.00 (median)
reading level was subtracted with the number of months that had passed between T1 and T2 for each individual child). T-tests were performed for each subtest and for a constructed index representing the average level of reading age when results from both subtests were combined. Additionally, effect sizes were calculated, i.e. pre-group mean was subtracted from the post mean and divided by the standard deviation at T1, the usual effect size calculation for one-group pre–post designs (Lipsey & Wilson,2001).
The sample was too small for performing meaningful multivariate analyses. However, simple subgroup comparisons were conducted by means of different bivariate analyses.
Compiling quantitative data in previous evaluations
In order to compare the results with previous studies, the pre/post measurements of the children’s reading ability in previous British evaluations were crudely compiled (the compilation did not include a critical assessment of the quality of the evaluations).
These evaluations were based on the same national literacy test (The Neale Analysis of Reading Ability: Neale,1997). The test is age-standardised with a mean score of 100 and a standard deviation of 15. Pre–post changes were retrieved from the articles/reports (when pre and post means were presented by subgroups, a sample mean was estimated based on this information). When possible, effect sizes were calculated. Effect sizes in the pre/post evaluations were calculated in the same manner as in the Swedish trial previously described. Results from the RCT were calculated by taking the mean pre–post change in the treatment group minus the mean pre–post change in the control group, divided by the pooled pre-test standard deviation, which is the recommended measure for studies with unequal sample size within a pre–post-control design (Morris,2008).
Interview study Sample
Due to practical reasons, the interviews were based on a convenience sample (Robinson, 2014), including all children whose placement was located in or around Stockholm (n = 19). After an initial assessment by the caseworkers, one child was ruled out due to ‘turmoil’. The remaining children and foster families were contacted by their case- workers, who informed them about the interviews and secured consent, which was confirmed by the researcher prior to and during the interviews. In one case, it was not possible to retrieve consent from the child’s birth parent. Additionally, four foster families declined. In studies that use voluntary participation, individuals who do not consent may be different to those who do (Robinson, 2014). In this case, reasons for declining included: time constraints, turmoil, and not wanting to expose the child to
‘authority personnel’. Another two carers agreed to participate themselves, but did not want their children to be interviewed, since they had not liked the programme. Thefinal sample was therefore comprised of 11 children and 13 carers, with presumably different experiences of the programme.
The child sample was quite representative of the total participant group in terms of age, gender, and native language. However, they had longer placement durations, and higher pre and post reading age levels compared to the total participant group mean.
Data on the carer sample are limited, but the sample comprises 10 women and three men. Two were kinship carers (grandmothers), four had vast experience of caring for
many children both in emergency, and in short- and long-term placements, and the rest had cared for fewer children, usually long-term.
Interviews
Two carers were interviewed by telephone. The other interviews took place at the foster carers’ homes. Both the child and the carer interview guides consisted of questions aimed to capture attitudes towards reading, use of and engagement with the parcels, and overall views and experiences of the programme. Most questions were open-ended.
Additionally, photos of the books, and pictures of ‘smileys’ (happy, neutral, sad) were used as prompts during the interviews with the children.
After a period of‘getting to know each other’, the planned structure was to interview the child, and thereafter turn to the carer, but the interviews took on somewhat different forms. Four children were interviewed separately with the carer in a nearby room.
Another four had the carer present but ‘inactive’ when questions were directed to the child. The other three child interviews were characterised by carers who actively joined the conversation. Furthermore, while some children immediately started talking about the Letterbox Club, others needed stimuli and/or direct questions. Most children were present when the carers were interviewed, and sometimes joined in, e.g. objected or supported a statement.
The interviews were recorded and ranged in length from 20 to 90 minutes, although each visit usually lasted for 2–3 hours.
Analysis
Following transcription, the interview data were analysed with the aim of conden- sing the rich textual data into a brief, summary format, which describes the chil- dren’s and carers’ use of and engagement with the parcels, and overall views and experiences of the programme in comparison tofindings in previous British evalua- tions. In this process (which was guided by the ‘general inductive approach’:
Thomas, 2006), the raw text was repeatedly read to identify themes and categories.
A coding frame was developed and segments of the text were labelled into cate- gories. The categories were repeatedly refined and the transcripts were reread in line with the revised coding frame. Furthermore, text irrelevant to the evaluation objec- tives was dismissed. The coding process resulted in five summary categories in which subcategories were sorted, and include contradictory points of view and comparisons with previous evaluations. The summary categories capture positive and negative notions about the concept of the programme, the book-gifting aspect, and the content in the parcels as well as experiences of reading engagement and adult involvement.
Results
Reading age development
Table 2shows the average development of reading age when comparing test results at T1 to time adjusted results at T2. On the subtest‘Decoding of non-words’, the average child had made a significant improvement in his/her reading age of 4.31 months above
the expected time effect in comparison to the same-aged national average. The corre- sponding result on the subtest‘Decoding of words’ shows a small, but non-significant improvement.
The decoding index shows the participants had on average improved their reading age by 2.51 months (p=0.01). This corresponds to an effect size of 0.18, which might be considered‘small’. However, in educational policy contexts, effect sizes of around 0.20 are often of policy interest when they are based on measures of academic achievement (Hedges & Hedberg,2007). The standard deviation was relatively high on all measures, i.e. some children had made large gains whilst some performed at a lower level following the intervention in comparison to the expected reading development in the national norm.
As can be noted, not all participants are included in Table 2. The missing numbers (n = 7 on‘Decoding of non-words’, n = 1 on ‘Decoding of words’) account for a measure- ment limitation known as the‘ceiling effect’. These children were close to or reached the
‘ceiling’ of the pre-test and can therefore not show their improvement in reading age.
Nevertheless, it is possible to detect trends of growth or decline by comparing their T1/
T2 raw scores. On‘Decoding of non-words’, four children had increased their scores, two remained at the same level, and one had decreased. On‘Decoding of words’, the child’s scores had increased (not shown in tables).
Figure 1shows that the small improvements in this study echo the results in previous British evaluations of the Letterbox Club (seeAppendix A). The black straight line with arrowheads represents the average reading level in the general population. All other
Table 2.Average development of reading age (months), comparisons adjusted T2 vs. T1 with paired samplesT-tests, n = 65/71.
Test Mean SD p-value ES
Decoding of non-words 4.31a 10.75 0.002 0.29
Decoding of words 0.68b 7.01 0.416 0.04
Decoding index 2.51a 7.62 0.010 0.18
Notes:an = 65;bn = 71.
86 88 90 92 94 96 98 100 102 104
T1 T2
National average Wal -11 (7-9) +2.5 Eng -07 (7-9) +4.35 Eng -07 (9-11) +2.53 Eng -08 (7-9) +4.4 Eng -08 (9-11) +3.5 Wal -09 (9-11) +3.5 Wal -11 (9-11) +3.2 Wal -09 (7-9) +4.4 Eng -10 (11-13) +3.0 NI -09-10 (7-11) +3.6 NI RCT (7-11) +2.15
Figure 1. Results for intervention groups at T1 and T2 in previous studies in comparison to the national average, change in average scores at the Nealy analysis of reading ability (reading accuracy).
lines show the results for different participant groups (Dymoke & Griffiths,2010; Griffiths, 2012; Griffiths et al.,2010; Mooney et al.,2016; Winter et al.,2011). All groups start with an average reading level below the national average. Following the programme, all groups have increased their score (ranging from 2.15 to 4.4 points), thereby decreasing the performance gap to their same-aged peers. All improvements in the pre/post evaluations are reported as significant in comparison to the national average. With the exception of Mooney et al., both the Swedish and British studies were not randomised controlled studies and the findings from them do not therefore allow conclusions implying causation.
The smallest improvement among the intervention groups is reported in the RCT study (the dotted black line). Their results were being compared to a matched control group, whose average score had declined (−1.15 points: Mooney et al., 2016). The gap between the control group and the national norm had thus increased over time, which reflects the expected development among foster children in general (Sebba et al.,2015).
A comparison of the pre/post means in the RCT intervention and control groups yielded an effect size of 0.25 (Appendix A). The authors report that the differences between the groups are not significant. However, they also note that the trial was sufficiently powered to detect a minimum effect size of 0.47, and that it therefore was under- powered to detect effect sizes within the anticipated range of 0.20–0.30 (Mooney et al., 2016). Accordingly, the absence of statistically significant differences was expected at the onset of the trial.
Subgroup differences
Comparisons of different subgroups in the Swedish trial revealed a significant associa- tion between age and the level of development, i.e. younger children had made larger gains. Thisfinding is similar to the results in a study of another Swedish intervention targeting foster children’s reading ability with a paired reading programme (Vinnerljung et al.,2014). Furthermore, although it has not been possible to examine any statistical subgroup differences in previous evaluations of the Letterbox Club, a trend is larger gains among younger age groups (7–9 years) compared to older (9–11 years) (Figure 1).
Region accounted for another in-sample variation. Children from Stockholm had made larger gains compared to participants from Jönköping. Differences between the groups were significant on all measures. These results are most likely due to differences in the timing of the intervention and the pre/post measurements. Children in Stockholm received their parcels during the autumn and spring semester, whilst the intervention period for children in Jönköping included 10 weeks of the summer holiday, a period when attainment usually drops, especially among children from lower socioeconomic backgrounds (Allington & McGill-Franzen,2003).
No noticeable differences were found between boys and girls, or with regards to native language or placement duration. Furthermore, there were no clear associations between reading age development and the raw scores or age-standardised level of reading age at baseline, indicating poor and strong readers developed equally.
Child and carer experiences A likeable programme?
Both children and carers liked the idea of getting learning material in personalised packages through the post. Children typically received the parcels in their home after school. Their reaction was described with feelings such as excitement, joy, and anticipa- tion. This was partly attributed to the surprise moment of not knowing what the parcel would contain, but also to the delivery in itself. Getting books through the post was described as more exciting than going to the library or a bookstore. The personalised delivery of the parcels also seemed to make the children feel noticed and special. Mostly carers, but also some of the children, said that these aspects had evoked an interest in, and encouraged increased reading, which is similar to findings in earlier studies (Griffiths,2012; Griffiths & Comber,2011; Hancock & Leslie, 2014).
Compared to previous evaluations, the club membership was not as central to their experiences of the programme. Whilst some carers applauded the fact that it targets foster children, saying it was good that they‘could feel special in a positive sense’, most children seemed to be unaware of this concept, and did not talk about it as a club at all.
In those cases, it seemed as if neither carers nor caseworkers had properly explained what the parcels were about or where they came from, which points out the importance of informing children about the programme (Roberts, Winter, & Connolly,2017).
In contrast to these mainly positive views, it is important to acknowledge that two carers talked about negative child responses.‘He just wants to be like any other child, not pointed out as a child in need of extra support just because he doesn’t live with his biological mum’, one of them explained. The other carer described her child as a reluctant reader who thought the programme was‘lame and childish’.
Who would not like to get books?
With some exceptions, children and carers talked about the book-gifting aspect in positive terms. Carers thought it could encourage children to read more, and some explicitly said that it had. In particular, a subgroup of children described as ‘book deprived’ seemed to value the gifts and expressed that it had enabled them to read more at home. An enthusiasm for books was also apparent among children in well- provided environments. Children who loved to read seemed to have a continuous hunger for books. Another group’s indifference to the Letterbox Club’s contribution was mainly attributed to not enjoying reading in general. More worrying, one boy said:
‘When I first got some books—it was good, but then it came loads of them and I started thinking I would never be able to read them all’. After further exploring, this was interpreted as he had felt overwhelmed by the number of books. Hence, the current study reflects both positive results regarding book-gifting from some evaluations (Griffiths,2012; Griffiths & Comber,2011; Hancock & Leslie,2014), and a critical notion of this aspect reported by Roberts et al. (2017), who introduced the term ‘book burdened’.
An appealing content?
The next important aspect of how the programme was received is related to the content in the parcels. Both children and carers had experienced the range of the book selection
as positive. In particular, the mixture offiction and non-fiction was appreciated. Carers said that it could, and some meant that it had, promoted an interest in new genres, which was supported by some children. In general, the books were referred to as‘high quality’, and some carers talked about ‘enthusiasm of books’ and how it had resulted in increased reading. However, not all children liked all books. Some books simply did not appeal to the child’s interests in reading. Older children in particular expressed that they would have preferred a scheme that enabled them to influence the selection. Another reason for not liking a book was that it was not on a developmentally appropriate level.
Overall, these findings are similar to results in earlier evaluations (Dymoke & Griffiths, 2010; Hancock & Leslie,2014; Roberts et al.,2017).
The letters in each parcel were perceived in varying ways. While the children appre- ciated letters from authors, they typically described the side letters as boring. Carers on the other hand could say it was valuable to receive information about the content.
Although the stationery items might not be seen as central, they seemed to play a role in the children’s attitude. They enjoyed getting extra gifts such as pens, note- books, and bookmarks. Some carers felt as if the stationery items had encouraged the children to engage in writing or other literary activities. It enhanced the excitement of receiving the parcels and could spur somewhat reluctant children into engaging with the books.
Reading engagement
The interviews revealed varying degrees of reading engagement across the sample.
Some children reported that they had read all or most of the books. Another group had read a smaller selection of books, and the aforementioned child who thought the programme was lame had not read any at all. A high level of reading engagement was related to a positive attitude towards reading in general, and/or being enthusiastic about the concept of the programme. Yet, some of the more reluctant readers had also engaged in reading when their carers had taken an active part in encouraging and sometimes pushing them to.
Book sharing, i.e. reading together, also varied. One group of children had a daily routine of reading the books with their carers. In other cases, book sharing happened occasionally. A third group only read by themselves. It seemed as if this was either a reflection of the child being an able reader who preferred independent reading or a result of a carer’s unwillingness and/or inability to engage in the reading process.
There were also examples of children reading to other children in the family.
Adult involvement
As demonstrated, carer involvement seemed to be of importance to the children’s overall engagement in the programme. When carers were asked about their role, somewhat conflicting attitudes regarding their responsibility emerged. Whilst some thought they were obliged to support the child’s engagement, others emphasised that the programme was directed at the child. However, the perceived impact of the carers’ involvement was not clearly related to the level of the involvement in itself, but rather their ability to engage at a suitable level for each individual child. This could mean regular shared reading or merely asking questions about the books and showing interest in the parcels.
Some carers seemed confident in their role of supporting the child, whereas others expressed a need for guidance. As one carer said‘I loved the idea of the programme, but he’s struggling, and I didn’t know how to help him, so he didn’t read as much as I’d hoped for’. Thus, in line with past evaluations (Dymoke & Griffiths, 2010; Griffiths &
Comber, 2011; Hancock & Leslie, 2014; Roberts et al., 2017), the current study both shows the importance of carer involvement and the need for carer support.
Although the Letterbox Club is not connected to the school setting, the interviews revealed that children could initiate a connection by telling teachers about the pro- gramme and bringing the books to school. Links to schools have been noted in some earlier evaluations (Griffiths,2012; Griffiths & Comber,2011; Hancock & Leslie, 2014). In this case, children and carers had positive experiences of teacher involvement including encouragement to engage in reading, and relating schoolwork to the books.
Discussion
In recent years, there has been an increased interest in ways of improving the educa- tional outcomes of children in out-of-home care. As a response, interventions targeting foster children’s academic skills have been developed. One initiative that has received considerable attention is the British book-gifting programme the Letterbox Club. By reporting the results from a Swedish trial of the programme, and compiling findings from previous evaluations, the purpose of this article was to further our understanding of the programme’s potential impact on foster children’s reading attainment.
The Swedish trial encompassed 72 foster children aged 8–11. The pre/post evaluation showed the participating children had on average improved their reading age by 2.5 months more in comparison to the national norm of same-aged peers. This is equivalent to an effect size of 0.18 and similar to findings in previous British evaluations (Griffiths & Comber, 2011; Griffiths, 2012; Griffiths et al., 2010). However, there were differences across the two subtests used. The largest gain was on the subtest that mainly measures alphabetical reading skills, whilst the improvement on the subtest that measures orthographical reading was not significant. This might be due to the former skills being a basis for the latter in a child’s reading development (Elwér et al.,2016). Moreover, there were variations across the participant group, demonstrating that it cannot be expected that all foster children will benefit from the programme. Subgroup analyses revealed that younger children had made larger gains. The compilation of previous British evaluations showed a similar trend, indicating that younger children might be more susceptible to the programme. Furthermore, the group who received the intervention during the summer months had made smaller improvements. This is expected since attainment usually drops during the summer holiday (Allington & McGill-Franzen, 2003). However, intervention research has found that children benefit from receiving support in different contexts (Ferrer-Wreder, Stattin, Lorente, Tubman, & Adamson, 2004). Thus, this finding could also indicate that the programme has better potential to support children’s reading development when they attend school. Teacher involvement was highlighted as impor- tant by children and carers in both the Swedish and previous evaluations (Griffiths,2012;
Griffiths & Comber,2011; Hancock & Leslie,2014).
Interviews with some of the participating children and their carers provided some further insights into the programme’s potential impact. In general, the programme idea
was perceived as likeable, the book-gifting was appreciated, and the content was appealing. There were also indications that these aspects could result in increased reading engagement and that participation in the programme could inspire carers to get involved in the process. Yet, there were also examples of more neutral and even negative responses to the programme. In comparison to previous studies, these results thus lend support to positive notions put forward in some reports (e.g. Griffiths,2012) and the more critical ones brought forward by Roberts et al. (2017). Overall, the differences in how the programme was received support the previous notion that it cannot be expected that all participants will benefit, and might explain some of the variation in reading development across the group.
The current study thus shows that the Letterbox Club seems to work in a similar way in a Swedish context and implies that it is transportable across countries, but based on the current empirical base, what can we actually say about its potential impact on foster children’s reading attainment? Should the programme be seen as a potentially potent intervention targeting foster children’s academic skills or should it rather be seen as a programme with the potential of providing basic supplementary support to this academically vulnerable group?
The Swedish and previous British evaluations do show that participation in the Letterbox Club is associated with significant improvements in foster children’s reading skills in comparison to the national average of same-aged peers. However, the study design does not allow for causal interpretations. Since the performance gap between foster children and their peers tends to grow over time, one could argue that compar- isons with a national average could actually underestimate an intervention’s true impact.
On the other hand, comparing foster children—a group that in many ways differ from the population as a whole—to a national average could be problematic in itself.
Furthermore, it is not possible to eliminate other causal interpretations to noted improvements in the intervention groups. Unfortunately, the only evaluation of Letterbox Club that used a matched control within a randomised design was under- powered and therefore not able to detect effect sizes within the anticipated range (Mooney et al., 2016). A Danish RCT did report that a similar book-gifting programme had a positive effect on socially disadvantaged pre-school immigrant children’s literacy skills. However, unlike the Letterbox programme, parents were expected to, and were supported in taking, an active part in the delivery of the intervention (Jakobsen &
Andersen,2013), which complicates any generalisations of those positivefindings.
Another problem with interpreting the Letterbox programme’s potential impact is the lack of long-term follow-up studies. All evaluations have focused on the children’s reading attainment directly following the intervention. Thus, we do not know if noticed improvements are sustained over time.
Yet another limitation relates to the programme’s lack of an explicit theory of change.
The programme founder has suggested that the Letterbox Club could act as a‘catalyst’
and start a ‘virtuous circle’ of attainment (Griffiths, 2012). Indeed, the interviews pro- vided examples of children and carers whose experiences of the programme supported such ideas. Yet, the process from receiving the parcels to possibly engaging in more reading is by no means automatic. In fact, as noted by Roberts et al. (2017), both individual and contextual factors seem to be of great importance in determining for whom and under what circumstances the programme has the possibility to begin
a virtuous circle of improved attainment. Moreover, the mechanisms behind such improvement remain unclear. With that being said, this study points to the importance of adult—carer—involvement.
Previous research has shown that carer support has a positive impact on foster children’s school performance (Flynn, Tessier, & Coulombe, 2013; Jackson & Martin, 1998), and adult involvement was a part of the previously mentioned successful Danish book-gifting programme (Jakobsen & Andersen,2013). In its current form, the Letterbox programme does not build on carer involvement. Nevertheless, the current study indicates that carers, in varying degrees, do get involved in the process. Similar findings have been reported in previous evaluations and might explain noted improve- ments. However, carers need support in their supporting role (Dymoke & Griffiths,2010;
Griffiths & Comber, 2011; Hancock & Leslie, 2014; Roberts et al., 2017). With proper support from the care and/or educational system, carers can be actively involved in interventions targeting foster children’s academic skills (Flynn et al., 2012; Forsman, 2017; Vinnerljung et al.,2014).
Conclusions and implications
The current empirical base suggests that the Letterbox Club has a small impact on foster children’s literacy. However, most evaluations are based on pre/post measurements with age-standardised instruments, which do not allow for causal interpretations. Long-term outcomes are unknown. The programme lacks an explicit theoretical foundation, and the implementation is dependent on individual and contextual factors. If we, despite these uncertainties, are to take the improvements reported in the current and previous evaluations at face value, we can conclude that they are not on a large scale (ES:
0.18–0.26). Yet, when interpreting the significance of a particular effect size, the magni- tude has to be placed into an appropriate context (Durlak, 2009) and in educational policy contexts, effect sizes of around 0.20 often have practical value (Hedges &
Hedberg, 2007). Furthermore, since the performance gap between foster children and their peers tends to grow over time, one could argue that simply not getting worse could be seen as successful. An important goal for interventions targeting foster chil- dren’s academic skills could therefore be formulated in terms of preventing an expected deterioration over time.
Moreover, the impact of an intervention is not only a product of its measured efficacy.
From a public health perspective, the impact of an intervention can be defined as the programme’s reach, and its efficacy (impact = reach × efficacy). Following this, less effica- cious interventions that reach large numbers of people may have a larger overall impact compared to high-efficacy interventions, which tend to be intensive, expensive, and deliv- ered to a small proportion of the target population (Glasgow, Vogt, & Boles,1999).
In child welfare, the continued pursuit of evidence-based practice has led to the development of manualised evidence-supported interventions. Regrettably, researchers find many barriers to adopting, implementing, and maintaining such interventions in practice (Barth et al., 2012). In comparison, the Letterbox Club already reaches thou- sands of children in care across the UK. The concept has been spread internationally, inspiring similar initiatives in Canada (‘The Bookworm Club’: Brady, 2013), Denmark (‘Klub Penalhus’, currently being tested by the National Board of Social Services), and
now also Sweden. Thus, looking at practical significance, the programme has the potential to reach a large number of children. Furthermore, the programme is simple to administer and low-cost. It has been well received by both children and carers— which has a value in itself—and although it will not appeal to all, the risk of harm is low.
Based on the above, this article suggests that the programme in its current form could and should be seen as a basic, yet useful supportive measure, such as e.g. libraries, but specifically targeted at an educationally vulnerable group. From this perspective, the Letterbox Club’s potential is best understood as a supplement to other more intense and theoretically driven interventions. However, by further exploring the potential role of links to the educational system, and in particular, promoting carer involvement, it might be possible to strengthen its potential impact on foster children’s academic skills.
Disclosure statement
No potential conflict of interest was reported by the author.
Funding
This work was supported by Bokspindeln AB and FoUrum; Region Jönköping County.
Notes on contributor
Hilma Forsmanhas a master’s degree in social work and is a PhD student at the Department of Social Work, Stockholm University. Her main PhD project concerns foster children’s school perfor- mance and focuses on both risk factors and interventions.
ORCID
Hilma Forsman http://orcid.org/0000-0001-5082-388X
References
Allington, R. L., & McGill-Franzen, A. (2003). The impact of summer setback on the reading achievement gap. Phi Delta Kappan, 85(1), 68–75.
Barth, R. P., Lee, B. R., Lindsey, M. A., Collins, K. S., Strieder, F., Chorpita, B. F., . . . Sparks, J. A. (2012).
Evidence-based practice at a crossroads: The timely emergence of common elements and common factors. Research on Social Work Practice, 22(1), 108–119.
Berlin, M., Vinnerljung, B., & Hjern, A. (2011). School performance in primary school and psycho- social problems in young adulthood among care leavers from long term foster care. Children and Youth Services Review, 33(12), 2489–2497.
Brady, E. (2013). The bookworm club: The implementation story of an evidence-informed literacy program for children residing in out-of-home care in Ontario. Child Welfare, 92(5), 137–149.
Brännström, L., Vinnerljung, B., Forsman, H., & Almquist, Y. B. (2017). Children placed in out-of- home care as midlife adults: Are they still disadvantaged or have they caught up with their peers? Child Maltreatment, 22(3), 205–214.
Burnett, C. (2017). Acknowledging and interrogating multiplicities: Towards a generous approach in evaluations of early literacy innovation and intervention. Journal of Early Childhood Literacy, 17(4), 522–550.
Burnett, C., Daniels, K., & Bailey, C. (2014). The contribution of early years bookgifting programmes to literacy attainment: A literature review. London: Booktrust.
Durbeej, N., & Hellner, C. (2017). Improving school performance among Swedish foster children: A quasi-experimental study exploring outcomes of the Skolfam model. Children and Youth Services Review, 82, 466–476.
Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34(9), 917–928.
Dymoke, S., & Griffiths, R. (2010). The letterbox club: The impact on looked-after children and their carers of a national project aimed at raising achievements in literacy for children aged 7 to 11 in foster care. Journal of Research in Special Educational Needs, 10(1), 52–60.
Elwér, Å., Fridolfsson, I., Samuelsson, S., & Wiklund, C. (2016). LäSt: Test i läsförståelse, läsning och stavning för åk 1–6 [LäSt: Test in reading comprehension, reading and spelling for grade 1-6].
Stockholm: Hogrefe.
Evans, R., Brown, R., Rees, G., & Smith, P. (2017). Systematic review of educational interventions for looked-after children and young people: Recommendations for intervention development and evaluation. British Educational Research Journal, 43(1), 68–94.
Ferrer-Wreder, L., Stattin, H., Lorente, C. C., Tubman, J. G., & Adamson, L. (2004). Successful prevention and youth development programs: Across borders. New York: Kluwer/Plenum.
Flynn, R. J., Marquis, R. A., Paquet, M.-P., Peeke, L. M., & Aubry, T. D. (2012). Effects of individual direct-instruction tutoring on foster children’s academic skills: A randomized trial. Children and Youth Services Review, 34(6), 1183–1189.
Flynn, R. J., Tessier, N. G., & Coulombe, D. (2013). Placement, protective and risk factors in the educational success of young people in care: Cross-sectional and longitudinal analyses.
European Journal of Social Work, 16(1), 70–87.
Forsman, H. (2017). Foster carers’ experiences of a paired reading literacy intervention with looked-after children. Child & Family Social Work, 22(1), 409–418.
Forsman, H., Brännström, L., Vinnerljung, B., & Hjern, A. (2016). Does poor school performance cause later psychosocial problems among children in foster care? Evidence from national longitudinal registry data. Child Abuse & Neglect, 57, 61–71.
Forsman, H., & Vinnerljung, B. (2012). Interventions aiming to improve school achievements of children in out-of-home care: A scoping review. Children and Youth Services Review, 34(6), 1084–1091.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327.
Griffiths, R. (2012). The letterbox club: An account of a postal club to raise the achievement of children aged 7 to 13 in foster care. Children and Youth Services Review, 34(6), 1101–1106.
Griffiths, R., & Comber, C. (2011). The letterbox club in Wales: Evaluation report 2009–2011. Leicester:
University of Leicester, School of Education.
Griffiths, R., Comber, C., & Dymoke, S. (2010). The letterbox club 2007–2009: Final evaluation report.
Leicester: University of Leicester School of Education.
Hancock, A., & Leslie, M. (2014). Letterbox club Scotland. Edinburgh: The University of Edinburgh.
Harper, J., & Schmidt, F. (2016). Effectiveness of a group-based academic tutoring program for children in foster case: A randomized controlled trial. Children and Youth Services Review, 67, 238–246.
Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87.
Hickey, A. (2018). Improving academic outcomes for children in foster care through tutoring or working memory training: Three randomized trials. Ottawa: University of Ottawa.
Jackson, S., & Martin, P. Y. (1998). Surviving the care system: Education and resilience. Journal of Adolescence, 21(5), 569–583.
Jakobsen, M., & Andersen, S. C. (2013). Coproduction and equity in public service delivery. Public Administration Review, 73(5), 704–713.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousands Oaks, CA: Sage.
Mooney, J., Winter, K., & Connolly, P. (2016). Effects of a book gifting programme on literacy outcomes for foster children: A randomised controlled trial evaluation of the letterbox club in northern Ireland. Children and Youth Services Review, 65, 1–8.
Morris, S. B. (2008). Estimating effect sizes from pretest–posttest-control group designs.
Organizational Research Methods, 11(2), 364–386.
Neale, M. D. (1997). Neale analysis of reading ability—Revised. Windsor: NFER-Nelson.
Osborne, C., Alfano, J., & Winn, T. (2010). Paired reading as a literacy intervention for foster children. Adoption & Fostering, 34(4), 17–26.
Roberts, J., Winter, K., & Connolly, P. (2017). The letterbox club book gifting intervention: Findings from a qualitative evaluation accompanying a randomised controlled trial. Children and Youth Services Review, 73, 467–473.
Robinson, O. C. (2014). Sampling in interview-based qualitative research: A theoretical and prac- tical guide. Qualitative Research in Psychology, 11(1), 25–41.
Sebba, J., Berridge, D., Luke, N., Fletcher, J., Bell, K., Strand, S., . . . O’Higgins, A. (2015). The educational progress of looked after children in England: Linking care and educational data.
Oxford & Bristol: University of Oxford & University of Bristol.
Slavin, R. E., Lake, C., Chambers, B., Cheung, A., & Davis, S. (2009). Effective reading programs for the elementary grades: A best-evidence synthesis. Review of Educational Research, 79(4), 1391– 1466.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data.
American Journal of Evaluation, 27(2), 237–246.
Tideman, E., Vinnerljung, B., Hintze, K., & Aldenius, A. (2011). Improving foster children’s school achievements: Promising results from a Swedish intensive study. Adoption & Fostering, 35(1), 44–56.
Tordön, R., Vinnerljung, B., & Axelsson, U. (2014). Improving foster children’s school performance:
A replication of the Helsingborg study. Adoption & Fostering, 38(1), 37–48.
Trout, A. L., Hagaman, J., Casey, K., Reid, R., & Epstein, M. H. (2008). The academic status of children and youth in out-of-home care: A review of the literature. Children and Youth Services Review, 30 (9), 979–994.
Vinnerljung, B., & Hjern, A. (2011). Cognitive, educational and self-support outcomes of long-term foster care versus adoption. A Swedish national cohort study. Children and Youth Services Review, 33(10), 1902–1910.
Vinnerljung, B., Tideman, E., Sallnäs, M., & Forsman, H. (2014). Paired reading for foster children:
Results from a Swedish replication of an English literacy intervention. Adoption & Fostering, 38 (4), 361–373.
Winter, K., Connolly, P., Bell, I., & Ferguson, J. (2011). Evaluation of the effectiveness of the letterbox club in improving educational outcomes among children aged 7–11 years in foster care in Northern Ireland. Belfast: Centre for Effective Education, Queen’s University Belfast.
Wolfendale, S., & Bryans, T. (2004). Evaluation of the looking after project in Kent for children in public care. London: University of East London.
StudyCountryStudydesignStandardisedtest instrumentYearNAgePre(SD)Post(SD)Results (SD)ES Griffithsetal.,2010;Griffiths, 2012EnglandPre/postwithoutcomparison groupNeale(readingaccuracy) Mean:100(SD15)20073167–996.4b 100.75b +4.35 9–1196.4b 98.93b +2.53 20084497–993.7b 98.1b +4.4 9–1194.3b97.8b+3.5 20103811–1390.6b93.6b+3.0 Griffiths&Comber,2011WalesPre/postwithoutcomparison groupNeale(readingaccuracy) Mean:100(SD15)2009118a7–989.9b94.3b+4.4 9–1191.8b95.3b+3.5 201145a7–998.7b101.2b+2.5 38a 9–1191.8b 95.0b +3.2 Winteretal.,2011Northern IrelandPre/postwithoutcomparison groupNeale(readingaccuracy) Mean:100(SD15)2009–20102687–1189.5(13.9)93.1(15.6)+3.60.26c Mooneyetal.,2016Northern IrelandRCTNeale(readingaccuracy) Mean:100(SD15)2013Interv:56 (51)7–1090.0(14.7)92.15 (15.14)+2.150.25c Control:6092.3(11.3)91.15 (14.26)−1.15 ThisarticleSwedenPre/postwithoutcomparison groupLäSt(readingaccuracy) Readingagein months
201672(65)8–11115.25 (15.06)119.55 (18.71)+4.31 (10.75)0.29 72(71)116.62 (16.62)117.30 (16.57)+0.68 (7.01)0.04 72(65)114.74 (14.30)117.25 (15.99)+2.51 (7.62)0.18 Notes:a Unclearnumber. bFullsamplemeanestimatedbasedoninformationonsubgroupmeansinthepublishedarticle/report. c Calculatedbasedoninformationonpre/postmeansandstandarddeviationsinthepublishedarticle/report.
AppendixA.CompilationofevaluationsoftheLetterboxClub
