https://doi.org/10.1007/s10797-019-09538-9
Increasing the take-up of the housing allowance among Swedish pensioners: a field experiment
Per Engström1· Eskil Forsell2· Johannes Hagen3,4 · Arnaldur Stefánsson5
Published online: 22 April 2019
© The Author(s) 2019
Abstract
Using a randomized field experiment in the Swedish pension system, we investi- gate whether receiving an information letter affects the take-up rate of the housing allowance for pensioners. We also investigate whether the framing of the information letter affects take-up. The results show that simple information letters had a significant effect on the application rate and subsequent take-up rate: The baseline application rate in the targeted control population was only 1.4%, while the corresponding rates in the different treatment groups were between 9.9 and 12.1%. However, while the applications in the control group were accepted in almost 3 out of 4 cases, up to 50% of the applications in the treatment group were declined. The lower conditional acceptance rate in the treatment group seems to be largely driven by wealth, which the Pensions Agency cannot observe prior to submission. Information campaigns aimed at increasing benefit take-up therefore need careful design in situations with imperfect targeting.
Keywords Benefit take-up· RCT · Nudge
B
Johannes Hagen johannes.hagen@ju.se Per Engströmper.engstrom@nek.uu.se Eskil Forsell
eskil.forsell@phdstudent.hhs.se Arnaldur Stefánsson
arnaldur.stefansson@nek.uu.se
1 Department of Economics and UCFS, Uppsala University, Uppsala, Sweden 2 Department of Economics, Stockholm School of Economics, Stockholm, Sweden 3 Jönköping International Business School, Jönköping University, Jönköping, Sweden 4 Department of Statistics and UCFS, Uppsala University, Uppsala, Sweden 5 Department of Economics, UCFS and UCLS, Uppsala University, Uppsala, Sweden
JEL Classification H24· H55 · C93 · D90
1 Introduction
A common feature of various means-tested social benefit programs is that the targeted individuals fail to take up their benefits (Currie2006). Empirical estimates typically indicate low take-up (see, e.g., Blank and Ruggles1996; U.S. Department of Health
& Human Services2015; Department for Work and Pensions2008; Riphahn2001;
Gustafsson2002).1Incomplete take-up reduces the effectiveness of social programs and limits the ability of the government to reduce poverty. It is therefore of central policy importance to understand how take-up can be affected. The various factors that might explain why individuals do not apply for the benefits they are entitled to have generally been categorized into lack of information, information costs, transaction costs, stigma and complexity/non-transparency (Currie2006; Remler et al.2001).
However, little is known about the relative importance of these factors in different parts of the population.
Using a randomized field experiment in the Swedish pension system, we investigate whether information letters with different framing affect the take-up rate of housing allowance for pensioners. Specifically, from a sample of about 95,000 single pensioners (10% of all single individuals in the age group 65+ in Sweden) with sufficiently low income to potentially qualify for the housing allowance, about 10,000 individuals were randomly selected to receive any of four different information letters.2One of the letters contained basic information about the housing allowance, which was reproduced in the other three letters with a slight addition that relates to the hypothesis we wish to test. One of the framed letters addressed the role of eligibility myths and stigma while the remaining two letters aimed at clarifying the eligibility criteria. An application form was enclosed in all mailings. Therefore, the letters simultaneously lowered both information costs and transaction costs related to applying for the benefit. Individuals who received no information mailing make up the control group.
The housing allowance is a means-tested benefit for pensioners with low income and wealth that pays maximum SEK 5000 (SEK 1≈ USD 0.11) per month with an average payment of SEK 2400. Previous studies indicate substantial under-utilization of this benefit and the foregone benefits are large. According to the Swedish National Audit Office, half of the non-claiming individuals are estimated to forego SEK 900 per month, an amount that individuals with a monthly pension income of SEK 10,000 typically could qualify for (Riksrevisionen 2013). While only 15% of the current pensioner population receive housing supplement, as many as 30–35% are likely to be eligible.
Our study is closely related to three previous studies in the empirical literature on benefit take-up. From a methodological perspective, our study is inspired by Bhar- gava and Manoli (2015) who conduct an extensive information intervention aiming to
1 An exception is Mood (2006) who finds relatively high take-up rates for means-tested benefits in Australia.
2 The Swedish Pensions Agency, which administers the housing allowance, cannot perfectly define eligi- bility because of data limitations prior to collecting the submissions. The lack of data relevant for eligibility concerns individual wealth, cohabitation status, occupational pension income and housing costs.
increase the take-up of the US EITC. We distinguish our project from Bhargava and Manoli (2015) by studying take-up among a group of non-working individuals in great economic need with low take-up, namely poor elderly. From a policy perspective, it is important to know whether the effectiveness of information letters varies across individual characteristics like work status and age. Furthermore, as noted by Korpi and Palme (1998), the pensioners are of particular interest since public transfers play a larger role in their income than they do for the working population. The second study is by Matikka and Paukkeri (2016) who analyze the effect on the take-up of the min- imum guarantee for pensioners in Finland by sending information on eligibility and an application form to eligible individuals. Our study is similar to the Finnish study in the sense that they also focus on poor elderly. However, in contrast to Matikka and Paukkeri (2016) where the letter recipient status was based on pre-determined pension income status, we randomly assign individuals to different information letters. Fur- thermore, while Matikka and Paukkeri (2016) study the introduction of a new benefit, we study a benefit that has long been in place. As a result, we do not have to account for potential effects of media coverage and time on take-up.
Our study is also inspired by Guthmuller et al. (2014) who conduct a randomized information experiment to increase the take-up rate of a health-insurance voucher program for the poorest in France. An important distinction between our study and Guthmuller et al. (2014) is that they evaluate the effects of two treatments (varying the subsidy amount and invitations to information meetings) against a basic information letter. In our experiment, we have two control groups: one that received no letter and one that received a basic information letter. This allows us to evaluate both the effects of receiving any information at all as well as the specific contents of such a letter.
The results show that simple information letters had a significant effect on the application rate and subsequent take-up. The baseline application rate in the targeted control population was only 1.4%, while the corresponding rates in the different treat- ment groups were between 9.9 and 12.1%. The only letter that had a significantly different effect from the basic information letter was the letter that addressed com- mon misconceptions about the benefit’s eligibility criteria. This group had the highest application rate of 12.1% compared to the basic letter of 9.9%. The effects are partic- ularly large for old pensioners (above the age of 80). We also find that the economic impact of the treatment was substantial. Using an IV analysis, where application is instrumented by treatment, we estimate an income effect of approximately SEK 750.
This means that the applicants, induced by the treatment (i.e., the compliers), increased their income by roughly 10%.
However, the information treatments also had negative effects in the form of sub- stantially lower conditional acceptance rates. The applications in the control group were accepted in almost 3 out of 4 cases, while up to 50% of the applications in the treatment group were declined. These findings suggest that there is strong positive self-selection among the applicants in the treatment group and that the information letters pushed some individuals to apply even though they were not eligible. The lower conditional acceptance rate in the treatment group seems to be largely driven by wealth, which the Pensions Agency cannot observe prior to submission. However, all the other unobservables—occupational pension, cohabitation status and housing expenses—also seem to contribute to lower eligibility among the treated applicants.
Declined applications are costly both to the individuals in terms of false hope and the effort of applying, but also to taxpayers in general through the administrative costs of processing applications for ineligible applicants. The substantial decrease in conditional acceptance rates among the treated is somewhat surprising since all letters aimed at simplifying the pensioners’ own evaluations of eligibility. We can only speculate about the reasons for this decline in self-selection among the treated. One reason could be the well known fact that Swedes (and Scandinavians in general) report a very high level of trust toward public institutions [see, e.g., Rothstein and Uslaner (2005)]. It is possible that a high level of trust may misfire, so that an individual who is informed by an institution that she may be eligible simply disregards her own knowledge and presumes that the authorities know best. If this, arguably speculative, story is correct, public institutions with high levels of trust need to be extra careful when designing information campaigns aimed at increasing benefit take-up in situations with imprecise targeting. It could also be the case that the information treatments simply lowered the costs of submitting an application to the point where many applied even though they thought that they were not likely eligible.3
The results also indicate that far from all letter recipients applied (the non- application rate is close to 90%). Many probably refrained from applying because they knew that they were not eligible due to any of the aforementioned unobservables.
Even though this group did not waste any effort on applying, the misdirected letters may have a cost in terms of bad publicity for the Pensions Agency. Of course, the high non-application rate also implies large direct costs of sending letters to non-eligible people. This adds to the necessity of more accurate assessment of the eligibility of indi- viduals before scaling up these interventions to the broader population of (potentially eligible) non-claimants.
Our finding that information letters increase take-up is consistent with studies that examine the impact on benefit take-up of receiving an information letter compared to receiving no letter at all. Bhargava and Manoli (2015) find large effects on take-up from simple reminders about the EITC. Matikka and Paukkeri (2016) also find that the targeted information letters about a new guarantee pension system (introduced in 2011) significantly increased take-up and prompted pensioners to apply sooner.4To our knowledge, the only related study with similar difficulties in reaching the eligible population is Guthmuller et al. (2014). While the submission rate in their experiment was slightly higher than in our case (16.7%), the conditional acceptance rate was very similar (55.2%).
Regarding the different explanations for why some individuals do not apply for the benefits they are entitled to, we conclude, based on the modest differences between
3 We cannot evaluate the role of transaction costs since all letters had an attached application form. However, survey evidence suggests that they are not perceived as the main reason for not applying (Försäkringskassan 2007).
4 Other examples of recent information studies that find positive effects from information letters concerning social benefits include Mastrobuoni (2011) and Liebman and Luttmer (2015). Some studies, on the other hand, find no or small effects from benefit information. Bettinger et al. (2012) find no effect on take-up of financial aid to college students from information alone, albeit information combined with assistance did have a positive effect. Zantomio (2015) also finds that a number of measures intended to encourage take-up, including simplified application procedures, application assistance and targeted mailings, had no impact on take-up of the pension credit in the UK.
the different letter framings, that the primary explanation for the low take-up rate among poor elderly is low awareness.5An interesting exception, however, is that the conditional acceptance rate for the Rule of Thumb Letter was about 20% higher than that of the Base Letter. Demonstrating the eligibility criteria using simple examples might therefore be the most effective tool to get eligible individuals to apply. This is in line with the finding in Bhargava and Manoli (2015) that simplified information about the benefit had larger effects.
The remainder of the paper is organized as follows. Section2provides a background of the Swedish pension system and the housing allowance for pensioners. Section3 describes the experimental setup, including the sample selection and the contents of the information letters. Section4reports the results from the experiment, and Sect.5 concludes.
2 Housing allowance and take-up
Sweden’s pension system has two main pillars: a universal public pension system and an occupational pension system for workers whose employer is tied to some occupational pension plan. The public pension is the most important source of pen- sion income for most people. The relative importance of the public pension typically decreases with the income level of the individual because of a progressive feature of the public pension (Hagen2017).6
The public pension system has in itself three tiers. The first two tiers are income- related and are referred to as the income pension and the premium pension.7The third tier is called the minimum guarantee and is paid out to pensioners above 65 years of age who have low or no earnings-related pension. As shown in Fig.3, the reduction is taken in two steps: For low incomes, the minimum guarantee is decreased by the full amount of the earnings-related pension; for higher incomes, the guaranteed pension is decreased by only 48%. Thus, a single pensioner with a monthly earnings-related pension of SEK 11,343 or more received no guaranteed pension in 2016, the year of the experiment.
Another component of basic security in the Swedish pension system is called the housing allowance. The housing allowance is means-tested against income (pension, labor and capital income) and wealth. It is also a function of housing expenditures. If the individual is married or has a cohabiting partner, eligibility depends on the income and wealth situation of the household. The maximum and average benefit level per month is SEK 5000 and SEK 2400, respectively.
5 Bhargava and Manoli (2015) also emphasize low awareness as a key reason for low take-up. Moreover, Guthmuller et al. (2014) find that a voucher only marginally increased take-up while information meeting invitations actually discouraged it slightly compared to receiving a basic information letter.
6 Specifically, the public pension system contains a ceiling on the income qualifying for pension rights.
The ceiling is currently at 7.5 income base amounts (≈ USD 52,000). For 2019, this means that no pension rights are earned for the monthly wage portion that exceeds SEK 40,250 (≈ USD 4 340), which is far above what the typical housing allowance recipient would earn.
7 See Hagen (2017) for a more detailed description.
Take-up is more common among women than men (18 and 7% of the pensioner population, respectively). This reflects the fact that women have lower pensions on average. Around 80% of those who are entitled to the minimum guarantee are women.
Estimates of the share of non-applicants in the population of potentially eligible pensioners range between 25 and 35% (Riksrevisionen2013; Pensionsmyndigheten 2019). A survey sent out by Statistics Sweden to potentially eligible non-applicants in 2007 offers some explanations for this low take-up (Försäkringskassan 2007).
It showed that around half of the non-applicants did not know about the housing allowance. This suggests that an information letter, regardless of framing, is likely to have a large impact on the submission rate. The survey also showed that one-third of those who knew about the allowance falsely believed that they were automatically disqualified because they were house owners (as opposed to living in an apartment).
A significant share also believed they were disqualified because they had too high income and/or wealth. Thus, the framing of an information letter, particularly with respect to the eligibility criteria, may affect submission (and acceptance) rates. The low take-up could also be due to social stigma associated with receiving means-tested benefits, which obviously is difficult to elicit from surveys.
The Pensions Agency is responsible for administrating the housing allowance. This means that they collect the incoming applications and decide whether the allowance should be granted or not. The application form can either be submitted by mail or online and a decision is typically made within 1–2 months (the average waiting time in the experiment is 34 days).
There are two application forms: one for single households and one for two-person households. Applicants are required to provide information on things that affect eli- gibility (i.e., household members, type of housing, housing costs, assets, debts and income). Before making a decision, the Pensions Agency verifies the reported informa- tion by requesting information about the applicant(s) from the relevant banks, mortgage lenders and the Tax Agency. The information is always double-checked against the banks if the reported wealth exceeds SEK 100,000. If reported wealth is below this level, the responsible desk officer at the Pensions Agency has the mandate to make a decision without verifying the information. Recipients should notify the Pensions Agency of any changes in the financial, living or family circumstances that might affect the housing allowance.
3 Experimental design
3.1 SampleThe sample for the field experiment was drawn in May 2016, three months prior to the mailings. The selection criteria for the original sample have been chosen so as to capture elderly who are likely to be eligible for the housing allowance, but for some reason never applied. The individuals in the original sample satisfy the following conditions as of May 2016. First, the pensioners are at least 65 years of age (the housing allowance eligibility age). Second, the pensioners are registered as unmarried (single, divorced or widowed). This restriction is motivated by the fact that more than 90%
May 2016 Sampling
August 5, 2016 Pensions Agency’s reference date
for death Week 35 Week 36 Week 37 Week 38 Week 39
May 1, 2017 Pensions Agency’s
reference date for pension Pensions Agency sends out letters, Aug 29 – Oct 2
Fig. 1 Timeline. Note: The sample for the field experiment was drawn in May 2016. The Pensions Agency did not send letters to sampled subjects who were deceased before August 5, 2016. Therefore, subjects that deceased before August 5, 2016, were dropped from the sample. Three months after the sampling, during week 35 (August 29–September 4) to week 39 (September 26–October 2) 2016, the four different information letters were sent out. Finally, subjects with a missing value for pension are dropped. Most of these deceased before end of April 2017, while 59 subjects (of which 7 receive a letter) have missing value for pension for an unknown reason
of the estimated eligible non-receiving population are made up of single households (Riksrevisionen2013). Third, the pensioners have started to withdraw 100% of their public pension, which is also an eligibility criterion, which amounts to less than SEK 10,833 (average public pension in 2016 was SEK 12,300). This implies that virtually everyone in the sample is eligible for the minimum guarantee.8Finally, the pensioners did not apply for the housing allowance in the two preceding years.
It is important to note that these selection criteria do not guarantee eligibility. There are four main limitations to the data that prevent the Pensions Agency from targeting the truly eligible. First, there is no information in the data on wealth. The size of the housing allowance decreases rather quickly with increasing levels of wealth.9Second, there is no information on pension income from other sources than the public pension system, such as occupational pension. Third, the selection criteria made sure that no individuals had a (living) spouse but cohabitation status is not a priori known to the Pensions Agency. As a result, some individuals in the sample may be disqualified because their cohabiting partner has high income and/or wealth. A fourth limitation is the lack of information on housing expenditures. Thus, it is expected that some (ineligible) letter recipients choose not to apply and that some fraction of submissions are rejected. We get back to the issue of imprecise targeting in more detail in Sect.4.3.
The resulting sample consisted of 96,481 individuals, which corresponds to about 10% of the Swedish population of single individuals 65 years old or older. We then dropped 969 individuals who were deceased by August 5 from the sample, i.e., roughly a month before the first letter was sent out. From the remaining sample of 95,512 individuals, henceforth referred to as the original sample, 10,013 individuals were randomly selected to receive any of the four different information letters during the fall of 2016 (week 35–week 39).10The remaining subjects received no letter and hence make up the control group. Figure1provides a timeline of the events in the experiment.
8 In order to claim the full minimum guarantee, the individual must have lived in Sweden for at least 40 years. The minimum guarantee is reduced by 1/40 for every year the individual has lived elsewhere.
9 As seen from one of the letters in Fig.7, pensioners with a monthly income of SEK 10,000 are ineligible for the housing allowance if they have SEK 500,000 in wealth. With a monthly income of SEK 12,000, only SEK 300,000 in wealth is required be disqualified.
10 The Pensions Agency budget could afford approximately 10,000 letters. This allows us to detect a minimal effect of four percentage points with a power of 80%, assuming that we compare two groups with 2500 subjects in each, and a mean outcome of 50%.
We make two restrictions on the original sample to arrive at our analysis sample.
First, 3822 individuals were dropped because information on their public pension was missing. Of these, 3763 had deceased before May 2017, when the Pensions Agency collected information on pension, while 59 individuals had missing information for pension for an unknown reason. Second, 933 subjects were dropped because they submitted an application before the experiment started.11
The final sample, called the main sample, consists of 90,757 individuals. Of these, 9534 received a letter, henceforth referred to as the treatment sample, and the remaining 81,223 are henceforth referred to as the control sample. We include applications that arrived at the Pensions Agency before January 1, 2017, i.e., 4 months after the first set of letters were sent out. A decision is generally taken within 1–2 months after the submission.
3.2 Interventions
During the fall of 2016 (week 35–week 39), the Swedish Pensions Agency sent out letters to the treatment sample.
Four types of letters were sent out, approximately to 2500 subjects each and 2000 letters each week, i.e., about 500 of each treatment each week. The information in the most simple letter, which we refer to as the Base Letter, was reproduced in the other three letters with a minor addition that relates to the hypothesis we wish to test. An English version of each letter is provided in the Appendix. The two application forms were enclosed in each mailing.
The Base Letter informed that many pensioners who might be eligible for the housing allowance had not yet applied for it and that the recipient might be one of those. Furthermore, there was information on an income level for eligibility, where to apply, how the allowance is paid out and what it is. The letter also included a link to an online tool that pensioners can use to get a preliminary check of their eligibility status.12 The title of the Base Letter was “Have you heard about the housing allowance?”.
The Myths Letter had, in addition to the information provided in the Base Letter, information aiming to correct for four widespread myths about the housing allowance:
The letter explained that the allowance does not depend on the type of housing (tenancy, condominium or own property) nor the value of the residence; that those with low income can have a certain wealth and still be eligible for the allowance; that those who share residence with others may be eligible; and that certain life changes that pertain to housing, income or marital status, might affect eligibility. The purpose of the Myths Letter was also to de-stigmatize the take-up of means-tested benefits. The title of this letter was: “280,000 pensioners receive housing allowance today—are you eligible, too?”
The Rule of Thumb Letter showed, in addition to the information provided in the Base Letter, three examples of the possible level of allowance given three different
11 This is expected because the sample was drawn three months prior to the experiment. The ex post sample restrictions, i.e., missing pension information and early submission, are balanced across the treatment and control group, as well as across the different treatments (letters).
12 Unfortunately, our data do not allow us to see which pensioners made use of this tool.
Table 1 Descriptive statistics by treatment status
Original Main
Full Control Treated Full Control Treated
All Base Myths RoT Table
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
Female .730 .729 .735 .729 .728 .735 .725 .747 .748 .720
(.444) (.444) (.441) (.444) (.445) (.441) (.447) (.435) (.434) (.449)
Age 76.3 76.3 76.2 75.9 75.9 75.8 75.9 75.9 75.7 75.8
(8.9) (8.9) (9.0) (8.7) (8.7) (8.8) (8.7) (8.8) (8.7) (8.8)
Pension 8531 8526 8574 8532 8527 8576 8556 8566 8,610 8,572
(2278) (2279) (2269) (2280) (2281) (2270) (2268) (2219) (2314) (2279)
N 95,512 85,499 10,013 90,758 81,224 9534 2378 2348 2421 2387
This table reports means and standard deviations (in parentheses) of subjects by treatment status. Columns 1–3, Original, show the originally sampled pensioners who had not passed away before August 5, 2016.
Column 1 shows the full Original sample, column 2, Control, is the subset of Full that does not receive any letter (the control group), and column 3, Treated, is the subset that receives letters (the treatment group).
Columns 4–10, Main, are the sample used in the analysis and consist of the subjects in the Original sample that (1) had not submitted an application before the experiment started (the first letters were sent out) and (2) did not have a missing value for public pension. Column 4 shows the full Main sample, column 5, Control, is the subset in the Main sample that does not receive any letter (the control group), and column 6, Treated is the subset that receives letters (the treatment group). Columns 7–10 show the treatment groups in the Main sample by letters; the Base, the Myths, the Rule of Thumb (RoT ), and the Table Letter, respectively.
Pension is monthly public pension amount in Swedish Krona (SEK)
combinations of income and wealth. The purpose of this letter was to make the eli- gibility criteria with respect to income and wealth more transparent. Previous studies have shown that a significant share of the eligible non-receiving population have false beliefs about how much income and wealth you may have and still qualify (Riksrevi- sionen2013). The rules of thumb might help people overcome the rather complicated eligibility calculation.
Finally, the Table Letter showed nine potential combinations of income and wealth and the resulting housing allowance. Thus, this letter was similar in spirit to the Rule of Thumb Letter except that it included more detailed information on wealth and income criteria (and presented in table form).
3.3 Data and descriptive statistics
The data for the analysis were provided by the Swedish Pensions Agency. For all individuals in the main sample (see Sect.3.1), we have information on age, gender, marital status, and public pension income. There is additional information for those who actually applied for the housing allowance. This information is obviously related to the eligibility criteria of the housing allowance and include second and third pillar pension income, income from capital, labor and self-employment, cohabitation status and housing costs.
Table1reports summary statistics by treatment status. As expected, the majority in the sample are women (around 73%). The average age is 76, but the spread is quite large
(10% are 89 years or older). The average monthly public pension is approximately SEK 8500.
We run a number of balancing tests, implemented through a series of regressions, to ensure that the treatment samples are similar across the observed variables. The first hypothesis is that receiving a letter does not have predictive power for the level of covariates. Therefore, for each covariate xjand using the main sample, we run the following regression
xj i = α + μjLetteri+ i, (1) and the null hypothesis is that for each j , H0: μj = 0. The second hypothesis is that all letters have the same predictive power for the level of covariates. Therefore, for each covariate xj and using the treated sample, we run the following regression
xj i = α + μj 1Mythsi + μj 2RoTi+ μj 3Tablei+ i. (2) and the null hypothesis is that for each j and k, H0: μj k = 0.
Columns 1, 3 and 5 in Table 2 show the balancing of receiving a letter, that is, estimates of regression (1). While there are no significant differences between the control and the treatment group with respect to gender and age, the treatment group individuals receive a significantly higher public pension than the individuals in the control group. However, the difference, SEK 49.5, is economically insignificant, or less than 0.6% of the average monthly pension in the control group.13We therefore conclude that these two groups are very similar and that randomization into control and treatment has been successful. When considering the randomization of letters separately, Eq. (2), the significant difference in pension disappears (see columns 2, 4 and 5 in Table2). Women are, however, more likely to receive the Myths and the Rule of Thumb Letter, but again, the differences are economically insignificant, with women being 75% of those receiving the Myths and Rule of Thumb Letters as compared to 73% in the control sample. We therefore conclude that the randomization has in general been successful.
4 Results
Figure2a shows the share of pensioners who submitted an application by treatment.
There is a clear difference between the control group, on the one hand, and the dif- ferent treatment groups, on the other hand. While only about 1% in the control group submitted an application during the experiment period, more than 10% of those who received a letter applied. The submission rate is highest among pensioners receiving the Myths Letter, but otherwise very similar.
Figure2b shows the share of accepted applications conditional on submission, also by treatment. A striking result is that the conditional acceptance rate is much lower in all treatment groups compared to the control group. Thus, the substantial increase in
13 The difference is no longer significant when we exclude pensioners with public pension income above the sampling threshold of SEK 10,833 (i.e., outliers). The reason that some have income above the sampling threshold is that the public pension is measured in May 2017, one year after the sampling (see Fig.1).
Table 2 Balancing tests
Female Age Public pension
(1) (2) (3) (4) (5) (6)
Received a letter 0.0062 − 0.069 49.5**
(0.00478) (0.0948) (24.59)
Myths 0.022* − 0.028 9.77
(0.0128) (0.255) (65.27)
RoT 0.023* − 0.19 53.9
(0.0127) (0.251) (66.14)
Table − 0.0052 − 0.13 16.2
(0.0130) (0.254) (65.88)
Constant 0.73*** 0.72*** 75.9*** 75.9*** 8526.5*** 8555.9***
(0.00156) (0.00916) (0.0305) (0.179) (8.003) (46.51)
R2 0.000 0.001 0.000 0.000 0.000 0.000
N 90,758 9534 90,758 9534 90,758 9534
Sample Main Treat. Main Treat. Main Treat.
F test
Joint significance 0.054 0.872 0.859
Myths = RoT 0.935 0.533 0.502
Myths = Table 0.037 0.703 0.922
RoT = Table 0.029 0.811 0.569
Balancing tests for randomization of letters. Columns 1, 3 and 5 show the balancing of receiving a letter (results from specification (1)). Columns 2, 4 and 6 show the balancing of letters separately (results from specification (2)) among subjects receiving a letter. Subjects receiving the Base letter are used as a base group, and compared to subjects receiving the Myths, the Rule of Thumb (RoT ) and the Table letter.
Heteroskedasticity robust standard errors in parentheses
***( p< 0.01), **(p < 0.05), *(p < 0.1)
application rate, induced by the treatments, has clearly come at a cost since the letters seem to have caused many ineligible pensioners to apply. We will get back to this issue in Sect.4.3, in which we analyze the probable causes for ineligibility.
4.1 Response to receiving a letter
In the first specification, we estimate the effect of receiving a letter in contrast to not receiving any letter,
yi = α + δLetteri+ εi (3)
where yi is indicator for either an application submission or an acceptance of an application for subject i . The indicator variable takes the value 100 if the condition holds and zero otherwise, where the variable for acceptance is coded as 0 if a subject does not submit an application. Letteriis a dummy indicating that a letter was received.
The results from estimating specification (3) are shown in Table3. In the first column,
0 2 4 6 8 10 12 14
Share (%)
Control Base Myths RoT Table Group
(a)The share of pensioners with submitted applications by treatment
0 10 20 30 40 50 60 70 80
Share (%)
Control Base Myths RoT Table Group
(b)The share of pensioners with accepted applications conditional on submission, by
treatment
Fig. 2 The share of pensioners with submitted and accepted applications by treatment. Note: a shows the share with submitted applications, and b the share with accepted applications. Each bar represents the share of either the Control group or a treatment group. The treatment groups are subjects who receive the Base, the Myths, the Rule of Thumb (RoT ) or the Table letter. The capped spikes show the 95% confidence interval of each share
Table 3 The effect of receiving a
letter on application submission Application submission Application accepted
(1) (2) (3)
Received a letter 9.28*** 4.35*** − 22.95***
(0.32) (0.23) (2.05)
Constant 1.39*** 1.02*** 73.29***
(0.04) (0.04) (1.32)
R2 0.035 0.012 0.056
N 90,758 90,758 2144
Sample Main Main Submitted
The effect of receiving a letter on application submission (column 1) and acceptance (columns 2 and 3) from specification (3). Application submission takes the value 100 if the subject submitted an application and zero otherwise, and Application accepted takes the value 100 if the subject submitted an application that was accepted and zero otherwise.
Heteroskedasticity robust standard errors in parentheses
***( p< 0.01), **(p < 0.05), *(p < 0.1)
submission is used as the outcome, while acceptance of an application is the outcome in the second and third columns.
Receiving a letter almost sevenfolds the probability of a pensioner submitting an application. The probability increases by 9.3 percentage points, from 1.4% (column 1).
Due to the increased submission rate, a letter fivefolds the probability of a pensioner receiving housing allowance. The probability increases by 4.4 percentage points, from 1.0% (column 2). The last column in Table3shows that the acceptance, conditional on submission, is significantly lower among subjects who received a letter than in the control group, 50% and 73%, respectively. That is, about half of the treated subjects who submitted an application for housing allowance were not eligible to receive an
Table 4 The effect of receiving a letter and submission on housing allowance
HA 100· HAP
(1) (2) (3) (4) (5)
Received a letter 69.0*** − 870.6*** − 644.6***
(4.88) (58.39) (66.29)
Application submission 743.2*** 10.20***
(43.11) (0.76)
Constant 24.2*** 1744.0*** 2379.5*** 13.9*** 0.20***
(0.95) (45.46) (44.91) (1.05) (0.02)
R2 0.005 0.092 0.062 0.367 0.251
N 90,758 2144 1338 90,758 90,757
Sample Main Submitted Accepted Main Main
Model OLS OLS OLS IV IV
The effect of receiving a letter and submission on monthly housing allowance (HA) in Swedish Krona.
Columns 1–3 show results from OLS specification (4). Columns 4 and 5 show results from IV estimation of specification (5). The final column also shows results from IV estimation of specification (5), but with housing allowance replaced by the fraction of housing allowance to public pension (P), scaled by 100.
Heteroskedasticity robust standard errors in parentheses
***( p< 0.01), **(p < 0.05), *(p < 0.1)
allowance. Point estimates are stable to the introduction of covariates (see Table7in Appendix A3).
We also investigate the effect of receiving a letter on the benefit amount. The specification we estimate is
HAi = α + δLetteri + εi, (4)
where HA is the monthly housing allowance amount in Swedish Krona (SEK).
The first column in Table4shows that the information intervention increased the average housing allowance by SEK 69. In line with our previous finding that individuals in the treatment group are less likely to have an accepted application, Table 4also shows that the treatment group applicants on average receive smaller amounts than the control group applicants. Conditional on acceptance (column 3), the control group receives on average SEK 2380, in contrast to the treatment group’s average of SEK 1735. The difference of SEK 645 corresponds to almost 30% of the average housing allowance among the control group recipients.
The rather small effect of receiving a letter on the benefit amount (SEK 69) reflects the fact that only one-tenth of the letter recipients chose to apply. Another interest- ing outcome is therefore the corresponding effect on those who were induced by the treatment to apply. To investigate this, we run an IV estimate of submission on hous- ing allowance and the ratio of housing allowance to pension, where submission is instrumented with treatment status. That is,
HAi = α + δH Aˆsi+ i, (5a)
where ˆsi is the predicted value of submission from the first stage,
si = αs+ δsLetteri + i. (5b) We also use the fraction of housing allowance to public pension as a dependent vari- able.
The results, displayed in columns 4 and 5 in Table4, show that those who were induced by the treatment to apply, on average received housing allowance of around SEK 743 (columns 4), or about 10% of the amount they had in public pension (column 5). Note that this is slightly lower than the average housing allowance of SEK 873 among treatment group applicants (column 2). The reason is that those who would have applied even without the treatment receive higher housing allowance than those who are induced to apply by the treatment. Point estimates are again stable to the introduction of covariates (see Table8in Appendix A3).
4.2 Response to each letter
We now turn to the specific responses to the different letters. To make the result tables more tractable, we drop the control group and use those who received the Base Letter as the reference group. That is, we separately estimate the effect of receiving the Myths, Rule of Thumb (RoT) and the Table letter, in contrast to receiving the Base Letter,
yi = α + δ1Mythsi+ δ2RoTi+ δ3Tablei+ εi. (6) The results from estimating equation (6) are shown in Table5.
Subjects respond strongest to the Myths Letter and weakest to the Base Letter.
The share of submitted applications among pensioners receiving the Myths Letter and the Base Letter is, respectively, 12.1% and 9.9%, and the difference is statistically significant. Furthermore, when compared to the Rule of Thumb and the Table Letters, the treatment effect of the Myths Letter is significantly different at the 5% and the 10% level, respectively (see results from F tests in column 1).
Turning to acceptance, column 2, we see again that the Myths Letter generates the largest response compared to the Base Letter. The average effect on the number of accepted applications is 32% larger among pensioners receiving the Myths Letter than among those receiving the Base Letter, 6.0% and 4.6%, respectively.
Finally, from the last column we see that the acceptance rate conditional on sub- mission is significantly higher among treated subjects receiving the Rule of Thumb Letter compared to the other letters. This suggests that demonstrating the eligibility criteria using simple examples may be an effective tool to get eligible individuals to apply. Point estimates are stable to the introduction of covariates (see Table9in Appendix A3).
We have also examined whether the letters had a significant effect on take-up within various subgroups, including gender, age and public pension income. Appendix A4 presents the estimated equations and the corresponding estimates. In short, the elderly (80+) and the poor (below median pension) were more likely to apply and submit an accepted application. We show that these effects are more likely driven by age
Table 5 The effect of receiving each letter on application submission
Application submission Application accepted
(1) (2) (3)
Myths 2.21** 1.46** 3.64
(0.91) (0.65) (4.40)
RoT 0.28 0.99 8.47*
(0.87) (0.63) (4.54)
Table 0.51 0.69 4.42
(0.88) (0.63) (4.54)
Constant 9.92*** 4.58*** 46.19***
(0.61) (0.43) (3.25)
R2 0.001 0.001 0.003
N 9534 9534 1017
Sample Treated Treated Treated and submitted
F test
Joint significance 0.075 0.142 0.319
Myths = RoT 0.034 0.487 0.266
Myths = Table 0.064 0.252 0.858
RoT = Table 0.794 0.649 0.367
Results from specification (6), showing the effect of receiving each letter, in comparison with the Base Letter, on application submission (column 1) and acceptance (columns 2 and 3). Application submission takes the value 100 if the subject submitted an application and zero otherwise, and Application accepted takes the value 100 if the subject submitted an application that was accepted and zero otherwise. Heteroskedasticity robust standard errors in parentheses. The last four rows show p values from F tests
***( p< 0.01), **(p < 0.05), *(p < 0.1)
than income. The probability of submitting and getting accepted was highest among females who received the Myths Letter, while among males, the Base Letter had the largest effect on the probability of submission.
4.3 Imperfect targeting
As noted previously, the substantial increase in the application rate, induced by the treatment, clearly came at the cost of a drop in the conditional acceptance rate. In the control group, only one out of four applications was declined, while as much as half of the applications in the treatment group were declined. To understand what disqualifies the treatment group applicants, we compare them to the control group applicants in terms of the covariates that are most crucial for eligibility.
Table6shows five sets of results from regressing a treatment dummy (i.e., receiving a letter) on the amount of public pension, indicator for having positive occupational pension income, indicator for having positive wealth, indicator for joint submission with a cohabiting partner and annual housing expenditures, respectively.14This com-
14 We use dummies for having positive occupational pension and wealth since only 40% of the applicants have positive occupational income, and around 50% have positive wealth.
Table 6 Differences between treatment and control group conditional on submission Public
pension
Positive occupational pension
Positive wealth
Joint submission with partner
Housing expenditure
(1) (2) (3) (4) (5)
Received a letter − 16.35 5.87*** 15.55*** 5.98*** − 5180.6***
(94.43) (2.12) (2.14) (1.41) (849.11)
Constant 8411.21*** 37.89*** 43.74*** 8.96*** 59,209.9***
(64.89) (1.45) (1.48) (0.85) (566.53)
R2 0.00 0.00 0.02 0.01 0.02
N 2144 2144 2144 2144 2036
Sample Submitted Submitted Submitted Submitted Submitted
Results from specification (3) with outcome replaced by the amount of public pension (Swedish Krona), indicator for having positive occupational pension income, indicator for having positive wealth, indica- tor for joint submission with a cohabiting partner and annual housing expenditures (in Swedish Krona), respectively. The indicator variables take the value 100 if the condition holds and zero otherwise. Het- eroskedasticity robust standard errors in parentheses
***( p< 0.01), **(p < 0.05), *(p < 0.1)
parison is only possible to make for the applicants since individual level information on occupational pension, wealth and cohabitation status is collected by the Pensions Agency upon application.
We first conclude that the higher rejection rate in the treatment group cannot be attributed to differences in public pension. The coefficient for belonging to the treat- ment group in the first column is insignificant and reflects a very small amount. The second column shows that treatment group applicants are on average more likely to have positive occupational pension income, 44% compared to 38% in the control group. The differences with respect to wealth are even more pronounced, however;
the share of applicants with positive wealth is 16 percentage points higher in the treat- ment group than in the control group. Thus, the lower conditional acceptance rate in the treatment group seems to be largely driven by wealth, a conclusion also reached by the Pensions Agency (communicated through email correspondence).
A third explanation for the low conditional acceptance rate in the treatment group is that a substantially larger share of treated applicants reported that they had a cohab- iting partner. As seen in the second last column of Table6, 9% of the control group applicants used the application form for cohabiting couples. In the treatment group, the corresponding share is 15%. According to the Pensions Agency, the majority of applicants that jointly submitted an application with a cohabiting partner would be eligible on their own, so it should be the income and/or wealth situation of the partner that disqualify them for housing allowance. Furthermore, the conditional acceptance rates among cohabiting applicants in the treatment and control group are 15% and 42%, respectively. Acceptance rates are therefore generally lower among cohabiting applicants, but the difference is larger in the treatment group.
A final explanation for the higher rejection rate in the treatment group could be dif- ferences in housing expenditures. In fact, the last column in Table6shows that annual
housing expenditures are on average 9% lower among treatment group applicants than among control group applicants (10% when controlling for cohabiting partner).
We want to emphasize two implications of the results that the treatment group applicants have higher rejection rates and lower benefit levels. First, many letters were sent to ineligible individuals. Basing selection only on public pension income and marital status clearly misses important financial information that might disqualify individuals for the housing allowance, such as wealth and actual cohabitation status.
Second, many individuals applied despite being ineligible. This could either be due to difficulties in calculating eligibility status and taking the letters as a signal of being eligible (complex benefit criteria) or that the letters decreased the cost of applying, or a combination of both.
5 Discussions and conclusion
This paper reports the results from a randomized field experiment investigating whether receiving an information letter affects the submission and acceptance rate of a means- tested benefit in Sweden, the housing allowance for pensioners. We also investigate whether the framing of the information letter affects take-up. The experiment was carried out in collaboration with the Swedish Pensions Agency
The sample for the field experiment consisted of 96,481 single, low-income pen- sioners and was drawn in May 2016, three months prior to the mailings. Four different letters were sent out to 10,000 subjects (2500 subjects each), and those who received no letter make up the control group. The information in the most simple letter, the Base Letter, was reproduced in the other three letters with a slight addition about com- mon misconceptions about the benefit (the Myths Letter), three simple examples of the possible level of allowance for different combinations of income and wealth (the Rule of Thumb letter), and nine potential combinations of income and wealth and the resulting housing allowance (the Table Letter). Applications were collected up until December 31, 2016.
The results from the experiment are mixed. On the positive side, the results show that simple information letters had a significant effect on the application rate and subsequent take-up. The baseline application rate in the targeted control population was only 1.4%, while the corresponding rates in the different treatment groups were between 9.9 and 12.1%. The only framed letter that had a significantly different effect from the Base Letter on the probability of submission was the Myths Letter. The unconditional acceptance rates also increased substantially, from around 1% in the control group to about 5% in the treatment groups. The economic consequences of the treatment were substantial. Our analysis shows that those individuals who were induced by the treatment to apply, increased their monthly income by around 10% on average.
However, the information treatments also had negative effects in the form of sub- stantially lower conditional acceptance rates. The applications in the control group were accepted in almost 3 out of 4 cases, while up to 50% of the applications in the treatment group were declined. The reasons for decline seem to be that the applicants in the treatment group had higher occupational pension, higher wealth, lower housing
expenditures and were more often cohabiting compared to the applicants in the control group. Even though the letters were supposed to give simple information about the eligibility criteria, they thus seem to have misled some applicants to believe that they were eligible even though they were not. Another interpretation is that the treatments simply lowered the cost of applying to such a low level that many applicants applied even though they were aware that the chance for acceptance was low.
It is reasonable to assume that a policy maker would like to increase take-up among the eligible population but at the same time also reduce the share of declined applica- tions. Declined applications are costly both to the individuals in terms of false hope and the effort of applying, but also to taxpayers in general through the administrative costs of processing applications for ineligible applicants. There are also indirect costs asso- ciated with the non-application of eligible individuals because of uncertainty about their eligibility. It is very hard to put a price tag on any of these costs and benefits.
We can thus conclude that, from the perspective of the policy maker, these informa- tion interventions could be an effective tool to increase the take-up rate of the housing allowance. Scaling up these interventions to the broader population of (eligible) non- claimants can be done at relatively low cost, e.g., by including the information in the personalized Orange Letter that is sent out yearly by the Pensions Agency to all individ- uals who are part of the pension system (savers as well as pensioners). However, if this is to be done, it is important that the eligibility of individuals can be assessed properly.
One way of achieving better targeting would be to allow the Pensions Agency to gather more data from other public agencies and banks in order to make a more accurate a priori assessment of the eligibility status. Another option could be to send out this information to pensioners who experience an important life change which typically increases eligibility, such as the loss of a husband or wife. If enhanced precision is not feasible and one still wants to make use of information interventions, then our results suggest that demonstrating the eligibility criteria using simple examples may be the most effective tool to get eligible individuals to apply.
Acknowledgements Funding from the Swedish Social Insurance Agency (nr 028986-2013), Handels- banken (nr P2015-0147), Forte (nr 2013-2482 and 2017-00092) and the NORFACE Welfare State Futures Programme (nr 462-14-010) is gratefully acknowledged. Special thanks are due to Hanna Karlsson Ruiz and Hanna Linnér for providing the data, and Lena Lantz and Emma Akther for managing the experiment, and also to Ohto Kanninen and seminar participants at IIPF 2018, Jönköping University and the Swedish Pensions Agency for useful comments. We are also grateful to an anonymous referee and the editor whose comments contributed to the improvement of the paper.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 Interna- tional License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Appendix
A.1 Minimum guarantee in the public pension system See Fig.3.
45◦
0 1.141.26 2.72 3.07
1.90 2.13 2.72 3.07 Income pension
+ Minimum guarantee
Income pension Married
Unmarried
Minimum guarantee (gray shaded area)
Fig. 3 The relationship between the income pension and the minimum guarantee in the public pension system. Note: Annual pension in price-related base amounts (1 base amount in 2016 = SEK 44,300). The corresponding monthly amounts in SEK 2016 are: SEK 4208 (1.14), SEK 4652 (1.26), SEK 7014 (1.90), SEK 7863 (2.13), SEK 10,054 (2.72) and SEK 11,343 (3.07). The gray-shaded area denotes the size of the minimum guarantee (the vertical distance between the dashed 45◦line and the solid lines)
A.2 Letters
See Figs.4,5,6and7.
Date Personal number
Have you heard about the housing allowance?
Why we contact you
We know that there are many pensioners who may be eligible for housing allowance, but who have not applied for it. Information that the Pensions Agency has show that you could be one of them and that is why we contact you.
A simple rule-of-thumb that we have, which applies to single households, is that if you have low wealth and a monthly after-tax income per month of SEK 13,000 or less you might be eligible for housing allowance. If you are married, or have a co-habiting partner, the income and wealth thresholds are somewhat lower.
Make a calculation before you apply
Before you decide to apply we encourage you to make a simple calculation on our homepage: www.pensionsmyndigheten.se/BeraknaBostadstillagg.html. This calculation tool gives a preliminary indication of your eligibility status. If you do not have access to a computer you can contact our customer service at 0771-776 776 who will assist you in making the calculation.
Applying for the housing allowance
The easiest way to apply is via our homepage or by filling out one of the enclosed application forms. Which form you should use depends on whether you have co-habiting partner or not.
Disbursement
After you have submitted your application you will be notified whether you are eligible for the housing allowance or not. If you are eligible, you will also receive information on the size of the allowance.
What is housing allowance?
The housing allowance is a supplement to the public pension and you turn to us at the Pensions Agency to apply for it. Whether you are eligible, and how much you would get if so, depends on your housing costs, your wealth and your income.
Best regards, The Pensions Agency Fig. 4 The Base Letter
Date Personal number
280,000 pensioners receive housing allowance today – are you eligible, too?
Did you know that:
• It does not matter whether you live in a rental apartment, a condominium, a house that you own or in some other type of housing – you may still apply for the housing allowance. The value of your house or condominium does not count as wealth.
• If you have low income you may still have some wealth (bank deposits, funds/stocks, summer house etc.)
• If you live together with others you may still apply for housing allowance
• Surveys have shown that many retirees lose this pension supplement because they never applied
• If something changes in life – for example if you move, your income changes or start to live on your own – this might affect your eligibility status
[Base letter content (from first heading)]
Fig. 5 The Myths Letter
Date Personal number
Have you heard about the housing allowance?
Why we contact you
We know that there are many pensioners who may be eligible for housing allowance, but who have not applied for it. Information that the Pensions Agency has show that you could be one of them and that is why we contact you.
The examples below give you an indication about your chances of getting the housing allowance, depending on your income, potential wealth (bank deposits, bonds/stocks, summer house etc.). The examples apply to single households. You may also qualify for the housing allowance if you are married or have a co-habiting partner, but then other income thresholds apply.
You can get a housing allowance of… if you have…
SEK 900 SEK 10,000 per month after tax + net wealth of SEK 300,000 SEK 1,000 SEK 12,000 per month aer tax +
net wealth of maximum SEK 100,000 SEK 1,900 SEK 7,000 per month aer tax +
net wealth of SEK 500,000
Note that these examples use approximate numbers. The numbers are therefore not applicable to everyone, but they can give you an indication of how income and wealth affect the housing allowance. The numbers are for example not correct if your housing costs are below SEK 5,000.
[Base letter content (from second heading)]
Fig. 6 The Rule of Thumb Letter
Date Personal number
Have you heard about the housing allowance?
Why we contact you
We know that there are many pensioners who may be eligible for housing allowance, but who have not applied for it. Information that the Pensions Agency has show that you could be one of them and that is why we contact you.
Whether you are eligible, and how much you would get if so, depends on your housing costs, your wealth and your income. In the table below, there are several examples that show the size of the housing allowance that you can get, depending on your income and wealth (bank deposits, bonds/stocks, summer house etc.). The examples apply to single households. You may also qualify for the housing allowance if you are married or have a co-habiting partner, but then other income thresholds apply. The top row shows monthly after-tax income while the left-most column shows wealth.
Monthly housing allowance (SEK)
Monthly income Monthly income Monthly income
Wealth 7,000 10,000 12,000
0 – 100,000 5,090 2,300 1,000
appr 300,000 3,300 900 0
appr 500,000 1,900 0 0
Example: If your monthly aer-tax income is SEK 10,000 and your wealth is SEK 300,000 you may be eligible for a housing allowance of SEK 900 per month.
Note that these examples use approximate numbers. The numbers are therefore not applicable to everyone, but they can give you an indication of how income and wealth affect the housing allowance. The numbers are for example not correct if your housing costs are below SEK 5,000.
[Base letter content (from second heading)]
Fig. 7 The Table Letter
