Education, skills and gender: The impact of a grading reform and the business cycle on labor market outcomes

(1)

Education, skills and gender

The impact of a grading reform and the business cycle on labor market outcomes

Charlotta Boström

Charlotta Boström Education, skills and gender

Swedish Institute for Social Research 103

Doctoral Thesis in Economics at Stockholm University, Sweden 2019

Department of Economics

ISBN 978-91-7797-817-6 ISSN 0283-8222

(2)

Education, skills and gender

Charlotta Boström

Academic dissertation for the Degree of Doctor of Philosophy in Economics at Stockholm University to be publicly defended on Wednesday 25 September 2019 at 13.00 in

Nordenskiöldsalen, Geovetenskapens hus, Svante Arrhenius väg 12.

Abstract

This thesis consists of three self-contained essays in economics, all concerned with different aspects of education and labor market outcomes. The abstracts of the three studies are as follows.

A flight of hurdles? Effects on graduation and long-term labor market outcomes of a nationwide grading reform.

In the academic year 1994/1995, a grading reform was implemented in Swedish upper secondary schools. The reform replaced norm-referenced grading with criteria-referenced grading that raised the hurdle to graduate on time. By exploiting exogenous variation in exposure to the reform due to exact date of birth coupled with implementation date, the effects on upper secondary school completion and subsequent long-term labor market outcomes are explored in a difference-in- discontinuity design. Results indicate that the probability to graduate from upper secondary school decreased throughout the ability distribution, with the strongest effects at the left tail. Furthermore, many of these individuals still lack a degree at age 33. Nevertheless, the grading reform does not seem to have had any clear effects on long-term labor market outcomes.

The effects of graduating from college in a recession: The case of Sweden

This paper studies the long-term labor market consequences of graduating college into the Swedish economic crises of the 1990s. I use a sample of Sweden born men who graduated college between the years 1985 and 1998. I estimate the effects of labor market conditions at the time of graduation on labor market outcomes using a panel covering 12 years post- graduation. Since the timing of graduation might be affected by economic conditions, I instrument the unemployment rate at graduation using the unemployment rate at age 25, which is the modal age of graduation. I find a significant negative effect on real annual earnings that last up to 5 years after graduation before fading out. The heterogeneity analysis reveals that graduates in the lower end of the distribution of cognitive abilities experience a substantial earnings loss that persists for at least eight years before fading out, while individuals with high cognitive ability are unaffected. Furthermore, I find that graduates well-endowed with noncognitive abilities, individuals we would expect to perform well on the labor market, also experience significant earnings losses.

Gender and field of study: The impact of graduating college into a recession.

The aim of this paper is to investigate if there are gender differences from entering the labor market during an economic downturn. Using a sample of Swedish college graduates who completed their first college degree between 1996 and 2007, I estimate short- and medium-term effects of graduating into adverse labor market conditions on a range of labor market outcomes such as annual earnings, nonemployment and skill-mismatch. I find that the overall differences between the genders of graduating college into a recession are driven by the choice of field of study and the fact that females outnumber male graduates with degrees aimed towards occupations in the public sector. The analysis shows only small differences between the genders when I compare outcomes within Business, Law and Engineering graduates, degrees leading to occupations that typically require workers to maintain a high degree of labor market attachment.

Stockholm 2019

http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-171508

ISBN 978-91-7797-817-6 ISBN 978-91-7797-818-3 ISSN 0283-8222

Department of Economics

Stockholm University, 106 91 Stockholm

(3)

(4)

EDUCATION, SKILLS AND GENDER

Charlotta Boström

(5)

(6)

Education, skills and gender

Charlotta Boström

(7)

©Charlotta Boström, Stockholm University 2019 ISBN print 978-91-7797-817-6

ISBN PDF 978-91-7797-818-3 ISSN 0283-8222

Printed in Sweden by Universitetsservice US-AB, Stockholm 2019

(8)

To everyone I count as family. You are many.

(9)

(10)

vii

Swedish summary

Sammanfattning

Denna avhandling består av tre självständiga uppsatser i nationaleko- nomi, som alla analyserar utbildnings- och arbetsmarknadsutfall ur olika perspektiv. Nedan följer en kort sammanfattning av de tre upp- satserna.

En lång rad hinder? Effekter på slutförande av gymnasiestudier och långsiktiga arbetsmarknadsutfall till följd av en nationell be- tygsreform.

Under skolåret 1994/1995 reformerades betygssystemet i den svenska gymnasieskolan. Reformen ersatte ett relativt betygssystem med ett målrelaterat betygssystem som gjorde det svårare för eleverna att gå ut gymnasieskolan på utsatt tid. Denna studie utforskar effekterna av denna förändring på slutförande av gymnasiestudier och långsiktiga arbetsmarknadsutfall. Genom att utnyttja den exogena variation i be- handling som uppstår till följd av studenternas exakta födelsedatum och implementeringsdatum identifieras kausala effekter med en så kallad difference-in-discontinuity design. Resultaten visar att sanno- likheten att fullfölja gymnasiestudierna minskade över hela fördel- ningen av förmågor, här mätt som genomsnittsbetyg från grundskolan.

Dock är effekterna starkast bland de elever som hade lägst betyg från grundskolan. Många av dessa individer saknar dessutom fortfarande vid 33 års ålder en gymnasieexamen. Däremot verkar inte betygsre- formen ha medfört några tydliga effekter på långsiktiga arbetsmarknadsutfall.

(15)

xii

Effekter av att ta en högskoleexamen under rådande lågkonjunk- tur: Det svenska fallet.

Denna uppsats analyserar effekter på långsiktiga arbetsmarknadsutfall av att ta en högskoleexamen under den djupa ekonomiska kris som inträffade i Sverige under början av 1990-talet. Jag studerar ett urval av svenskfödda män som tog ut en examen mellan 1985 och 1998.

Effekterna av arbetsmarknadens tillstånd vid examen skattas genom att följa männen i en panel de 12 närmast följande åren. Resultaten indikerar att det finns en signifikant negativ effekt på individers reala löneinkomster som varar upp till sex år efter examen. En heterogeni- tetsanalys visar dessutom att effekterna inte är jämt spridda över individers olika förmågor. Individer med låga kognitiva förmågor, det vill säga det vi vanligtvis betraktar som förmåga till abstrakt tänkande, lä- rande, och problemlösning, tenderar att uppvisa större minskningar av sina löneinkomster än individer med goda kognitiva förmågor. I denna grupp varar effekten upp till tio år efter inträdet på arbetsmarknaden.

Jag skattar också effekter över icke-kognitiva förmågor, det vill säga förmågor vi ofta beskriver i termer av sociala och beteendemässiga attribut så som uthållighet, samarbetsförmåga och pliktkänsla. Ana- lysen visar att de individer som har goda icke-kognitiva förmågor, individer vi vanligtvis väntar oss skall ha goda utsikter på arbetsmarknaden, också uppvisar signifikanta minskningar av sina löneinkomster i upp till fem år efter examen.

Kvinnor, män och studieval: Effekter av att ta en högskoleexamen under rådande lågkonjunktur.

Denna uppsats syftar till att analysera om det finns könsskillnader i arbetsmarknadsutfall om inträdet på arbetsmarknaden sker i en låg- konjunktur. Med ett urval av svenska universitetsstudenter som tar sin första högskoleexamen mellan 1996 och 2007, skattar jag de kort- och medelsiktiga effekterna av att slutföra en högskoleexamen under en recession. Jag studerar en rad arbetsmarknadsutfall såsom årlig ar-

(16)

xiii betsinkomst, sysselsättning och matchning till yrken som kräver hög- skolekompetens. Jag finner att de övergripande skillnaderna mellan män och kvinnor drivs av skillnader i utbildningsinriktning och det faktum att fler kvinnor än män tar examina som leder till yrken inom offentlig sektor. Analysen visar att det endast finns små könsskillna- der i arbetsmarknadsutfall mellan individer som blir civilekonomer, jurister eller civilingenjörer – yrken som traditionellt kräver en hög grad av arbetsmarknadsanknytning.

(17)

xiv

(18)

xv

Acknowledgments

This thesis could not have been achieved without the knowledge, support, generosity and love of many others.

First and foremost, I want to express my gratitude to my supervisors.

I first met Anne Boschini when writing my master’s thesis. Her positive attitude and generosity made the experience of writing that thesis so much fun that I once and for all decided to apply to the PhD program. Ever since then, I have felt that she as an academic professional has been embodying what I want life in academia to be – generous, curios and committed to making research more than a one woman’s job. Thank you Anne for always believing in me and giving me support. Thank you for telling me to write when I got caught up in the wonderful world of data. I would not have managed this without you.

I am also very grateful to have had the opportunity to be co-supervised by Peter Fredriksson. Peter generously gave me the opportunity to work with Swedish register data and to write the first study of this thesis. His guidance, sharp insights and patient support was extremely helpful. Peter is an excellent economist and a highly skilled econome- trician. So, to put it simply - I learned a lot. Thank you!

Matthew Lindquist co-supervised the second study of this thesis.

Ever since I first ended up in the same seminar room as Matthew he has proven to be enthusiastic, quick-minded, fair and humble. Simply a fantastic combination of qualities that I, and surely many others, ap- preciate a lot. Thank you for taking me under your wings, sharing your time and giving me advice. I have listened!

(19)

xvi

I’m also heavily indebted to the great academic and social climate at SOFI. Thank you all for letting me become a part of SOFI. I have thrived in this environment where the doors are always open and the smiles are never far away. You have all made a difference, sociolo- gists as well as economists. Nevertheless, I want to give some extra credit to some not already mentioned: Anders Björklund, Karin Hede- ros, Markus Jäntti, Erik Lindqvist, Johanna Rickne, Dan-Olof Rooth, Anders Stenberg and Marianne Sundström. Thank you for comments on my work and support of various kinds! And of course, SOFI would never run without excellent administration: Maria Mårtensson, Caro- line Hertzman, Katarina Hagelin, Elma Sose and Tara Navabi. Ap- plause!

Further, I would like to thank everyone who has been involved at the Department of Economics at Stockholm University. Thank you for giving me the opportunity to pursue a PhD and giving me valuable teaching opportunities. I especially want to thank Jonas Häckner, Astri Muren, Anita Karlsson, Anne Jensen and Ingela Arvidsson.

During these years I’ve met many wonderful PhD students who made this experience much more enjoyable. Of course my former office mates have had a special impact. Linnea Wickström Östervall, Emma von Essen, Paola Montero Ledezma and Tamara Sobolevskaia.

Thank you for sharing the ups and downs of life and PhD life with me!

This would have been so much harder without you. Although not an office mate, Maria Cheung also became close. Moreover, there are many others who took courses with me, have supported me or in other ways just brightened up my days. I especially want to acknowledge Lisa Laun, Martin Nybom, Martin Olsson, Johan Egebark, Montasser Ghachem, Pedro Soares Brinca, Miri Stryjan, Manja Gärtner, Sara Fo- gelberg-Lövgren, Andre Richter, Niklas Kaunitz and Marit Widman.

Last but definitely not least, I want to acknowledge the importance of family and friends who have stuck by my side for better and for worse. I owe everything to my parents for never ever letting me think that there exist boundaries for what I can do. Dad, I’m sad you are no longer with us to see the end product of this. Mom, you are one of my

(20)

xvii best friends and thank you for always being here.Thank you Mikael for staying by my side all those years and for being such an excellent co-parent to our children. Thank you Henry and Hedda for bringing perspective and laughter to my life. I love you! And thank you Martin for your endless support and for choosing to be that significant other in my life <3. And to my friends: now is a good time to catch up! :)

* * *

I am very grateful for support from IFAU, as well as access to their database.

(21)

xviii

(22)

1

Introduction

This thesis consists of three self-contained essays in economics, all concerned with different aspects of education and labor market outcomes. The abstracts of the three studies are as follows.

A flight of hurdles? Effects on graduation and long-term labor market outcomes of a nationwide grading reform.

In the academic year 1994/1995, a grading reform was implemented in Swedish upper secondary schools. The reform replaced norm-referenced grading with criteria-referenced grading that raised the hurdle to graduate on time. By exploiting exogenous variation in exposure to the reform due to exact date of birth coupled with implementation date, the effects on upper secondary school completion and subsequent long-term labor market outcomes are explored in a difference- in-discontinuity design. Results indicate that the probability to graduate from upper secondary school decreased throughout the ability distribution, with the strongest effects at the left tail. Furthermore, many of these individuals still lack a degree at age 33. Nevertheless, the grading reform does not seem to have had any clear effects on long- term labor market outcomes.

The effects of graduating from college in a recession: The case of Sweden

This paper studies the long-term labor market consequences of graduating college into the Swedish economic crises of the 1990s. I use a sample of Sweden born men who graduated college between the years 1985 and 1998. I estimate the effects of labor market conditions at the

(23)

2

time of graduation on labor market outcomes using a panel covering 12 years post-graduation. Since the timing of graduation might be affected by economic conditions, I instrument the unemployment rate at graduation using the unemployment rate at age 25, which is the modal age of graduation. I find a significant negative effect on real annual earnings that last up to 5 years after graduation before fading out. The heterogeneity analysis reveals that graduates in the lower end of the distribution of cognitive ability experience a substantial earnings loss that persists for at least eight years before fading out, while individuals with high cognitive ability are unaffected. Furthermore, I find that graduates well-endowed with noncognitive abilities, individuals we would expect to perform well on the labor market, also experience significant earnings losses.

Gender and field of study: The impact of graduating college into a recession.

The aim of this paper is to investigate if there are gender differences from entering the labor market during an economic downturn. Using a sample of Swedish college graduates who completed their first college degree between 1996 and 2007, I estimate short- and medium- term effects of graduating into adverse labor market conditions on a range of labor market outcomes such as annual earnings, nonemployment and skill-mismatch. I find that the overall differences between the genders of graduating college into a recession are driven by the choice of field of study and the fact that females outnumber male graduates with degrees aimed towards occupations in the public sector.

The analysis shows only small differences between the genders when I compare outcomes within Business, Law and Engineering graduates, degrees leading to occupations that typically require workers to maintain a high degree of labor market attachment.

(24)

3

Study 1

(25)

4

(26)

5

1 A flight of hurdles? Effects on graduation and long-term labor market outcomes of a nationwide reform

^*

1 Introduction

Since schooling is assumed to raise human capital levels and produc- tivity, promoting education has long been an essential part of govern- ments' policies to enhance sustainable economic development. Fur- thermore, by dropping out of school, many early school leavers are bound to lack the skills demanded in the labor market and therefore will face a higher risk of unemployment. Since schooling affects individuals’ labor market opportunities, understanding the mechanisms that determine investment in human capital is of utmost importance to help reduce difficulties in finding a job and potential experiences of social exclusion. Among the many causes of disengaging from education are the personal experiences of schooling, which in turn are influ- enced by school policies and different teaching pedagogies. There is an increasing amount of empirical evidence that teachers, schools and standards of curricula affect student performance.¹ The effects of different grading practices are, however, largely an unexplored field since variation within a given school system is rare.

∗ I thank IFAU for the access to their database.

1 See for instance Grönqvist and Vlachos (2008) on the effects of teachers’

cognitive and noncognitive abilities on student achievement, Fredriksson et al. (2013) on the long-term effects of class size, Pop-Eleches and Urquiola

(27)

6

This study analyzes the effects of a Swedish nationwide grading reform implemented in upper secondary schools (USS) in 1994. The reform replaced norm-referenced grading and progressive assessment with criteria-referenced grading in a course framework. The effects on USS completion and subsequent long-term labor market outcomes are analyzed in a difference-in-discontinuity setting using a sample of individuals born between 1975 and 1980. It is shown that the introduction of criterion-referenced grading increased the hurdle to graduate² on time, while the long-term effects on individuals' labor market outcomes are less clear-cut. This paper is the first to analyze such a large- scale implementation of a new grading system.

The few existing studies on feedback within the education system indicate that providing information on relative position improves student performance. For instance, Azmat & Iriberri (2010) show that the introduction of relative performance feedback in addition to grading on an absolute scale led to an increase in performance of treated students at a Basque high school. A more recent study by Jalava et al.

(2015) studies the effects of supplying nonfinancial incentives on test performance in a randomized field experiment using Swedish sixth graders. They find support for the idea that grading students according to an absolute scale does not motivate students to perform at their best but rather that ranking students has a particularly large motivational power on test performance. There are also a few studies on providing feedback in general. Bandiera et al. (2015) show that interim feedback improves student performance at a leading US university. Sjögren (2010) investigates whether the introduction of grades in Swedish compulsory schools affects students’ long-term educational attainment and earnings in adulthood. Sjögren finds that the effects on edu-

(2013) on the effect of school quality, and Hall (2012) on the prolongation of vocational tracks in upper secondary school.

2 Completing and graduating are used interchangeably. In Sweden students do not graduate in the same sense as in other countries, but rather complete studies at USS and then receive their final grades.

(28)

7 cation are heterogeneously distributed by gender and parental education but finds no evidence of average effects on educational attainment or earnings in the long term. Fachionelli (2016) studies the same reform and finds that high-ability students exert more effort when graded and that high-ability students from low-SES environments invest more in education, but that this investment does not translate into any effects on long-term earnings.

Closely related is also a strand of literature on grading and graduation standards. Raising standards is likely to induce behavioral re- sponses, since students will try to affect which grade they receive by deciding how much to invest in the production of a test score (Becker and Rosen, 1992). For the investment to take place, the benefits of a certain grade must exceed the cost of exerted effort and time spent on studies. Hence, the net effects of increasing standards depend on how the costs and benefits of schooling are affected. Studies typically indicate that increasing standards may lead more-capable students to increase their effort and perform better, while they may cause less able students to lower their effort when standards move beyond their reach.³ Altogether, these studies suggest that whether and how feedback is supplied, and how high the standards are set, matter for individuals’ performance. They also suggest that effects may be heteroge- neous across both predetermined individual characteristics and the ability distribution.

This study will contribute to the literature by providing new evidence on how individuals are affected both in terms of academic attainment and in terms of long-term labor market outcomes. Even though it will not be able to disentangle the exact mechanisms, it will investigate to what extent there were consequences from replacing norm-referenced grades with criteria-referenced grades. As a policy

3 See for instance Betts and Grogger (2003) and Figlio and Lucas (2004) on effects of increased grading standards, Dee and Jacob (2006), Ou (2010), and Marcotte (2011) on effects of the introduction of high school exit exams, and finally Lilliard and DeCicca (2001) on effects of minimum course require- ments.

(29)

8

matter, the impact of a grading reform over the range of the ability distribution is relevant since the risk of dropping out of school likely increases as we move down the ability distribution. Hence, this study adds new evidence by examining effects over a gradient of ability.

Results indicate that the criteria-referenced grading standard raised the hurdle to graduate from upper secondary school, with the strongest effects found at the left tail of the ability distribution. Consequently, many individuals still do not have a degree at age 33. Nevertheless, the grading reform does not seem to have any significant effects on long-term labor market outcomes.

The paper proceeds as follows. In the following section, I provide background on the institutional context and the grading reform. In Section 3, I discuss the empirical strategy, and in Section 4, I present the data and the measurements used. Sections 5 and 6 present the results and provide a specification analysis, respectively. In Section 7, the results are discussed in relation to the related literature. Finally, Section 8 concludes.

2 Institutional context and the reform

The Swedish compulsory schooling law requires individuals to complete a minimum of nine years of education, independent of when they start. Children usually enter in the academic year they turn seven, with a school entry cut-off date of 1 January, implying that season of birth has no effect on years of compulsory schooling attended. Around the years of the grading reform in 1994, a large fraction of students, approximately 96 percent of each birth cohort, completed compulsory school at the expected age of 16. Grade advancement and grade retention is rarely practiced within Swedish lower secondary school (LSS).

The vast majority, above 99 percent, of the students apply immedi- ately to USS with the intention of adding another two or three years of education before entering the labor market or tertiary education.

The programs at USS were, and still are, divided into general tracks

(30)

9 that aim at preparing students for higher education and vocational tracks that focus on skills relevant to the labor market. The vocational and a few academic tracks consisted of two years at the end of the 1980s but were gradually prolonged to consist of three years.⁴ As of 1994, they are all three years long. Students are entitled to enter USS until the year of their 19^th birthday. After that point, they are referred to the public adult education system to take courses at the upper secondary level, either to obtain a full degree or to complete their previ- ous studies.

Up until the mid-1990s, Swedish schools graded students using a norm-referenced system. This system was based on the idea of ranking students and grading them according to a normal distribution, with the intent of systematically and credibly guaranteeing a fair selection to further education. Hence, the distribution of grades within a class was based on the distribution of scores from nationwide tests in three sub- jects – Swedish, mathematics and English. Grades were given on a numerical scale ranging from 1 to 5, 1 being the lowest grade and 5 being the highest grade.⁵ Grade 3 represented the mean. Students were graded as long as they did not experience long-term absence, did not show neglect or by some other special circumstance did not provide their teachers with a basis for assessment. Grades were given by subject each semester, fall and spring and upon graduation from LSS or USS; the final grade summarized a student’s progression. A school- leaving certificate (diploma) was acquired when a student had obtained a final grade in each subject in the curriculum.

As of the academic year 1992/1993, a short series of changes were made regarding the grading practices in USS. The first change was meant to introduce criteria-referenced grades, though the grades were

4 See Hall (2016) for a more detailed overview of the reform.

5 As of the curriculum set in 1980, there were no given percentages for the distribution of grades. However, grade 3 was supposed to reflect the average performance of students in the entire country, and grades 2 and 4 should be given more frequently than were grades 1 and 5.

(31)

10

still given according to the same scale, ranging from 1 to 5. However, the means and standard deviations of the final grades did not shift relative to earlier cohorts (Fredriksson and Vlachos, 2011), which indicates that teachers and schools did not change their grading behavior in practice.

The second and more substantial change, which is subject to eval- uation in this paper, was implemented as of the new curriculum of 1994. The norm-referenced system had been criticized for not relating to the accumulation of knowledge and application of skills obtained but rather comparing students’ achievements. As a response, a new criteria-referenced grading system was formulated. In this new system, grades were distributed based on well-defined learning goals and performance standards for each grade level. Thus, it was an absolute grading system. Furthermore, students were subjected to a course system, which divided the content of each subject into shorter courses by which they received a final grade. This grading system would make it possible to track changes in performance levels in the population over time by the use of standardized tests. The grading scale was changed to contain 4 levels – fail, pass, pass with distinction and pass with honors. A school-leaving certificate was acquired only when all courses facing the student within his or her path of study were com- pleted and graded. Note that the grade fail implied that a student was graded. Given a fail or lacking a grade, students were given one chance to pass the course again to obtain a (higher) grade.

It seems as though the combination of criteria-referenced grades and the course system increased the hurdle to graduate USS on time.

Figure 1 depicts the share of students present in education proxied by the transfer of student aid.⁶ The timing of the reform is indicated by

6When registered in a program in USS, students receive a smaller amount of government funded student aid. I use registers of these transfers to generate a proxy for being present in education, since there is no other register spanning these cohorts that indicates presence in USS. Variation between years 1-3 within the same cohort are to some extent explained by students registered in

(32)

11 the vertical line, and the figure also displays graduation rates. Before the reform, it seems as though most students remained registered at USS until graduation. After the reform, the trends are similar even though we could suspect an increase in the dropout rate in the third year. Variability in the measure before the reform makes it hard to visually detect any specific trend. Approximately 85 percent of each birth cohort graduated on time before the reform, but graduation rates then fell substantially, by approximately 10 percent.

Moreover, the effects of the reform on graduation seem to have been heterogeneously distributed. Figure 2 shows that graduation rates fell for the cohorts entering USS after the reform (birth cohorts of 1979 and later) and that the impact of the new grading system was highest at the bottom of the ability distribution while leaving the top largely unaffected. Hence, while the reform does not seem to have caused large increases in the dropout rate during USS, the reform seems to have increased the hurdle to graduate on time.

3 Empirical strategy

Criteria-referenced grading was implemented at the same point in time across all municipalities in Sweden. Simple cohort comparisons before and after the reform may be subject to bias because of omitted time or cohort effects. To resolve this identification problem, I will exploit exogenous variation in the assignment of treatment status created by an individual’s date of birth and the implementation date of the reform. Individuals born before 1 January 1978 were mainly exposed to norm-referenced grading, and individuals born on or after 1 January 1978 were mainly exposed to criteria-referenced grading.

However, the grading reform moves in tandem with the School Entry law, which stipulates that an individual will start school if born as of

exchange programs abroad. Student aid is not given if there is an equivalent program in Sweden.

(33)

12

1^st January the academic year s-he turns seven. Since there is empirical evidence that an older school starting age (SSA) raises educational attainment and may also affect labor market outcomes (Fredriksson and Öckert, 2014, and Black et al. (2011), it is possible that this confounding policy will generate bias in an ordinary regression discontinuity setting. Hence, I will implement a difference in discontinuity design (diff-in-disc). The idea is to identify a time period when the confounding policy occurs in isolation and then take the difference between the joint effect of the two policies and the effect of the confounding policy. This approach will achieve an unbiased estimate of the policy of interest, in this case, the grading reform.⁷

Assuming that exact day of birth is randomly distributed has the implication that individuals starting school according to the Swedish Compulsory schooling law are randomly distributed across the school entry cut-off date. This point suggests that individuals born just before and just after the cut-off are on average identical except for exposure to the treatments. Hence, I mimic a randomized experiment and identify the causal effects of the grading reform. I define cohorts to run between July and June, and the effect of the reform is identified by the discrete level change (“jump”) at the cut-off in the cohort consisting of individuals born between 1 July 1977 and 31 June 1978. The re- maining cohorts included in the sample identify potential SSA effects when no changes are made in grading practices.

Since all individuals in the sample receive the treatments or are assigned to the control conditions, the analysis will be based upon a sharp regression discontinuity design. Since the assignment variable in this case is discrete, I implement a parametric strategy and estimate treatment effects based on a specific functional form of the smoothing function between the outcome and the assignment variable. Day of

7 Appendix Figure A.1 plots the average LSS GPAs by month of birth. These graphs are indicative of school starting age effects. Therefore, taking this effect into account is crucial for identifying an unbiased effect of the grading reform.

(34)

13 birth is normalized such that the treatment effect at the cut-off corresponds to the coefficient on the assignment variable. To assess the effects of the grading reform, I estimate the following outcome equation:

= + × +

+ + + +

where for individual i belonging to cohort c, is a treatment indicator identifying the timing of reform. = ! ≥ 0 is a treatment indicator identifying SSA effects, where D denotes exact day of birth. is a k^th-order polynomial in day of birth that will take care of any smooth relationship between day of birth and the outcome of interest, . Finally, denotes cohort fixed effects, and is the error term. The coefficient is the diff-in-disc estimand, which will capture the causal effect of the grading reform on the outcome. The slope of the smoothing function is allowed to be different before and after the school entry cut-off and potentially also to vary by cohort. Standard errors are clustered on day of birth (Lee and Card, 2008). To explore heterogeneity over a gradient of ability, I will estimate effects by quintiles of percentile-ranked LSS grade point averages.

4 Data and measurement

The analysis exploits rich register data originating from Statistics Sweden. The sample used for the analysis covers the Swedish population born during the years 1975-1980 and spans the years 1990-2013.

Included is a large set of individual characteristics such as exact day of birth, gender and parental background. School registers contain in- formation on what years individuals completed LSS and their grades upon completion. They also provide details on when and where an

(35)

14

individual began USS and which tracks were chosen. To these registers, I match information on annual earnings and wages.

The sample is restricted to individuals for whom I have exact date of birth and who graduate the compulsory level on time, i.e., in the year they turn 16, and commence directly to USS the following semester. These individuals were graded according to the same norm- reference system in LSS, which has the implication that the probability of being admitted to USS is unaffected. Individuals included in the sample enter both two- and three-year-long programs at USS. It would be compelling to compare students who enter programs of the same length, but during the pre-reform era, students had the possibility of choosing among two- and three-year-long programs, and the availa- bility of two-year-long programs decreased over time. To minimize potential bias due to self-selection of high-ability students into the longer tracks during the pre-reform period, I will include individuals entering programs of both lengths in the analysis. Furthermore, I will restrict the sample downwards to individuals born in 1975, since sev- eral municipalities participated in a pilot scheme that affected students at vocational programs in the preceding cohorts. Students included are furthermore entering a regular program, implying that students who started an adjusted study program, due to incomplete LSS grades or disability, are excluded.⁸ Since graduation rates varies slightly over different birth cohorts and I shift cohorts to identify the effects of the reform, I will include multiple cohorts in the analysis, from both the pre- and post-reform period, to estimate an average of the SSA effect.

The short-term outcome of interest is graduation from USS, i.e., graduating within the length of the program students enrolled in.

8 Excluding individuals that lack information about which program they enrolled in and then matching exact day of birth from the historical birth registers results in a sample consisting of 494 619 individuals. Excluding individuals in adjusted study programs further reduces the sample to 475,385 individuals. Shifting cohorts implies that the basis for the analysis consists of 394,460 individuals. Up until age 33, the sample is further reduced, mainly due to migration.

(36)

15 Long-term educational attainment is explored by assessing effects on the probability to have at least the equivalent of an upper secondary degree at age 33, which corresponds to measuring outcomes for the youngest cohort at the end of the year 2013. The reason for focusing on upper secondary schooling and for instance not exploring effects on years of schooling is that after the reform, all tracks at USS were gradually prolonged to consist of three years, and this change would mechanically affect such an outcome measure. Hence, based on the available school registers, I create an indicator for graduating USS on time. An indicator for holding the equivalent of a USS degree at age 33 is based on the information about highest level of education.

The long-term outcome of main interest is annual earnings. Annual earnings is a function of hourly wages and hours worked and as such summarizes the overall labor market performance. This earnings measure includes annual taxed income and other reimbursements for which the employer returns payroll taxes. I will estimate effects on the logarithmic transformation of earnings and on the probability of having positive earnings. Furthermore, I will include a measure of employment status to explore effects on the labor supply.⁹ In Sweden, men generally work full time, while women more often work part time. Thus, I concentrate on an employment measure that would capture movements in and out of part time employment and that would capture individuals who move in and out of temporary jobs. Since the data do not allow me to measure employment directly, I use the map- ping of the distribution of annual earnings and wages to generate an earnings cut-off to indicate employment status. To construct this indicator, I first divide the data into cells by level of education, gender and year of birth. The earnings cut-off is then defined as the mean of annual earnings of those with earnings greater than 6 times the wage in the first percentile.

9 Employment status could also be measured by the probability of having positive earnings but is in this case included, since zero earners are excluded in the logarithmic transformation of annual earnings.

(37)

16

Background information used in the specification analysis includes percentile ranked LSS grade point averages, whether an individual has an immigrant background (defined as both parents having immigrant status), the maximum value of either parent’s level of education, gender and municipality of residence at age 16. National unemployment rates used in the long-term analysis stem from Statistics Sweden and the Labour Force Surveys.

5 Effect of the reform on upper secondary school completion

This section presents the evidence on the effects of introducing criteria-referenced grading on USS completion. It includes a specification analysis and then presents the results from estimating the model.

5.1 Specification analysis

To credibly estimate causal effects in a regression discontinuity setting, we need to know how flexible the functional form of the treatment assigning variable needs to be. Generalized cross-validation pro- cedures such as Aikake’s information criterion are in this case incon- clusive with regards to the best fitting order of the polynomial. The goodness-of-fit test suggested by Lee and Lemieux (2010) implies that a second-order polynomial or higher is not over smoothing the data.

Given this ambiguity, I plot the LOWESS curve of graduating USS by day of birth in Figure 3.¹⁰ By inspecting the curve, I find that a third- order polynomial fits the data-generating process quite well. The ef-

10 The assignment variable is binned in days of 15. See Lee and Lemieux (2010) and Jacob and Zhu (2012) for details about two different types of test used to assess which bin size to use in the graphical representation of the data.

A bin size of 15 days is not over smoothing the data. This bin size is further used to assess the goodness of model fit.

(38)

17 fect of the grading reform is visible in panel (b) as a jump in the probability of completing USS on time. By inspecting panels (a) and (c), we also notice that we may expect a negative effect of school starting age on the probability to graduate on time. The effect of this confounding policy is smaller before the grading reform (panel (a)) than after (panel (c)), potentially because of a shift in the slope. According to the literature, for instance Fredriksson and Öckert (2014) and Black et al.

(2011), we would expect the school starting age effect to be positive, but when shifting cohorts, small variations in USS graduation rates by birth cohort may lead to negative school starting age effects. In this case, a negative effect would not lead me to overestimate the effect of the grading reform on USS graduation but rather underestimate it.

Results from estimating a baseline specification (see Appendix Ta- ble A.1) show that the estimated effect on graduation is invariant to the order of the polynomial. Estimates including a third-order polynomial are also invariant to different window widths and to interacting the polynomial with cohort.The fact that some individuals are retained and some individuals are advanced in LSS could potentially bias results. In this sample, individuals born in the later months of the year are more frequently retained, and individuals born in early months of the year are more often advanced.¹¹ To assess the impact of grade advancement and grade retention, I run a specification on the so-called donut sample where I exclude individuals born in December and Jan- uary (Barreca et al.,2011). The results are invariant to the exclusion of these individuals.

In the difference-in-discontinuity and in the regression discontinuity setting, the estimates of the model have a causal interpretation if

11 Appendix Figure A.2 plots the share retained and share-advanced individuals at the end of the ninth year of compulsory school, by year and month of birth. Individuals born at the end of each calendar year are to a much higher degree retained in compulsory school. Individuals born near the beginning of a calendar year seem to be advanced to a higher degree even if the pattern is not as pronounced. The pattern is fairly invariant over different birth cohorts.

(39)

18

the individuals are unable to precisely manipulate the assignment variable, implying that variation in treatment near the threshold is approximately randomized if there are no not-accounted-for differences across the individuals on either side of the cut-off that are correlated with the outcome. One approach to testing this assumption is to explore whether a set of baseline covariates are equally distributed just above and just below the cut-off or, in other words, are balanced. The balancing of covariates at the cut-off created by the grading reform is explored in Table 1.

Columns (1) to (3) show results from a regression of the diff-in- disc estimand × on LSS grade point averages, gender, immigrant background and parents’ education. Covariates seem balanced at the cut-off; the p-values of the F-tests for joint significance range from 0.317 to 0.443, and I cannot reject the hypothesis that these predetermined individual characteristics are unrelated to the grading reform. Column (4) examines the correlation between each individual characteristic and the diff-in-disc estimand. Each row reports a P- value of a t-test obtained by regressing the individual characteristic on the estimand. The P-values indicate that I cannot reject the hypothesis that the individual characteristics are unrelated to the grading reform.¹²

McCrary (2008) suggests another test of the imprecision of control over the assignment variable by examining whether there is a discontinuity in the density of the assignment variable itself at the cut-off. In this multiple-period setting, it is difficult to implement the exact test proposed by McCrary. However, I may test for the presence of manipulation by exploring whether the fraction of individuals born on either side of the cut-off differs between the reformed cohort and the unre- formed cohorts as a consequence of the reform. The results from such a regression are presented in Table 2. Column (1) presents the results using all cohorts in the sample. The coefficient on the diff-in-disc estimand is very small but nevertheless significantly different from zero.

12 This result is invariant to polynomial order and whether I allow the slope of the polynomial to vary with cohort.

(40)

19 Given that covariates seem to be balanced, I commence by exploring whether any specific cohort is driving this effect by separately comparing the reformed cohort to the other cohorts. The results are presented in columns (2) to (5), and we can see that the youngest cohort possibly is driving this effect.

These results indicate that there might be sorting around the school entry cut-off in the youngest cohort included in the sample. However, the coefficient on the estimated parameter in column (5) is very small, and in this case, the presence of a discontinuity is not sufficient to claim that there was manipulation, because a precisely estimated jump may result randomly from noise in the data.

Since the results are basically invariant to polynomial order, con- ventional balancing tests indicate no imbalance in the covariates, and there is no consistent evidence of manipulation of the assignment variable, I proceed by estimating the model using a third-order polynomial in day of birth.

5.2 Results

Figure 4 illustrates the cohort-wise estimates of being born after the school entry cut-off on the probability to graduate USS on time. The reform is implemented at the cohort born between July 1977 and June 1978 and is visible as a downward jump. The estimated effect implies a decrease in the probability to graduate on time by 10.7 percentage points. In a difference-in-discontinuity setting, we would like the effect of the confounding policy to lie on a parallel trend to achieve an unbiased estimate of the policy of interest (Eggers et al., 2015). Both before and after implementation, the effects of the confounding policy, the School Entry law, lie on a roughly parallel trend, even though the effect sizes differ somewhat. These differences are driven by shifts in graduation rates among the different birth cohorts in the sample.¹³

13 In this sample, graduation rates vary between 82 and 85 percent among the four birth cohorts, 1974-1977, preceding the reform. Among the three birth

(41)

20

In Table 3, I present the results of estimating the effect of the reform on graduating USS on time. Column (1) presents the average effect, and columns (2)-(6) present the effects by ability, measured as quintiles of percentile-ranked LSS grade point averages. Column (1) shows that the effect of introducing criteria-referenced grading amounted to an average 8.9 percentage point decrease in the probability of graduating on time. As is clear from inspecting the estimates in columns (2) to (6), the reform seems to have the highest impact at the bottom of the ability distribution. In the first quintile, column (2), the estimated effect implies that the probability to graduate on time decreased by 17 percentage points. Column (3) shows that in quintile 2, there is also a large negative impact of approximately 9.8 percentage points, and columns (4) to (6) also show the presence of negative effects in the top half of the ability distribution, ranging from 8.2 to 5.3 percentage points. Thus, the grading reform seem to have raised the hurdle to graduate from USS. Furthermore, it seems as though the hurdle was raised throughout the entire ability distribution, with the greatest impact at the left tail.¹⁴

The introduction of criteria-referenced grading by course, instead of progressive norm-referenced grading, seems to have affected students of different aptitude by different amounts. Since failing to obtain a grade in a course also made it necessary to pass it at a later date, the future work load was increased – something we would suspect was harder to overcome for students with lower aptitude or poor motiva- tion. Nevertheless, holding a USS degree has positive implications for the future. First, formal education is a determinant of labor market opportunities. Different levels of education generate different sets of po- sitions in the labor market upon entry but may also affect the career path. Second, grades serve as a gateway to tertiary education, where selection and admission is made mainly upon USS performance.

cohorts after the reform, 1978-1980, graduation rates vary between 72 and 75 percent.

14 One could potentially consider clustering the standard errors by cohort and month of birth. Doing so does not affect the results.

(42)

21 Hence, those expecting benefits from further investments in education should be induced to complete their USS degree within the formal adult education system at a later point of time. For this reason, I also explore effects of the reform on holding a USS degree at age 33.

Figure 5 presents the average effects at different ages. Please note that due to a time series break in the data, the youngest cohort is excluded from the sample.¹⁵ Over time, the negative effect of the reform on USS graduation is decreasing, even though on average, individuals never catch up entirely. Until age 23, approximately 4 years after expected graduation, the rate of change is the fastest, which implies that most individuals who have an incentive to enter adult education do so early on.

Figure 6 in turn depicts the cohort-wise estimated effects of being born after the school entry cut-off on the probability to hold a USS degree at age 33. The jump at the reformed cohort (cohort 1977/1978) implies that the reform decreased the probability of holding a USS degree at age 33 by 4.6 percentage points. However, taking the SSA effects into account, the estimated effect of the grading reform on the outcome will likely be lower. Results are presented in Table 4.

Column (1) of Table 4 presents the average effect of the reform.

The estimate implies that the probability to hold a USS degree at age 33 decreased by 2.5 percentage points. Viewing the results in columns (2) to (6), we note the same pattern as in Table 3 – the effect of the reform is of greatest magnitude in the bottom of the ability distribution. The estimated coefficient of the diff-in-disc estimand suggests that the probability to hold a USS degree at age 33 decreased by 7.7 percentage points in quintile 1. The effect gradually diminishes as ability increases, in this case leaving the top quintile unaffected by the

15 On a trial basis, Statistics Sweden began collecting grades for individuals without a complete grade card as of the birth cohort of 1980. Some of these individuals were then assigned a level of education equivalent to three years of upper secondary schooling even though they had not in fact graduated and obtained a school-leaving certificate. Including this cohort would inflate graduation rates and hence bias results due to measurement error.

Education, skills and gender: The impact of a grading reform and the business cycle on labor market outcomes

Education, skills and gender

Charlotta Boström

Education, skills and gender

Education, skills and gender

Charlotta Boström

Contents

Swedish summary

Acknowledgments

Introduction

Study 1

1 A flight of hurdles? Effects on graduation and long-term labor market outcomes of a nationwide reform

1 Introduction

2 Institutional context and the reform

3 Empirical strategy

4 Data and measurement

5 Effect of the reform on upper secondary school completion