Essays on the Promotion of Foundational Literacy and Numeracy in Developing Countries

DISSERTATION OVERVIEW

Primary Education in the Developing World

Primary schools in developing countries serve a large share of the global population. Nine of every ten children under 15 live in a low- and middle-income country (LMIC), and 91% of them attend primary school (World Bank, 2021a, 2021b). These figures are expected to further increase in coming decades, as the population growth rate in these contexts is four times faster than that in high-income countries (World Bank, 2021c). In all, over half a billion children – 8% of the world population – are currently enrolled in a primary school in a developing country (UNESCO, 2019), and a much larger share of the global population will have been through primary school in a LMIC at some point in their lives. Given that education has long been considered an engine towards economic growth and social mobility (Chetty et al., 2020; Cunha and Heckman, 2007; Heckman, 2006; Montenegro and Patrinos, 2014), especially in contexts where extreme poverty and inequality are rampant, improving the quality of basic education can be both an effective policy goal in itself, and also a wide-reaching tool to improve human welfare more broadly (Hanushek and Woessmann, 2007; Ozturk, 2001). Yet, education policymakers working in these contexts face pernicious constraints in terms of the institutional capacity, initial human capital of the populations they serve, and the evidence base on which they can draw to design thoughtful policies. As such, creating research that is grounded in LMIC contexts is a valuable step towards improving educational quality, and through this dissertation, I offer three chapters on the issue of learning in developing countries.

More specifically, this dissertation focuses on foundational literacy and numeracy (FLN) outcomes in primary schools in LMIC, and aims to build on the body of evidence on improving and measuring these skills. Strengthening these skills has received increasing research and policy attention in recent decades as one of the most pressing policy goals to improve educational quality in developing contexts more broadly (Evans and Hares, 2021). There are at least three key reasons why FLN should be front and center in the agendas of education policymakers and researchers in developing contexts – namely, the weak current levels of learning, the potential gains in equity achieved through stronger FLN, and the broader systematic diagnostics policymakers can obtain by measuring these skills.
The “Learning Crisis” and Weak FLN Outcomes Around the World
There is ample and growing documentation that the modal experience for a child in the developing world is “schooling without learning” (Pritchett, 2013), or the fact that in spite of the large increases in school enrollment around the world, most children never end up acquiring FLN. For instance, while primary school enrollment in India was at 97% in 2019, up from 79% in 1971 (World Bank, 2021d), only little over a quarter of all grade 3 children in rural India can read at the grade 2 level, and by grade 8, 27% of children in rural India still cannot read at the grade 2 level (ASER, 2019). This experience is likely to be worsened by the pandemic-induced school closures, as preliminary evidence from the Indian state of Kartanaka suggests that as the share of grade 3 rural students being able to read a grade 2 passage fell from 19% in 2018 to 10% in 2020 (ASER, 2021). This “learning crisis”, as it is called in policy circles, is particularly worrying due to the extent to which FLN shapes later stages in people’s academic and personal lives. For those in LMIC that leave the education system early, there is evidence suggesting that fewer than half of young adults who only reached primary school are functionally literate (Kaffenberger and Pritchett, 2020). This implies that governments and families invested valuable and limited resources on sending these children to school, with little return in terms of foundational skills for the majority of them. On the other hand, for those that do persist in the educational system, acquiring these skills can put them on a higher learning trajectory which ensures that they have a strong academic performance in later grades (Bau et al., 2021; Carter et al., 2020). Therefore, regardless of students’ paths after the early grades, addressing the current gaps in foundational literacy and numeracy across much of the developing world is a promising path to improve life outcomes for most students in these contexts.

Pursuing Equitable and Efficient Policies by Targeting FLN

In the process of designing policies, decisionmakers typically face a dire trade-off between elements that promote “equity” and elements that promote “efficiency” (Okun, 2015). However, interventions that focus on improving FLN may be among the rare set of policies that can simultaneously address both. On the equity side, early-grade interventions in developing countries – such as those that focus on FLN – are likely to reach a wider range of socioeconomic backgrounds than other educational programs implemented in later grades. While this is the case in most educational settings, the significantly higher rates of dropout in LMIC make this phenomenon particularly pronounced. For instance, in Kenya and Tanzania, countries where two chapters of this dissertation are based, for every child from the poorest income quintile in the first two grades of primary school, there are 0.5 and 0.8 children from the richest income quintile respectively. By the first two years of secondary school, this ratio becomes 2 and 4.4 children from the richest income quintile, respectively. In Tanzania, by the last grade of secondary school, fewer than 1% of all students come from the poorest income quintile (DHS, 2014, 2015-16). These differences in socioeconomic composition also overlap with large achievement gaps between socioeconomic groups. For instance, Spaull and Kotze (2015) suggest that the gap between the wealthiest income quintile and the three poorest quintiles in South Africa grows by over one grade-level between third and ninth grade. Yet, there is a growing body of evidence which suggests that interventions that simply aim to improve FLN averages can indeed also reduce learning inequality in the early grades, both in terms of achievement and socioeconomic gaps, and especially so when baseline performance is weak as it is in much of the developing world (Crouch et al., 2021; Rodriguez-Segura et al., 2021; Asim, 2020). Therefore, policies that act in the early grades to improve FLN are likely to be significantly more progressive than interventions targeted at later grades once the student body has a wealthier composition.

Given that FLN underpin much of children’s educational and cognitive development, strengthening FLN can also lead to more productive citizens and stronger economic growth as these students join the workforce, advancing the “efficiency” side of policy planning. For instance, literacy is correlated with higher incomes (Hanushek, 2015;; Valerio et al., 2016), greater educational benefits for the next generation (Banerji et al., 2017, Andrabi et al. 2012 or Abuya et al. 2015), better health outcomes (Mathew, 2012; Mensch et al., 2019; Taylor et al., 2016), and higher agricultural productivity (Paltasingh and Goyari, 2018). Similarly, the development of these skills can lead to more cohesive societies with higher civic engagement, as skills such as reading with comprehension and performing mathematical operations for real-world scenarios are key steps towards becoming a self-sufficient citizen and member of society. In aggregate, these gains are likely to increase the economic well-being of individuals and countries as a whole. As policymakers consider areas to invest, they should be aware that the promotion of FLN can yield gains on both sides of the dreaded policy trade-off between gains in equity and efficiency.

FLN as a Policy Target and a Diagnostic Tool

The fact that educational systems worldwide have a universal need to foster FLN makes these skills promising candidates to act as standardized benchmarks for the overall performance of educational systems, and as metrics to set clear learning goals for specific interventions. In practical terms, using FLN to track educational progress has at least three tangible benefits. First, as countries move away from education as a signaling or credentialing system and towards education being a path to acquire tangible skills, directly tracking the acquisition of these skills —as opposed to indirect outcomes like school enrollment— is a valuable way to ensure that educational systems indeed deliver on their promise. Second, using educational inputs such as school construction or expenditures as benchmarks for performance may be, at best, a noisy indicator of educational outputs. This is in part due to the fact there is now strong evidence that the mere availability of these inputs, without accompanying systemic or pedagogical changes, does not lead to improved academic performance (Masino and Niño-Zarazúa, 2016; Murnane and Ganimian, 2017; Evans and Mendez Acosta, 2021). Instead, monitoring outputs like learning can also incentivize a more informed and deliberate use of inputs, as ensuring mastery of FLN requires a minimum level of cohesion and alignment of these inputs with stakeholders and institutional policies (Azevedo et al., 2021b). Finally, using learning as an outcome measure can also pressure policymakers to focus on equity. There is now extensive evidence that educational systems in LMIC, through implicit and explicit incentives, tend to cater to high-performing students (Glewwe et al., 2009; Glewwe and Muralidharan, 2016; Pritchett and Beatty, 2015; among others). Therefore, tracking inputs like “textbooks” may not reflect how these inputs are used, particularly along the lines of equity. Instead, monitoring learning requires policymakers to focus on the full range of students, as the large lower tails in the current distributions of learning in LMIC make it so that moving the overall sample average may be easier by focusing on the large number of low-achieving students than on the few high achieving students (Crouch et al., 2021; Rodriguez-Segura et al., 2021). In this sense, using system-level measures of performance on FLN can create strong incentives to get these systems to use resources more effectively, equitably, and ultimately to deliver adequate learning for all students.

These potential benefits of using FLN as trackers of learning have already moved some policymakers and donor organizations to adopt harmonized metrics of FLN achievement for national benchmarks of educational progress. For instance, the World Bank developed the notion of “learning poverty” (Azevedo et al., 2020a) – a metric which captures deficits in reading achievement and school enrollment. This specific metric was created to both bring attention to the extent of the “learning crisis”, and to be able to set actionable goals for education projects with this indicator as a target. This indicator was developed to mirror important global commitments such as the Sustainable Development Goal 4 (United Nations, 2021), which have placed strong and equitable FLN outcomes at the forefront of global educational goals. Importantly, Azevedo et al. (2021a) also show that the “learning poverty” metric is a good proxy for aggregate future performance and contemporaneous learning in other subjects. Furthermore, although the concept of “learning poverty” has grown in prominence and popularity, it only serves as an example of the use of FLN to track overall educational performance. In fact, learning poverty is not the only option that policymakers have pursued, and for instance, policymakers in Liberia and Kenya have also used a well-known metric of reading fluency, “correct words per minute”, as another system-wide metric of educational achievement. These differing metrics illustrate that FLN can be a meaningful way to track progress and align incentives towards the needs of all students.

This Dissertation

As I argue here, foundational literacy and numeracy are pivotal for both child development and the broader strengthening of educational systems starting in the early grades. Investments in FLN can promote student well-being in the short- and long-run, and can do so while also advancing goals of equitable educational development. As such, in this dissertation I study topics around the promotion and measurement of foundational literacy and numeracy in LMIC through three studies in Costa Rica, Tanzania, and Kenya.

My first two chapters investigate how curriculum reforms affect learning, particularly the acquisition of FLN, in the early primary grades. These two chapters respond to the broad need to find large-scale interventions that can effectively improve FLN even by governments with relatively weak implementing capacity. Much research – such as the early research agenda around the “Teach at the Right Level” movement – has centered on documenting the principles of better content alignment with students’ proficiencies through targeted and relatively small-scale interventions. However, there is not much evidence to date on how curriculum reforms play out when deployed at a large-scale. This is partly due to the relative rarity of this scale for educational intervention, but also due to the difficulty of rigorously evaluating interventions that almost by definition are rolled out all at once. To fill this gap, I study nationwide curriculum reforms aimed at improving FLN and implemented by the governments of Costa Rica and Tanzania in 2014 and 2015 respectively. In both cases, I leverage fairly unique nationally representative panel data, and the exogenous variation created by the grades that were targeted and the timing of the policies to derive causal estimates of the effects of the reforms on student outcomes. Interestingly, these policies differed significantly in terms of the level of expectations from teachers and complexity levels for a successful implementation as designed. In turn, these differences led to very different achievement outcomes in both countries, yielding negative unintended consequences in Costa Rica, and positive effects on learning in Tanzania. Below, I describe these two chapters in greater depth, and provide some ideas on the potential policy features that led to their main results.

My first chapter is titled “Strengthening early literacy skills through social promotion policies? Intended and unintended consequences in Costa Rica.” This chapter was published in the International Journal of Educational Development (Rodriguez-Segura, 2020). Historically, grade repetition has been between 4-7% for grade 1 students in Costa Rica, a higher rate than for any other primary school grade. This pattern is typically attributed to children not attaining sufficient literacy skills to move on to grade 2. In 2014, the government implemented a curricular reform with the goal of improving early literacy skills for children. This reform was coupled with a social promotion policy, abolishing grade repetition in grade 1, with the goal of allowing students to develop their reading skills during grades 1 and 2 before they had any high-stakes outcomes linked to their reading proficiency. The curriculum reform outlined new instructional approaches teachers were expected to follow as they taught early literacy, but the policy did not have any accountability mechanisms to ensure changes were implemented. Ultimately, the social promotion portion of the policy was the only feature that ended up being closely followed by teachers. Using difference-in-differences methods, I find that instead of allowing children more time to catch up, this policy simply delayed grade repetition for many students, leading to increases in grade repetition once they were in grades 2 and 3. This pattern was most pronounced for the most socioeconomically disadvantaged communities. This paper highlights the role of high-quality support for teachers and schools as they implement a complex instructional change.

My second chapter is called “Back to the Basics: Curriculum Reform and Student Learning in Tanzania”, co-authored with Dr. Isaac Mbiti. In response to very low learning levels in Tanzania, and a curriculum that was “overloaded” with subjects for grades 1 and 2, the Tanzanian government passed a curriculum reform which narrowed the instructional scope teachers were expected to cover so they could focus on early literacy and numeracy. This reform mandated that the other subjects were left to either be taught during Kiswahili class as readings, or postponed until grade 3. Observational data shows the policy increased instructional time for early literacy and numeracy by about 2 hours in a 15 hour-week. Difference-in-difference estimates illustrate that the reform improved performance in math and Kiswahili by approximately 0.2 SD, and also decreased the overall dropout rate. Larger gains were (noisily) correlated with teacher training on the new curriculum. Four years after, the policy still caused more students to stay in school, although this was accompanied by decreased passing rates in the standardized national test in grade 4 – highlighting that even though curriculum reforms can yield higher learning outcomes at-scale, they may not be sufficient towards helping all children meet certain desired benchmarks. To the best of our knowledge, this is the first evaluation of a successful curriculum reform at a national level in a LMIC, serving as empirical support for the large but mostly theoretical literature on the benefits of better aligning the curriculum with students’ needs, and as evidence that such a reform can indeed be carried out by governments at a national scale.

My third chapter is called “Assessors influence results: Evidence on enumerator effects and educational impact evaluations”, and it is co-authored with Dr. Beth Schueler. This chapter is a methodological paper which investigates the issue of “enumerator effects” — inconsistent practices between the interviewers who administer questionnaires that can lead to increased measurement error— in a phone-based assessment on foundational numeracy. The issue of enumerator effects is particularly relevant for the measurement of FLN, as internationally-validated exams like the Early Grade Reading Assessment (EGRA) — adapted for over 65 countries (Dubeck and Grove, 2015) —, or the International Common Assessment of Numeracy (ICAN), used in the nationally representative Annual Status of Education Report (ASER) in India and Uwezo in East Africa, and follow this approach. In turn, these statistics have been widely used in important pieces that have laid out the landscape of education in developing countries in recent years (World Bank, 2018; Pritchett, 2013). Therefore, understanding the extent of “enumerator effects” in educational assessments of FLN, and how they might bias the results emerging from these data collection efforts is important as it could shape, to some degree, our understanding of the state of FLN in the developing world.

In fact, it is well-documented that enumerator effects can be a key source of error in survey data collection. However, it is currently less understood whether this is a problem for academic assessments or performance tasks. Furthermore, even if researchers have a reason to suspect that enumerator effects do exist in educational assessments, less is known about how they might bias potential estimates of interest, like treatment effects in impact evaluations. This is especially true when the assignment of assessors is carried out at a level of disaggregation higher than the student, so there are more opportunities for measurement error to cluster around certain aggregate units like classes or schools. Our contribution in this paper is that we find evidence that the foundational numeracy assessment was indeed prone to enumerator effects, and we use simulation to show that these effects were large enough to lead to spurious results at a troubling rate in the context of impact evaluation when assessors are not assigned at the level of the student. In all, this paper highlights the need to thoughtfully weigh the logistical and empirical trade-offs when assigning enumerators at different levels of aggregation, and proposes some alternative solutions like norming or allowing for enumerator fixed-effects to minimize the extent of the bias that enumerator effects might introduce.

The Limits of Policies which Focus on FLN

As LMIC aim to strengthen their current systems by promoting foundational literacy and numeracy, policymakers need to also be mindful of some of the potential pushback towards these policies. First, focusing on tangible educational inputs, like textbook delivery or school construction, has clear political advantages (e.g., Williams, 2017 or Ejdemyr et al., 2018) but it is less clear if politicians reap similar political rewards from focusing on education quality. For instance, Harding and Stasavage (2014) find that politicians in Kenya benefit less from electoral promises around the improvement of educational quality relative to those focusing on educational inputs, and Sandholtz (2021) finds that after a large national education reform in Liberia, politicians were rewarded and punished in their local elections accordingly with the improvement, or lack thereof, in the quality of education as a result of the reform. This trade-off may be starker when the intended policies targeted at educational quality compete for significant resources with input-augmenting policies. This, in turn, could be an advantage of policies like curriculum reforms which mostly use already existing resources in the system. Still, when deciding which educational policies are implemented, policymakers may have to balance promoting their future political careers, and developing policies that advance educational quality. The current evidence suggests that these policies, contrary to the broadly popular input-enhancing programs, can be at best a “high risk, high reward” endeavor for politicians, and in essence could weaken their motivation to pursue this type of policy.

Another drawback of policies that focus on FLN is that, as any other policy, there are opportunity costs. Especially when states have weak implementing capacity, and they can only run a few programs at a time, the focus on FLN should not distract from other important policies that are equally needed to build strong educational systems. For instance, schools in the developing world still experience high dropout rates in upper primary – clearly an issue that is just as worthy of the attention of policymakers. Similarly, as the name indicates, FLN are simply the foundation of learning in other subjects, and as such, educational systems must ensure that once students have mastered FLN, they can keep learning in the higher grades. Even beyond learning and enrollment, schools must also foster socioemotional learning (SEL) to ensure that children can successfully navigate their environment and personal relationships once they leave the educational system. In this sense, the two policies that I study in chapters 1 and 2 of this dissertation are fairly narrow, as neither focused explicitly on outcomes beyond foundational learning. Yet, there are other areas of student development that are as valuable as FLN, and therefore, policymakers in developing contexts need to carefully balance the promotion of FLN while also minding these other crucial tasks, all the while staying within their tight resource constraints – no easy feat.

Conclusion

In all, this dissertation aims to build on the body of evidence that explores ways to improve and measure foundational literacy and numeracy in the developing world. The contrast between a mostly successful and an unsuccessful curriculum reform in two contexts begins to populate a largely missing literature on rigorous evaluations of nationwide early grade reforms. Similarly, these two chapters in tandem can also shed light on the curricular features that could maximize the likelihood of success for this type of program. On the other hand, measuring FLN outcomes is the first required step towards improving them. We find that the issue of “enumerator effects” could introduce significant measurement error into the quantification of FLN in LMIC. Therefore, researchers need to carefully consider the logistical and empirical trade-offs of different approaches to assign enumerators to subjects, and the implications that this choice might have on the accuracy and precision of their results and policy recommendations. Ultimately, the creation and dissemination of research is a valuable step towards the design of better-informed policies, and as a result, I hope that this dissertation is a small move in the right direction towards effectively promoting FLN in developing contexts.