As part of our work on the independent evaluation of the GEC II, the team measured girls’ learning losses in Kenya and Nepal in March 2021. This was compared to projects’ midline data from 2019 to estimate changes in girls’ learning during the prolonged school closures.
The impact of school closures on the girls in our sampled schools has been stark. We find that girls have suffered very large learning losses in both maths and reading in both countries. The magnitude of these losses suggests substantial support will be needed to ensure they can resume learning and successfully progress. These losses are sadly not surprising given the length of time girls were out of school and the low levels of reported time spent studying during closures, with a quarter of girls in Kenya and 14% of girls in Nepal reporting not studying at all. Of those who did study, only a small share studied every day.
The findings suggest that many of the challenges and barriers that existed prior to COVID-19 have been exacerbated by the pandemic. There were substantial challenges reported by girls on remote access to learning, particularly when it required technology. They also reported being relatively uninformed. Many of these girls, all of whom were in secondary school, remained unaware of learning initiatives launched by government and supported by projects and schools. Where they could study, they mostly used their textbooks, with many girls reporting issues or challenges in trying to engage with radio or TV lessons to learn.
Kenya confirmed its first case of COVID-19 on 13 March 2020. Schools closed shortly afterwards, only re-opening in October for exam grades and then fully in January 2021. This impacted learning for over 18 million children1. Nepal was a similar story, with schools closing in March 2020 and reopening in stages from September (those in rural areas with no Covid-19 cases), with most schools fully opened from the end of January 2021.
Whilst schools were closed, girls received varying levels of learning support from the GEC II projects in their areas.
EDT in Kenya worked with Community Health Volunteers to visit households and raise awareness on government-supported televised and radio educational programmes. Local radio talk shows were also run by the project to guide parents and caregivers on how to support girls’ learning at home, in addition to tutorials, homework and tests that were shared with parents to support girls in continuing with their learning. Some households were also provided with light-emitting solar radios to help girls listen to radio lessons and study after dark.
Mercy Corps in Nepal supported girls with distance learning classes delivered through local radio stations. Girls preparing to sit their Grade 10 secondary education examinations received radio classes in English, mathematics, science and Nepali to enable them to continue studying during school closures.
To measure learning loss before and after school closures, we compared reading and maths test scores for girls in February to March 2021 with their scores in 2019. We returned to the same schools that projects visited as part of their midline evaluations2 to ensure comparability, selecting girls3 in the same grades at random. We find (statistically) significant learning losses in both Kenya and Nepal, in both subjects.
In Kenya, we find a large fall in learning levels – for those of you who are technically minded, we measured this as 0.37 standard deviations (SD) in maths, and 0.86 SD in reading. In Nepal, learning fell even more - by 0.72 SD in maths and 0.88 SD in reading. For context, the target for improvement in the previous GEC projects was 0.25 SD over the control group, suggesting substantial support will be needed. Historically, learning gains pre-COVID-19 were quite small between grades (e.g., in Nepal, there were between one and four percentage points difference between girls in Grade 9 and girls in Grade 10). So, the relatively much higher learning losses due to COVID-19 (18 percentage points in Nepal) suggests a great amount of time spent learning is required to catch up, unless instruction becomes more targeted.
These averages mask large differences across schools, and in some schools, we actually found learning levels had increased. We found this for 20% of schools for maths in Kenya, and 11% of schools in Nepal. There does not seem to be any correlation between changes in learning level and geographic region (e.g., urban versus rural areas). In Nepal, we saw greater declines in the more difficult tasks, while in Kenya the pattern was less obvious.
We find girls struggled to access learning opportunities. Girls spent very little time studying. In Kenya only around one in three girls studied every day during school closures, although one in four did not study at all. In Nepal more girls studied – but only 58% every day, and 13% of girls reported not studying at all during closures.
The girls were also asked how else they used their time. Surprisingly, in Kenya, girls reported less time on household chores or other caring responsibilities during COVID-19, but they were more likely to have helped with the family business than before. In Nepal, in addition to helping out more with the family business, girls were more likely to report having to spend time on caring activities, agricultural work and fetching water during school closures than before.
The findings related to high levels of learning loss matter, as there is a high likelihood (already realised in some countries) that schools will close again. The low use of time spent studying, and girls’ experiences, provides an opportunity to reflect on the modalities of support during closures, many of which are one-way and rely on the assistance of adults in the home (who may not have been to secondary school themselves); and girls’ continued motivation to study and learn at home. In simple terms, as time drags on with schools closed, girls increasingly reported losing motivation from not having the learning support they needed. This also worried headteachers, particularly when the uncertainty of when schools would reopen increased. Learning losses were not uniform though, showing that learning during school closures is possible.
Regression analysis revealed that the only factors that were significantly associated with learning were the availability and usage of reading materials at home, including textbooks, either with the help of parents (in Kenya) or by reading from textbooks on their own (in Nepal). This suggests that traditional ways of delivering information – textbooks and through household members – might actually be the simplest pathway. It also means that ensuring schools and children have sufficient textbooks which they can take home in a crisis (as opposed to sharing books between students, which can be common for cost savings reasons) can help build resilience to shocks like COVID-19.
Even where schools remain open, the learning losses mean that schools need to recognise that many girls need the chance, and support, to catch up. Simply resuming the curriculum where children left off assumes that children have not forgotten their existing knowledge, which our evidence suggests might not be the case.
For full access to the Access and Learning Report click here.
For further reading about how we used machine learning to predict girls at risk of dropping out from school click here.
++++++++++++++++++++++++++++++++++++++++++++++++++++
1. State Department of Early Learning and Basic Education, May 2020, Kenya Basic Education Covid-19 Emergency Response Plan
2. These were assessments conducted by projects’ external evaluators in 2019 and included learning assessments to compare changes in learning levels since the start of the project (baseline).
3. Kenya: 4,402 girls spread across Forms 1 and 2; Nepal: 1,470 girls in grades 9 and 10 plus 113 from the panel of girls