Yves here. Finally, a solid study on how much children’s education suffers during lockdowns. The answer is a lot.
By Per Engzell, Researcher, Leverhulme Centre for Demographic Science, University of Oxford, and Swedish Institute for Social Research, Arun Frey, PhD candidate in Sociology, University of Oxford and Mark Verhagen, PhD student, Leverhulme Centre for Demographic Science, University of Oxford. Originally published at VoxEU
School closures have been a common tool in the battle against COVID-19. Yet, their costs and benefits remain largely unknown. This column estimates the ‘learning loss’ that occurred when Dutch schools closed for eight weeks, using national exams that took place just before and after lockdown and similar data from previous years. On average, students lost out on a fifth of a year’s worth of learning. Losses were especially marked among those from disadvantaged homes.
School closures during the first wave of the COVID-19 pandemic raised concerns about how well children were learning out of school, and which groups were most likely to fall behind. According to the UN, some 95% of the world’s school population have been affected by now, constituting the largest disruption to education in history (United Nations 2020). Many observers have pointed to the potential harm in terms of learning progress, and the increased care burden on parents (Burgess and Sievertsen 2020, Ilzetzki 2020, Kuhfeld et al. 2020, Moroni et al. 2020). In countries like England, Germany, or France, leaders have opted to close bars and restaurants while letting schools stay open when going into a second lockdown. In other countries, stimulus packets are being directed at the economy while children are still being kept at home.
As the debate rages, reliable evidence on the costs of school closures has been hard to come by. There are several reasons for this. Data from schools are usually not collected and disseminated in real time, unlike data on economic activity, hospitalisations, or fatalities that have dominated the debate on the impacts of the pandemic (Chetty et al. 2020, Dong et al. 2020). Another challenge is inferential: test scores can fluctuate from one year to the next, so valid inference ideally requires data from before schools closed and after they reopened. We also need a relevant comparison group, and information to ensure that the two are similar in relevant respects. For example, if advantaged families keep their children from going back to school, average test scores may decline even if there is no loss of learning.
In this column, we provide new information on how children’s learning suffered when primary schools closed for eight weeks in the Netherlands (Engzell et al. 2020). We meet the above challenges by drawing on exceptionally rich data and nationally standardised tests that took place just before and after lockdown. This lets us implement a difference-in-differences design where we compare progress during school closures to that occurring in a normal year. We adjust for selection bias by matching on a large set of observables, but also with fixed-effects models that compare students within the same school and family. The Netherlands is especially interesting as a ‘best-case’ scenario with a relatively light impact in the first pandemic wave, a short lockdown, and world-leading rates of broadband access (Di Pietro et al. 2020).
Figure 1 Estimated learning loss during spring 2020 by parental education under various specifications
What happened when Dutch schools closed? Despite a light lockdown and excellent infrastructure for remote learning, our results are dire. Figure 1 shows that, across a range of specifications, primary school students lost on average 3 percentile points in the national distribution relative to a normal year, equivalent to 8% of a standard deviation. Measured as progress in a typical year, this implies that students lost out on a fifth of a year of quality-adjusted schooling (Azevedo et al. 2020). As this corresponds almost perfectly to the time that schools remained closed, the upshot is that the average student made little or no progress while learning from home. Worryingly, losses are particularly concentrated among students from less-educated homes – here, the learning loss is up to 55% larger than among their more advantaged peers.
Are these results a temporary setback that schools and teachers can eventually compensate? Only time will tell whether students rebound, remain stable, or fall further behind, but if no action is taken, small losses can cumulate into large disadvantages with time. What we do know is that these losses are in fact due to students learning less during the pandemic, and not due to unfamiliar testing environments. Performance on school tests that do not assess curricular content also declined, but to a much lesser extent than student progress on curricular subjects such as maths, reading, and spelling. This discrepancy suggests that actual knowledge learned is the main explanation for why students are performing worse when returning from lockdown than in a normal year.
To what extent are these learning losses driven by school closures over other detrimental implications of the pandemic? Exposure to stress and anxiety also increased during the lockdown, while children’s parents were more likely to face job loss and sudden economic hardship. The pandemic presents a unique historical event which makes it difficult to address these questions directly with data. Nevertheless, these concerns are arguably much smaller in the Netherlands than elsewhere – the country only suffered mild consequences during the first wave of the pandemic, and the government pursued a famous ‘intelligent lockdown’ relying on voluntary cooperation and allowing ordinary life to continue as much as possible (Tullis 2020).
As large parts of the world are heading into a second wave of the pandemic, it is vital to know how school closures impact on students’ progress, and be aware of the disproportionate damage to students from disadvantaged homes. With new school closures currently being discussed as a way to combat the pandemic, our study gives crucial input for decision-makers. The results are sobering: even in the ‘best-case’ scenario of a short lockdown and good infrastructure for remote learning, students learned little or nothing from home. Overall, these results highlight the urgency of meeting students’ educational needs and implementing measures to compensate for the progress already lost.
See original post for references
29 comments
Comments are closed.
Almost all of Donald Trump’s cabinet-level appointments were wretched, but there was one appointment that I really liked — namely Dr Scott Atlas for Covid affairs.
Dr Atlas has been opposed to school closings from day one. His interview with Peter Robinson in June is still worth reading…..
https://www.hoover.org/research/doctor-scott-atlas-and-efficacy-lockdowns-social-distancing-and-closings
Atlas feels there are no dangers to children from schools, at least no more danger than the annual flu seasons. Teachers who are over 55 are at some risk, but they could be reserved the online jobs that will be necessary.
I do not feel that the damage from school closings is really horrid yet, but we cannot keep closing them for every wave of Covid.
While there may be little risk to the children themselves, they can spread contagion by bringing COVID home to their extended families.
And the teachers can get infected from children who carry the virus.
I suspect online learning can eventually be better than in class for learning theoretical stuff, including things like math. Worse for labs though. It’s just that the education system was unprepared to transition to online learning, so what it ended up was not an optimal form.
Better for what age? I have a kindergartner and a second grader, and online learning is pretty useless for them – they need to learn socially, not just through abstraction.
I am most curious how online learning could eventually be better than in class learning — especially for theoretical stuff like math.
I imagine a tutor working with a student one-on-one probably works best as learning and teaching situation. A small classroom adds a useful social dimension, especially for socializing younger children, and that might offset some of what is lost in replacing the tutor with a classroom teacher. Perhaps a mix of both might be most suitable for the early years in developing the potentials of our children. I believe many problems with the classroom model come in as administrators push the classroom system toward analogies with a Taylorist-factory floor.
I suppose the giant lecture halls and limited section reviews that had characterized ‘higher’ education might offer some but too often too little advantage over online learning — particularly as I remember some most unremarkable teaching from the podium [but also remember a few remarkable lecturers who still teach me as I recall and apply what they taught me — an advantage no online learning system I’ve ever seen can claim].
But to end — I strongly agree with the quote from Einstein about education:
“It is a miracle that curiosity survives formal education.”
I would expect the well educated homes to be populated with the parents/guardians that can flex their schedules and navigate the tricks of online education. I am in one of those homes.
More difficult to measure perhaps is how each school system adjusts to teachers totally transforming how they educate in a tightly compressed window. Here in Massachusetts we went all remote with little rigor in the spring and very little support for teachers. The semester was a punt that largely just kept the kids connected to the school and let them see peers on streaming meetings. And I’m a fortunate home/community where this could largely happen.
In the fall we have mixed responses all over the state. My kids are in hybrid learning, they’re able to attend school 2 days a week. K-4 students are able to attend school 4 days a week, soon to be five. But communities around me are still fully remote and planning for hybrid or have gone back to remote after large high school parties are discovered and inadequate contact tracing is not possible. It’s a mess.
My strategy is to help the kids as much as possible, but for the duration of this to also get them reading more independently. I’m not too concerned about a perceived loss during this unprecedented time, but more with them being able to remain engaged if they go back to remote. It’s a model that just doesn’t work for anybody concerned.
Awhile back someone awarded “Guillotine Watch” status to a handful of parents who were trying to hire a teacher for their young children.
I didn’t understand the guillotine then, still don’t. Who knows what sacrifices the parents were making? But why would it matter, at least their priorities were straight.
Anyone who’s had kids knows how fast they can backslide in a summer, kids with learning challenges all the more so.
Now there’s also a big slide in socialization at critical ages. Often people wearing masks walking by won’t even look at kids and nod, and I see trepidation in kids’ eyes.
Meanwhile, I’ve a “kid” in social work grad school and one in undergrad art, now confined to online rigors, including field work. It’s not the same. They’re missing much of the college experience now, including another stage of socialization. Their schools haven’t reduced costs a cent. It ain’t easy all around. I hope the razor finds mercy.
It’s because most people can’t afford an expensive tutor, and if you can’t, your kid is stuck with the crapified learning provided by the public school system.
My question is, if so many people think their kids are getting a better education with an online tutor than with the online courses the schools are offering, why don’t the schools adjust course? And I do realize that the public schools are often constrained in what they do but mandates from state legislatures, but what’s stopping the legislatures from stepping up to the plate? What did we elect these clowns for in the first place?
if I’m thinking of the same post, it’s because the mother in question was seeking a nanny of Poppinsean stature who would care for their children for no pay, and in addition would have to pay room and board as well (or something equally ridiculous and tone deaf, I’m going by memory here and we might be thinking of different things). That was certainly guillotine-adjacent.
No, not the same post.
Details have floated, but basically it was a small group of parents wanting to hire a teacher for in-person teaching of their children, probably at one of their homes or backyards. No room and board in the deal. Beyond salary, I think they offered to throw in a sweetener of some sort, perhaps a debit card or gift card to some fancy product store. Probably a decent deal for a teacher who was cut loose or who didn’t want to brave Covid Elementary.
Of course they are. My kid has the worst of all worlds program – in school two days a week where it’s more likely to contract covid, and then “distance learning” for the rest of the week where despite the millions our district has spent on technology over the years, now that they actually need to use it they are only able to keep children engaged on a very limited basis. On two of the three ‘distance learning’ days they keep my kid busy for about 2-3 hours, and on the third day even less.
You can’t fit in a full year’s curriculum when you’re teaching less than 60% of the normal amount of time. But the PMCs who run my district are hoping that my understanding of math came from the current 21st century obfuscatory teaching methods and perhaps I won’t notice that 60 is less than 100.
It doesn’t have to be that way though. In talking to coworkers from around the country, I’m finding that some of their kids are doing full time distance learning that keeps the kids engaged from 8:30 to 3:00 every day. I’m sure the ability to teach properly still takes a hit, but at least they’re putting in the effort and at least trying to keep kids engaged for the same amount of time as if they were in the classroom.
I was one of those kids who had the attention span of a gnat. Especially when I was bored in school, and that happened a lot.
The problem? I don’t learn well when I have to sit at a desk and listen to someone talk. And talk. And talk.
I think that I would quickly chafe under the online-only system of education. I’d hear birds calling in the trees outside the Slim family home, and outside I would go.
The number of experienced teachers leaving the profession is staggering. If a teacher has reached retirement, many are taking it. A friend has an elementary school aged child doing online. Teacher had 25 years experience at top public elementary school in the city. She resigned last week. 4 experienced teachers leaving at Christmas break at my teacher friend’s high school. Lost several in August before school started. I know 2 teachers with 25 years+ experience leaving at end of this school year. Had not been planning that prior to doing online teaching this year.
What happened in the Netherlands does not have a lot of relevance to the United States where our policies to mitigate the spread of the virus and return students to schools are not under centralized control. Because I have a background in educational research and testing, I looked at many of the references in this and related papers. Some are dated well before 2020 and produce nothing not already known to educators and educational researchers.
Perhaps the most important problem in this and related studies is that scores on standardized tests in two subjects are aggrandized as indicators of learning and these misleading indictors of learning tell us not much about learning except– how well students have learned to take these tests and usually with the benefit of long hours of test prep. This is certainly true in the United states where federal laws governing public schools require such tests.
More recent studies, like the one from the Netherlands, are interesting only in this respect: a similarity to calculations in the United States performed by economists and with scores on standardized tests in math and reading used to calculate a statistical fictional called days (or years) of learning gained or lost. These estimates have no relevance to the reality that students who attend school or participate in online or homeschooling have varied daily time distributions for study and they encounter far more to learn than reading and math. I have some familiarity with Dutch tests from several decades ago. These national tests had video segments, were heavily reliant on a knowledge of Dutch history, literature, scientific and technical accomplishments, and the arts–including popular culture. The tests were placed into the video stream where adults could test their own knowledge.
This article begs the question: Is home schooling bad for children?
I’ve lived in states where a sizeable proportion of people home school for various reasons. In one state, it was because of the closure of rural schools and the need for children to be bussed for 3+ hours a day. In another city I lived in, people home schooled to keep their children away from the prevailing religion – others homeschooled so that their children would only be exposed to their religion. And so on.
I’ve searched the net and so far I can find no evidence that home schooling hurts children – in fact, they seem to do as well or better on the standardized tests that are supposed to measure how well our educational systems are doing.
Wikipedia pretty much sums up all the info – and I purposely ignored all the home school propagandists – that I could find.
https://en.wikipedia.org/wiki/Homeschooling
It is also interesting that the Netherlands is opposed to home schooling – might that have some effect on how the data was interpreted?
https://en.wikipedia.org/wiki/Homeschooling_international_status_and_statistics
Personally, I have grandchildren that are home schooled right now because of Covid and grandchildren that go to school every day. My home schooled grandchildren are doing as well as my grandchildren in schools – one grandson even gets his trombone lessons online and he’s going to be quite the musician when this is all over!
There are a lot of articles wanting schools opened, but I think that is because of cultural and economic bias, rather than what it does to our children. Sure, some on-line learning is horrible because Covid came upon us too rapidly for the schools to adapt, but it will get better as time goes on if we don’t beat this disease.
What is happening right now for millions of children is not home schooling.
Home schooling is a parent or parents making a huge commitment to taking over all of the responsibility of a complete schooling experience. Many if not all go beyond the average curriculum and include nature, ecology, conservation, biology, and lots of global studies.
What is happening now is maybe 10% of that.
Home schooling is probably fine but it can’t work if parents have jobs to go to everyday. Not many families are lucky enough to have a single income that is able to support them so that one spouse can teach. And not everyone is cut out for teaching.
…..good infrastructure for remote learning????
It took at least three weeks before schools offered some e-learning/ e-courses.
Both my kids (11 and 13) have not been in a classroom since early March. They are by my choice in remote only education. Almost all their friends are now back in “hybrid” education or full-time school. I have a choice to put them back into hybrid/full-time starting in February. I’ve told my wife I would like them back in actual school.
I think both their social and academic progress is impeded by online only. They are not as engaged in school in remote only and frequently forget assignments. Basically, one child is on the couch all day learning, and the other is in his bed. I hear bits of their classes. Their teachers are generally good to excellent. But my son’s 2nd year Spanish class seems like a mess. I have yet to hear that teacher speak any Spanish to them. There is no person-to-person interaction whatsoever in that class. I am about ready to recommend that my son retake the class in his first year in high school. He’s also in a “Robotics” class without any robots.
Also, the boys don’t see any friends except on the weekends or through their video games. It is kind of sad, and 10.5 months of “online only” schooling is about all I can subject them to even though I still fear they could bring the virus home to me or my wife. I think the schools are doing a “ok” job of mitigating the threat through full masking, so I hope to convince my wife to let them go back to school next February.
I definitely believe the results of the Dutch study and think periods longer than 8 weeks likely would result in larger losses.
My high school junior is currently on-line only. We are hoping to go to hybrid in January. It is a real mixed bag. Some classes are fine like his AP US History Class taught by a recent “Teacher of the Year.” The Teacher is very engaged and comfortable with technology and my son is interested. However, his AP Physics class is a disaster. Teacher almost 70 years old and very uncomfortable with the on-line format and technology. Kids are telling her how to fix issues with her camera or her computer, and of course, no labs to help illustrate the concepts. I’m super disappointed with it.
As a positive, The school has started athletic practices back up 2X/week with social distancing and my son is enjoying doing soccer ball drills outside in the sunshine with his teammates.
It is all very sad, and I know that for the most part my son isn’t getting the education he would have by physically being in school.
What I have been advocating is reopen schools with at least 2 college grads for each classroom and the regular teacher attending remotely. This removes most of the virus risk, gets the grads involved, experienced, and is a big benefit for the kids.
Annecdotal experience here: my three grandchildren, aged 13, 9 and 8.
Although she doesn’t like distance learning, the 13 year old is doing well. She was a good student to begin with, she is old enough to be part of a circle of friends who are also good students and their friendship goes beyond classroom presence. So she has a ready made support group that kibbitzes over the lessons.
The 9 and 8 year old are so close in age, they’re almost like twins. But they are in different grades and being separated in school and, with the social support in school. they are at somewhat loose ends trying to learn at home. Both their parents are working at home and, so, have some control. But the reality is, the two younger kids are struggling.
I would think that vouchers will expand greatly — so that parents can take their tax money and send their child to a private school that does stay open.
This will devastate the teachers’ unions in some locales, but it does seem like some unions have “asked for it” by putting their own interests way above the good of the children.
Your comment takes my breath away. My first impulse is to wish you good fortune in your desire to expand sending children to private schools where they can attend in person. Given the characteristics of the Corona flu, your proposal might reduce the population of those who might share your opinion [the flu spreads through super-spreaders and through close contacts — like family] — which might not be a bad thing. On reflection though, I am embarassed and chastened by my initial impulse — it violates my principles and my value of human life and well-being.
Your acid feelings toward teachers unions seems odd when I consider how little teachers’ unions have come to the support of their rank-and-file. You should be far more concerned about wild-cat actions and sick-outs. Have you forgotten why unions were tolerated and so carefully domesticated by Big Money? Your contention that teachers put their own interests “way” above the good of the children through those teachers’ desire not to get the Corona virus repulses me. Have you given no thought to how much the spread of the Corona virus might be helped by in person teaching and the aggregation of students in a classroom?
Remote ‘learning’, at least for my 7 year old, is a complete farce. She’s gone from loving school to actually hating it: “the teachers just talk and talk and we don’t learn anything” she says. I said, well that sounds like a lot of college lectures I’ve been in!
But let me set the real stage:
I am a single parent, so she goes to the Boys and Girls club where a lot of kids go to do remote “learning”. There, they mask up. They sit together, do their classes together, play together during breaks, eat lunch together, play again together, do projects together.
What do you call this? It’s called school! They are doing it there, so why can’t a school do it? It really is farcical
Our school system has been infected by Neoliberalism. Children — Humankind — develops at different rates. The powers that controlled our school systems before the Corona pandemic assumed that students are like beets or carrots and should progress and mature emotionally and intellectually within a prescribed growing season — allowing for some very limited variance. Those who don’t are culled and cast aside. The powers that control our school systems assume the acquisition of ‘education’ can be measured like the temperature under the awning at noon or the amounts in their bank accounts at the same hour. All that is needed is the proper test, just as all one needs to measure temperature or money is a reliable thermometer or bank statement. These ideas about learning and education were flawed before the Corona pandemic but grow glaringly false on inspection in context of the pandemic.
Humankind — both child and adult — learns at its own speed. Some of us can learn more quickly, some more deeply, and some a little more slowly but adequately when given sufficient time and nurture. Neoliberalism offers and can offer very little in the way of nurture. Does the Corona pandemic and remote learning — online learning — damage education? I believe it does and much more than damages education and learning. Tied to online-“learning” It damages curiosity and independent thinking which were once things included in the meaning of the word ‘education’.
If there is any real desire to minimize the impacts of the Corona pandemic the educational system must forego its ties to the Taylorist model of learning. Many students will be behind but can and will catch up given the time. Education and learning cannot and SHOULD not be optimized like a factory assembly line.
Well said!
My two bits, as a teacher trainer, teacher, and professional student of some years.
The study shows that on-line learning is a failure, proven to be so by the formal assessment of what the student understands and can do. (This assessment is the acid test of learning, it ain’t learning if you can’t prove what you learned.) Why on-line learning fails is due to 3 areas that affect a student’s learning: 1. the pre-requisite skills for learning (the ability to see that something is to be learnt, to be able to focus on the task, see how to respond to errors, etc.; 2. more innate factors such as personal commitment, curiosity, confidence, etc.; & 3. the physical requirements of learning-the horizonal and vertical space needed (desk, white board, shelves), an area under physical and emotional control-no interuptions, no items being randomly moved or replaced, etc. It is a scandal that schools, long expert in the design and implementation of the physical environment of teaching and learning, does nothing to help their students or their parents (and relatively little for their teachers) to set up a learning / teaching environment outside the physical bounds of the school.
Long story short, the US’s “on-line teaching” is good for keeping students and teachers in some kind of educational rhythm, but for true learning of content and skills, that will have to be done face to face at a later date.