School leaders warn against overgeneralization, say some sub-groups may have suffered more
JOHNSON CITY, Tenn. (WJHL) – Tennessee’s Department of Education offered a stark warning last fall – data showing COVID 19’s impact on schools could produce 50 percent decreases in third grade reading proficiency and 65 percent in math.
Governor Bill Lee called it “an alarm bell.” Now early testing data are in and News Channel 11 met with two area school administrators. So far they’re pleasantly surprised.
“Although we do expect to see some decline we certainly don’t expect it to be as dramatic as was originally feared,” Washington County Superintendent Jerry Boyd said.
Boyd and Johnson City Schools Supervisor of Instruction Roger Walk both warned against reading too much into early data. It has yet to be “normed,” which accounts for outlying scores and gives systems an even more accurate view.
Both also said while aggregate data is helpful, it’s by no means the most important use of the tool.
Aggregate scores let systems compare against prior years and to other systems and the state and nation as a whole. But a more important advantage of all the data comes at the individual level.
“I think we have to be really careful about making decisions based on the big data,” Walk said. “It’s just another opportunity for us to drill down and look at individual students and their challenges and opportunities.”
But both said that from kindergarten to the all-important third grade year (the first for TCAP testing) up through high school ACT scores, teachers and students appear to have weathered COVID remarkably well, academics-wise.
“We found that our kindergarteners in the spring of ’21 had a slight decrease in performance based upon their kindergarten peers from the spring of ’19,” Walk said.
Those internal, nationally standardized tests for kindergarteners weren’t performed in 2020.
They compare how well students know their letters, the sounds the letters make, phonemic awareness.
“It was about a drop of about 7 percent as a district but what we found within that is there were schools actually that improved,” Walk said.
Impact also appears small in the data-critical third grade year when students take their first TCAP tests.
“Looking at that data, that raw data comparing it to previous data of the same kind, we don’t see any significant decrease,” Boyd said.
Raw high school ACT scores also show small declines at the systems.
“We anticipate there could be a slight drop in our composite number for that,” Walk said. Johnson City’s 2020 average composite score was 22.7.
“Just looking at the preliminary numbers I think it’s 21.5, in the upper, maybe 21.8.”
Walk said it’s hard to determine whether the expected decline is due to students not receiving all the information they needed in the classroom. Some national test centers that allow kids to take tests multiple times didn’t operate during the summer.
Boyd said the county also is seeing a drop. But he sees a silver lining even for the students who graduated in 2021 after losing practically a year of in-class instruction.
“Even though those ACT scores for the graduating class might have dipped I think they’re certainly going to be able to build off the strength they were able to attain just through going through the adversity they went through this past year.”
Concern at the individual and subgroup level
Boyd and Walk both think last year’s data may show some students in subgroups whose test scores traditionally lag the average have fallen further behind.
Those include economically disadvantaged students, Black and Hispanic students, students with learning disabilities and English language learners.
“We found the same thing,” Walk said. “Those groups are still struggling in their performance relative to their peers.”
But it’s not a one-size-fits-all situation, Walk stressed.
“You have to be very careful about making those broad decisions as a grade level or as a whole group,” he said. “When we do drill down to the results themselves we see vast differences.”
Especially for students who are struggling academically, the Johnson City system brings teachers together monthly and looks at all the bits of data for a particular student to determine what’s best for that student, Walk said.
Walk said some early elementary students who were virtual had an enhanced experience.
“Every day they had an opportunity to have an adult at home sitting beside of them who was to there to help them and guide them and give them extra assistance almost like an educational assistant for that student.”
Often, though, socioeconomics played into that.
“We have younger students who were home with an adult who couldn’t manage that for whatever reason, whether there was a younger sibling that they had to attend to, and so it varies.”
Boyd said his biggest academic concern was the inability to offer the traditional suite of “wraparound” services for students.
“When they weren’t there I think this past year it was certainly highlighted that those services are needed and all of us as we have a chance to get back to what might be called normal we look forward to that.”
Boyd said families realize their responsibilities and want to meet their children’s needs. It just isn’t always that simple and the pandemic made it less so.
“Our families are struggling too to overcome the past 18 months,” he said.
“As an organization serving our community we have the expertise, we have the experience doing that and when we can get back to a point where we have daily access in a more normal way with our students we can certainly support them academically and then on the non-academic, the mental health side as well.”
Boyd echoed Walk’s assessment that the more at-risk subgroups are likely to have suffered the most – at least in the aggregate.
“There’s always that gap and our role is to close that gap.”
Walk said it’s tough seeing students and families that were already challenged face possible greater levels of learning loss. He said parents did the best they could.
“But some students at home did not have the same opportunities as their peers who were also learning at home, so those are the students we’re going to look at and give as many extra resources in the coming years as possible.”
But he said struggles were by no means limited to students from specific groups or of specific academic abilities.
That included high achieving students. “For whatever reason they might not have performed at the level at home, and to get down to why is very complicated.”
When all the data settles, Walk said benefits from the data have to be folded into a more complex recipe for success and improvements.
“The numbers themselves are important but also just the anecdotal information we hear from our teachers and our parents and our students.
“All of that comes together with the numbers to help us make decisions as far down close to the student as possible.”
For those seen to have struggled most, the system hopes to address that challenge by steering federal stimulus money to what he called opportunities.
“We’ve been given a great deal of resources and extra funds with which to address learning loss, and so for us it’s an opportunity to devise plans and programs … for those students we identify at that level to have extra time … to be exposed to the educational opportunities that they’ve missed over the past 15 months.”
