Project Activities
The researchers will survey principals and parents and link survey responses to student formative test scores and administrative data from the three Atlanta schools to examine the following:
- how variation in instructional mode (virtual/hybrid/in-person) and teacher characteristics (such as experience and prior use of technology) affected student achievement
- how the effects varied by sex, race/ethnicity, FRL status, English Learner status, disability status, subject and grade level
- how instructional delivery method (concurrent vs. sequential delivery to in-person and remote learners) and the degree of student online engagement (log-in frequency) during remote learning affected the incidence of student misbehavior once students returned to reopened schools
- how the combination of virtual learning experiences and subsequent recovery strategies affected longer term outcomes such as high-school dropout, high-school graduation and college enrollment.
Structured Abstract
Setting
Three Atlanta metro-area school districts that encompass both urban and suburban areas and serve a diverse set of students.
Sample
The sample includes approximately 0.25 million students, including 56 percent who identify as Black, 18 percent Hispanic, 8 percent Asian and 15 percent White.
The researchers will examine malleable program and policy features including remote learning groupings, teacher assignments to remote/in-person instruction, teacher professional development, in-person and online tutoring programs, extended learning time in core subjects during the school day, supplementary instruction after school, and summer learning opportunities.
Research design and methods
For each district recovery strategy employed, the researchers will examine the near-term immediate impacts on student achievement growth, student behavior and student attendance and how the effects varied by student characteristics, subject and grade level. In the first phase of the project, which focuses on school year 2020/21, the researchers will survey parents about family dynamics during the pandemic and at-home learning experiences and combine this information with administrative data from district partners to assess the relationship between instructional mode (in-person/virtual) and both student achievement growth and attendance/engagement. The researchers will analyze a variety of moderating factors, including home environment, student characteristics (e.g., race/ethnicity, sex, English learner status, disability status and household income) and teacher characteristics (e.g., experience, licensure status, educational attainment, and prior use of remote instruction). The researchers will also investigate mediating factors that explain the variation in student outcomes associated with remote learning, such as changes in peer group composition, simultaneous versus sequential instruction of in-person and remote learners, refugee status, and parental preferences for remote learning. The second phase of the project will use data from school years 2021/22 – 2023/24 to analyze the relationships between multiple district recovery/acceleration initiatives and both short-run (student achievement) and long-run (high school completion and college attendance) student outcomes. Sub-group analyzes will also be conducted, with an emphasis on the extent to which programs and policies promote learning and positive long-run outcomes for historically marginalized groups that suffered the largest negative effects from the pandemic.
Control condition
The researchers will use a comparison group to estimate the factors associated with differential success in remote learning by comparing the measured performance of students with similar characteristics and test score histories in the same schools who are exposed to differing levels of remote instruction. The researchers will use program eligibility criteria (for example, low test scores) to create a comparison group for estimating the relationships between recovery strategies and student outcomes. In some cases, the team will compare student achievement and educational attainment for students who just meet the eligibility threshold with those who fall just below the threshold. In other instances, the team will compare outcomes for students who receive the intervention with observationally similar students who do not receive the treatment.
Key measures
In the first phase the researchers will measure remote learning participation and engagement variables (such as days of remote instruction, attendance and extent of online logins, self-reported emotional and cognitive engagement), student characteristics (such as race/ethnicity, sex, English learner status, refugee status, disability status, household income, ), individual test-score histories, peer groups (such as sex mix and prior behavioral incidents of classmates), teacher characteristics (such as experience, advanced degrees, licensure status, and prior use of online instruction), remote learning delivery (simultaneous with in-person instruction or separate), parental preferences, and student outcome variables (such as nationally normed formative assessment scores, student behavior). In the second phase, in which the team will investigate the efficacy of district recovery strategies, they will incorporate information on program characteristics and eligibility criteria and expand measured outcomes to include high school graduation and college attendance.
Data analytic strategy
The researchers will use a variety of cumulative achievement regressions to examine the relationship between remote learning participation and engagement and student test scores as well as mediators and moderators of remote instructional mode effects. To estimate the relationship between district recovery/acceleration strategies and short- and long-term student outcomes, the researchers will employ a mix of analytic strategies, including regression discontinuity, difference-in-differences, and linear regressions with statistical controls for student characteristics.
People and institutions involved
IES program contact(s)
Products and publications
The research will yield a variety of products that will be tailored to stakeholders' needs and the timelines of decision makers. The researchers will produce research briefs for district and state leaders, which will also be shared with the public, and rapid-response presentations of research findings. The research team will create longer research reports for policymakers, practitioners, and other stakeholders and scholarly work for academic conferences and peer-reviewed journals.
Additional project information
This application was submitted to the FY 2022 Education Research Grants (84.305A) competition and awarded in FY 2023 when funds were available.
In FY23, this project joined the Prekindergarten Through Grade 12 Recovery Research Network as a research team.
Supplemental information
Co-Principal Investigators: Capers, K. Jurée; Darling-Aduana, Jennifer
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
