Project Activities
This evaluation uses a mixed-methods design to investigate the impacts of Transform Me enrollment on student SEL, behavior, and academic achievement. The impact analyses will use a cluster randomized control trial (RCT). The implementation analyses will help to explain program impacts and will provide information to inform program developers' support of schools. Using surveys, interviews, and case studies, the researchers will examine student and staff experiences with the program to characterize facilitators and barriers to success.
Structured Abstract
Setting
The research will take place in 29 high schools from three local school districts in the South Queens area of the New York City Department of Education). The schools include a variety of program types, including magnet schools, large comprehensive high schools, and smaller theme-based schools. The intervention will be implemented in advisory periods in each school.
Sample
Participants will include students in all 9th and 10th grade advisory periods in each school, with half of the advisory classrooms randomized into treatment and half into control. They will randomly assign a minimum of 8,750 students in 350 advisories across all participating high schools. On average, the districts are 35% African American, 23% Asian, 28% Latinx, and 76% of students are economically disadvantaged.
The intervention is a blended course called Transform Me. This semester-long course focuses on improving self-awareness, self-management, social awareness and responsible decision-making. Each of four units consists of twenty half-hour lessons that satisfy adolescents' developmental needs for autonomy, competence and relatedness.
Research design and methods
The researchers will use a 3-level randomization design. Advisories in each school will be randomized to receive Purpose Prep programming ("treatment"), or to conduct business-as-usual ("control") in the first year of the study. Students will participate in the Transform Me course in the 2022-2023 academic year. To study implementation, they will use case studies, curriculum review, and software usage data.
Control condition
The control condition will consist of advisory periods that implement "business as usual" programming without access to the Transform Me curricular materials.
Key measures
Outcome measures include state Regents Exams in Algebra and English, GPA, chronic absenteeism, suspensions, and four SEL competencies (self-management, growth mindset, self-efficacy, and social awareness) as measured by validated survey instruments.
Data analytic strategy
The researchers will use multi-level modeling with random effects for the outcomes. They will examine the role of mediating variables using Barron and Kenny's model, such as motivation, to better understand how the course affects outcomes.
Cost analysis strategy
The research team will estimate the total cost of the course considering all cost components, including resources that support the design, delivery, monitoring and sustaining of the intervention. They will incorporate information on resources required for implementation based on actual expenditures and off-budget items. They will collect similar data on the control condition.
People and institutions involved
IES program contact(s)
Products and publications
The researchers will engage in a broad dissemination strategy that is designed to reach national research audiences, as well as policymaker and practitioner audiences. This includes presentations to research partners to inform policy, practice, and future decisions about effective program implementation strategies. They will also conduct workshops, webinars, or give presentations at national conferences targeted at superintendents and school leaders. They plan to present findings at academically oriented professional conferences. They will also produce a final peer-reviewed RAND Research Report. They also plan to publish results in highly ranked journals in relevant fields.
Supplemental information
Co-Principal Investigator: Steiner, Elizabeth
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
