Assessing ELA Curriculum Shifts: A Practical Guide for Measurement and Progress Monitoring
IN PARTNERSHIP WITH:
Across the country, districts are making big bets on the promise of high-quality instructional materials (HQIM) and professional learning (PL) offerings to support curricular shifts.
How will we know whether these efforts are working? Right now, districts, PL providers, and states are all using different tools to gather data even when they are trying to measure the same things. By coming together, we can enhance data consistency and completeness to better track these efforts.
This toolkit offers consensus recommendations on a series of key measures — compiled by PL organizations and researchers — that we can all use to monitor progress on curriculum implementation in English Language Arts across grades 3-12.
This toolkit supports PL providers, district leaders, and coaches who want to track implementation progress and PL quality across the first several years of ELA curriculum shifts. The measures here have been selected by the Research Partnership for Professional Learning (RPPL) in partnership with PL provider organizations and researchers as a set of common metrics that can be collected across multiple locations to improve progress monitoring and comparison across multiple systems and organizations. The measurement tools can be used either as a comprehensive suite or tailored to specific needs (i.e., use cases), including in-depth evaluations of overall curriculum implementation, monitoring ongoing progress, supporting cycles of improvement, or identifying and learning from bright spots in the work.
Get the latest on Toolkits, Instruments, and more
Sign up for the newsletter
Key Areas to Measure When Assessing ELA Curriculum Shifts
School & System Conditions
HQIM implementation is supported by and integrated with existing infrastructure
Effective implementation doesn’t happen in isolation. It requires alignment with district values, structures, and resources to ensure that new materials can be fully adopted and used to their potential.
Professional Learning
PL engages teachers in opportunities to build core skills for implementing HQIM
High-quality PL is essential for teachers to build the skills they need to use HQIM effectively. We focus on key elements of PL—guided practice, opportunities for reflection, and targeted feedback—that evidence suggests can help teachers implement new instructional practices with confidence. One enabling condition of successful curricular shifts is that PL supports use of student data and (supplementary) materials to adapt for student needs and identities to engage all students in rigorous, grade-level tasks.
Instructional Practice
Teachers implement HQIM with integrity while adapting to their students' needs and backgrounds
HQIM is only as effective as its implementation. While fidelity to the curriculum is important, teachers must also have the flexibility to adapt materials in ways that resonate with their students’ diverse needs and backgrounds, maintaining the balance between consistency and responsiveness.
Teacher Beliefs & Mindsets
Teachers see a core part of their work to be preserving the integrity of the curriculum while also addressing students’ needs and identities
Teachers believe that HQIM can be a key tool for maintaining high expectations for all students while simultaneously incorporating their individual needs and identities.
STUDENT SOCIAL-EMOTIONAL OUTCOMES
Students have positive learning experiences
Students learn best in an environment where they feel that their identities are respected and affirmed. We include measures of social-emotional outcomes like belonging and affirmation to ensure that HQIM implementation fosters both academic and personal growth.
STUDENT ACADEMIC OUTCOMES
Students master grade-level content
At the heart of HQIM is the belief that all students, regardless of background, can master grade-level standards when provided with rigorous, well-designed materials. We focus on ensuring that students are not only exposed to grade-level content but are actively engaging with it in meaningful ways.
Overview of Recommended Tools
The full Assessing ELA Curriculum Shifts Toolkit includes 58 individual items that are administered via six data collection instruments.
School Leader Survey
Recommendations:
- Minimum 70% response rate
A survey administered to school and district leaders to understand how well HQIM is integrated with existing infrastructure
PL Observation
Recommendations:
- Minimum 70% response rate
An observation checklist completed by an observer to understand the extent to which PL provides opportunities for guided practice, reflection, and targeted feedback
Teacher Survey
Recommendations:
- Completed by 100% of teachers in attendance at PL session
A survey administered to teachers to understand how well HQIM is integrated with existing infrastructure and to understand teacher perceptions of the PL they receive
Classroom Observation
Recommendations:
- Before each observation window starts, observers practice rating examples together before beginning observations
- PL activities are randomly selected for observation to ensure a representative sample
- During each observation window, each selected PL activity is observed on two different occasions, each by a different observer
- 5% of all PL observations are rated by two raters at the same time to determine whether different raters are making similar observation choices
An observation rubric completed by a classroom observer to understand how teachers are implementing and adapting HQIM and how students are engaging with ELA content
Teacher Pulse Check
Recommendations:
- Before each observation window starts, observers practice rating examples together before beginning observations
- Classrooms are randomly selected for observation to ensure a representative sample
- During each observation window, each classroom is observed on two different occasions, each by a different observer
- 5% of all PL observations are rated by two raters at the same time to determine whether different raters are making similar observation choices
A brief survey administered to participating teachers at the end of an individual PL session to understand their experience of that session
Student Survey
Recommendations:
- Completed by 80% of students in attendance when administered
A survey administered to the students of teachers that participate in PL to understand students’ sense of belonging and engagement with ELA content
How Was the Toolkit Developed?
This toolkit represents consensus recommendations developed by the Research Partnership for Professional Learning (RPPL) working with a set of key organizations all focused on building the conditions for more effective curriculum-based PL.
We are advancing educational equity for every student, including those who have been historically pushed to the margins of our education system. Together, we study teacher learning to identify, share, and enact PL that improves teachers’ practices and students’ classroom experiences, well-being, and academic growth.
We bring together scholars to tackle persistent challenges in education with a focus on reducing educational inequities. Through applied research training, engaged scholarship, sustained partnerships, and efforts to generate actionable knowledge, our goal is to improve the creation, synthesis, and application of educational research.
We provide actionable support to educators. For nearly 20 years, we’ve partnered with schools, districts, and systems to create classrooms where every student can thrive.
We work to fundamentally shift the paradigm of teacher PL for educational equity.
We partner with school systems to build and sustain the conditions, teaching, and leadership to ensure that the students furthest from opportunity succeed in school and in life.
We strengthen instructional leadership in schools, school systems, and states to ensure teachers have the support they need to improve learning experiences and learning outcomes for students—with attention to students of color, students experiencing poverty, multilingual learners, and students with disabilities.
We work to end the injustice of educational inequality by providing excellent teachers to the students who need them most and by advancing policies and practices that ensure effective teaching in every classroom.
We empower educators to eliminate the predictability of student outcomes by race, language, and socioeconomic status through evidence-informed, engaging, affirming, and meaningful grade-level instruction—so that all students succeed academically.
Tool Selection Process
The toolkit is a result of a year-long consensus process that balanced academic rigor with practical feasibility.
The toolkit is a result of a year-long consensus process that balanced academic rigor with practical feasibility.
Working group members identified a series of shared constructs and sub-constructs that each organization considered central to its theory of action about how high-quality instructional materials and curriculum-based PL could improve classroom outcomes.
The group conducted a review of potential measures for the constructs used in research and practice, culminating in a comprehensive literature review that identified over 4,000 articles, reports, and briefs and focused on a close reading of more than 500 selected sources.
Research leads in the working group conducted an initial screening for relevance, quality, and feasibility. Then they created a shortlist of 4-8 tools per sub-construct, attending especially to content alignment, psychometric evidence, and usability in practice.
Participants began by individually reviewing tools by sub-construct, assessing the strengths and weaknesses of each one. Then, the group collaborated to build recommendations from the short-listed options, selecting tools that would effectively meet measurement needs while maintaining a manageable number of items that could be used across different formats and respondents.
Recommendations for Use
Comprehensive Annual Calendar
We recommend offering baseline surveys to school personnel and to students as the year begins, tracking PL quality across the course of the fall and winter, observing classrooms in the spring, and administering final surveys near the end of the year but before end-of-year testing.
SAMPLE CALENDAR
Use Cases
Below, we outline a few potential use cases and highlight the data collection tools that will be most useful depending on the use case.
Use the toolkit for in-depth diagnosis
Use the toolkit for ongoing monitoring
Use the toolkit to facilitate cycles of improvement
Use the toolkit to identify and leverage existing bright spots
In-Depth Diagnosis
Purpose: Obtain a detailed picture of HQIM implementation over time and identify potential intervention points for improvement
Measures to Use
- All measures in the full toolkit
Administration Schedule
- Administer during at least three timepoints over the course of the school year
Key Inferences to Draw
- How is HQIM implementation changing over time?
- Where are the key points for improvement in the system?
- How does HQIM implementation vary across groups (e.g., schools, grade levels)
Ongoing Monitoring
Purpose: Obtain a high-level snapshot of HQIM implementation
Measures to Use
Administration Schedule
- Administer at a single point during the school year to develop a snapshot of HQIM use
Key Inferences to Draw
- How is HQIM implementation proceeding overall?
- How does HQIM implementation vary across groups (e.g., schools, grade levels)?
- How are teacher perceptions of PL related to teachers’ implementation of HQIM and to students’ learning experiences?
Cycles of Improvement
Purpose: Track the results of targeted improvement efforts
Measures to Use
- Select 1-2 measures aligned to a key lever for change and select 1-2 outcome measures aligned to the selected lever for change
Administration Schedule
- Administer within improvement cycles (at least 2-3 times within a school year)
Key Inferences to Draw
- How do changes on the key lever identified within an improve- ment cycle relate to changes in an immediate outcome of interest?
Bright Spots
Purpose: Identify and leverage existing successes to inspire, guide, and sustain effective improvement efforts
Measures to Use
- All measures in the full toolkit
Administration Schedule
- Administer once in contexts that have been identified as bright spots
Key Inferences to Draw
- How does HQIM implementation look in places that are experiencing success?
Comparing Data Between Organizations
Coming Soon …
Sign up for the EdInstruments newsletter to get the latest updates on Toolkits, Instruments, and more.
SICS
Andrew J. Thayer et al., “Construct Validity of the School-implementation Climate Scale,”
- All items from the School Leader Survey
RPPL Mindset Study
Hannah Carter, Heather Hill, Brendon Krall, and Nate Schwartz, “Mindset Study Report,” RPPL, December 2024.
- All items from Foundational Supports and Leadership Expectations in Teacher Survey
Survey of Enacted Curriculum Items on Collaboration
JeanMarie Farrow, Sarah Kavanagh, and Preeti Samudra, “Exploring Relationships Between Professional Development and Teachers’ Enactments of Project-Based Learning,”
- Items #2 and #3 from Collaboration in Teacher Survey
RAND AIRS Survey
RAND, “American Instructional Resources Survey (AIRS), Spring 2023,” https://www.rand.org/education-and-labor/survey-panels/aep/surveys/items/american-instructional-resources-survey-airs-spring-2023.html.
- Item #1 from Overall Value in Teacher Survey
Soine & Lumpe
Karen M. Soine and Andrew Lumpe, “Measuring Characteristics of Teacher Professional Development,”
- Items #1 - 4 from Specific Supports in Teacher Survey
- Items #1 - 5 and #10 from Active Learning in the Classroom in PL Observation Checklist
Collaborative Professional Development Survey (CPDS)
PMR2, “Collaborative Professional Development Survey (CPDS),” n.d., https://www.pmr2.org/_files/ugd/3d8a27_3ec8dcf6b85e49bdb9d59ac0f00a6c08.pdf.
- Item #1 in Teacher Pulse Check
HQPD Checklist
Amy S. Gaumer Erickson, Patricia M. Noonan, Jennifer Brussow, and Kayla Supon Carter, “Measuring the Quality of Professional Development Training,”
- Items #1 - 3 from Feedback in PL Observation Checklist
CRIOP
Kamau Oginga Siwatu, “Preservice Teachers’ Sense of Preparedness and Self-Efficacy to Teach in America’s Urban and Suburban Schools: Does Context Matter?”
- Items #6 and #9 - 11 from Instructional Practice in Classroom Observations
BCI
Jamaal Matthews et al., “Belonging-Centered Instruction: An Observational Approach Toward Establishing Inclusive Mathematics Classrooms,”
- Item #7 from Instructional Practice in Classroom Observations
E-Squared Core Question 3A
Student Achievement Partners, “E2 Instructional Practice SuiteTM - Student Achievement Partners,” August 30, 2024, https://learnwithsap.org/e2/.
- Item #8 from Instructional Practice in Classroom Observations
IPG
Student Achievement Partners, “Achieve the Core: Instructional Practice Guide,” n.d., https://achievethecore.org/page/1119/instructional-practice-guide.
- Item #5 from Instructional Practice in Classroom Observations
- Items #1 and #2 from Student Outcomes in Classroom Observations
Panorama Student Survey
Hunter Gehlbach and Harvard Graduate School of Education, “Panorama Student Survey | Panorama Education,” August 2014, https://www.panoramaed.com/products/surveys/student-survey.
- Items #1 - 5in Student Survey
- Items #1 - 5 in Student Survey
Created Internally
- Items #1 and #4 from Collaboration in Teacher Survey
- Items #2 and #3 in Teacher Pulse Check
- Items #6 - 9 and #11 from Active Learning in the Classroom in PL Observation Checklist
- Items #1 - 4 from Instructional Practice in Classroom Observations
- Item #1 from Engagement with Content in Student Survey