On July 27, ERN hosted “Assessments 101: Assessment Types, Their Uses, and Building Balanced Assessment Systems,” the first in a series of three webinars that we’re calling the Assessment Bootcamp. This first session was designed to provide advocates and policymakers with a shared language and knowledge base about different types of assessments and the purposes they are—and are not—designed for and how these various assessments can work together to create coherent, balanced assessment systems.
Summative, Statewide Assessments Have a Unique and Irreplaceable Role
Stuart Kahl, President of Kahl Balanced Assessment Practices, started by noting that statewide summative assessments are essential for program evaluation improvement, “You have an awful lot of comparisons that are allowed by state tests, and the results should raise questions like: Why are we not performing as well as other schools in our state that serve the similar population of students? Why is this subgroup of ours not performing as well as the same subgroup at other schools across the state?”
Susie Feliz, Vice President of Policy and Legislative Affairs at National Urban League, noted that statewide summative assessments are about “ensuring that all students are taught to the same high standards and the statewide assessments are aligned to the states’ college and career readiness standards,” while local assessments–though valuable for addressing other needs–are not necessarily aligned to these standards.
Summative, Statewide Assessments Can’t Do Everything
Kahl cautioned against using statewide summative tests for the more fine-grained work of diagnosing specific student learning needs or directly informing day-to-day adjustments to instruction. Instead, local level assessments, such as teacher-created formative assessments or standardized interim assessments, are better designed to produce information needed for this work.
But, it’s a lack of understanding about these discrete purposes that can sow confusion about assessments—and produce data that doesn’t answer the questions we want answered—Paige Kowalski, Executive Vice President at Data Quality Campaign, pointed out, “Different assessments yield different types of results and for different purposes and the trick is understanding what you’re going for—what kinds of questions you’re trying to answer—so you can pick the right assessment instrument.”
Summative, Statewide Assessments Are Comparable Across Districts and Subgroups
Throughout the conversation, each of our expert panelists continued to highlight that it’s the ability to make apples-to-apple comparisons which make statewide summative assessments so valuable.
Kahl noted: “Test results are only meaningful in comparison to something: in comparison to a pre-established standard, in comparison to previous performance, or in comparison to some other groups…They give you an external perspective on how well your program is doing.”
Similarly, Kowalski emphasized that the “statewide summative assessment data is the only data point that a state has to look out across their own districts to do program evaluation, to understand how groups of students are doing, to understand which high schools are preparing their students for college or careers.”
This is particularly important because—through federal accountability systems under the Every Student Succeeds Act (ESSA)—schools that consistently underperform on statewide assessments are eligible for additional support and resources to aid school improvement.
Local Assessments Should Not Be Used in Lieu of Statewide Assessments
Asked about states who have expressed interest in using local assessment data in lieu of statewide summative assessments, Kahl stated that “In my mind, whether it’s for accountability or for program evaluation and improvement, that lack of comparability is a serious problem with that approach.” Kowalski added that “You could have a state with 500 districts and 500 district scores sent up to the state and they can publish that but what is that data set? What can you do with it? What questions can you really answer? And it will get misused because it’s not even apples and oranges, it’s apples and eggplants.”
The panel’s moderator ERN’s VP of K12 Policy, Charles Barone, pointed out that without statewide summative assessments, “you can have some districts look good because they set a lower bar and other districts look not as good because they set a higher bar… Without the apples to apples comparability you really can’t do the resource part of this in any effective way. You’d actually have a fairly regressive distribution of resources.”
Statewide Summative Assessments Play an Important Role in Driving Equity
Building off the discussion on comparability, Feliz very clearly highlighted the importance of statewide summative assessments in advancing educational equity, “It’s really about shining the light on academic disparities, particularly for students of color and other vulnerable students. …For Urban League affiliates and their civil rights partners on the ground, the use of standardized tests has helped reveal these long standing racial disparities in academic opportunity which has armed them with the evidence that they need to advocate for change.”
Without statewide summative assessments, these disparities wouldn’t be visible. The resulting growth data is, Kowalski notes, “our only comparable data point about performance and it is our only equity indicator on performance in existence right now.”
And federal requirements to disaggregate statewide assessment data—first included as a part of federal law in 2001—are equally important to promoting equity. “The disaggregation of data… was a critical and pivotal point in education reform: being able to see the disparities by student group by Black students, Latino students, English Learners, low income kids, students with disabilities,” Feliz stated. “Now we can see as parents and community members and advocates how well our students are performing against these statewide college and career readiness requirements.”
Assessment Bias and Systemic Racism
At the conclusion of the webinar, our panelists tackled the issue that while many view assessments as a key tool for advancing racial equity in our schools, some observers claim that these assessments are inherently racist or biased.
Kahl, who has been involved in the design of assessments for decades, pushed back on the idea that statewide assessments are biased by design. “From the beginning we’ve had multiple steps in the process of test development to ensure [cultural bias] doesn’t happen. We have bias and sensitivity reviews. But even before we get to that stage of committees looking for just those things that can lead to issues and differential performance of kids, the test developers are trained on all these factors they have to avoid in these items, so there a great deal of effort in the development process to make sure that tests are unbiased.”
Feliz pointed out that we do need to acknowledge the history of racial discrimination in education, including assessment, such as when SAT and IQ tests were used as “evidence” of Black students’ intellectual inferiortiy. But, ultimately, she says statewide assessments are a critical tool in continuing to improve our schools:
“We have to acknowledge the progress that’s been made through these important reforms in our education system…We believe that the standardized statewide tests, by shining a light on academic gaps, are intended to promote equity and hold a system accountable for educating all students…As a parent and a community member I am trusting my public school system to do right by my child, and if history is precedent then I have to do my part to hold my schools system accountable. And there’s still a little mistrust in our communities when it comes to not just testing, but the government’s role in promoting discriminatory policies that disadvantage students of color. …I think we’ve made a lot of progress and we just need to keep moving towards advancing equity in our education system.”
You can view the full recording of the webinar here.
Stay tuned for information about session two and three where we’ll discuss innovations in assessment and accountability and continue to provide advocates with the information and talking points they need to make the case for statewide summative assessments.