Assessment at the Instructional Core

“I gave it a strong 2.”

“So did I.”

“I gave it a 3. The student clearly drew from both passages and the video in supporting her analysis.”

And so began another spirited discussion of a student’s response to one of the extended, open-ended tasks on the recent spring test. More than 330 tasks were scored over five days by 78 teachers from five states and education systems, all of whom were gathered in Austin, Texas, to review New Meridian test content to be used in their state summative test designs.

I had the privilege of sitting in and observing several scoring rooms across grade levels in both math and English language arts. What I saw reminded me of why high-quality assessment is so critical to deepening students’ engagement with high-quality, challenging academic tasks.

In one of the seventh-grade math scoring rooms, teachers evaluated the accuracy and complexity of students’ reasoning in solving a problem that required them to model a real-world situation. Not only do students have to solve the problem, they have to explain their solution strategy, often using multiple versions of how the situation could be represented. The teachers discussed each student response, evaluating the depth of understanding and clarity of the explanation.

In a ninth-grade English language arts scoring room, teachers discussed their own personal experience teaching tone in their classrooms—and how difficult it is for students to grasp this nuanced concept. The teachers’ personal experience trying to teach the concept informed how they interpreted the scoring rubric and thus how they scored the student responses.

I was struck by the deep intellectual engagement, the regular reference to their own classroom experience, the familiarity with the standards, and the deep understanding of students’ development at each grade level that informed their scoring. What I witnessed was a quintessential professional event: teachers as professionals drawing on their experience and expertise to turn an otherwise lifeless test into a living, human, thoughtful evaluation of students’ readiness to meet the challenge of the tasks before them.

Reflecting on this work, I thought of Richard Elmore’s model of the instructional core, in which student learning happens where skilled teachers foster deep student engagement with challenging academic tasks. The quality of academic tasks, the knowledge and skill of teachers, and the level of student engagement define a school’s instructional core.

Students learn when they and their teachers engage together in thoughtful exploration around the challenges of meaningful academic tasks.

The instructional core was at the heart of the scoring sessions I observed: While the students were not personally present in the scoring rooms, their work was, and these teachers were engaged deeply in evaluating both the academic task in light of their own classroom experience and the level of student engagement with the task as reflected in their responses.
 
Participating in that day reminded me that it is the teachers across this country who every day seek to foster deeper student engagement with meaningful tasks that will advance student learning. And that is why teachers are such a critical part of our test design and development process. They help us ensure we are challenging students appropriately in ways that reflect what is being taught in the classroom and to a level that will prepare this generation to meet the challenges of this century.
 

Changemakers

Examining how states and systems are rethinking education, assessments and accountability to improve student outcomes

Propel America Connects Students with Opportunity—Right in Their Own Communities 

Roughly 20 million Americans were enrolled in two- and four-year colleges across the United States last year, a figure that sounds impressive until you realize that only about 57 percent of U.S. students actually graduate. More than a third never receive a credential of any kind—and the impact is profound.

Students who drop out of college are more likely to default on student loans, make less money throughout their lifetimes and more likely to find themselves in poverty, studies show. It was a cycle Paymon Rouhanifard could recognize.

“I can see the faces of my old students,” he said. “They did everything we asked them to do. But I believe the system failed so many of them.”

Rouhanifard, the former superintendent at Camden City School District in New Jersey, and John White, the current Louisiana State Superintendent of Education, decided to do something about it. The two veteran educators, who met each other years ago while working in the New York City schools, launched Propel America earlier this year.

Propel America has an ambitious goal: to bring high schools, local employers, and educational facilities like training centers and colleges into alignment in order to provide students with education and an upwardly mobile first job, right in their own communities.

“We came to understand that setting students up for success meant thinking beyond graduation rates,” Rouhanifard and White wrote in a letter announcing the organization in May. “Every year, we saw students graduate with immense potential—and every year, too many of those students floundered. They had completed high school successfully, but were left without a viable, affordable pathway to a strong first job. They had been presented with a false choice: forgo income and accrue significant debt while pursuing a traditional 2- or 4-year degree; or forgo education for low-wage work without upward mobility.”

The model that White and Rouhanifard created, which is being tested in Louisiana and New Jersey, is designed to address local employment needs and provide students with an educational path that yields a solid job in the near term and a career path that is long term.

For example, a high school graduate can train to be a Certified Medical Assistant, a job that paid a median annual wage of more than $33,600 nationwide in 2018 and garnered as much as $47,000 in some areas, according to the U.S. Bureau of Labor Statistics.

Yet, that is only a beginning. In a few years, with more education, that CMA can earn an associate degree, obtain certification and licensing and become a Radiologic Technologist, making more money still. The median annual pay in 2018 was about $61,000 and as much as $99,000 for top earners, according to the BLS.

The result is a student who received training and started with a job right out of high school, continued to pursue education as they traveled a career path in their chosen industry, and significantly increased their skills, their value and their salary along the way.

Research suggests that if colleges could increase their graduation rates from 57 percent to the rate of U.S. high schools, which is 84 percent, for even one class, the impact would be massive. It would increase wages for two-year degree holders by an average of almost $5,000 a year; increase wages for four-year degree holders by an average of almost $19,000 a year; and keep roughly 48,000 out of poverty, according to a study by the think tank Third Way.

Of course, jobs in healthcare and other vital fields are available in most communities—especially now, with unemployment low and employers in need—and most offer solid health benefits. The key is to put students on the path right from high school and support them as they transition.

“The answer is right under our noses,” Rouhanifard said, noting that there are similar paths in many industries, from building trades to information technology. “We’ve got to create a system to support these students.”

The system that Propel America created begins in high school with a semester-long course senior year designed to teach employment skills and familiarize students with local employers and opportunities.

Propel America then offers a program that connects students with training in their field, often at a community college; introduces them to a local employer who has job openings; and gives them a stipend of about $60 a day while they are in training. The goal is to provide students with a stable way to pursue an education that makes them highly employable.

In places like Camden and Pennsauken, New Jersey, and Baton Rouge and Jefferson Parish, Louisiana, Propel America has pilot programs that are attracting students. “They understand that this is a path that involves college,” Rouhanifard said. “That is the biggest selling point. It’s free and they get paid to go to school, but that’s not the primary reason (they join). The primary reason is college.”

Propel America will be training roughly 50 students in its program across Louisiana and New Jersey. “It’s small, but we’re learning a ton,” Rouhanifard said, adding that, “We wanted to get inside schools to understand what scales.”

The first cohort of students will not graduate from training until later this year, entering the workforce in the first quarter of 2020. Propel America will keep robust data on the students, tracking the credentials they obtain, the jobs they land—even metrics such as debt and credit ratings. “We believe in being data driven,” Rouhanifard said.

As Forbes observed in April, “Propel America is more a template than a program.” That’s a characterization that the organization embraces. If the pilots yield results, the model can be exported to other states.

As Rouhanifard put it, “We believe this is a template that you can bring anywhere in the country.”

Item Spotlight

A look at the unique characteristics that make New Meridian assessment items the best in the industry

Several times a year, The Prime gives readers an in-depth look at the thought process that goes into the production of New Meridian test questions and performance tasks. The goal is to better understand how test items align with academic standards to produce high-quality assessments.

The Question: In a retired Research Simulation Task (RST) developed for a 5th-grade assessment, students are asked to read passages about pandas. One of the passages is about giant pandas. After reading, the students are asked a series of two-part, multiple-choice questions. One particular question asks students to identify the two main ideas in the article and then choose two quotations or key details that support those main ideas.

The Expert: Laura Beltchenko is a literacy and educational professional from Libertyville, Illinois, one of several dozen educators involved in New Meridian’s collaborative process to develop high-quality, engaging assessment items.

The Analysis: Here’s what we asked Beltchenko and the answers we received:

What is this question designed to measure?

“This two-part question is addressing multiple ELA Standards:

— drawing explicit and inferential evidence from the text (RI 5.1);

— determining two main ideas (RI 5.2.1);

— how those main ideas are supported by key details (RI 5.2.2); and finally

— how the author identifies reason and evidence to support main ideas (RI 5.8.2).

“Two-part questions are asking students to understand the information and then use their skills as readers to support their understanding with evidence. In general, authors of informational text want the reader to get the main points of what they have to say about a topic. As readers it is up to us to determine not only the main ideas but the key details that describe the overall purpose of the article.”

How does this 5th-grade RST differ from what we expect of 4th-grade students?

“Students in Grade 5 are becoming more independent readers and writers. In Grade 4, students are expected to ‘refer’ to details and examples from the text. In other words, students may find specific evidence to support an answer and, in a written response, they may paraphrase or insert that information. However, by Grade 5 students are expected to ‘quote accurately’ from a text. This moves students into the true ‘close reading’ mode.

“As educators we must ensure that students understand the difference between what is explicitly stated and what they infer the author is saying. This becomes especially important when students are drawing evidence from multiple sources. Students need to be able to manipulate two or more resources and then gather evidence on the main ideas and key details. Quoting accurately helps students understand that the author must receive credit for their research. This Grade 5 standard also helps students understand the difference between personal opinion and researched work.”

Which aspects of this task do you like?

“First and foremost, let’s look at what the students are asked to read. What’s not to like about pandas! On the journey to creating authentic assessment, passage selection is extremely important. The passages in this RST present a main idea with similar concepts throughout. They also present visual media (photos), which provide students visual literacy information.

“Questions to ask when planning for assessment are: Is the topic of the article or video interesting to a specific grade level? Is the article authentic, meaning it is a published piece written to inform and found in a student magazine, periodical or text? What is the complexity of the articles and media? And, do the directions for the task tell the students exactly what is expected of them?

“What I admire about the task, as well as the academic standards themselves, is the way they can be broken down into measurable parts.”

Why is the question constructed as it is?

“The comprehension skills tested by an RST support a scaffolded method of building knowledge.

“The standards we use have a progression of learning expectations. Grade 3 students are asked to find and explain information explicitly. Grade 4 students are expected to explicitly and inferentially draw information from text. In Grade 5, the learning trajectory requires that explicit and inferred evidence be quoted accurately. Viewing the standard as a journey, each step on that path has students traveling closer to becoming independent learners. The goal is to transfer the information learned at each grade level and build upon it.”

Laura Beltchenko has been in the profession of educating students and educators for more than 35 years. Her career in public education includes experience as a classroom teacher, a reading specialist, and a teacher and coordinator of gifted education programs, as well as an Associate Superintendent for Curriculum and Instruction in a suburban Chicago K-12 school district.

 

Inside New Meridian

New Meridian Leaders Moderate NCSA Panels 

New Meridian is proud to participate in the National Conference on Student Assessment sponsored by the Council of Chief State School Officers (CCSSO) next week.

More than 1,000 educators, administrators, assessment experts and state officials will gather in Orlando for a three-day conference June 24 to 26. This year’s theme is, “Measure What Matters, and Create Accountability for Equity.” Participants will share innovations, research, resources and strategies to improve state assessment across 90 individual sessions.

“Conferences like this are vital to our industry,” said New Meridian CEO Arthur VanderVeen. “They allow us to share information, learn from the best and get a first-hand look at the innovation taking place. I always leave the conference feeling energized.”

VanderVeen will moderate a June 25 panel at 11 a.m. titled, “Communicating about Growth at the District Level Using Median Growth Percentiles.” The session focuses on how student growth percentiles, aggregated to the district-level, can be used to support district and state decision makers.

The panel features A. Rae Clementz, director of assessment and accountability for the Illinois State Board of Education, as well as Nathan Dadey and Damian Betebenner of the Center for Assessment.

New Meridian Chief Assessment Officer Tracy Gardner, who has more than 20 years of experience in measurement, assessment design and development, and psychometrics, will also moderate a panel.

“Implementing Pre-Equating to Allow for Faster Score Reporting to Support Assessment Utility for Educators” will take place at 4 p.m. June 24. It will explore the merits of taking a “pre-equating” approach, in which scoring tables are created prior to operational administration of an assessment, versus a “post-equated” approach, in which scoring tables are produced after operational testing.

The panel will feature Clementz, the director of assessment and accountability from Illinois, and Art Thacker, principal staff scientist at HumRRO (the Human Resources Research Organization).

The session will explain the results of a study that compared post-equated item parameters and student scores with pre-equated results for the same students and test items on the ELA assessment. Panelists will also discuss strategies for assessment literacy and the educational and political climates in their states that led to policy decisions to reduce the time between testing and score reporting.

“The conference is a wonderful opportunity to share information, collaborate and learn,” Gardner said. “We are excited to be there.”