Q&A: Amy Reilly, Director of Product Management

Amy Reilly is the Director of Product Management at New Meridian. We caught up with her in late June to talk about test design, test management and the creation of the new Start Strong Assessment.  

You are passionate about product design and management. What inspires you about this kind of work and why did you choose to design assessments?

I began working in educational assessment when the principal at the school where I taught asked me to be the testing coordinator. Having a background as a special education teacher introduced me to various diagnostic assessments and I was fascinated by what the school psychologist could learn about student strengths and challenges from these tests. I left the school district and began working with the Texas Education Agency as an ELA content specialist and have since had the benefit of working on a number of innovative assessment research projects where I learned a lot about the design process. 

What inspires me the most is the opportunity to understand and measure what students know and can do in different and unique ways that don’t always look like a traditional multiple-choice question. Being able to present results that support decision making without being burdensome is also important. So is blending assessment and learning in such a way that assessment is less of a disruption to the natural flow of instruction. To me, that is the ultimate goal.

You were instrumental in creating New Meridian’s new Start Strong test. Tell us about the process.

 We started thinking about the idea for Start Strong around the time that schools closed, realizing that without the summative assessment data or even grades, educators would be without key data points for making beginning-of-year decisions. I could see through my own three children that their online experience was vastly different based on individual teachers, course design, learning style and what our home internet connection was like day to day. It became an eye opener to me how big a challenge next school year will be. 

We thought really hard about the type of decisions various stakeholders would need to make to efficiently and effectively support students, and we let that drive the test design and product reporting features. At the same time, we couldn’t ignore that testing at home in August may be a real possibility and that the items and passages needed to be viewed through a new lens for sensitive topics. We put plans in place to address these types of issues—and to do it really, really quickly.

What are the most important considerations for educators as students head back to school after the summer?

I think there are a few critical elements of assessment that educators should consider as they plan for back-to-school. The most important is probably being thoughtful and explicit on the specific purpose or use for an assessment, and understanding the assessment design. All too often, educators become frustrated with an assessment because the design and subsequent information doesn’t support the intended outcomes or decisions that need to be made. 

For example, all stakeholders should be clear on standards alignment and the depth to which standards are measured. There is no time to waste on a poorly aligned test or one that covers content students have not yet had the opportunity to learn. I don’t think you can ever go wrong asking yourself, “is this test worth taking for the outcome I’m trying to achieve?”

It’s also important to consider the amount of time to devote to assessment, knowing there’s a trade off with time devoted to instruction. We’re all feeling uncertain about the potential for more disruptions, so there is a real need to understand where students are quickly and then adjust accordingly.

Finally, I understand that many districts are planning for multiple scenarios for returning to school. Having an option to administer the assessment at home, or simply more flexibility with the testing window, is another key consideration.

You have three children. How does being a parent impact your work?

Being a parent, as well as a former teacher, allows me to gut-check our assumptions when thinking through product ideas and features. In the spring, I was the parent who was struggling to keep up with work and my kids’ online learning. At times, I was overwhelmed by all of the learning products and tools available as I looked for resources to support my son in Spanish or my daughter with reading. It is difficult to determine what really works. I became worried that I didn’t really know how much they were learning and whether they would be prepared for next year. In talking to my friends and colleagues, I also knew that I wasn’t alone. That perspective helps me consider what I want to know about Lilly, Claire, and Jack when they go back to school and what would help me the most when trying to support them.