Last spring, New Meridian hosted a dozen chief state school officers at our annual convening on assessment literacy, which focused on assessment as a lever for equity and change. Interestingly, the topic that prompted the most interest this year was risk and how to manage the many complex components of a state testing program.
Statewide testing is a complex, large-scale operation for every state. Assessment companies take extraordinary steps to ensure every student has an opportunity to demonstrate his or her mastery of the state’s learning standards in ways that are fair, consistent, unbiased, and appropriate. Testing has become even more complex as assessments have moved online; computer-based testing allows students to demonstrate their knowledge and skills in ways that better reflect how they will process information and solve problems in the real world, but such systems are complex, and things inevitably go wrong.
Several high-profile incidents over the last two years have either delayed reporting of test scores or even invalidated statewide test scores entirely, resulting in a significant waste of resources and time, causing frustration among students and parents, and raising questions as to whether the state can competently manage its budget and operations. Chief state school officers are keenly aware of these responsibilities and rightly concerned about how to reduce the risk associated with assessing students with high-quality tests.
Some states are addressing this problem by separating test design and development from test administration, scoring, and reporting. Texas recently issued an RFP that broke out its testing program into 10 separate components, with test development separated from test administration. The state took this approach—unprecedented in its history—explicitly to attract innovative new approaches from a broader pool of providers, beyond the handful that can manage the massive operational scale required to administer a statewide test to 3.5 million students annually.
The states in the Smarter Balanced Assessment Consortium have always taken this approach: SBAC designs and develops the assessment in cooperation with its member states, and states procure their test administration vendors separately. Christyan Mitchell, chief operating officer at SBAC, says that such an approach has enabled them to focus on quality rather than having to invest in large-scale operational capacity.
Specialization like this is a proven approach to managing complex development processes. In the construction industry, you have architects and builders. A good architect will propose solutions and develop a design that reflects a client’s vision and goals. They will consider innovative developments in construction technologies that builders may not be aware of. (In assessment terms, this would be similar to considering an array of possible item and test designs, including technology enhanced items, multimedia-based items, state-of-the art accommodations, fixed test forms, item-level or multistage adaptive test forms, etc.). They will also help the client anticipate and manage the implications of their design for the construction phase in terms of cost, technical challenges, materials availability, and timeline. Finally, they will act as the client’s agent throughout the construction process, making sure the building is constructed to spec and performs as intended.
Having a partner that specializes in test design and development—like an architect—dramatically reduces risk and brings greater peace of mind. “It’s one of the best decisions we ever made,” notes John White, state superintendent of education for Louisiana. “We have greater control over the test design and a very responsive partner who knows our goals and priorities.”
Finally, as in construction, architects and builders must work as a cooperative team on behalf of the client. Tests must be designed and developed to function flawlessly in the administration vendor’s system. The test developer and the test delivery vendor must work together to help the state evaluate results and ensure technical quality. As states move toward greater specialization across their assessment partners, new models for cooperation and coordination will be required.
As a test design and development architect, New Meridian works with states to adopt this new approach to ensure they are getting the highest quality assessment program for their significant investment of tax dollars. We work closely with every major test administration vendor to ensure seamless coordination across workflows and systems. Ultimately, our goal is to help states develop the highest-quality assessment for the best value to provide educators, families, and students with the information they need to keep learning and growing.
