Buying Time: ACT abandons aggressive timeline to implement Computer Adaptive Testing

The ACT, Inc. has postponed its plans to migrate all international test takers to a Computer Adaptive Testing (CAT) format. The move, initially slated for this fall, will now take place at a later date, likely over a year from now. The announcement of the delay came as something of a relief to students, counselors and those in the test-prep industry who were concerned because the ACT, Inc. had not yet released prep materials, software demos, or implementation plans for the impending new digital testing platform. CAT is coming: to the ACT, to the SAT, and eventually to most high stakes standardized tests. It is a more efficient, more personalized, more secure form of assessment, and it will have its day, albeit a later day than planned.

Delays of this nature are not uncommon when a major change to testing is in the works. The College Board had initially planned to release its redesigned SAT in March of 2015, but did not actually launch the test until a full year later. The ACT, for its part, has been forecasting a move to CAT testing for years. In 2013 we learned that the target date for computer adaptive ACTs would be 2015. Then that was pushed to 2016. In July of 2016, the ACT, Inc. announced that starting in the fall of 2017, all overseas students would take computer-adaptive ACT tests and the paper option would be retired outside of the US. Observers who believed this timeline was too aggressive were right.

Pushing back another year is a smart play for the ACT. A rocky rollout for a flagship test would have real world consequences. The ACT, Inc. experienced such consequences when when it moved too quickly to digital testing with its ASPIRE product. The ASPIRE launch was plagued with early implementation issues, glitches, reporting issues, and consumer complaints. This led to a rapid drop in state-wide contracts for the testing giant and, a few years later, the ACT is now trying to recapture the ground it lost by offering a paper-based Pre-ACT product. There is wisdom in having a measure of caution in changing testing formats.

If the ACT moves too quickly with its digital product and releases the software before the support systems are in place to faithfully implement the new test, students will quickly migrate to the SAT. Now that the tests are so closely aligned, there is less room for error: customers can vote with their feet, and if they are fed up with one test, they’ll quickly switch to the other.

The reason the ACT initially felt compelled to move quickly to CAT is simple: test-security. Every new overseas testing scandal threatens the validity of the assessment. There’s a spillover effect. Test security breaches and cancellations in Egypt, Saudi Arabia, Hong Kong, China, and South Korea fuel the the growing perception that these tests are easy to cheat and therefore unreliable. Over the last 18 months, the overseas cheating scandals have been coming fast and furious for both the SAT and ACT. Going digital is the natural solution.

We already have a working platform for CAT. Look no further than the GMAT or the GRE to observe CAT in action. And the the ACT is no stranger to these platforms. Their team has 20 years of experience with CAT through its other products and its partnership with the GMAT, which has been offered digitally for years. But creating a digital platform for the ACT and the infrastructure to support it is something different.

The ACT has the resources to build a large enough problem bank for adaptive testing, and it has the psychometricians to norm the test and create reliable testing scales. The challenge lies on the ground, at the testing sites. Implementation issues are most likely what stalled the rollout of the Computer Adaptive ACT.

On-the-ground logistics for the GRE or the GMAT are easier. Students go to test centers at their convenience, thereby distributing demand. Some students test Monday, others Tuesday, others on the weekend. Having hundreds of testing dates decreases the need for computers and personnel. The ACT, however, is currently sticking to its model of fixed test-dates which creates major logistical challenges. There are large test-centers filled with hundreds of seats ready for paper-based tests, but these seats are not as easily made ready for CAT. Preparing the hardware- the banks of testing-computers, the servers, the security of the network will take some time.

And once the ACT solves these problems overseas, the CAT format will come to the US. The announcement that overseas CAT will be delayed to September 2018, at the earliest, most likely means that any kind of rollout in the US will be several years behind. The ACT will want to troubleshoot the system overseas (a relatively smaller market for the ACT) before it puts its domestic market at risk.

CAT will eventually come to dominate the testing landscape. It solves the issue of question exposure, and prevents test forms from being easily copied and shared. As long as the testing giants recycle paper test forms, which they currently must do to serve students across all the world’s time zones, there will always be a risk of cheating. The College Board and ACT have been increasing their security, but cheaters will find work-arounds. Students take a test in California, within minutes, students in Singapore can learn the answers on that test. An employee in China breaks the seal on a test-packet and, with the touch of a smart phone, students across a wide network can have access to the answers on that test. The only solution is to abandon the recycled test form, which means either dramatically increasing the investment in new tests, and writing hundreds of official tests per year, a prohibitively expensive proposition, or moving to the more efficient and secure model of CAT.

The ACT will lead, and the College Board will follow. Paper-based standardized tests will, someday, go the way of the slide rule. Dynamic, engaging, personalized digital assessment is the future. It’s only a matter of time until it arrives.