Cheating Scandals: Why the ACT and SAT Need to Adopt Computer Adaptive Testing

For years we’ve known about the widespread international cheating scandals that have rocked the SAT and ACT, leading to cancellations of scores from entire countries. It’s a problem that has been well-documented by the likes of Reuters and other news agencies. South Korea has been singled out for massive breaches in test security: in one instance the ACT was cancelled for every student in the country. In April I had the privilege of speaking to a colleague who lives and works in Seoul. He explained the grim details of the culture of cheating that presently exists in his home market. His detailed account of cheating in Seoul demonstrates the necessity of moving away from recycling test forms and making the move to computer-adaptive testing.

In April I interviewed Charles Ahn, an experienced tutor and educational consultant, who expressed tremendous frustration regarding a broken system in South Korea where all those who choose to play by the rules are at a profound disadvantage. Rampant cheating is made possible by the business decision of the College Board and ACT, Inc. to recycle test forms. These test forms are used in different markets, across different time zones, for multiple years, until they are retired during one of the three annual test return dates. Both the College Board and ACT Inc. typically have several dozen active test forms in circulation at any one time.

The problem of cheating due to recycled test forms is by no means limited to the SAT and ACT. According to Ahn, cheating also takes place with the SAT Subject Tests, the TOEFL and AP Exams. Any test form that is recycled across markets and time zones is vulnerable. He mentioned the unique problem of foreign nationals attending boarding schools in the US. Many students spend time doing illegal test preparation in their home markets during winter or summer break, and then return to boarding schools in the US where they will take their SATs, ACTs, and Subject Tests with an incredibly unfair advantage.

ACT Cheating Spiked After the SAT Redesign

For decades, the SAT was the primary focus of tutors and tutoring shops willing to use illegal and unreleased tests for preparation. There were dozens and dozens of SATs being spread around, until these old tests were rendered worthless when the new SAT arrived in 2016. The absence of released tests drove many students towards the ACT. At that point, the ACT had achieved greater international penetration, and the dishonest test prep providers turned their attention to the ACT. As the SAT is now in its 2nd year post-redesign, more and more compromised test forms are appearing on the internet and in the hands of dishonest operators, and students looking to gain an unfair advantage are returning to the SAT.

The Thumb-Drive: Cheating Made Easy

Ahn revealed that many dishonest test prep providers in his home market have thumb drives with some 30-40 ACT unreleased test forms that they use to prepare students. These test forms remain in active circulation, and any one of these forms could appear at any time when a student goes to sit for an official ACT.

Tutors will use this cache of tests to prepare students for the official tests. With enough repetition and exposure, most students can memorize or become supremely familiar with the 30-40 tests. When they go to sit for the official testing administration, if they receive a test they have not memorized, they can immediately cancel that score and take the next test administration. Eventually they’ll hit a test they have memorized, and their performance will be effectively guaranteed.

Shadow Operations Are Prevalent

Ahn spoke of how shadow operations have infiltrated the test prep culture to the extent that even “official” and supposedly regulated entities are getting involved.

Ghost Rooms

The “ghost room,” a shadow approach that is apparently used with some frequency, is an underground cheating operation that emerges and disappears before anyone can take notice of its existence. A “tutor” with thumb drive in hand will travel to a location and find 8-10 students whose families are looking for a quick and easy path to a high ACT score who have the resources to pay for it. Everything about this operation is secretive. Marketing is entirely word of mouth: no advertisement, no website. The tutor will rent out a quiet and inconspicuous, occasionally windowless, apartment room or studio, rented under a different name, totally off the radar.

The tutor prints up dozens of copies of the exams from the thumb drive. None of the printed material will ever leave the ghost room. For the month students will meet there to drill and memorize the stolen exams. They will meet seven days a week, sunrise to past sunset. Ahn spoke of some tutors who hold kids until 2:00 to 3:00 AM. And then the students are back at 9:00 AM to begin again. Rote memorization all day long. Take a test. And then take a clean version of the test again. A student might take the same practice exam 15 times to lock it into memory.

When the month of memorization is up, all of the paper copies are destroyed and shredded. The place is packed up, and the “tutor” moves on to a new location to find another 8-10 kids. This makes it incredibly hard to police. As soon as there’s a lead, the operation is wrapped up and on its way to a new location and set of students.

Hagwons

The official tutoring academies of South Korea, the hagwons, also play a role in this illegal preparation. Some official hagwons, regulated by the government, run underground programs to escape scrutiny. Other entities register as information centers or research centers to avoid the strict regulation imposed upon the official academies. The regulated academies cannot stay open past 10pm, but those with a different type of registration don’t have that limitation and can run at all hours.

The Economics

There is a range of economic options for test prep in Seoul. Legitimate test prep without the use of illegal tests can be found for $10-$12 per hour in large classrooms with 50-60 students. Other parents secure a spot for their children at a hagwon for $2,000 for a month, or a more exclusive academy for as much as $6,000 for a month.

What the illegal operators offer is time. Ahn said that many of the affluent families care more about time than money. A student could spend $10k for 3 months of preparation over the summer. Or that same student could spend the $10k for 1 month of memorizing unreleased test forms, and get that other 2 months back for some other activity. Efficiency is the primary selling point. The family will get time back for the summer, and the tutor will clear $20,000 for the month, after all expenses are factored in.

Securing Unreleased Tests

Ahn mentioned that many of the dishonest providers using unreleased tests are using the exact same set of tests, and he proposed that these test forms are likely coming from the same brokers who aggregated the tests.

These brokers don’t have to work that hard to break the security of the College Board or ACT, Inc. They employ teams of students willing to make a few bucks. A team of students will take a test, with each student responsible to memorize a few questions. One student will use a hidden camera to photograph the diagrams and passages. Everything is sent to an aggregator, uploaded, and the test is recreated. In some cases, brokers can glean info shared by students on Tumblr, College Confidential or other sites. Ahn said that some of the PDFs of unreleased tests in circulation are flawless copies, while others are a bit rougher with snapshots taken from cell phones.

It is imperative that the College Board and the ACT, Inc. address this issue and secure unreleased test material. As Reuters reported, the inaugural version of the new SAT, first administered in March 2016, was circulating in Asia in advance of its official administration there. This story is not new, it is simply ongoing. Ahn also mentioned the administration of the April 2017 SAT, which was posted online before it was administered internationally the following month.

Focus On The Verbal

Ahn mentioned that the verbal sections of the tests are of particular interest in the international market, and therefore particularly vulnerable to the cheating practices that take place. The underground prep shops essentially ignore the Math and Science sections of the ACT because English and Reading are the key sections holding most of the students back from getting the score they need they need for elite US schools. The drilling is designed to yield a “just right” score of 34 or 35, as a 36 in English or Reading might raise a red flag.

The Challenge

For those who seek to play it fair and offer honest tutoring services, the fact that flagrant cheating exists is profoundly troubling. Ahn spoke of his personal challenge in Seoul: “These are my competitors: how am I going to go up against the guy who has the tests if I want to make money?” Many students have the mindset: if this tutor has the tests, then I’m going with him; he must know what he’s doing. There’s also an idea that Ahn has heard in his home market: “It’s not cheating unless you get caught.” This unfortunate viewpoint and the rampant cheating that it spawns is demoralizing for Ahn, as it is for the students who refuse to cheat. Ahn was worked with students who are frustrated with the state of things: “I spend three months studying and score a 1480 on the SAT. My neighbor spends one month prepping and gets a 35 on the ACT. We need to solve this.”

The only fix, and the hard fix, is to abandon the practice of recycling test forms and reusing them across time zones. Computer-adaptive testing, with massive problem banks, too large to memorize, is the only way to avoid this kind of practice. Even if the ACT shifts to digital testing, and even if they mix up sections from different tests, students who memorize the 40 tests in circulation will have a significant unfair advantage over honest test takers.

We need hundreds of test forms or problem banks so full of items that no student could ever memorize them all. The ACT is investing in software to allow it to generate a limitless supply of math problems, and this is a step in the right direction. Artificial intelligence may ultimately help us write a near limitless supply of verbal items. We need to do much more to ensure the security of our tests and the validity of the scores. Of course there will always be people who will find new ways to cheat any system, but we need to make it much more difficult to accomplish this. It’s far too easy given the current circumstances. ACT, Inc. and the College Board have plenty of work ahead of them, but they must do that work to ensure there is fairness and validity in college admissions testing.