The Myth of the Perfect Test Date

March is the worst! Avoid it like the plague! Don’t you dare take October; that’s when the smart seniors are sitting. All the jocks take December, that’s the surest bet for a high score!

If you spend long enough in this business, you will hear chatter about choosing test dates strategically to garner a testing advantage. One of my independent educational consultant friends recently reached out to me. She heard a rumor from another mom, while sitting in the basketball bleachers, that certain tests are considered "easier" based on the students taking them at that time. She wanted to know if she should have her clients prioritize any particular date.

How wonderful would it be to have the insider track on picking the right test date! Unfortunately, in spite of the allure of being “in the know,” there is no statistical advantage in selecting one test date over another. That’s simply misinformation, being spread like wildfire in basketball bleachers across America.

I must admit that I too have been a purveyor of this particular brand of misinformation, driven primarily by having too small of a data set to draw any legitimate conclusions. Years ago, when I started keeping tabs on my students' performance, I noticed some trends. Back in 2002-2003, poring over my excel spread sheet, I noticed something. My students' March scores were abysmal, but their May scores were amazing! Going off my extremely limited data set, I advised all my students: "Take May! May is the Way!" A few years later I changed my tune, and I was proselytizing June!

These days our data set has grown considerably. As a company, Applerouth has always been results driven and data focused. In our current data set, we have over 7400 official SAT and 2900 ACT results. Our findings do not come from flimsy anecdotal evidence: we are working from a legitimate data set. And this data set quickly wipes away any notions of preferential test dates.

Is there no variance in difficulty?

Although there is no consistent pattern, year over year, it is true that certain tests are objectively harder than others, and within a given test, certain sections are much more challenging than others. Having taken over a dozen SATs and ACTs, I’ve experienced this first-hand. I have been slammed by particular science sections on the ACT, flummoxed by Math sections on the SAT. But the statistical equating process the College Board and ACT Inc. use, it turns out, is effective. The harder sections and test are balanced out by a different curve.

Additionally, if you take this test long enough, you realize that these companies recycle their content. SATs and ACTs are very expensive to create and calibrate; the testing companies are wise to get as much bang for their buck before retiring a test. When a testing form is finally administered on one of the test-return dates - October, January or May for the SAT - December, April, June for the ACT- that particular test, now externally released, leaves circulation. On the December 2012 ACT, I was given the exact test form that I had already taken in 2010. As test forms are recycled throughout the calendar year, the notion of easy and hard dates loses further credence.

The Data tells all

Individuals may give more weight to the most recent data- suffering from recency bias: "the last two October administrations sure were tough! Avoid October!" Or they may privilege emotionally charged data points: "Billy tanked on December, but went up 300 points in May! Everyone should take May!" We need to take a long view with a robust data set; only then can we circumvent these cognitive errors and come to a clear understanding of the trends.

To dispel the myth of differential test date difficulty, I’m going to lay out the full data set as an appendix to this article. But I’ll summarize the data here. All of these results are from official testing administrations, reported by our students and their parents.

Historic SAT Data 2006-2012

‍

Historic ACT Data 2007-2012

As you can see, the notions of an easy or hard test date do not bear out. The test dates are quite consistent in their level of difficulty. The appendix charts go into more granular detail. The only instances in which our data show a real swing in test difficulty correspond with a smaller than normal sample. Sampling bias in effect!

A few notes on the data
There is a slight maturation effect/test-retest effect, as students taking the early fall tests are more likely to be seniors who have already taken the test than are students taking the early winter tests. This effect is relatively weak, because for every test date, we have students taking the test for the first time, others for the final time, some having prepped with us, others having gone in to the test without any prep. Each sample is a mixture, and the law of averages wins in the end. Our sample of students pursuing test prep, by no means a random sample, is clearly skewed toward the higher end of the score spectrum.

In Conclusion
The next time you hear bleacher-parents talking about easy and hard test dates, please burst their bubble. Let them in on the big secret: it doesn’t matter which test date you sit for, simply choose test dates that make the most sense for your child, and for your child’s schedule.

Download the Data