A Harder SAT: College Board release foreshadows the most challenging test to date

For those of you searching for a more challenging college admissions test, look no further than the new SAT. In late December, the College Board writers released the most comprehensive problem set to date drawn from the new SAT and PSAT, providing some “flesh and bones” to the test specifications released last April. The level of difficulty and sophistication of numerous test items was surprising. If this practice set is an accurate reflection of the new test in development, it will be hardest SAT we've ever seen, and significantly harder than the ACT.

This test will mark the arrival of Common Core assessment on a national scale. Adhering to the Common Core State standards, test writers are delivering more advanced content and shifting to more sophisticated question types. While success on the current SAT is predicated upon a student’s ability to solve math, reading and writing items, the new SAT seeks to measure fluency and gauge a student's deeper understanding of tested content. The latter approach will yield a more complete picture of a student’s comprehension, but it makes for a test that may scare off many potential customers.

Reading and Writing

While the Writing section of the new SAT looks remarkably like ACT English, the reading level of the passages is more advanced. The new Reading section includes passages more typically found on the harder, more complex SAT Literature test. The political science and literature passages, in particular, would challenge students who were not highly fluent readers.

In order to gauge deeper comprehension, 3 of the 24 reading items don’t just test a student's ability to spot a right answer, but require a student to explain Why that answer is correct. For these new “show me the evidence” questions, students are first asked an inference question and then asked to identify where they found support in the passage for their answer. These question pairs will be especially tricky for students: miss the first question, and you’ll likely get the second question incorrect in a “domino effect.” These items are also more time intensive than traditional line-reference questions because they require a student to scan multiple areas of the passage.

To our surprise, and counter to David Coleman’s assurance that the SAT was moving away from the advanced vocabulary that has been a hallmark of the SAT for nearly a century, a significant number of “SAT” words remained on the new SAT, appearing throughout both the passages and the questions. Words like "acuteness," "partisan," and "empirically" appeared in correct answer choices, revealing that after nearly 90 years of privileging advanced vocabulary, the College Board may have a hard time saying goodbye to old friends.

The passage evoking the “Founding Documents and the Great Global Conversation” demanded a great deal of external knowledge. The Constitutionally-inspired passage requires a more nuanced understanding of the U.S. political system than many American or certainly international students would possess. Lacking knowledge of the Federalist Papers, the political party system in the U.S., or the branches of government, an international student would be at a serious disadvantage.

Given the elevated textual complexity, ongoing presence of advanced vocabulary, and time-intensive “command of evidence” question types, there is no doubt that this new Reading test is harder than its equivalent on the current SAT or ACT.

Optional Essay: harder, but better

The “optional” essay with its focus on analyzing the manner in which an author uses logic, structure and rhetorical skills to build an argument is markedly more challenging and analytical than the current required essay. I’m willing to go out on a limb and declare this new essay superior in every regard to the current SAT essay. On this new essay, students will shoot for a maximum score of 12, with a possible 4 points each for Reading, Analysis, and Writing. A profound departure from today’s SAT essay, this essay is better suited to assess a student’s readiness for college-level writing and analysis.

Math: Awaken the slumbering giant

Of all the sections, the one that was the most surprising, and even a bit shocking, was the math section. Emphasizing math fluency and conceptual understanding, the math questions required students to model real-world situations with complex formulas. Typically tests like the SAT, ACT, GRE and GMAT assess math skills in a particular way, requiring a student to solve given equations. A student can usually solve most math problems on these tests using an accurate set-up, methodical problem solving, basic heuristics, and occasionally some logic or creative techniques. These new Common Core inspired question types move us into a new realm, beyond the ability to solve, into the ability to understand the underlying math fundamentals of particular real-world scenarios.

In particular, the calculator-free section was the most radical departure from anything we’ve ever seen on the SAT or the ACT: it was something brand new. Questions in this section require a different method of perceiving a math problem. Students who are well-versed in the logic and language of the current SAT may, in fact, struggle on the new SAT because of their training. If you are looking for the standard set up–find the relevant info in the problem, structure the work left to right, top to bottom, and solve– you may be scratching your head, wondering how to proceed when facing these new kinds of problems. Students trained on the current SAT will have to unlearn strategies to succeed on the new SAT, eliminating the prospect of transitioning seamlessly between the current and new SAT. Take note, sophomores!

This new math content reveals a new paradigm for assessing math skills. Reflecting the Common Core standards and their emphasis on fluency, this test is far more conceptual than the current SAT. Word problems are everywhere, and students will need to step back to understand what the variables and the constants signify. For many of the questions, the ability to solve a math problem is subordinate to one’s ability to “read” the problem and understand the function of the various components of the problem. This is math fluency. Examples of these questions include: What does X stand for? How do you interpret the constant 25 in the equation? What does the negative term represent?

Other items that require a solution take many more discrete steps and significantly more time than do typical SAT problems. Additionally, students must now, for the first time, wade through extraneous information—and distractors—to find information relevant to the question.

Finally, the new SAT math content is much broader in scope than its predecessor. Students will be accountable to know more definitions, equations, and math structures on this new test. The variety of new concepts covered by the two new math sections was unprecedented: deep trigonometry knowledge, deep planar geometry knowledge and much more. I felt like I was taking a high school Algebra II/Trig final rather than taking the SAT. The redesigned math section also featured language that may be unfamiliar to current SAT test takers: margin of error, line of best fit, coefficients, constants, smooth curves, and “solutions” to quadratic equations. Extrapolating from the depth and variety of content covered in this small sample of problems, students might be accountable for knowing over a hundred new math concepts, well beyond the 40 or so discrete topics assessed on the current SAT.

If the College Board writes a complete test at this level of difficulty, it will need a very forgiving curve to round out the normal scoring distribution. A student might be able to miss numerous problems and still attain a high score. This would be in stark contrast to the current SAT, in which a student could miss a single problem and drop from an 800 to a 750 as was the case on the November, 2014 SAT.

More time per item: more of a necessity than a gift

While I was initially excited by the prospect of having more time per question on the new SAT, assuming the College Board was backing off processing speed, I now realize that much of the benefit from the additional time will be negated by the new, harder problem types. The new kinds of questions will demand a greater time investment. Replacing speedy sentence completions with cumbersome command-of-evidence questions will put the focus back on processing speed. Turning quick math problems into lengthy, complex word problems will have a similar effect. Though this test still seems like it will be gentler on processing speed than the race that is the ACT, the benefit will not be as substantial as I had once hoped, given the changes in question type. Students have more time per item, and they will need it.

A taste of online testing

For those interested in the rise of online testing, the College Board developers made some significant strides towards an era in which these tests can be taken on any manner of digital device: tablet, small laptop, large screen desktop. Instead of releasing a PDF or static webpage to disseminate the new items, the College Board created what is essentially an app to take the test. Our head of Software Engineering was impressed by its functionality, particularly the reading passage line-references that are adaptive to a student’s screen size. Shrink the screen, and line 10 becomes line 12 in the passage and the questions. The College Board has clearly been investing time and resources in this new product. We can anticipate digital testing coming soon from the College Board, and digital practice coming through the Kahn Academy partnership.

A Common Core

The College Board had no choice but to change, facing shrinking market share to an ACT that was objectively better aligned with the Common Core and better meeting the demands of the marketplace. The College Board embraced the challenge of deeply integrating the Common Core into its flagship test. All of the stated learning objectives for each new SAT item in this problem set align almost perfectly with the published Common Core standards. The SAT has now been reconfigured as a Common Core test. And I’m honestly conflicted about it.

On one hand, I’m at heart a Common Core apologist. I’ve been defending the Common Core for several years, citing our low national scores on the Program for International Student Assessment (PISA), and witnessing the proportion of high school students entering college needing to be funneled immediately into remedial classes. Our educational system is in need of repair, and I believe accurate assessment and accountability are key components of the solution. The Common Core State Standards take steps to give us an accurate measure of student academic preparedness. We need good assessments, just as we need a good curriculum, proper training and effective implementation. All of this will take time, and there will be bumps in the road.

As the College Board is vying to make the SAT into a Common Core assessment, it will inevitably reveal skill deficits in many of our students and school districts. The fact that a Common Core assessment may reveal shortcomings in the academic preparedness of students is not a reason to discard the assessment. Shooting the messenger to conceal a deficit is far from optimal.

So Which Test is Best?

Do I believe the better-aligned SAT will give us a better prediction of college preparedness? Yes, I do. Based on my knowledge of the skills required to succeed in college, and viewing the Common Core standards as a decent measure of those skills, this test seems like a better yardstick of preparedness. We’ll certainly have lots of new data by 2018 to verify this hypothesis. Despite this, given my obligation to help students find the most expedient route to college, and viewing the alternatives- the current SAT (through January, 2016) and the ACT- as easier pathways, how can I in good faith guide students to a harder assessment? My inner idealist and pragmatist are in conflict.

Though I can foresee top students thriving on this harder test, many of our students will struggle with this harder content. And as long as college admissions offices are willing to accept the SAT and the ACT on equal footing, I will be inclined to guide many of our students towards the current SAT (while it’s available) and the ACT. If the data bears out that the new SAT is, in fact, a better predictor of collegiate success, and college admissions offices come to perceive it as such, we will certainly adjust our position.

Broader Implications

As we at Applerouth are grappling with how to counsel our students regarding the changes coming to testing, every SAT/ACT tutor and every college admissions counselor in the country must come up with guidance for their students who will soon be facing the new SAT. Some of my colleagues are advising all of their students to avoid the new SAT and migrate en masse to the ACT. Many others have adopted a wait-and-see approach. The College Board, anticipating some attrition, is actively soliciting larger contracts to offset some of the individual losses. Just last week College Board reps cut a state-wide deal with the Michigan Department of Education, guaranteeing that every junior in the state will take the new SAT. Budgetary concerns, particularly savings of $15.4 million, were a major driver of that decision, but others cited the superior Common Core alignment of the new SAT as a meaningful decision factor. If the price is right, more states may follow this path.

The ides of March: waiting for confirmation

Our present analysis is based off of the problems that we have been given by the College Board. The redesigned SAT will feature 52 reading, 44 writing, and 58 math problems. For our analysis, we were given only 24 reading, 22 writing, and 48 math problems, 61% of a full-length test. From our limited sample, we cannot accurately predict the full breadth of content that the College Board will include on its new test. This test is still a work in progress. The poorly calibrated difficulty levels of many test items were one indication of that. The College Board is still figuring this test out, and it could certainly change by the time we see full practice tests in March 2015 and official administrations in March of 2016. We’re going off of the data that we have, but know that this test is clearly evolving.

Potential Implications and timelines

Assuming the new SAT aligns with the content we've seen in this current sample, and the writers are not merely showing off their hardest possible material, we anticipate significant changes in student behavior. Many students will flock in droves to alternatives: the current SAT and the ACT. There are few incentives for an average student to choose the redesigned SAT when the content is less familiar and more challenging, and the initial March and May SAT results will be delayed until 6-8 weeks after the test. Perhaps the only incentive is for the strong testers who might have a chance to “clean house” on the new SAT if there is a talent flight towards the ACT and the current SAT. Those intrepid souls willing to brave the waters of the new SAT may benefit from their efforts, but I forecast many, many students migrating away from this test.

For those current sophomores who will look to use the current SAT for college admissions, they must finish testing by January of their junior year. This would accelerate their testing timeline, which typically ends in June of junior year and occasionally spills over into the fall of senior year. These students will need to complete their preparation and take 2 or 3 official SAT administrations by January of next year. Some of my college counseling colleagues are even encouraging sophomores to take their first SAT in May or June of this year! Traditionally I’m opposed to commencing test prep sophomore year, but in this year of transition, I’m softening my position. If a student wants to finish testing by January 2016, and has a tough fall schedule, it’s okay if he or she wants to sit for the June or possibly the May SAT. After this year, we go back to the normal guidelines: keep testing in the junior year, and save senior year as insurance.

Summary

In closing, I’m impressed with the big gamble the College Board is taking, embracing the mantra “Go Big or Go Home” and betting all its chips on the Common Core. Will this pay off? Can the SAT reclaim its number one position? Will the SAT be as challenging as this problem set makes it seem? We’ll know much more as the College Board continues to release official material. All the while, we’ll look to the colleges to see how they embrace this new assessment. We promise to keep you fully informed as we navigate the new landscape of admissions testing together.