Deep Dive on the 4 Digital SAT Practice Tests

The digital SAT will debut for international students in March 2023 and the College Board has released four official practice tests, providing our first full view into this new test. We shared our analysis of these tests shortly after the release in a webinar for our educational partners and international students. In case you missed the webinar, here’s our in-depth take on what international students and educators need to know about the test.

In short, our initial analysis of the first digital practice test has been confirmed by the release of four official digital practice tests and four linear practice tests. The structural changes to the test, making it shorter, adaptive, and slower-paced, will make it a more popular test for many students and educators.

Scoring

Now that we have official practice tests, we have a better sense of the official scoring algorithm. Students might ask, “how many correct answers do I need to get a 650?  A 720?”  When it comes to classical testing, i.e., the current SAT, the total number correct will correspond directly to a scaled score. When it comes to adaptive testing, scoring is much more nuanced, as individual items, in particular sections, have differential weights.

The College Board refers to the differential weighting as “item response scoring.” In the Digital SAT Assessment Framework, the College Board mentions that “the use of item response theory (IRT) and ability estimation will provide the basis for calculating scale scores in near real time.” At the College Board Forum, our colleagues at Summit Education spoke with the lead psychometrician at College Board, Tom Proctor, who announced that in 2023-2024 College Board will release a technical manual explaining the new SAT’s complex scoring system.

We don’t yet have any breakdown of the differential item weighting on any of the official tests, but we have tinkered with the scoring scales enough to understand that students who are routed to the harder adaptive sections can attain fewer correct answers, but higher scores.

Consider the following scores on the first official Reading and Writing test:

Reading and Writing Scoring Digital Test 1

The top two students were routed to the harder adaptive section (green highlight) while the bottom student was routed to the easier adaptive section (blue highlight). The student who was routed to the easier adaptive section and answered 43 items correctly,  attained a scaled score of 560, while the student who answered 32 items correctly, but was routed to the harder adaptive section, attained a score of 580. Adaptive testing, by its nature, is more efficient in terms of administration, but more complex in terms of scoring.

In a similar light, here are some sample scores from the math section:

Math Scoring Digital Test 1

Notice that the student who attained a total of 18 of 44 total correct answers attained a 360, while the student who attained 15 total correct answers and was routed to the harder adaptive section earned a score of 460. Similarly, the student with 36 total correct answers scored a 650, while the student with 35 total correct answers scored a 540. Going forward, the number of correct answers will not be as instructive as it has been in the past, under the classical testing paradigm.

Here is a scoring scale to show how a student with a perfect baseline score will achieve different scores based on performance on the harder adaptive section.

Scoring Reading and Writing Digital Test 1 (perfect baseline)

And here is a scoring scale to illustrate how a student will achieve different scores on the harder adaptive math section.

Scoring Math Digital Test 1 (perfect baseline)

Easier and Harder Adaptive Modules

We found that there is a pronounced difference in average difficulty level between the easy and hard adaptive module. Students who find themselves on easy street for the adaptive section have most likely landed in the lower-level module. By our estimate, 16 out of 22 questions on the easier math module were Easy/Medium difficulty compared to only 5 out of 22 questions on the harder module. Here are the shapes of the baseline, easy, and hard sections. These present very different mountains to climb.

Baseline Math Module

Easy Adaptive Math Module

Hard Adaptive Math Module

Note that every section will have easy, medium and hard items, but the ratios are quite different for the easy and hard sections.

Test Form Variation

Although the four test forms available on College Board’s Blue book app appear static, which allows us our preliminary analysis, when it comes to official administrations, students will receive a unique assortment of test items, drawn from a vast problem bank. One student’s question 17 will be different from another student’s question 17, making cheating much more difficult, and these tests dramatically more secure.

Insights on the Math Section

At 44 total problems, down from 58, the math section feels more manageable. There are fewer word problems, which is great for non-native speakers. On the digital SAT only 30% of math items will be in the context of science or social studies or real-world applications. The remaining 70% of items will be “pure” math problems.

Testing Math, Not Reading

Math phrasing has been updated by linguistic modification, which is an intentional edit pass of item phrasing specifically focused on removing colloquial artifacts (idioms, odd turns of phrase, unneeded compound/complex sentences) that are harder to understand for ESL students. Basically, the goal is to make sure the SAT is testing math skills, not reading skills (unless the point of the problem is specifically parsing a complex word problem).

More Focus on Geometry and Trigonometry

The Current SAT averages roughly 8% of questions from the domains of geometry and trigonometry. The digital SAT will increase the percentage of geometry and trigonometry questions to 15% of all test items.

Desmos is a Game-Changer

The powerful Desmos graphing calculator, baked into the app, plays a major role on this new test, and for many students will fundamentally change the experience of taking the test. Students who know how to use a graphing calculator could use it to answer the majority of math questions on the Digital SAT.

Math Content Distribution‍

‍

‍Problem solving and data analysis (which covers things like percentages, probability, organizing data, and basic statistics) has been cut the most compared to the paper test.  However, it looks like the full range of problem solving and data analysis topics that were tested previously will remain fair game on the new SAT.

In analyzing the content covered on each test, it’s clear that there is a fair degree of variability across tests and sections. Here are the distributions for the first two official math sections:

No two tests or modules are identical.

Insights on the Reading and Writing Section

As we mentioned in our initial analysis, reading is short, but tough on the new SAT. Short passages are not necessarily easier passages. The reading level on the practice tests is as high, if not higher, than other SATs in recent years.

Switching to a new topic, voice, and style every question requires a bit more mental agility. Counterbalancing this, the predictable grouping of questions by type (i.e., all vocabulary at once, followed by all craft and structure questions at once), does allow a student to get into a particular problem-solving mindset for a stretch of time.

Here are some additional notable aspects of the new reading and writing tests:

• Decreased emphasis on grammar overall, but an increased emphasis on logical transitions
• Increased emphasis on vocabulary, which may be tough for some ELL students
• Elimination of idiom questions, which is good for ELL students
• Greater diversity of ideas and tones with more and smaller passages
• Question types are grouped and come in the same order every time
• Significant increase in humanities passages, surveys of artists/musicians
• Poetry is new, but relatively rare: multiple poems from Shakespeare, a Walt Whitman poem, all early 1900s or earlier public domain poets

Distribution of Reading and Writing Content

The above analysis is based on the first two official practice tests. Note the variability across sections and tests.

Reading and Writing Passage Types Across the Four Digital Tests

In terms of the content of the passages, here is an analysis of the average number of passages per module across the first four official practice tests

‍

As this analysis shows, poetry is not a focus on the digital test, while students should be very prepared for natural science reading passages.

Vocabulary, But not “SAT Vocab”

The increased focus on vocabulary in context does not mean a return to flashcards.  Those will not be helpful. Students must, however, pay attention to how words are used in different contexts, a skill that is best built through reading.

Closing Thoughts

Now that we have official practice, our international students can begin their work of preparing for the inaugural March 2023 digital SATs. We recommend that international students schedule a time to speak with an Applerouth Advisor about private 1-1 prep using our digital materials.