UC Faculty Issue a Powerful, Data-driven Defense of Standardized Testing in College Admissions

The critics of standardized testing in college admissions are numerous and highly vocal, while the supporters willing to take a prominent stand in favor of testing are few. This is what makes the recent report delivered by the faculty of the University of California so surprising. On February 3rd, the Academic Council’s Standardized Testing Task Force released a comprehensive 228-page analysis of the role of testing at UC and recommended that the UC system continue to require test scores for admissions.

At the request of UC President Janet Napolitano, the faculty-based Academic Senate appointed the Task Force to investigate the use of testing at UC and make policy recommendations for the governing Board of Regents. Per its charter, the Academic Senate exercises control over conditions of admissions for UC; thus, it would be surprising for the Board of Regents to reject the Senate’s findings when it makes a final decision on UC admissions policy in May. It seems the tide of test-optional admissions will stop at the doors of the UC system.

Two Task Force Takeaways

Within the extensive report, two key findings emerge: that testing data is both useful in identifying candidates for admission and not detrimental to admission prospects for underrepresented students. Indeed, in some cases, testing may help support UC’s diversity and access objectives.

(1) The tests help identify candidates who will succeed at UC

The Task Force conducted a rigorous data-driven analysis of the performance of the SAT and ACT in admissions and determined that the tests are doing expressly what they were designed to do: help the university identify candidates who will succeed at UC. The Task Force found that test scores help predict college performance, beyond what can be predicted from high school GPA alone. Indeed, much of the data pointed to the declining utility of high school GPAs as stand-alone predictors of student performance in college.

(2) The tests do not exacerbate underrepresentation in admission to UC

The second key finding is that the tests did not exacerbate racial differences in the makeup of the student body. In its holistic review process, UC was able to use testing to help identify diverse students with a high likelihood of collegiate success. This is a critical finding given that much of the argument in favor of test optional admissions has focused on whether testing contributes to disparities in representation.

UC faces particular challenges in admitting a racially diverse class. Following a 1996 state-wide referendum outlawing the direct use of race in admissions, UC had to use other factors to indirectly build a diverse student body. UC has been under pressure to enroll a student body that more closely represents the state population. In 2019, 59% of California high school graduates were underrepresented minorities, but only 37% of California resident students at UC and 26% of all admitted students to UC were from this population. At the prestigious flagship campuses, representation is markedly lower, with Berkeley’s admitted students consisting of only 18% underrepresented minorities in 2019.

The pivotal question is what’s contributing to the gap between the proportion of minorities graduating high school in California and the proportion admitted to UC schools? Critics believe testing has exacerbated the lack of diversity at UC but, analyzing the 14 factors contributing to admissions outcomes, the Task Force found that testing played a relatively minor role in the representation gap.

The primary factor driving the lower representation of minorities was UC’s requirement that students complete a broad college preparatory curriculum (known as the “A-G requirements”) with a mean grade of B or higher. Underrepresented minorities had a lower than average completion rate of the required classes and this accounted for 40% of the representation gap in admissions. Lower high school graduation rates explained 10% of the gap, and lower application rates explained 25% of the gap.

The key point is that testing plays a much smaller role than many critics have assumed, at least at UC. Meanwhile, discrepant high school GPAs played a more significant role in the representation gap. The Task Force was concerned that if test scores are eliminated from the admissions equation, the UC system will inevitably have to rely more heavily on high school GPAs, a shift that could actually result in lower diversity in the UC system.

With these big picture takeaways in mind, we can explore the context and data supporting the Task Force’s findings and recommendation.

UC admissions in context: testing is only one piece of the puzzle, and helps many minority students

To fully understand the Task Force’s conclusions about testing, it’s important to have some basic knowledge of how the UC admissions system works. Within the context of the UC system’s admissions process, testing is merely one of many factors that contribute to an admissions decision.

(1) Comprehensive review

UC admits students in two steps: general eligibility for the university followed by selection by a specific campus using a comprehensive 14-factor review, which includes academic factors such as high school GPA and grade trends, test scores, and number of A-G and honors classes. Demographic factors include background, socioeconomic status, first-generation status, family income, and extracurricular activities. Using holistic review, no applicants are eliminated based exclusively on testing or GPA. UC does not assign fixed weights to any of the 14 factors; all factors are considered in the context of the applicant’s high school and the overall applicant pool.

The comprehensive 14-factor review appears to help compensate for the differences in test scores across various applicant groups. The Task Force found that applicants from “less advantaged demographic groups are admitted at higher rates for any given test score as a result of comprehensive review.” Testing is considered, but receives considerably less weight in the admissions process than other factors such as high school GPA. For every campus, the admit rate is much higher for applicants with higher grades and lower testing than for the strong testers with weak grades.

(2) Eligibility in Local Context and statewide eligibility.

UC offers two forms of guaranteed or automatic admission. Through Eligibility in the Local Context (ELC), students in the top 9% of GPA at their high school gain automatic admission. Test scores are completely irrelevant for students admitted through ELC. UC offers a second track, known as statewide eligibility or index eligibility, which combines test scores and GPA to admit the top 9% of high school graduates across the state.

In the 2018 applicant pool, 22,613 students were guaranteed admission via statewide eligibility, but not through ELC; their SAT/ACT scores made them eligible. Of this group, 4,931 were low-income, 5,704 were first generation, and 5,609 were under-represented minorities. One quarter of all low-income, first generation, and underrepresented minority students admitted to UC in 2018 secured admission through statewide eligibility but not ELC: their high test scores are what made the difference for guaranteed admission. Twenty four percent of Latino students, 40% of African American students, and 47% of Native Americans earned their guarantees of admission to UC as a result of their high test scores, leading campus admissions directors to cite test scores as “a useful tool to advance diversity of incoming classes.”

The Role of Test Scores and High School GPAs in Predicting UC Grades and Graduation

The Task Force report dedicates considerable time and effort to parsing the data on predictive validity: how well do high school GPAs and test scores predict future success at UC? The overall findings from this extensive data analysis are that high school GPAs are of declining utility, when used alone, to predict student success at UC and that adding test score data helps improve predictive validity. Adding test scores to the equation leads to a significantly better prediction of short and long-term outcomes.The data also show that “test scores were better predictors of outcomes for underrepresented groups than for majority groups.”

(1) The declining predictive power of high school GPAs

The Task Force found that high school GPA is an imperfect measure of academic preparedness, citing evidence of grade inflation, grade compression, and discrepant grading cultures at different high schools. The Task Force study references several high schools to illustrate variations in grading standards and the “reduction in the information provided by grades due to the clustering of students near the top possible GPA values.” In short, grade inflation and compression render high school GPA less informative than it once was.

One factor to consider is that the applicant pool at UC has improved in quality. Candidates are presenting higher GPAs than those from prior years, with more applicants from the top high school GPA quintiles. This shift in the applicant pool may have contributed to the declining variability in the high school GPA of enrolled students and the decrease in its explanatory power.

As the Task Force states in it’s report, the “predictive power” of high school GPA “has declined markedly over the last few years.” High school GPA “accounted for 17 to 20 percent of variance in the freshman GPA prior to 2007, and then 15 percent in 2012 and 13 percent in 2015. As high school GPAs become less useful, the “explained variance accounted for by test scores has increased over the time from 13 percent in 2001 to 20 percent in 2015.” The Task Force concludes:“we need additional measures of achievement beyond GPA more than at any other time in the last few decades.” Inflated and compressed GPAs are making testing more important to determine who will succeed at UC.

(2) Test score data helps improve predictive validity

The Task Force emphasizes the clear predictive value derived from testing:

"Contrary to the narrative that is commonly expressed, the information available in test scores is not (emphasis ours) redundant with the information available in grades. Even among students with similar high school academic records, we still see a substantial relationship between test scores and college retention, grades, and graduation."

Using high school GPA alone explains only 16% of the variance in grades. Testing alone explains 21% of this variance. Using GPA and testing in combination explains 26% of the variance.

For UC students admitted with SAT scores under 700, 35% had dropped out of UC within a year and only 50% graduated within seven years. Among students with test scores over 1400, 97% continued past freshman year and 92% graduated within seven years. The Task Force predicted that without admissions tests, the average student at UC would have “a lower first-year GPA, a lower probability of persisting to year 2, a lower probability of graduating within seven years, and a lower GPA upon graduation.”

(3) Test scores better predict outcomes for underrepresented students

For students with family incomes over $120,000 per year, high school GPA explained 17% of the variance in grades, and adding testing improved the prediction by 20%. For those in the lowest income quartile, adding testing to the high school GPA model improved the prediction by 60%. The predictive power of testing increases as income decreases. Similarly, testing is much more predictive for first-generation students, yielding a 75% increase in predictive power over high school grades alone, compared to a 40% increase for non-first generation students.

Test scores are also more predictive of performance for underrepresented minorities. Adding testing to high school GPA increases the prediction of freshman grades by 29% for Caucasian students, 60% for Asian students, 77% for Hispanic students and 96% for African American students. Testing similarly increases the prediction of graduation by 22% for Caucasian students, 39% for Asian students, 61% for Hispanic students and 56% for African American students.

The Task Force summarizes these findings in clear terms:

"Test scores are predictive for all demographic groups and disciplines, even after controlling for HSGPA. In fact, test scores are better predictors of success for students who are underrepresented minority students, who are first-generation, or whose families are low-income: that is, test scores explain more of the variance in GPA and completion rates for students in these groups."

For the students who are less represented at UC, testing demonstrates the greatest predictive strength. The Task Force found that “consideration of test scores allows campuses to select those students from underrepresented groups who are more likely to earn higher grades and to graduate on time.”

Other Findings in Favor of Testing

(1) Testing to identify those in need of academic support

Test scores were helpful for UC in identifying students at risk of dropping out without additional academic support, particularly for underrepresented students.

(2) Testing is highly predictive of performance in particular majors

In certain courses of study, testing was particularly powerful at predicting performance. In the natural sciences, GPA alone predicted a mere 4.7% of the variance in grades, while SAT math alone predicted 13.5% of the variance. Combining testing and GPA was much more powerful in yielding accurate predictions of performance. The explained variance in the humanities using a combined model was 2.7 times greater than the high school GPA model and 5.2 times greater for engineering. High school grades alone do not paint a clear enough picture of readiness for certain courses of study.

Recommended testing changes to UC Eligibility Criteria

The Task Force recommends that UC perform an updated analysis of the SAT and ACT to explore differential item functioning (DIF) - in other words, analyze the test data to ensure that individual test questions are fair and not biased against certain groups. The Task Force also proposed eventually developing a new assessment system for UC, which would be specifically designed to predict success in the university. This custom-made test could potentially decrease disparities along racial, ethnic, and class lines but the the timeline for this ambitious project puts widespread implementation at least nine years into the future.

The Task Force additionally explored the possibility of shifting from the SAT/ACT to the Smarter Balanced assessment but ultimately did not advocate for this path, citing assessment validity issues, test security, item bank size, item exposure, and inconsistent implementation across the state.

What to do about the SAT and ACT Essays?

The Task Force found that the SAT and ACT essays were relatively weak predictors of performance on UC’s Analytical Writing Placement Exam (AWPE). While the ACT’s English Language Arts scores and SAT’s EBRW (Evidence-Based Reading and Writing Scores) proved predictive and were accepted as methods of meeting UC’s Entry Level Writing Requirement, the essays were rejected in this capacity. UC researchers also found that adding essay results to SAT/ACT scores “does not increase the explanatory power of pre-admission measures on college success.” Relying on a variety of regression analyses that underscore the poor predictive power of the SAT/ ACT essays, the Task Force questioned whether the essays are worth the time and effort they require:

“With limited explanatory power added by the essay scores, it is worth considering the social costs of additional writing tests. It is not ideal that students spend lots of time preparing for various tests while they could be focusing their energy on more important academic and social activities that could benefit them in the future.”

This does not bode well for the fate of the essay. Given that the essays take time and resources yet provide very little in terms of predictive validity, it would not be surprising if the Regents ultimately decide to jettison the essay requirement.

What’s next for UC?

The Academic Senate will take comments from faculty members at the different campuses and give a final recommendation in April to President Napolitano. She is expected to bring the issue to the system's Board of Regents, which in May will make the final decision on the matter. The Regents will most likely accept the recommendation of the Academic Senate and the powerful evidence they’ve compiled in defense of testing. We expect the SAT and ACT requirement to stay and the essay requirement to go.

Implications for Higher Education

Over the past few years we have seen a wave of colleges and universities move to test optional admissions. This wave has taken on considerable momentum but the UC faculty Task Force results, which largely favor testing, are sure to shake things up, at least in California. The faculty Task Force endorsed the utility and power of testing to shape a desired class and the evidence behind their decision will give cover to those institutions who want to continue to integrate testing into their plan for admissions.

The Task Force speculated that test scores may have greater predictive power for UC than for other universities with different student populations, meaning not every finding at UC will translate to the unique circumstances elsewhere. Still, many schools have a diverse student body like UC, and the reasons driving UC to keep testing may influence other schools weighing the pros and cons of test-optional admissions. More schools will adopt a test-optional policy, for the benefits in application numbers and rankings, the ability to bring in more low-scoring full-paying students and the potential to increase diversity, but the UC defense of testing will give pause to other institutions concerned about these issues.

Grade inflation and grade compression are real, as are disparate grading cultures across the 30,000 US high schools, and, as the UC study has shown, having a standardized measure to compare students can be very valuable in the admissions process. Testing is not the only measure or even the most important one but, when placed in context among many other factors, it can be a powerful tool for schools aiming to build a diverse and academically prepared student body.