Wealth, the SAT, Access, and the New Adversity Score

For decades the SAT has grappled with its reputation as a “wealth test,” a phrase coined by Harvard University law professor, Lani Guinier. In the latest effort to distance itself from this image, the College Board has unveiled the Environmental Context Dashboard (ECD), a tool that will enable colleges (but not students) to see an adversity score along with each students’ test score. The ECD, several years in the making, enters the stage just as the public conversation around access, equity, race, wealth, and privilege in college admissions have reached a fever pitch. The story of the adversity score and the implications for students are wrapped in this larger debate.

Putting the “Wealth Test” in Context

Guinier is not alone in her sharp criticism. In a similar broadside, education and parenting author Alfie Kohn notes that “[Standardized test] results are highly correlated with socioeconomic status, to the point that they tell us less about the potential of the child than they do about the size of the house in which that child lives.”

Is it true that the SAT correlates highly with family income? Yes. The Guinier and Kohn view - that an SAT score reflects nothing more than wealth - tends toward the extreme and oversimplifies the issue. But the fact remains - score gaps exist between affluent and non-affluent students. This issue pervades education and is not limited to the SAT. Myriad other measures of academic success in America - ranging from academic rigor in high school, to high school completion rates, college grades and graduation rates, graduate school attendance, and beyond - are tied to economic background.

We as a society long have struggled with this connection. And rightly so - we want students of all backgrounds to reach their potential and achieve their academic dreams. After Varsity Blues, we are looking even more closely at the connections between wealth and educational access. While the families in Varsity Blues used their resources to cheat the system and participate in elaborate scams, crossing into territory that most parents would never even contemplate, the scandal has brought heightened scrutiny to all aspects of the system, such as legacy and athletic admissions, as well as the ways in which parents with means are able to provide economic, social, and academic opportunity for their students. It is no secret that parents use their resources to secure educational opportunities for their children, moving into more expensive neighborhoods with better public schools, or paying private school tuition, after-school enrichment and academic summer camp fees.

The effects of economic disparity reach every aspect of education, including the SAT. Since Stanley Kaplan entered the scene in the 1940s, tutoring students from his parents’ basement in Brooklyn, it became apparent that SAT scores could be improved with focused preparation, procured at a cost. Students who could afford private instruction could attain higher scores.

The College Board has long bristled against its reputation as an organization that benefits the affluent at the expense of the under-resourced. For decades the College Board and other testing giants went to great lengths to deny the coachability of their assessments.

When David Coleman was brought on as President of the College Board, he changed the organizational messaging completely. He announced what everyone already knew, that students could, in fact, prepare for the SAT and significantly improve their scores. Coleman’s pivot was that he was going to eliminate the element of privilege by providing free test-prep for all. The College Board allied itself with Khan Academy, and a partnership for free, universal test prep was born.

Unfortunately, the partnership with Khan is not a panacea and, while we have no doubt it has helped many students, a program like this alone will not be sufficient to fully eliminate score gaps between students from the highest and lowest socioeconomic backgrounds. These students have such divergent experiences, from their family’s pre-existing education level to the educational experiences afforded them from an early age. These are social disparities that play out on a national policy level.

Coleman and the College Board have now introduced the ECD, which will place SAT scores in the context of affluence and adversity and call attention to students who rise above their challenging economic conditions. This is not an entirely new strategy for the College Board, which made a similar move with its Strivers program in the late 1990’s. The program elicited a chorus of critics who were upset, in particular, by the privileging of race in identifying the Strivers, and it flamed out fairly quickly after its debut.

The College Board’s new “adversity score” intentionally avoids the integration of race or ethnicity data to avoid those critiques. But critics have emerged, nonetheless, to identify this adversity score as a thinly veiled proxy for race. And it is as a proxy for race that this score may offer the greatest utility. If the Supreme Court upends affirmative action and prohibits the consideration of race in admission, having a metric which is a reliable proxy for race may allow colleges to stay within the bounds of the law and build the classes they want.

The Data Informing the “Adversity Score”

What data points are the College Board using to produce the adversity score? Is the information specific to the student or based on aggregate information? The only piece of data specific to the student is the student’s SAT score; the rest of the data pulls from aggregate information about the student’s high school and the census tract data for the student’s neighborhood. The College Board has made clear: “all students living in the same census tract will have the same neighborhood data and all students attending the same high school will have the same high school data.”

The factors that load onto the adversity score and inform the ECD are available on the College Board’s website along with a mock-up of the dashboard. There are 31 discrete variables, which are given equal weight in determining the score. These factors inform the adversity score, officially named the “Overall Disadvantage Level,” a single score ranging from 1 to 100 with 50 as the midpoint, with scores over 50 demonstrating heightened disadvantage/adversity.

The adversity factors fall into two primary categories: neighborhood factors and school factors.

• Neighborhood factors include employment data, crime rates, poverty rates, food stamp usage, housing values and vacancy rates, rentals vs. ownership rates, single-family homes, and college attendance behavior.
• School factors include some of the same variables from the neighborhood factors in addition to rates of free and reduced lunch, curricular rigor, AP offerings, and AP performance outcomes.

The data is drawn from the American Community Survey from the Census Bureau, National Center for Education Statistics, the FBI, and the College Board’s own sources. Students living in the same census tract and attending the same high school will have identical data, apart from their test scores. Race and ethnicity are absent from the ECD, which will allow colleges that are race- and ethnicity-blind to use the tool in their admissions process.

The Two-Year Pilot Study: Parsing the Utility of the Adversity Score

When I first heard the news of the adversity score, I was quick to dismiss this as a PR stunt on the part of the College Board. Colleges clearly know how to identify affluent and less affluent students. They have numerous proxies for socioeconomic status at their fingertips, from the rate of free-and-reduced lunch at schools, to the average test-scores from high schools, to parental education levels, to the addresses of families, to the robust financial data provided on the FAFSA, CSS profile, and institutional methodologies for determining financial aid. Admissions offices have been making distinctions based on socioeconomic status for decades without the help of the College Board. So what does the College Board’s new tool add to the process?

The College Board, with the support of researchers from the University of Michigan and Harvard, conducted a study on a two-year pilot version of the dashboard and found that it “resulted in more offers of admission to low-income applicants.” Jeremiah Quinlan, Dean of Undergraduate Admissions at Yale, spoke to Yale’s increase in spots given to less affluent students following the pilot, and John Barnhill, Assistant Vice President for Academic Affairs at Florida State University, told the Wall Street Journal that FSU boosted nonwhite enrollment from 37% to 42% since the advent of the tool. Although colleges have been able to factor in class and socioeconomic status into admissions for decades, the inclusion of the ECD and adversity scores seem to be accelerating the inclusion of lower-income students and students of color.

Reading the quotes from some of the college administrators who have participated in the two-year pilot program helps us understand why: the ECD helps to standardize a formerly piecemeal process. Jeremiah Quinlan, Dean of Undergraduate Admissions at Yale, spoke plainly about the utility of the ECD and adversity score or admissions at Yale. He said the score was “literally affecting every application we look at,” helping to contextualize each student. While Yale could glean much of the needed data from different parts of the student’s application, having it standardized across applicants was very helpful, and clearly efficient for an overworked admissions office.

Many schools in the pilot study had not previously approached this data in a standardized, algorithmic manner. Some high schools readily broadcast school data on their school profiles, while others are not as transparent. According to the College Board, other participants in the pilot study found that the ECD and adversity scores were particularly helpful when students from less familiar high schools were evaluated. This makes sense. There are over 30,000 US high schools, and admissions officers cannot intimately know every high school in their region. Contextualizing the student’s SAT score data, placing the student’s score in the context of the quartile in that high school was also helpful. One participant in the pilot felt that this contextualized SAT score helped prevent the admissions office from “immediately comparing [the student’s score] to our middle 50.”

It is clear from Quinlan’s reflections that the standardization of the algorithm was really quite helpful for Yale. Even if the algorithm was imperfect, the colleges seem to prefer the efficiency it provides over the previous approach. For highly selective colleges, weeding through tens of thousands of applications, efficiencies matter.

Questions remain, however, about the College Board’s methodology, and whether the adversity score truly measures what it’s intended to measure. Marten Roorda, CEO of the ACT was critical of the standardization, noting that “the algorithm and research behind this adversity score have not yet been published,” making it impossible to “review the validity and fairness of the score.”

The ECD rollout will continue at a fast clip. Fifty schools participated in the two-year pilot, and an additional 100 schools will join in the beta testing phase during the 2019 admissions cycle. After that, the College Board plans to make this tool broadly and freely available to colleges and universities across the country.

Implications

While the College Board has created a tool that will likely be popular with many colleges (and it’s likely the ACT will soon offer something similar), it remains to be seen how broad of an impact this adversity score will have on the admissions landscape. The fact that schools as distinct as Yale and Florida State have found this tool compelling leads one to assume this will have relatively broad utility in the college admissions process. Many colleges have been actively pursuing greater economic diversity, hoping to admit more first-generation and lower-income students. The ECD will help them continue those efforts. In the event the Supreme Court outlaws the use of race in admissions, the ECD could become instantly valuable to colleges who want to continue to admit classes of their choosing.

Some affluent students may lose admissions spots with the inclusion of this adversity scoring. As FSU’s Barnhill told the Wall Street Journal:

“If I am going to make room for more of the [poor and minority] students we want to admit and I have a finite number of spaces, then someone has to suffer and that will be privileged kids on the bubble.”

There are economic realities, however, that limit how many spots most colleges can give to underprivileged students; colleges need to meet institutional goals and financial targets to operate. This economic reality is unlikely to change, given our steady path of publicly defunding higher education and placing a greater and greater financial burden on families and students to pay their way. Affluent students, in addition to their academic contributions, will have an important role in financially sustaining colleges and universities, allowing the schools to be more generous and inclusive.