AI Will Soon Be Reading & Evaluating Some College Admissions Essays

AI adoption continues to gain momentum as AI models become more powerful, better trained, and more useful in a wide variety of applications. The frontier Large Language Models (LLMs), ChatGPT, Claude and Gemini, are gaining proficiency at document analysis and summary, a function that could help beleaguered admission reps tasked with reading many thousands of personal statements and supplemental essays during an admissions cycle.

Given that applications to selective and highly selective colleges continue to rise, the need to expedite the review process has never been greater. While some admissions offices are loathe to insert AI into one of the most personal components of the application review, others, responding to the rising volume of applications and the corresponding stress on the admission officers, are eager to implement the new technological advances.

In one survey conducted by Intelligent.com, among the 300-plus education professionals surveyed who work in higher education and were very knowledgeable about their institutions’ admissions practices, 48% indicated their college or university planned to use AI to review student admissions essays. And these plans are not for the distant future. On a June 6 webinar hosted by Jeff Selingo, focused on AI and enrollment, John G. Haller, VP for Enrollment Management at the University of Miami stated that Miami will be “piloting [an AI application] for reading and reviewing essays in the fall.” Haller continued, “We are excited about the opportunity to try to use AI as an application reading feature for essays to see if it can help streamline the process to a greater degree.” The transition to AI-assisted application review is imminent for the early adopters.

AI applications can effectively evaluate and grade essays and have been doing so for years

GMAT

AI systems can easily evaluate the writing level of an essay, the level of vocabulary, grammatical correctness, length and variety of sentences and much more. The GMAT has been using AI to grade the Analytical Writing assessment portion of the GMAT for 16 years. In 2008 GMAC partnered with Vantage Learning, using its IntelliMetric essay-grading platform, and made the shift from two human graders to one human and one machine. The machines have proven to be highly proficient in scoring essays, and instances where the grades differ by more than one point (on a 6-point scale) occur in under 5% of cases. This is to say that machines have been successfully evaluating student writing ability, very similarly to humans, in the context of higher education admissions for more than a decade.

UPenn/UC Boulder study on using AI to score admissions essays for desirable traits

A research group based out of the University of Pennsylvania and the University of Colorado at Boulder developed a tool (i.e., modified a Facebook Large Language Model, ROBERTA) to analyze and evaluate student admission essays across seven variables, including teamwork, prosocial purpose, intrinsic motivation and leadership. Students whose essays scored positively for leadership were more likely to graduate from college in six years than those whose essays did not, even after controlling for differences in test scores, demographics and other factors. This research offers evidence that AI systems can effectively evaluate student essays for traits that are valuable to colleges.

Academic Study comparing essay scoring of ChatGPT to human graders

At this year’s annual conference for the American Educational Research Association, a paper was presented in which ChatGPT was pitted against human teachers in evaluating 1,800 student English and History papers. In one of the batches of 943 essays, ChatGPT was within a point (on a 6-point scale) of the human grader 89% of the time. This study took place before the most recent update to ChatGPT, and the accuracy is only going to improve with successive iterations of the model.

Texas is transitioning from human to AI graders for its state assessment

Texas is migrating towards AI essay grading for its State of Texas Assessments of Academic Readiness (STAAR) exams. Approximately four thousand human exam graders will be replaced by AI, resulting in annual cost savings in the $15-20 million range. Vice magazine reported in 2019, before the revolution in LLMs, 21 states were already using AI essay-scoring engines. Given the compelling economics and the rise of more powerful AIs, many more states will follow. Automated essay grading will soon affect students at all levels of education, K-12 and higher ed.

AI may be able to identify patterns in student essays that human readers miss

John G. Haller from the University of Miami explained that a technology team at U.M. is using AI to analyze the admissions essays from the 2022 admissions cycle, comparing those essays from students who persisted to those who did not. The team is exploring whether there are indicators within the essay, “keywords or common themes or topics of content that…raise a red flag that could potentially be applied to the subsequent year cohort.” The AI, when applied to the essays, may be able to help identify “those students that we think can be successful inside and outside of the classroom.”

Slate is preparing its essay Pre-Reader for mass distribution

The Slate Platform, a comprehensive admissions tool used by over 1,900 admissions offices across the country, has promised to deliver AI essay evaluation through its Pre-Reader: “the Slate Pre-Reader will summarize what a reviewer needs to know about a letter of recommendation, college essay, etc.” This function is still in development, but once rolled out, given the incredible market penetration of the Slate platform, is poised to have a significant impact on the admissions world.

How will admissions offices use these tools?

Generative AI has arrived in college admissions and is already affecting many of the processes involved in organizing and evaluating student applications. Admissions offices now have to decide which processes, including essay review, to automate, and which to keep firmly in the hands of human evaluators.

What does this mean for students?

Whether your essay will be read by a human or a machine trained to evaluate an essay like a human, there’s nothing, at this point, for you to do differently. When the GMAT began to use machine graders to evaluate student essays, our writing instruction didn’t change. The machines mimic human graders, and the algorithms are iteratively tweaked to align them more closely with human responses. If you write a good essay, you will get the credit, whether a human or machine intelligence evaluates the essay.

Colleges may try to mine student essays to identify patterns of future behavior/performance, but unless a college deploying the AI becomes very transparent in its findings and shares them publicly, there’s nothing for you to do differently. Trying to get ahead of a specific algorithm used by an AI reader is not a particularly useful exercise, as those algorithms will be changing regularly. Moreover, different admissions offices will use different AI applications, potentially highly customized, in their review processes. Be aware that admissions is evolving, but focus your energies where you can have an impact. Take the time to write a strong, authentic essay, in your voice, from your experience, and you will strengthen your application, no matter who, or what, happens to read it.

‍