Test-Optional Universities Threaten SAT/ACT Credibility: Part I

So you’re nearing the end of your long journey. High school graduation is on the close horizon. You studied hard. You rocked your classes. You killed it on extra-curriculars. You even put in a few excruciating volunteer hours at a local old folks home, though it meant spending your sunny afternoons playing canasta and watching “Murder She Wrote” reruns with cheek-pinching septuagenarians. Ironically, the SAT was designed to reduce cultural biases. Ha! You, my friend, are excellent university material.

Oh, there is just one more thing . . .

Every single thing you’ve done in your life until this point boils down to your performance on a single, socioculturally-biased, multiple-choice exam to be administered over four hours on a random Saturday morning as yet to be determined.

Whatever you do . . . don’t freak out!

You’ve gotta love the premise behind the SAT: Students simply aren’t under enough pressure to succeed day-in, day-out. Let’s impose an arbitrary diagnostic with the power to erase every accomplishment logged over more than a decade of public schooling. That’ll learn ‘em.

Ok, so that’s probably an over-simplification. But of course, standardized testing is all about over-simplification. In the case of the SAT, and its closest competitor, the ACT, that oversimplification manifests as a single catch-all score designed to encapsulate a student’s aptitude for learning.

Bailing on the Test

If you think it’s absurd that the potential of each and every U.S. student (that’s about 20 million for those of you scoring at home) could be effectively measured by a single instrument, you’re in good company.

More than a few highly-regarded universities are seriously reconsidering the importance of test scores in their admission decisions. This past summer, George Washington University joined a growing set of schools both private and public when it decided to make such testing optional for its applicants.

According to the National Center for Fair and Open Testing, there are, today, more than 800 institutions of higher learning that take a test-optional approach to admissions.

What’s the reason for this approach?

Well if George Washington University is to be taken at face value, diversity is the top motive. The SAT has long been the subject of criticism for its latent racial and cultural biases. Critics argue that the test’s content and makeup inherently disadvantage impoverished, immigrant, and minority students.

Enrollment personnel at George Washington say that the new test-optional approach to admissions “should strengthen and diversify an already outstanding applicant pool and will broaden access for those high-achieving students who have historically been underrepresented at selective colleges and universities, including students of color, first-generation students and students from low-income households.”

A Brief History of the SAT

This many years removed from the fact, it’s easy to miss the irony. But in fact, the test which is today so often criticized for its cultural biases was originally proliferated as a way to heighten inclusiveness in the nation’s top universities.

Tracing its origins to an I.Q. test typically administered to new army recruits in the early 20th century, an experimental college-readiness assessment was distributed to a select group of students in 1926.

Then in 1933, Harvard President James Bryant Conant sought a way to expand upon a pool of recruits which was traditionally drawn from only a select number of elite boarding schools. The SAT emerged as perhaps the best way to evaluate intelligence among prospective students that hadn’t necessary come through Harvard’s traditionally exclusive paths to admission.

At first, the member schools of the non-profit College Board collectively agreed to use the test to evaluate scholarship applicants. But by the time the U.S. entered into World War II, the military-derived exam had supplanted all preexisting college admission tests. The Educational Testing Service was chartered in 1948 to continue the development and administration of the test. From there, its ascendance to standardized supremacy was all but assured.

The American College Testing (ACT) exam emerged as a competitor in 1959 and, today, enjoys roughly half of the college admissions market. In fact, in 2011, the number of ACT-takers surpassed the number SAT-takers for the first time ever. Historically, the ACT and SAT have enjoyed separate regional dominance, the former in the South and Midwest, the latter on the East and West Coasts.

The ACT casts a somewhat wider disciplinary net, including a section on science and folding social studies content into its reading section. Though the ACT has endured much of the same criticism that have hounded the SAT, its rising relevance and acceptance among universities illustrates that the College Board’s monopoly on student credibility is vulnerable to competition.

Diagnostic Failure

One of the lowlights of the SAT’s history was notched in 2003, when the Harvard Educational Review published an article called "Correcting the SAT’s Ethnic and Social-Class Bias: A Method for Reestimating SAT Scores.” In a now famous study alleging racial bias in the nation’s leading college-readiness diagnostic, long time employee of the Educational Testing Service, Roy Freedle observed a gap of more than 200 points between average scores for white and black students. The finding underscored the deep-seated flaws resident in our most treasured educational testing instrument.

Naturally, this was exceptionally problematic with more than 1.6 million students taking the exam in a given year. Though the Educational Testing Service did its darnedest to refute Freedle’s findings, the researcher’s conclusions have proven difficult to erase from our collective assumptions about the SAT.

Freedle identified what he called a “differential item functioning” (DIF), a phenomenon whereby reading comprehension is influenced by any number of cultural factors. The research deduced that this phenomenon almost certainly played a determinant role in the gap between the scores of black and white students.

The evidence was even more compelling, Freedle resolved, when one considered that black students routinely scored higher on those test items classified as harder. Minorities scores were lower on allegedly easier items. Freedle identified “evidence of an unintended but persistent cultural and statistical bias in the verbal section of the SAT that adversely affects African Americans.”

Neither the College Board nor the ETS was particularly excited about the publicity.

Without getting into the rather wonky debate over statistical analysis that ensued, it should come as little surprise to anybody that the Educational Testing Service immediately jumped into defense mode. A flurry of statements and studies emerged from the belly of the testing beast, largely dedicated to undermining Freedle’s findings.

A 2004 study commissioned by the ETS asserted in its abstract that Freedle’s “claims, which garnered national attention, were based on serious errors in his analysis. In Dorans and Zeller (2004), we demonstrated that the effects Freedle reported are reduced substantially when the data are analyzed correctly.”

This undercurrent of dispute remains a stubborn presence in the conversation over testing. In fact, despite its protestations, the ETS would subject its flagship test to considerable alterations in the following year. By 2005, a revised test emerged with a new scoring system, without those hoary and dreaded analogies, and including a new essay segment.

These changes were designed to improve the diagnostic nuance of the test but, according to more recent research, failed to remedy issues of racial disparity.

Another study published in the Harvard Educational Review, in 2010, rehashed Freedle’s research in relation to the new test. Once again, researchers revealed that black students of otherwise equal academic performance and qualification scored equal to their white counterparts on harder verbal items while scoring significantly lower than their white counterparts in supposedly easier reading portions of the SAT. Researchers surmised that black students often fared worse on simpler reading items, arguably those that seemed largely adapted from white expressions and colloquialisms.

The study underscored concerns that unilateral dependency on SAT or ACT scores perpetuates already-present racial inequality.

As it did when Freedle released his study, College Board once again argued that its tests are not unequal, that the outcomes simply reflect an inequality that already permeates society.

Going Optional

Whether Freedle’s original claims are true, false, or something in between, they do prove that universities are selling themselves short by relying too heavily on test scores. The decision at George Washington University suggests that some admission boards know they can do better.

Whether or not the SAT is racially or culturally fair seems almost beside the point. The greater question now is just how valuable the test is and just how much faith we should place in its results. The rational person would observe that no single diagnostic is likely to serve as a true measure of human potential in its myriad and unpredictable forms.

Diversity is a notion unto itself, separate from questions of race, nationality, or gender. Diversity implies that our differences are infinite. And within the gates of any given campus, we presume that diversity also implies an infinite array of possibilities for ingenuity, innovation, and invention. Allowing this diversity to be stifled by a machine-graded gatekeeper is to deprive a given campus the full spectrum of intellect, talent, culture, and creativity that make university life so deeply rewarding.

This is merely a question of logic. Is the SAT doing a sufficient job of ensuring that colleges are drawing the best and brightest applicants? Or alternately, is this creating a monochromatic prism for student evaluation, one that might prevent us from cultivating the most talented pool of students? One could make a compelling argument that this question applies to the ACT, the GRE, and any number of other standardized tests that stand like unflinching sentries before the threshold of a higher education.

The gathering tide of testing optional colleges suggests that credible universities are coming to recognize the disservice that is done not just to their prospective students but to the variegation of their respective campuses by placing too much stock in standardized scores. While it may be too soon to conclude that this change is producing greater diversity, evidence does at least exist to suggest that schools are not suffering academically for the decision.

A recent study by the National Association for College Admissions Counseling surveyed the performance of students at 33 testing optional schools, comparing those who had submitted SAT scores to those who had not.

In a study of 123,000 participants, the difference in cumulative GPA between submitters and non-submitters was five one-hundreds of a point. The difference in graduation rate was three-fifths of a percent.

Sooooooo, statistically speaking, the difference between test-takers and non-test-takers was nil.

From the perspective of the student, that sure does seem like a lot of stress for nothing.

It’s not that the SAT tells us nothing about a student. Taken on its own merit, the diagnostic should help us to understand how test-takers address specific problem-solving challenges within the scope of just two disciplines. This means that the SAT could be a useful way to help students choose disciplines most compatible with their abilities.

But as a measure of everything a student is and might be? Well that just seems . . . wrong. Not ethically (although there is a case to be made). It just seems incorrect.

The SAT actually tell us much less about students than its importance suggests. High school students leafing through an infinite stack of glossy college pamphlets understand all too well how much is riding on that random Saturday morning. The admissions game is a fiercely competitive one. For many students, the SAT score will either be a lock or a key, preventing or granting access to the university of choice.

But the National Association for College Admissions Counseling study argues that the single greatest determinant of a student’s success in college is his or her success in high school. If you think about it, that story hangs together pretty well.

It also tells us that the Defcon 1 intensity couched in the SAT is pretty incongruous with its actual predictive value. Research supports the idea that colleges already have all the information they need on a student to make a wise admissions decision without ever looking at the test scores.

The Future of Testing Optional

In the wake of Roy Freedle’s much-debated study, a story in The Atlantic suggested that such research may at least stimulate alternative ways of thinking about college admissions testing. The article noted that “In recent years the College Board has supported research, by the Yale psychologist Robert Sternberg and others, into alternative tests, of creative and practical skills. Sternberg said that his initial results produced smaller ethnic test-score gaps than the SAT, but he acknowledged that his test still needed years of work.”

Perhaps this suggests a future in which the SAT is merely one of many exams in a robust marketplace of varied student diagnostics tailored to unique areas of academic and professional interest.

If the current testing regime has rendered higher education a gated community, then perhaps it is best to supplant it with a set of doors, each a portal for testing knowledge, skill, and aptitude specific to a given student’s disciplinary destiny.

Those universities who are going test optional have sent a powerful message to the testing sector, namely that students and universities alike deserve diagnostics that are fair, that are comprehensive, that are rich in statistical value, and that actually tell us more about our students than we already know.

But will this message have a ripple effect or will it simply marginalize the universities delivering it? What impact will going test-optional have on the credibility of such universities? On their ranking and enrollment numbers?

We’ll do our best to address these questions and more in our next feature on the consequences of going test optional.

Test Optional Universities: Part 2