Making Sense of College Rankings

For colleges and universities, reputation is everything. In the fiercely competitive and increasingly global higher education sector, perception is nearly as important as performance. With more than 183 million college students hitting the books across the globe, unnumbered eyes will behold countless college ranking publications.

These publications—which academics take special pleasure in deriding and universities take particular pains to satisfy—are a major driver of endowment decisions, enrollment figures, and employment opportunities. Rankings play a direct role in how aspiring college students select schools, how researchers build reputations, and how colleges position themselves in the marketplace.

This gives the leading rankers in the game considerable power in shaping student decisions, university priorities, and even national policy-orientation. To wit:

These quick facts merely scratch the surface. Clearly, though, there's a lot riding on college rankings.

But can we trust them? Do any of these rankings truly measure what they claim to? Can you judge your likely educational experience by using these rankings?

Well, that depends. No one ranking system is perfect. Each is vulnerable to critique and each employs a methodology that can be readily deconstructed and parsed for flaws. But if you understand these methodologies, if you recognize these flaws, and if you know how to distill the information that these rankings do have to offer, you may be able to learn quite a bit.

Whatever objections persist—and they are numerous—annual college ranking releases are stitched into the fabric of the university hierarchy. This means that you are better off learning how to navigate them than you are ignoring them.

Each year, the major players release their annual college ranking lists. The occasion of each release is marked by anticipation and media saturation. In the aftermath of each release, the winners celebrate their accomplishment, the losers issue apologies or rationalizations, and critics from all corners of academia scrutinize the rankers themselves.

In the face of this critique—and alongside the effort that colleges extend to perform well in these annual rankings—an industry of rankers battle it out to present what each sees as the most objective, meaningful, and engaging way to compare colleges.

The ranking game is dominated by a few major players, as well as a bevy of new entrants and innovators. In the coming months, we'll take a closer look at each of these players, their ranking methodologies, and some of the leading points of critique facing each of them. The goal is to create a growing set of resources on how to read, interpret and get the most out of each of these ranking systems both in light of their strengths and in spite of their flaws.

But first, let's get the lay of the land.

Hereafter, we offer a basic primer on college rankings, including a brief history of ranking, a glance at the major players in the game today, a look at the criteria driving the leading rankings, and a run-down of the leading points of criticism against them.

Full Disclosure

Transparency is an important part of ranking. Credibility extends in part from the ability of each ranking service to effectively show and defend its methodology. In that spirit, we feel it important to tell you that we are, in fact, in the ranking business as well. The Quad is the editorial arm of TheBestSchools.org, which provides reasonably objective, methodologically sound, and unbiased college and university rankings in a broad spectrum of categories.

As such, The Quad is also partnered with a tech startup, InfluenceRankings.com, which uses machine learning and search algorithms to rank universities and colleges based on influential faculty and alumni associated with them.

As the editorial arm of TheBestSchools.org, we at The Quad operate with complete and total independence and in accordance with our own Mission. Our opinions and the content we create at The Quad are neither dictated by, nor produced in coordination with, the rankings that drive either TheBestSchools.org or InfluenceRankings.com. The findings here are reported with a concerted emphasis on objectivity and with editorial freedom from the ranking apparatuses employed by TheBestSchools.org and InfluenceRankings.com.

A Brief History of Rankings

The college ranking business comes from humble beginnings. A century ago, rankings were largely honorary in nature and hardly carried the fanfare or economic consequence of modern ranking lists.

Among the first recognized rankings was a study called Where We Get Our Best Men, published by Alick Maclean in 1900. MacLean's text was mostly dedicated to profiling those so-called Best Men. However, its most consequential feature was an index which ranked universities based on “the absolute number of eminent men who had attended them.” Four years later, a fellow Englishman named Havelock Ellis compiled a similar list, this one based on the number of affiliated “geniuses,” as opposed to “eminent men.”

Neither list proposed to measure or rank the quality of universities based on this metric. However, as this methodology made its way across the Atlantic, the American spirit of competition took hold. In his 1910 publication, American Men of Science, author James Cattell made explicit the connection between alumni achievements and comparative university excellence, particularly in a table called “Scientific Strength of the Leading Institutions.”

Cattell even issued his publication with the advice that “students should certainly use every effort to attend institutions having large proportions of men of distinction among their instructors.”

This would be the guiding principal behind the dominant ranking strategy of the next several decades. Beginning in 1930 and lasting through the 1950s, Prentice and Kunkel published an annual report that ranked colleges based on how many of their alumni appeared “in the social bible Who's Who.”

Though their ranking was considered an outcomes-based way of evaluating colleges, Prentice and Kunkel were objectively honest about the inherent weaknesses in their approach. In a 1951 study, they conceded that their rankings were likely skewed because their primary source, Who's Who, suffered from an overrepresentation of ministers and college professors and an underrepresentation of engineers.

This concession would serve as prelude to a major transformation in the ranking business. Gradually, the outcomes-based approach to ranking would fall out of fashion as reputation-based measures became de rigueur. The Center for College Affordability traces this reputation-based approach to a 1959 ranking in which the University of Pennsylvania sought to measure itself against other American research universities. The study's author identified chairpersons at 25 top universities, all members of the Association of American Universities. These chairs were consulted as raters, a model which would ultimately lay the groundwork for the emergence of survey-driven, reputation-based rankings.

Over the next several decades, scholars from all around the globe offered their own methodological refinements to the process of reputational ranking. But for the better part of the 1960s and 1970s, these rankings were of greatest interest to academics. Rankings hadn't yet penetrated the mainstream consciousness yet. That was all about to change dramatically.

The Modern Ranking Sector

1983 was an inflection point. That was when a magazine called U.S. News & World Report published its first ranking list of “America's Best Colleges.” Driven entirely by survey responses, these reputational rankings would have an immediate and profound impact on the higher education marketplace. They would also set into motion the development of the broader ranking industry. The magazine began publishing the report annually starting in 1987 and since that time, has become the most frequently quoted of American college ranking outlets. Today, these annual rankings hold powerful sway over how colleges are perceived by students, parents, alumni, and employers. Each year, their ranking lists are met with equal parts excitement and critique.

At its inception, the US News & World Report ranking was entirely subjective, collecting its survey responses from university and college presidents. Starting in 1988, U.S. News undertook an effort to incorporate more meaningful quantitative data in its rankings. Since that time, chief data strategist Robert Morse has presided over an ever-evolving methodology.

By 2010, its lists had become so popular that U.S. News moved largely away from its news magazine format, making franchise ranking the dominant part of its business strategy.

For most of its history, U.S. News has focused solely on American colleges. Only in 2014 did the ranker unveil its Best Global Universities rankings. By this time, a number of leaders had already emerged in the international ranking game. The Academic Ranking of World Universities (ARWU) was first among them.

The ARWU was originally compiled and issued by Shanghai Jiao Tong University and is thus commonly referred to as the Shanghai Ranking. The inaugural Shanghai Ranking was issued in 2003 and, as an empirical ranking of universities on a global scale, was the first of its kind. It remains highly influential in the international ranking sector. Not only was Shanghai distinct from U.S. News in its global scope but it also forged a ranking methodology that eschewed reputational metrics in favor of a return to strictly outcomes-based metrics. Shanghai arrives at its rankings using criteria that are quite distinct from those used by U.S. News. Its outcomes are, likewise, quite different.

So too are the ranking outcomes produced by Quacquarelli Symonds (QS) World University Rankings, the next prominent player to emerge in the global ranking sector. QS began producing its rankings in conjunction with the publication Times Higher Education (THE) in 2004. Based out of Britain, QS is something of a hybrid, combining Shanghai's global focus with U.S. News & World Report's mix of reptutational and quantitative data. As critics are often quick to point out, QS relies more heavily on reputational data than any of its competitors.

Each year between 2004 and 2009, THE and QS collaborated to release an annual table of international university rankings. However, the two parties separated in 2009, with THE citing empirical weakness in the QS ranking methodology. Beginning in 2010, Times Higher Education began publishing its own THE World University Rankings.

THE would quickly emerge as another major player in the ranking sector, applying its own methodological refinements while employing a similar mix of reptutational and quantitative indicators.

Today, the field of college rankings abounds with competitors. Other notable ranking outlets include Forbes, The Princeton Review, Bloomberg Business, and the Washington Post, which only recently began producing rankings based on a composite of other rankings. There's also a ranker called PayScale, whose College ROI Report simply measures the cost of your college education relative to your likely earnings upon graduation.

The Department of Education has also gotten into the business, with its College Scorecard distilling certain practical indicators meant to evaluate the economic value of each college or university in its ranking.

Also, as noted from the outset, The Quad is affiliated directly with TheBestSchools.org. TheBestSchools.org is unique among ranking services for the sheer (and ever-expanding) variety of rankings it offers, across a broad range of categories relating to discipline, academic quality, affordability, and model of education.

InfluenceRankings.com, a new entrant in this sector and a partner with TheBestSchools.org, is also wholly unique among existing services. Using an algorithm that assigns influence scores to notable academic figures, and ranks university programs by their affiliation with these influencers, InfluenceRankings.com produces a ranking strictly quantified according to apparent influence across the internet.

This mix of major players and new entrants suggests a competitive and expanding college ranking sector.

Significance of Rankings

Why is this market so competitive and expansive? The short answer is that many students, parents, colleges and employers all take college rankings seriously and base important decisions on these rankings.

While we can debate the accuracy, reliability, or trustworthiness of rankings, their popularity is beyond dispute. To wit, the day that U.S. News & World Report issued its Best Colleges rankings for 2014, its website attracted 2.6 million unique visitors and 18.9 million page views.

And people aren't just looking. They're making active, life-altering decisions based on this information. According to a 2014 report from the American Educational Research Association, students do make critical application decisions based on these rankings. For instance, schools landing in the U.S. News and World Report's Top 25 will experience a 6-10% increasing in the volume of incoming applications. Schools making Princeton Review's Top 20 Best Overall Academic Experience will see a 3.2% surge in applications. And colleges that move just one percentage point upward in U.S. News rankings will see a corresponding 1% increase in applications received.

In spite of the considerable academic criticism aimed at them, college rankings make a lot of noise, and college students hear this noise as they send applications out into the world. Rankings have a direct impact on student decisions regardless of their empirical reliability.

But students aren't the only ones basing important decisions on ranking indicators. Institutions of higher learning are quite aware of the influence that rankings have on student applications and equally aware of the factors that enter into such rankings. This means that, regardless of how well a given ranking actually measures academic quality or student experience, most universities feel a certain pressure to compete.

An article in Christian Science Monitor notes that many administrators recognize a connection between rankings and the caliber of student their respective universities attract. The importance of reputation looms large, particularly when it comes to attracting international students who might lack informational resources beyond these rankings. Leading historian of American education and Harvard professor emerita Patricia Albjerg Graham observed that “rankings have become particularly important to college and university administrators, who are anxious for their school to rise in the rankings. They believe that such as rise will bring more and better prepared students.”

Of course, it isn't merely the pressure to look good that drives universities. It is the equal pressure to avoid looking bad. College rankings hold such influence on the public's perception that universities who slide in the rankings often find themselves compelled to rationalize such occurrences to their various publics. With the release of this year's U.S. News rankings, for instance, New York University saw a drop in the standings. In response, Peter Henry, dean of the university's Stern School of Business, issued a personal email apology to the university's students, taking responsibility for the failure to submit certain critical data to the rankers. He promised to tighten procedures and avoid such lapses in the future.

Henry's mea culpa underscores the pressure that colleges are under to at least participate in the ranking game. But this pressure extends even beyond what colleges experience. On the international scale, rankers like Shanghai have had a direct impact not just on the way that global universities make decisions but on the priorities that national governments emphasize in higher education.

According to the New York Times, some nations have even undertaken educational initiatives specifically designed to play to the Shanghai Ranking. For instance, “Indonesia, Malaysia, South Korea and Taiwan announced programs to lift at least one university into the top 100; Nigeria pledged to put two universities in the top 200. The ranking's biomedical bias makes it particularly influential in the developing world, where science, technology, engineering and math are seen as holding the key to economic prosperity.”

This suggests that rankings carry an influence that permeates higher education at every level, driving student decisions, college priorities, and even nationwide initiatives. Of course, this is merely a glimpse of the role that rankings have come to play in the higher education ecosystem, but we can already get a sense of their enormous power.

Ranking Criteria At a Glance

Is that power warranted? In order to answer that question, one must first understand what these rankings actually measure.

According to Simon Marginson, a professor from the Centre for the Study of Higher Education at the University of Melbourne, we can divide today's leading college ranking strategies into three categories:

Since the three leading rankers mentioned above (U.S. News & World Report, ARWU/Shanghai Ranking, Quacquarelli Symonds) serve as prime examples of three distinct approaches, we've outlined their respective criteria here for your consideration. Check out our individual write-ups on these and other ranking services for more in-depth explanations of ranking criteria and methodologies. In the mean time, note the distinct variations that drive each of these ranking formulae.

The U.S. News & World Report Best Colleges

TheU.S. News ranking system is based on two pillars: quantitative measures proposed by education experts as reliable indicators of academic quality and the magazine's own research on the indicators deemed to matter most in education.

The following criteria drive U.S. News & World Report's rankings:

Once these variables have been calculated, weighted scores are assigned and combined. All scores are then scaled to fit within a spectrum of 100 points.

ARWU/Shanghai Rankings

According to its own methodology, Shanghai evaluates every university which boasts Nobel Laureates, Fields Medalists, “Highly Cited Researchers,” and those universities with major papers published in the journals Nature and Science. Universities with significant numbers of papers indexed by Science Citation Index-Expanded (SCIE) and Social Science Citation Index (SSCI) are also considered.

These weighted indicators are used to derive a composite score. The top ranking institution from the list is given a score of 100. All other scores are calculated as a percentage of the top score.

Quacquarelli Symonds World University Rankings

The QS World University Rankings identify four areas around which university performance is assessed: research, teaching, employability, and internationalization. Evaluation of these areas is informed by data relating to six indicators: four based on “hard data” and the other two, on “Major global surveys.”

Weighted items are consequently combined to determine a composite score. In 2016-2017, the QS World University Rankings assessed over 3,800 universities, ranking more than 900 of them. The first 400 schools ranked are given individual rankings while those ranked 401 and up are assigned numerical bands.

Here below, we've included a few helpful tables so that you can observe these rankings in action. We've compiled and compared ranking lists from all three of the above publications. Below are the three global ranking lists measuring overall rank, rank in mathematics, and rank in medicine, for the two most recent years available. Note the differences in ranking, both from publication to publication and from year to year. These variations can tell you much about the way that differences in criteria and methodology can affect outcomes.

Top 50 Universities Worldwide

  1. Harvard University ARWU / Shanghai Academic Ranking of World Universities
  2. Stanford University
  3. Massachusetts Institute of Technology (MIT)
  4. University of California, Berkeley
  5. University of Cambridge
  6. Princeton University
  7. California Institute of Technology
  8. Columbia University
  9. University of Chicago
  10. University of Oxford
  11. Yale University
  12. University of California, Los Angeles
  13. Cornell University
  14. University of California, San Diego
  15. University of Washington
  16. Johns Hopkins University
  17. University of Pennsylvania
  18. University College London
  19. University of California, San Francisco
  20. Swiss Federal Institute of Technology Zurich
  21. The University of Tokyo
  22. University of Michigan-Ann Arbor
  23. The Imperial College of Science, Technology and Medicine
  24. University of Wisconsin - Madison
  25. University of Toronto
  26. Kyoto University
  27. New York University
  28. Northwestern University
  29. University of Illinois at Urbana-Champaign
  30. University of Minnesota, Twin Cities
  31. Duke University
  32. Washington University in St. Louis
  33. Rockefeller University
  34. University of Colorado at Boulder
  35. University of Copenhagen
  36. Pierre and Marie Curie University - Paris 6
  37. The University of Texas at Austin
  38. University of California, Santa Barbara
  39. University of North Carolina at Chapel Hill
  40. University of British Columbia
  41. The University of Manchester
  42. University of Paris-Sud (Paris 11)
  43. University of Maryland, College Park
  44. (Tie) The University of Melbourne
    (Tie)The University of Texas Southwestern Medical Center at Dallas
  1. Heidelberg University
  2. The University of Edinburgh
  3. Karolinska Institute
  4. University of Southern California
  5. University of California, Irvine

 
Source: Academic Ranking of World Universities (Shanghai)

  1. Massachusetts Institute of Technology (MIT) Quacquarelli Symonds Logo
  2. Harvard University
  3. University of Cambridge
  4. Stanford University
  5. California Institute of Technology (Caltech)
  6. University of Oxford
  7. UCL (University College London)
  8. Imperial College London
  9. ETH Zurich - Swiss Federal Institute of Technology
  10. University of Chicago
  11. Princeton University
  12. National University of Singapore (NUS)
  13. Nanyang Technological University, Singapore (NTU)
  14. Ecole Polytechnique Fédérale de Lausanne (EPFL)
  15. Yale University
  16. Johns Hopkins University
  17. Cornell University
  18. University of Pennsylvania
  19. King's College London
  20. The Australian National University
  21. The University of Edinburgh
  22. Columbia University
  23. Ecole normale supérieure, Paris
  24. McGill University
  25. Tsinghua University
  26. University of California, Berkeley (UCB)
  27. University of California, Los Angeles (UCLA)
  28. The Hong Kong University of Science and Technology
  29. Duke University
  30. The University of Hong Kong
  31. University of Michigan
  32. Northwestern University
  33. The University of Manchester
  34. University of Toronto
  35. London School of Economics and Political Science (LSE)
  36. Seoul National University
  37. University of Bristol
  38. Kyoto University
  39. The University of Tokyo
  40. Ecole Polytechnique
  41. Peking University
  42. The University of Melbourne
  43. KAIST - Korea Advanced Institute of Science & Technology
  44. University of California, San Diego (UCSD)
  45. The University of Sydney
  46. The University of New South Wales (UNSW Australia)
  47. The University of Queensland
  48. The University of Warwick
  49. Brown University
  50. University of British Columbia

 
Source: Quacquarelli Symonds

  1. Harvard University U.S. News & World Report Logo
  2. Massachusetts Institute of Technology
  3. University of California–Berkeley
  4. Stanford University
  5. University of Oxford
  6. University of Cambridge
  7. California Institute of Technology
  8. University of California–Los Angeles
  9. University of Chicago
  10. Columbia University
  11. Johns Hopkins University
  12. Imperial College London
  13. Princeton University
  14. (Tie)University of Michigan
    (Tie)University of Toronto
    (Tie)University of Washington
  1. Yale University
  2. University of California–San Diego
  3. University of Pennsylvania
  4. Duke University
  5. University College London
  6. University of California–San Francisco
  7. Cornell University
  8. University of Tokyo
  9. Northwestern University
  10. Swiss Federal Institute of Technology Zurich
  11. University of Wisconsin–Madison
  12. University of California–Santa Barbara
  13. University of Minnesota–Twin Cities
  14. (Tie)University of British Columbia
    (Tie)University of Texas–Austin
  1. (Tie)University of Melbourne
    (Tie)University of North Carolina–Chapel Hill
  1. Ohio State University
  2. University of Illinois–Urbana-Champaign
  3. New York University
  4. (Tie)Boston University
    (Tie)University of California–Davis
  1. Peking University
  2. University of Edinburgh
  3. Washington University in St. Louis
  4. (Tie)University of Hong Kong
    (Tie)University of Pittsburgh
  1. McGill University
  2. University of Sydney
  3. Pierre and Marie Curie University - Paris 6
  4. University of Queensland Australia
  5. University of Munich
  6. University of Manchester
  7. University of Southern California

 
Source: U.S. News & World Report

  1. Harvard University ARWU / Shanghai Academic Ranking of World Universities
  2. Stanford University
  3. University of California–Berkeley
  4. University of Cambridge
  5. Massachusetts Institute of Technology (MIT)
  6. Princeton University
  7. University of Oxford
  8. California Institute of Technology
  9. Columbia University
  10. University of Chicago
  11. Yale University
  12. University of California–Los Angeles
  13. Cornell University
  14. University of California–San Diego
  15. University of Washington
  16. Johns Hopkins University
  17. University College London
  18. University of Pennsylvania
  19. Swiss Federal Institute of Technology Zurich
  20. The University of Tokyo
  21. University of California–San Francisco
  22. The Imperial College of Science–Technology and Medicine
  23. University of Michigan-Ann Arbor
  24. Washington University in St. Louis
  25. Duke University
  26. Northwestern University
  27. University of Toronto
  28. University of Wisconsin - Madison
  29. New York University
  30. University of Copenhagen
  31. University of Illinois–Urbana-Champaign
  32. Kyoto University
  33. University of Minnesota–Twin Cities
  34. University of British Columbia
  35. The University of Manchester
  36. University of North Carolina–Chapel Hill
  37. Rockefeller University
  38. University of Colorado–Boulder
  39. Pierre and Marie Curie University - Paris 6
  40. The University of Melbourne
  41. The University of Edinburgh
  42. University of California–Santa Barbara
  43. The University of Texas Southwestern Medical Center–Dallas
  44. (Tie) Karolinska Institute
    (Tie) The University of Texas–Austin
  1. University of Paris-Sud (Paris 11)
  2. (Tie) Heidelberg University
    (Tie) Technical University Munich
  1. University of Southern California
  2. King's College London

 
Source: Academic Ranking of World Universities (Shanghai)

  1. Massachusetts Institute of Technology (MIT) Quacquarelli Symonds Logo
  2. Stanford University
  3. Harvard University
  4. University of Cambridge
  5. California Institute of Technology (Caltech)
  6. University of Oxford
  7. UCL (University College London)
  8. ETH Zurich - Swiss Federal Institute of Technology
  9. Imperial College London
  10. University of Chicago
  11. Princeton University
  12. National University of Singapore (NUS)
  13. Nanyang Technological University–Singapore (NTU)
  14. Ecole Polytechnique Fédérale de Lausanne (EPFL)
  15. Yale University
  16. Cornell University
  17. Johns Hopkins University
  18. University of Pennsylvania
  19. The University of Edinburgh
  20. Columbia University
  21. King's College London
  22. The Australian National University
  23. University of Michigan
  24. Tsinghua University
  25. Duke University
  26. Northwestern University
  27. The University of Hong Kong
  28. University of California–Berkeley (UCB)
  29. The University of Manchester
  30. McGill University
  31. University of California–Los Angeles (UCLA)
  32. University of Toronto
  33. Ecole normale supérieure–Paris
  34. The University of Tokyo
  35. Seoul National University
  36. The Hong Kong University of Science and Technology
  37. Kyoto University
  38. London School of Economics and Political Science (LSE)
  39. Peking University
  40. University of California–San Diego (UCSD)
  41. University of Bristol
  42. The University of Melbourne
  43. Fudan University
  44. The Chinese University of Hong Kong (CUHK)
  45. University of British Columbia
  46. (Tie) The University of Sydney
    (Tie) New York University (NYU)
    (Tie) KAIST - Korea Advanced Institute of Science & Technology
  1. The University of New South Wales (UNSW Australia)
    (Tie) Brown University

 
Source: Quacquarelli Symonds

  1. Harvard University U.S. News & World Report Logo
  2. Massachusetts Institute of Technology
  3. Stanford University
  4. University of California–Berkeley
  5. California Institute of Technology
  6. University of Oxford
  7. University of Cambridge
  8. Princeton University
  9. Columbia University
  10. University of California–Los Angeles
  11. (Tie) Johns Hopkins University
    (Tie) University of Washington
  1. University of Chicago
  2. Yale University
  3. University of California–San Diego
  4. University of California–San Francisco
  5. (Tie) University of Michigan–Ann Arbor
    (Tie) University of Pennsylvania
    (Tie) Duke University
    (Tie) Imperial College London
  1. University of Toronto
  2. Cornell University
  3. University College London
  4. University of California–Santa Barbara
  5. Northwestern University
  6. Washington University in St. Louis
  7. (Tie) New York University
    (Tie) University of California–Santa Cruz
  1. University of Wisconsin–Madison
  2. University of Texas–Austin
  3. University of British Columbia
  4. (Tie) Boston University
    (Tie) University of Colorado–Boulder
    (Tie) University of North Carolina–Chapel Hill
  1. Swiss Federal Institute of Technology Zurich
  2. (Tie) University of Edinburgh
    (Tie) University of Melbourne
  1. University of Minnesota–Twin Cities
  2. Rockefeller University
  1. (Tie) École Polytechnique Federale of Lausanne
    (Tie) University of Maryland–College Park
  2. University of California–Davis
  3. Ohio State University–Columbus
  4. University of Tokyo
  5. (Tie) King's College London
    (Tie) University of Sydney
    (Tie) University of Illinois–Urbana-Champaign
    (Tie) University of Pittsburgh
  1. University of Copenhagen
  2. (Tie) McGill University
    (Tie) National University of Singapore

 
Source: U.S. News & World Report

Top 50 Mathematics Programs Worldwide

  1. Princeton University ARWU / Shanghai Academic Ranking of World Universities
  2. Harvard University
  3. University of California-Berkeley
  4. Pierre and Marie Curie University - Paris 6
  5. Stanford University
  6. University of Cambridge
  7. University of Paris Sud (Paris 11)
  8. University of Oxford
  9. University of California–Los Angeles
  10. King Abdulaziz University
  11. Massachusetts Institute of Technology (MIT)
  12. University of Minnesota–Twin Cities
  13. Texas A & M University
  14. Swiss Federal Institute of Technology Zurich
  15. University of Michigan-Ann Arbor
  16. University of Wisconsin - Madison
  17. Kyoto University
  18. University of Washington
  19. Columbia University
  20. Duke University
  21. New York University
  22. University of California–San Diego
  23. University of Chicago
  24. University of Paris Dauphine (Paris 9)
  25. The University of Texas at Austin
  26. Moscow State University
  27. The Hebrew University of Jerusalem
  28. City University of Hong Kong
  29. University of Geneva
  30. Ecole Normale Superieure - Paris
  31. University Paris Diderot - Paris 7
  32. University of Warwick
  33. University of Bonn
  34. Pennsylvania State University - University Park
  35. University of Maryland–College Park
  36. University of Vienna
  37. California Institute of Technology
  38. Peking University
  39. Yale University
  40. Rutgers–The State University of New Jersey - New Brunswick
  41. Georgia Institute of Technology
  42. Cornell University
  43. The Chinese University of Hong Kong
  44. University of Illinois at Urbana-Champaign
  45. King Fahd University of Petroleum & Minerals
  46. The Imperial College of Science–Technology and Medicine
  47. Aix Marseille University
  48. Northwestern University
  49. Shanghai Jiao Tong University
  50. University of Pennsylvania

 
Source: Academic Ranking of World Universities (Shanghai)

  1. Harvard University Quacquarelli Symonds Logo
  2. Massachusetts Institute of Technology (MIT)
  3. University of Oxford
  4. (Tie) University of California, Berkeley (UCB)
    (Tie) University of Cambridge
  1. (Tie) Princeton University
    (Tie) Stanford University
  1. University of California, Los Angeles (UCLA)
  2. ETH Zurich - Swiss Federal Institute of Technology
  3. New York University (NYU)
  4. Imperial College London
  5. University of Chicago
  6. National University of Singapore (NUS)
  7. California Institute of Technology (Caltech)
  8. University of Toronto
  9. Carnegie Mellon University
  10. University of Texas at Austin
  11. Columbia University
  12. University of Michigan
  13. The Australian National University
  14. Ecole Polytechnique
  15. Yale University
  16. The University of Tokyo
  17. University of Waterloo
  18. The University of Warwick
  19. The Chinese University of Hong Kong (CUHK)
  20. Georgia Institute of Technology
  21. University of British Columbia
  22. Kyoto University
  23. Peking University
  24. (Tie) The University of Sydney
    (Tie) The University of Edinburgh
  1. University of California, San Diego (UCSD)
  2. University of Bristol
  3. Tsinghua University
  4. (Tie) Ecole Polytechnique Fédérale de Lausanne (EPFL)
    (Tie) UCL (University College London)
  1. Cornell University
  2. Universidade de São Paulo
  3. (Tie) Brown University
    (Tie) City University of Hong Kong
    (Tie) Rheinische Friedrich-Wilhelms-Universität Bonn
  1. University of Illinois at Urbana-Champaign
  2. Technical University of Munich
  3. (Tie) Ecole normale supérieure, Paris
    (Tie) The Hong Kong University of Science and Technology
    (Tie) The University of Hong Kong
  1. Nanyang Technological University, Singapore (NTU)
  2. Lomonosov Moscow State University
  3. The University of Manchester

 
Source: Quacquarelli Symonds

  1. University of California–Berkeley U.S. News & World Report Logo
  2. Stanford University
  3. Princeton University
  4. University of California–Los Angeles
  5. University of Oxford
  6. Harvard University
  7. King Abdulaziz University
  8. Pierre and Marie Curie University - Paris 6
  9. University of Hong Kong
  10. University of Cambridge
  11. Massachusetts Institute of Technology
  12. University of Minnesota–Twin Cities
  13. New York University
  14. University of Michigan
  15. University of Chicago
  16. University of Washington
  17. Peking University
  18. Columbia University
  19. Chinese University of Hong Kong
  20. Swiss Federal Institute of Technology Zurich
  21. Texas A&M University–College Station
  22. University of Wisconsin–Madison
  23. University of British Columbia
  24. University of Texas–Austin
  25. Fudan University
  26. California Institute of Technology
  27. Georgia Institute of Technology
  28. University of Illinois–Urbana-Champaign
  29. Université Paris-Sud
  30. École Polytechnique
  31. Imperial College London
  32. Lomonosov Moscow State University
  33. (Tie) Gyeongsang National University
    (Tie) Kyoto University
  1. Pennsylvania State University
  2. University of Toronto
  3. Zhejiang University
  4. Rutgers State University
  5. Tsinghua University
  6. Beijing Normal University
  7. National Sun Yat-sen University
  8. University of Tokyo
  9. Université Paris Diderot - Paris 7
  10. City University of Hong Kong
  11. National University of Singapore
  12. Universidade de São Paulo
  13. Brown University
  14. Duke University
  15. (Tie) Shanghai Jiao Tong University
    (Tie) University of Bonn

 
Source: U.S. News & World Report

  1. Princeton University ARWU / Shanghai Academic Ranking of World Universities
  2. Stanford University
  3. Harvard University
  4. University of California, Berkeley
  5. Pierre and Marie Curie University - Paris 6
  6. King Abdulaziz University
  7. University of Oxford
  8. University of California, Los Angeles
  9. University of Cambridge
  10. University of Paris-Sud (Paris 11)
  11. University of Minnesota, Twin Cities
  12. Massachusetts Institute of Technology (MIT)
  13. University of Warwick
  14. Swiss Federal Institute of Technology Zurich
  15. Texas A&M University
  16. University of Michigan-Ann Arbor
  17. Columbia University
  18. University of Washington
  19. University of Wisconsin - Madison
  20. Duke University
  21. The University of Texas at Austin
  22. City University of Hong Kong
  23. King Fahd University of Petroleum & Minerals
  24. New York University
  25. Aix Marseille University
  26. Ecole Normale Superieure - Paris
  27. Kyoto University
  28. Paris Dauphine University (Paris 9)
  29. University of Bonn
  30. University of California, San Diego
  31. Moscow State University
  32. University Paris Diderot - Paris 7
  33. The Hebrew University of Jerusalem
  34. University of Chicago
  35. Pennsylvania State University - University Park
  36. University of Maryland, College Park
  37. Lanzhou University
  38. Georgia Institute of Technology
  39. California Institute of Technology
  40. The Chinese University of Hong Kong
  41. University of Geneva
  42. Shanghai Jiao Tong University
  43. Peking University
  44. Rutgers, The State University of New Jersey - New Brunswick
  45. The Imperial College of Science, Technology and Medicine
  46. University of Colorado at Boulder
  47. Yale University
  48. Cornell University
  49. The University of Tokyo
  50. Harbin Institute of Technology

 
Source: Academic Ranking of World Universities (Shanghai)

  1. Harvard University Quacquarelli Symonds Logo
  2. University of Cambridge
  3. University of Oxford
  4. Massachusetts Institute of Technology (MIT)
  5. Stanford University
  6. University of California, Berkeley (UCB)
  7. Princeton University
  8. University of California, Los Angeles (UCLA)
  9. ETH Zurich - Swiss Federal Institute of Technology
  10. University of Chicago
  11. Imperial College London
  12. National University of Singapore (NUS)
  13. New York University (NYU)
  14. University of Michigan
  15. California Institute of Technology (Caltech)
  16. University of Toronto
  17. Ecole Polytechnique
  18. The University of Tokyo
  19. Columbia University
  20. (Tie) The University of Hong Kong
    (Tie)University of Waterloo
  1. University of Texas at Austin
  2. Yale University
  3. Carnegie Mellon University
  4. Georgia Institute of Technology
  5. Ecole Polytechnique Fédérale de Lausanne (EPFL)
  6. University of British Columbia
  7. University of California, San Diego (UCSD)
  8. City University of Hong Kong
  9. The University of Edinburgh
  10. The Australian National University
  11. Nanyang Technological University, Singapore (NTU)
  12. The Hong Kong University of Science and Technology
  13. (Tie) Cornell University
    (Tie) The University of Sydney
  1. Kyoto University
  2. (Tie) The University of Melbourne
    (Tie) University of Wisconsin-Madison
  1. Northwestern University
  2. Rheinische Friedrich-Wilhelms-Universität Bonn
  3. University of Pennsylvania
  4. Lomonosov Moscow State University
  5. The Chinese University of Hong Kong (CUHK)
  6. (Tie) Ecole normale supérieure, Paris
    (Tie) The University of Warwick
  1. (Tie) Peking University
    (Tie) University of Illinois at Urbana-Champaign
  1. Brown University
  2. Seoul National University
  3. Technische Universität Berlin (TU Berlin)

 
Source: Quacquarelli Symonds

  1. University of California–Berkeley U.S. News & World Report Logo
  2. Stanford University
  3. Princeton University
  4. University of California–Los Angeles
  5. University of Oxford
  6. Harvard University
  7. King Abdulaziz University
  8. Pierre and Marie Curie University - Paris 6
  9. University of Hong Kong
  10. University of Cambridge
  11. Swiss Federal Institute of Technology Zurich
  12. University of Texas–Austin
  13. University of Minnesota–Twin Cities
  14. Columbia University
  15. University of Chicago
  16. University of Wisconsin–Madison
  17. Texas A&M University–College Station
  18. Catholic University of Leuven
  19. Imperial College London
  20. Fudan University
  21. Peking University
  22. University of Washington
  23. University of Michigan–Ann Arbor
  24. University of British Columbia
  25. Pennsylvania State University–University Park
  26. Université Paris-Sud
  27. University Paris Diderot - Paris 7
  28. Chinese University Hong Kong
  29. Rutgers, The State University of New Jersey–New Brunswick
  30. Beijing Normal University
  31. Shanghai Jiao Tong University
  32. Brown University
  33. University of Illinois–Urbana-Champaign
  34. University of Bonn
  35. École Polytechnique
  36. University of Warwick
  37. École Polytechnique Federale of Lausanne
  38. (Tie) University of São Paulo
    (Tie) University of Vienna
  1. University of Toronto
  2. Tsinghua University
  3. Duke University
  4. Sapienza University of Rome
  5. (Tie) Georgia Institute of Technology
    (Tie) University of Tokyo
  1. Nankai University
  2. Kyoto University
  3. (Tie) Polytechnic University of Milan
    (Tie) University of Lisbon
  1. National University of Singapore

 
Source: U.S. News & World Report

Top 50 Medical Programs Worldwide

  1. Harvard University ARWU / Shanghai Academic Ranking of World Universities
  2. University of California, San Francisco
  3. University of Washington
  4. Johns Hopkins University
  5. Stanford University
  6. University of Cambridge
  7. Columbia University
  8. University College London
  9. The University of Texas Southwestern Medical Center at Dallas
  10. University of California, Los Angeles
  11. Yale University
  12. Karolinska Institute
  13. University of North Carolina at Chapel Hill
  14. University of Pittsburgh, Pittsburgh Campus
  15. University of Oxford
  16. Mayo Medical School
  17. University of Michigan-Ann Arbor
  18. University of Pennsylvania
  19. Vanderbilt University
  20. University of California, San Diego
  21. The University of Texas M. D. Anderson Cancer Center
  22. University of Minnesota, Twin Cities
  23. The Imperial College of Science, Technology and Medicine
  24. Duke University
  25. Cornell University
  26. University of Wisconsin - Madison
  27. University of California, Berkeley
  28. University of Toronto
  29. Northwestern University
  30. Boston University
  31. Leiden University
  32. The University of Melbourne
  33. Emory University
  34. University of Zurich
  35. University of Chicago
  36. University of Utah
  37. King's College London
  38. McGill University
  39. University of Amsterdam
  40. KU Leuven
  41. VU University Amsterdam
  42. University of Nottingham
  43. The University of Texas Health Science Center at Houston
  44. University of Oslo
  45. University of Gothenburg
  46. McMaster University
  47. Washington University in St. Louis
  48. Tufts University
  49. Kyoto University
  50. Massachusetts Institute of Technology (MIT)

 
Source: Academic Ranking of World Universities (Shanghai)

  1. Harvard University Quacquarelli Symonds Logo
  2. University of Oxford
  3. University of Cambridge
  4. Johns Hopkins University
  5. Stanford University
  6. University of California, San Francisco
  7. University of California, Los Angeles (UCLA)
  8. Yale University
  9. Karolinska Institutet
  10. UCL (University College London)
  11. Imperial College London
  12. University of Pennsylvania
  13. Duke University
  14. University of Toronto
  15. Columbia University
  16. Massachusetts Institute of Technology (MIT)
  17. The University of Sydney
  18. The University of Melbourne
  19. University of Washington
  20. The University of Tokyo
  21. National University of Singapore (NUS)
  22. McGill University
  23. University of California, San Diego (UCSD)
  24. King's College London
  25. Washington University in St. Louis
  26. University of British Columbia
  27. University of Michigan
  28. The University of Hong Kong
  29. New York University (NYU)
  30. Cornell University
  31. The University of Edinburgh
  32. London School of Hygiene and Tropical Medicine
  33. Erasmus University Rotterdam
  34. McMaster University
  35. Baylor College of Medicine
  36. Monash University
  37. University of Chicago
  38. University of Copenhagen
  39. Mayo Medical School
  40. Ruprecht-Karls-Universitaet Heidelberg
  41. National Taiwan University (NTU)
  42. University of Pittsburgh
  43. Northwestern University
  44. The University of New South Wales (UNSW Australia)
  45. University of Zurich
  46. Kyoto University
  47. University of Amsterdam
  48. Seoul National University
  49. The University of Queensland
  50. Leiden University

 
Source: Quacquarelli Symonds

  1. Harvard University U.S. News & World Report Logo
  2. Johns Hopkins University
  3. University of California–San Francisco
  4. University of Oxford
  5. (Tie) Duke University
  6. (Tie) University of Toronto
  7. University of California–Los Angeles
  8. University of Washington
  9. University of Pennsylvania
  10. Mayo Clinic
  11. Stanford University
  12. Columbia University
  13. University of Michigan
  14. Imperial College London
  15. University College London
  16. University of Pittsburgh
  17. University of Cambridge
  18. Karolinska Institute
  19. Erasmus University Rotterdam
  20. University of California–San Diego
  21. University of North Carolina–Chapel Hill
  22. Yale University
  23. Washington University in St. Louis
  24. Vanderbilt University
  25. KU Leuven
  26. (Tie) Cornell University
    (Tie) London School of Hygiene & Tropical Medicine
    (Tie) University of Chicago
  1. University of Copenhagen
  2. (Tie) Boston University
    (Tie) Kings College London
    (Tie) University of Copenhagen
  1. Boston University
  2. (Tie) Heidelberg University
    (Tie) Northwestern University
  1. University of Sydney
  2. University of Minnesota–Twin Cities
  3. (Tie) Charite - Medical University of Berlin
    (Tie) Icahn School of Medicine at Mount Sinai
  1. (Tie) Baylor College of Medicine
    (Tie) University of Amsterdam
    (Tie) University of Colorado–Denver
    (Tie) University of Southern California
  1. (Tie) McGill University
    (Tie) University of Barcelona
    (Tie) University of British Columbia
  1. (Tie) Leiden University
    (Tie) University of Melbourne
  1. McMaster University
  2. University of Alabama–Birmingham

 
Source: U.S. News & World Report

  1. Harvard University ARWU / Shanghai Academic Ranking of World Universities
  2. University of Washington
  3. Stanford University
  4. University of Cambridge
  5. Johns Hopkins University
  6. University of California–San Francisco
  7. Columbia University
  8. University College London
  9. The University of Texas Southwestern Medical Center at Dallas
  10. Yale University
  11. University of California–Los Angeles
  12. Karolinska Institute
  13. University of North Carolina at Chapel Hill
  14. University of Oxford
  15. Vanderbilt University
  16. University of Pennsylvania
  17. University of Pittsburgh–Pittsburgh Campus
  18. Mayo Medical School
  19. University of California–San Diego
  20. The University of Texas M. D. Anderson Cancer Center
  21. Duke University
  22. University of Michigan-Ann Arbor
  23. The Imperial College of Science–Technology and Medicine
  24. Cornell University
  25. University of Minnesota–Twin Cities
  26. The University of Melbourne
  27. Northwestern University
  28. University of Utah
  29. Washington University in St. Louis
  30. University of Wisconsin - Madison
  31. University of Toronto
  32. University of Amsterdam
  33. McMaster University
  34. Monash University
  35. KU Leuven
  36. (Tie) University of Oslo
    (Tie) University of Sydney
    (Tie) University of Zurich
  1. Boston University
  2. King's College London
  3. Leiden University
  4. VU University Amsterdam
  5. University of Chicago
  6. The University of Edinburgh
  7. The Ohio State University - Columbus
  8. University of Southern California
  9. University of California–Berkeley
  10. University of Nottingham
  11. Icahn School of Medicine at Mount Sinai
  12. University of Gothenburg

 
Source: Academic Ranking of World Universities (Shanghai)

  1. Harvard University Quacquarelli Symonds Logo
  2. University of Oxford
  3. University of Cambridge
  4. Stanford University
  5. Johns Hopkins University
  6. University of California–Los Angeles (UCLA)
  7. University of California–San Francisco
  8. Yale University
  9. UCL (University College London)
  10. Karolinska Institutet
  11. Imperial College London
  12. University of Toronto
  13. Columbia University
  14. Massachusetts Institute of Technology (MIT)
  15. University of Pennsylvania
  16. Duke University
  17. The University of Sydney
  18. The University of Melbourne
  19. University of California–San Diego (UCSD)
  20. University of Washington
  21. King's College London
  22. National University of Singapore (NUS)
  23. The University of Tokyo
  24. Washington University in St. Louis
  25. University of Michigan
  26. The University of Edinburgh
  27. McGill University
  28. University of British Columbia
  29. London School of Hygiene and Tropical Medicine
  30. The University of Hong Kong
  31. New York University (NYU)
  32. University of Chicago
  33. McMaster University
  34. (Tie) Cornell University
    (Tie) Erasmus University Rotterdam
    (Tie) Mayo Medical School
    (Tie) University of Amsterdam
  1. University of Copenhagen
  2. Monash University
  3. Ruprecht-Karls-Universitaet Heidelberg
  4. Baylor College of Medicine
  5. Northwestern University
  6. The University of Queensland
  7. National Taiwan University (NTU)
  8. University of Zurich
  9. The University of New South Wales (UNSW Australia)
  10. University of Pittsburgh
  11. Seoul National University
  12. Boston University
    (Tie) Kyoto University
    (Tie) The University of Manchester
  13. Aarhus University

 
Source: Quacquarelli Symonds

  1. Harvard University U.S. News & World Report Logo
  2. Johns Hopkins University
  3. University of California–San Francisco
  4. University of Oxford
  5. Duke University
  6. University of Washington
  7. University of Toronto
  8. University of Pennsylvania
  9. Stanford University
  10. Columbia University
  11. University of California–Los Angeles
  12. University of Michigan–Ann Arbor
  13. Imperial College London
  14. Massachusetts Institute of Technology
  15. University College London
  16. University of Cambridge
  17. University of Pittsburgh
  18. University of California–San Diego
  19. Karolinska Institute
  20. Yale University
  21. University of North Carolina–Chapel Hill
  22. Erasmus University Rotterdam
  23. Washington University in St. Louis
  24. Emory University
  25. Cornell University
  26. Vanderbilt University
  27. (Tie) Boston University
    (Tie) Catholic University of Leuven
  1. King's College London
  2. University of Chicago
  3. London School of Hygiene & Tropical Medicine
  4. University of Copenhagen
  5. University of Sydney
  6. University of Barcelona
  7. (Tie) Heidelberg University
    (Tie) Northwestern University
  1. (Tie) Mount Sinai School of Medicine
    (Tie) University of Amsterdam
  1. Baylor College of Medicine
  2. University of Southern California
  3. (Tie) McGill University
    (Tie) University of Alabama–Birmingham
  1. Charite - Medical University of Berlin
  2. (Tie) McMaster University
    (Tie) University of Melbourne
    (Tie) University of Minnesota–Twin Cities
  1. University of British Columbia
  2. (Tie) Maastricht University
    (Tie) New York University
    (Tie) University of Edinburgh

 
Source: U.S. News & World Report

Again, there are countless other ranking services in the game, each with its own unique formula for what constitutes excellence in higher education, be it post-graduate earning potential, faculty accessibility, or even campus beauty. We'll offer greater detail on each of their methodologies in our expanding series on college rankings.

The Trouble with Ranking

Now that you understand a bit of what goes into each of these rankings—and how distinct they are from one another—you might have a greater appreciation for some of the inherent challenges to the process.

Each ranking system has strengths, and each has weaknesses. Understanding and recognizing these strengths and weaknesses can truly illuminate the value of each ranking approach. Know what to look for (and know what you are unlikely to find), and you can learn quite a bit about a given college or university.

Ultimately, our critique is aimed not at discrediting the leading rankers but at identifying areas of need in the college ranking space and, more broadly, providing a reference for navigating this space.

Tinkering

Richard Holmes, an industry-leading ranking watchdog, administrates a reference site called University Ranking Watch. Here, he provides ongoing critique and analysis of the various university ranking systems in circulation today. He warns that for many ranking sites, methodologies are in somewhat constant flux. Either in response to academic criticism, or in the interest of refining existing strategies, or even with the intent of generating headlines, ranking services have a tendency to revise their strategies on a somewhat regular basis. This, said Holmes, can sometimes bring about notable and even problematic changes in outcome.

In some cases, these changes are modest on their face but can have significant real-world implications. According to the Atlantic, critics have charged U.S. News with imposing slight adjustments to its methodology every few years merely to give observers something new to talk about. This methodological tinkering can produce statistically significant movement in the standings for individual universities that have otherwise undergone no quantitative or material changes. This fluctuation can undermine our collective confidence in a ranking service's findings, particularly when they diverge significantly from prior findings.

Recent changes to the Shanghai Ranking offer a stark example of this pattern in action. Shanghai derives 50% of its ranking data from a source called Thomson Reuters' Highly Cited Researchers. This source underwent what would appear to be a dramatic revision over the last few years. Between 2014 and 2016, Shanghai made the gradual (though not gradual enough) transition to the updated source.

Richard Holmes notes that the rankings for 2013 reflected the old list of Highly Cited Researchers while the 2014 and 2015 rankings actually weighted the old and new lists equally. The goal in doing so was to ease the transition toward use of only the new list. All indications are that this was not the smooth transition they had in mind. In fact, the movement of some universities was so substantial that Shanghai might well have jeopardized much of the credibility it has earned to date.

Patterns throughout the Shanghai's Top 500 illustrate some dramatic ascents and descents. For instance, notes Richard Holmes, Rutgers University dropped from 39 to 96, University of Wisconsin Madison fell from 18 to 28, Virginia Polytechnic Institute plummeted from the range of 79-102 to 301-400.

Notably, Chinese schools fared particularly well in the transition. Whereas zero Chinese colleges were ranked in the top 100 in 2015, two broke through in 2016…and in a big way. Tsinghua University moved from the 101-150 range all the way to 58th spot in just one year. Peking University leapt from the same range up to 71st worldwide.

Overall, 54 Chinese schools made the Top 500 in 2016, which marked the addition of 10 Chinese schools in just one year.

Speaking free from cynicism, one could argue that most formula tinkering is done to prevent manipulation and gaming. Likewise, the web has given us an ever-expanding access to data. This means our ability to analyze these data are constantly improving. Thus, methodological changes are to be expected.

The Shanghai Ranking can at least be defended on the grounds that these changes will ultimately enhance the accuracy of its data analysis. Whether such dramatic one-year trends suggest bias is up for debate.

What is not up for debate is the empirical and psychological impact of such changes. These changes force us to question the trustworthiness of any ranking system that can undergo such dramatic data transformation in just on year. Even putting aside suspicion of bias, there is no statistical continuity between this year's rankings and last year's.

These kinds of sudden and dramatic movements can leave administrators scrambling for explanations before the public, their student bodies, their benefactors, and their alumni even when methodology and data sourcing—not academic changes—are at the root of these movements.

Lack of Scientific Rigor

In spite of their influence, most rankings lack proper experimental rigor or academic authority. For instance, in 1997, U.S. News commissioned the National Opinion Research Council to produce a comprehensive critique of its ranking methodology. The Council came to a problematic conclusion, finding that there was little apparent empirical justification for the weighting assigned to different variables. The importance ascribed to each seems to suggest little “defensible empirical or theoretical basis.”

In addition to questionable weighting of criteria, critics argue that the use of reputational metrics is itself inherently unscientific in nature.

QS uses its Global Academic Surveys to accumulate 40% of all the data underlying its `World University Rankings. With an additional 10% of its ranking attributable to Employer survey responses, a full 50% of the QS rankings are based on opinion. Granted, this is the opinion of a decidedly large sample population of respondents—the largest sample population of its kind according to QS—so the findings are not without validity. They are, however, highly vulnerable to empirical inconsistency.

In spite of the numerous steps that QS takes to control its findings, the unparalleled weight that it places on reputational survey responses makes it among the least empirically sound ranking methods in circulation. In fact, there are some startling anecdotal revelations from the data-gathering field that should give us pause. Indications are that its screening methods are somewhat porous, and inherently vulnerable to input from survey respondents that may be biased or downright unqualified.

Bias

Another common critique levied against college ranking is the inherent presence of bias, be it socioeconomic, cultural or even academic in nature.

For instance, the U.S. News & World Report rankings reward those colleges which, among other factors, demonstrate exclusivity. This, argues Salon, is used as a way to measure a college's worth, which is not only inherently inegalitarian but may also not be the truest indicator of a school's value.

As Salon phrases it, “these rankings exhibit a callous disregard for college affordability, prioritizing schools that spend more money on flashy amenities rather than scholarships and grants.” The article goes on to argue that “the magazine glamorizes selectivity, which creates a culture of exclusion that shuns low-income students the hardest.”

This ultimately tells us that, regardless of the quality of U.S. News & World Report's rankings, it is a highly flawed resource if affordability is a prospective student's top priority. This underscores one of the major takeaways from this primer on college rankings, that understanding such biases can help you navigate a densely populated ranking sector.

Consider, for instance, the implications of the Shanghai Ranking, which, even its very best, is limited in scope. From their inception in the early 2000s, the Shanghai Rankings were criticized for employing a methodology and a set of metrics that inherently favored scientific programs over humanities and liberal arts.

A quick glance at the criteria that drive the Shanghai Rankings demonstrates a clear and unmistakable focus on the natural and mathematical sciences. Its consideration of Nobel Prizes and Fields Medals is concentrated solely in the sciences and mathematics. The Shanghai Ranking also relies on scholarly citations concentrated entirely in science and mathematics.

Of course, this is not an accidental bias. In the interest of achieving what it views as an empirical set of metrics, Shanghai has dispensed with indicators that it believes can not be objectively defended.

Shanghai Jiao Tong University recognized the limitations in its own rankings in a 2004 study, conceding that “[m]any well-known institutions specialized in humanities and social sciences are ranked relatively low partly because of the imbalances in the production of articles among various subject fields. The Ranking Group tried hard but was unsuccessful in finding additional indicators that are special for humanities and social sciences.”

This academic bias is hardly insidious, but it reinforces an important theme. The better you understand the criteria, methods, and weighting behind each ranking, the better you'll understand what these rankings mean and whether this meaning is relevant to you and your needs. You'll need to decide what information can be gleaned from these rankings and what should be discarded.

Arbitrariness

This advice underscores another challenge that college rankers collectively face. The ranking business on the whole struggles with the problem of arbitrariness. As the leading industry examples demonstrate, there are various and considerably different ways that one can approach the question of college ranking. That each of the major rankings relies on its own wholly unique formula for measuring colleges against one another is itself quite revealing.

Though each of the major rankers has gone to great editorial lengths to justify its particular formula for ranking, the sheer variation of possibilities conspires to leave unanswered the one question that really matters to students: “Which college is right for me?”

According to a study by the Higher Education Research Institute (HERI), “How students use measures of institutional quality (e.g., SAT scores, student-faculty ratio, degree completion rates, etc.) is up for debate. While some research suggests students tend to factor in graduates' labor market outcomes, other analysis suggests that they rarely consider graduation rates and average student debt in their decision.”

Obviously, getting to the bottom of this debate remains a leading preoccupation for ranking enterprises. But the byproduct of this debate is problematic. Namely, many universities have become fixated not on the question of which school is right for you but on which school is right for this year's rankings. This is a recipe for misplaced priorities, most especially when the metrics and weightings used by rankers diverge from the variables that students consider most important.

Conclusion

Whether you use rankings as a key part of the decision-making process or you are merely curious where your top college choices stack up, you should know exactly what you're looking at. As this discussion should suggest, it is not enough to simply take your school's numerical ranking at face value. Dig deeper, consult methodologies, compare rankings and ultimately, remember that you are seeking the college that suits you best. Global standing aside, you must find the school that rates highest where it matters to you. Rankings are merely the starting point.

Speaking of which, this article is also merely the starting point for an ongoing series in which we feature (and scrutinize) both the major players and new entrants in this highly competitive ranking industry. Check the ever-expanding set of links below for a closer look at the major rankers in the game.