In May 1994, I came to Washington, D.C., one of thousands of college students who descend on the nation’s capital each summer, to intern with nonprofit organizations, on Capitol Hill, and with national media outlets. My home for the next three months: U.S. News & World Report.

Each weekday morning, along with half a dozen other interns, I reported to a small, windowless room in the basement of a sleek office building on N Street. From there, we would all stare at monochrome computer monitors as we made phone call after phone call to college campuses around the country.

Our task was to track down missing numbers, or in other cases, double-check questionable figures for the massive data collection that composed the secret sauce for the magazine’s annual college rankings guide. It was often a thankless task made better only by the camaraderie of twentysomethings, the stories we’d sometimes get to help out with for the magazine, and of course, the paycheck (when so many internships are unpaid).

We often joked that with a few keystrokes we could perhaps reorder the rankings, putting any of our colleges in first place, knocking out the perennial favorite, Harvard. (Two of the interns were from Yale and Princeton, respectively, so they would have loved it.)

Little did we know at the time that over the next 30 years the annual rankings, which were only 5 years old at the time, would become so influential that the number-checking task we performed would become so crucial to keeping colleges honest.

By now you probably know that when the latest edition of the U.S. News college rankings came out earlier this week, Columbia University plummeted from No. 2 to Number 18 after one of the university’s own math professors earlier this year questioned numbers submitted by the school. On Monday night, I was interviewed about Columbia’s drop in the rankings on CNN Tonight with Laura Coates. Coates is a former federal prosecutor and was surprised to learn the numbers that go into a list with such influence weren’t audited in any way.

Of course, if it were “Podunk U.” (a favorite on College Confidential for a no-name college) that submitted dubious data, I’m sure U.S. News would have sniffed out a problem long before the rankings were published. But this was Columbia, an Ivy League institution! No one seemed to question the “incorrect” data in two fields for Columbia: class size and the number of faculty with the highest degrees in their field.

One reason no one did is because we all expect Columbia to be among the top schools. That’s the so-called face validity of rankings. Few of us would buy magazine rankings that didn’t list Columbia in the top, nor would we buy rankings that remain the same every year—which is why U.S. News constantly tweaks its formula so there are a few changes on a regular basis near the top of the list.

The U.S. News interns, summer of 1994, with Bob Morse, who still leads the rankings.

The media firestorm about Columbia’s fall from grace this week reminds us of the dubious nature of the ranking in general.

Columbia is not suddenly a less desirable institution because it fell 16 spots. Yet when looking at the rankings, too many parents, students, and counselors think there are differences between schools ranked 10th and 30th or 40th and 70th. The differences are so subtle for the typical applicant—and most are applying to a set of schools within a certain range anyway.

Given their place in our culture, the college rankings aren’t going away no matter how much university leaders wish they would. I asked Colin Diver, former president of Reed College, which refuses to participate in the rankings, how they could be fixed.

Diver, a former dean at Penn’s law school, wrote a book that I highly recommend earlier this year called Breaking Ranks: How the Rankings Industry Rules Higher Education and What to Do about It.

Diver told me the “the perfect rankings system would consist of multiple equally influential rankings, each based on one important characteristic of higher education.”

So, there would be rankings of instructional quality and learning gains, social mobility, academic influence, affordability to low- and middle-income students, institutional wealth and spending, student selectivity, racial and ethnic diversity (of faculty and students), contribution to community wellbeing, and contribution to students’ future thriving and wellbeing. 

No matter how much we desire to have a “best” college or university, Diver told me that no “single ranking, no matter how well constructed and impartially administered, could possibly capture the rich variety” of higher education—in terms of varying missions of colleges but also in consumer preferences.

Other attributes of Diver’s “perfect” ranking system:

• It would published by an impartial non-for-profit organization, such as Consumer Reports—and not the government or a for-profit organization, like U.S. News.
• And after a numerical listing of, say, the top 25 schools, each ranking should report the remaining schools in unranked clusters to rid the rankings of their “faux precision.”

If you were rankings czar, what would you want to see in a new set of rankings? If you’re a parent, student, or counselor, how do you use the rankings in the college search? Hit reply and tell me. I’d love to hear from you and might include your response in a future newsletter.

Good morning. Thanks for reading Next. If someone forwarded this to you, get your own copy by signing up for free here.

For your calendar: Wednesday, September 28, at 2 p.m. ET/11 a.m. PT, is the Next Office Hour. Our topic: how colleges and universities can better meet the needs of an increasingly “nontraditional” learner population with a flexible and personalized learning design that is student-centered.

• You’ll hear from a panel of people with deep experience adapting programs to serve the needs of today’s learners—from recruiting adult students to incorporating 21st century skills.
• Joining me will be Sabrina Sanders, director of the Toro Reengagement Program at California State University, Dominguez Hills; Scott Ralls, president of Wake Technical Community College (NC), Diane Tavenner, author of “Prepared: What Kids Need for a Fulfilled Life,” and Barbara Gooch, a student at University of Maine at Presque Isle.
  Register here. (Support from the Charles Koch Foundation)

Campuses are following the retail and service sectors in using tech to transform their institutions.

The Promise of Digital Transformation

SUPPLEMENTS