How to Protect From Ranking-mania
By Jeffrey E. Stake, Professor of Law
Indiana University School of Law—Bloomington
There are lies, damned lies, statistics, and rankings. Rankings create hidden problems for people who are choosing a school, people who are affiliated with a school, and for society at large. They can discourage applicants, students, and teachers. They can cause applicants to choose the wrong school and schools to admit the wrong applicants. Rankings can even cause schools to teach less well by deflecting a curriculum from what matters to what will improve rankings. The points below are designed primarily, but not exclusively, to help students choosing a school by exposing some of the issues lurking within rankings.
Your values are Your values. If you have already played The Ranking Game, you have seen that rankings are highly dependant on the choice of factors and the weights applied to the various factors. (If you have not yet done so, try the examples.) What are the correct weights to use? There are no weights that are right for all people. That is one reason why rankings with arbitrarily fixed weights have the dangerous potential to mislead students and employers. If people in the legal education business do not rely on published rankings for making their decisions, why should you?
Many criteria matter.The Ranking Game includes data on nearly twenty criteria. There are many other factors that should be important to you, from academic considerations, such as quality of teaching (and research) done at the school, accessibility of faculty, and availability of clinical, international, interdisciplinary, and joint degree programs, to employment concerns such as strength of alumni network, to local leisure opportunities such as music, spelunking, and soccer. Because important factors are missing from all rankings, it would be unwise for you to rely solely on the rankings you create here or those published anywhere. Thus, a second reason that rankings are dangerous is that important parts of the picture are missing.
Data may be wrong.Third, rankings are dangerous because the underlying data are not reliable. Much of the information used by this program (and other rankings too) is supplied by the schools themselves. Different schools may have different ways of counting courses, faculty, students, or jobs. On top of those potential variations in approach, it is entirely possible for the administrator reporting the data to make a mistake that no-one catches.
Real differences are small.All rankings strip away information about differences in degree. In some cases this makes law schools look more similar than they are, but in most cases it makes schools look more different in quality than they really are. Law schools vary more in character and style than they do in quality. Notice that many closely ranked schools have very similar total scores, and even schools with quite different rankings can have similar total scores. With an insignificant change in the data, such as two additional students passing the bar exam, a school could jump over competitors with similar total scores. In other words, rankings are dangerous because they mask the fact that the differences between schools are negligible on many criteria. For schools that have close scores, whether the school colors look good on you is as important as the difference in rank.
Rankings wrongly make some schools look bad.Imagine a ranking of all NBA basketball players, and imagine that list being viewed by a person who had never seen an NBA game. If such a person were to read the ranking of NBA players, he might think that the dozen players at the bottom of the list were terrible at playing basketball. Little would he know that they can all play very well. Readers of school rankings might be similarly misled. Those choosing a school might be wrongly influenced to pursue some other career. Those in school (teachers and students) might be unfairly discouraged. So keep in mind that educational programs do not need to defeat other programs to be winners. Approved schools, just like NBA players, can all play the game.
Education has many winners.The effect of ranking schools like olympic divers may be to cause readers to forget that unlike games, which use losing as a concept to give them meaning, educational programs do not need to defeat other programs to be winners.
Do your homework.What should you look to choose a school if rankings are so unhelpful? To determine which school should be tops on your list, read what you can about all the schools under consideration. Look for information on internet web sites and in libraries. Other sources you might consult include The University of Richmond Pre-Law Handbook, the Boston College Online Law School Locator, and the listserv at email@example.com. More important, visit the schools. Find out what courses are regularly offered in specialties you consider important. Talk to the students. Find out if teachers are accessible and committed to devoting substantial energy to teaching. Find out if faculty members have published in areas important to you. Talk to faculty members. Only by doing those kinds of things can you begin to tell where you will fit in well. Along with the Deans of more than 150 law schools, I urge you not to rely on published rankings. Instead, do your own leg work and find the school that best meets your needs. It will be worth it.
Rankings can identify surprise candidates.Are rankings useful for anything besides a good laugh? Perhaps. You might allow your rankings to suggest schools you had not thought of, schools to be added to your list. But do not allow the rankings to be used for the opposite purpose, to eliminate schools. Do not abdicate decisionmaking authority. If you can remember that rankings are neither reliable nor necessarily relevant to your concerns, you will go a long way toward protecting yourself against their dangers.
It's a game. So have fun. But do not let the apparent objectivity of the data fool you into thinking that the rankings based on those data are objective. Ranking games played with real data are still just games.
Rankings are dangerous to all of us. Rankings of schools can harm society. Unfortunately, society cannot protect itself the way you can protect yourself. Click here for an explanation and examples.