Grad Applications and GRE Scores

It is getting close to the time for graduating seniors to submit applications to PhD programs, and I’ve been reflecting again on our discipline’s practices in this regard. Especially, I’ve been thinking about the role of GRE scores in the process.

The role of GRE scores in any given graduate program is fairly obscure, but it is fair to say that there are institutional pressures to recruit students with higher GRE scores. There are various ways to succumb to these pressures, from a crass preference for higher GRE scores to a system in which first cuts are made on the basis of GRE scores. I don’t mean to attach any significance to the question of whether such practices amount to succumbing to pressure, however; my concern is with the practices themselves.

My concerns about the role of GRE scores are two-fold: an issue of fairness, and an issue of substance.

It is by now well-known how malleable GRE scores are. Because of the nature of the game, I’ve told students with scores in the 1100’s that they must do whatever it takes to get their scores up. In some cases, the scores have gone from the 1100’s to the 1500’s.

So suppose a grad program is evaluating such a student: multiple takes of the GRE, scoring from the 1100’s to the 1500’s (maybe the multiple takes were of actual tests, maybe only of practice tests–it makes no difference to the case, though of course it would make a difference to what an admissions committee knows regarding the number of takes). Compare such a student with another student who takes the GRE once, scoring in the 1200’s. I have only anecdotal evidence for the following claim, but I’m quite confident the first student will pass GRE scrutiny at any of the top 20 PhD programs in philosophy, and the second student will fail nearly all the first cuts.

First, the issue of substance. I doubt there is any evidence in these two records to sustain a judgment that the first student is more likely to be a credit to a top 20 PhD program. Whatever evidence there is in favor of using standardized tests, I’m pretty confident that there is no data available to show that those who take the exam (or a practice exam) every day for three months have a better chance of success in graduate school and in the profession. It is true that the stamina and determination shown by a willingness to undertake such an effort is relevant to success in philosophy, but such characteristics alone are surely inadequate and much less important than evidence of philosophical ability.

This point leads to my second point. If one wishes to stake the relevance of higher GRE scores by multiple test-taking on the personal qualities displayed, there is a critical unfairness involved in such practices. We don’t know why the single test-taker, scoring in the 1200’s, took the test only once. But it is a certainty that those with lower income and less leisure time will have less opportunity to take practice tests, enroll in prep courses, etc. So not only is there a question of the relevance of comparisons between the best score by a multiple test-taker and the best score by a single-test taker, there is also a question of unfairness involved.

The question is what to do about this. One might argue for averaging the GRE scores of multiple test-takers, but such a practice will prompt even larger amounts of test-taking and test-preparation by the most advantaged, in order to continue to raise one’s average. I’m uncertain what practices to recommend here, but here’s a quick idea, and I solicit other ideas as well. Why not set a reasonable lower limit on GRE scores, high enough to predict success but low enough to discourage multiple test-taking? (Maybe 1400 at top 10 or 20 programs, maybe 1300 for those a bit lower, and maybe 800 for . . . U. of Dallas. Something of a joke, that is, to see if you are paying attention, but see the Hirsch number rankings in the right sidebar for the basis of the joke … 🙂 ) (One could insist, for example, that the lowest GRE score will be used, and then have an appeals process if anyone on the admissions committee is inclined toward the candidate in spite of the score, thus allowing students who claim to have had a “bad day” to make a special appeal and have the lower score not counted. Whatever policies are adopted, they’d have to allow room for exceptional cases, so one shouldn’t object to the general policy recommendation because of the need for exceptions–that’s just to favor the status quo policy, which, I’m arguing, is wrong both on grounds of fairness and substance.) Then applicants could be evaluated by the admissions committee on the basis of applicant information in which the GRE scores have been removed.

There is also the option of no longer using GRE scores, but I doubt that will happen anytime soon. In the meantime, I think there is both an argument from fairness and one from substance for minimizing the effect of higher GRE scores.

One other point worth making. If you knew that a student could get his GRE scores up enough to get into your program with enough effort, you’d surely prefer that effort to be devoted to philosophy itself rather than a standardized score. Of course, you don’t know that. But the preference itself counts as an additional consideration in favor of a negative view of multiple test-taking.

UPDATE: In response to some of the comments below: My concerns here are not meant to be general concerns about the role of standardized tests such as the GRE. My primary concern is rather with what I perceive to be the growing preoccupation with getting higher and higher GRE scores to compete at top departments. When prospective students spend thousands of dollars and hours trying to get scores higher than 1500, there is something seriously wrong. Such expense and effort is surely a waste in terms of the interests of our discipline in recruiting and developing the brightest and best who are interested in philosophy. It is a gross extravagance that responsible departments need to respond to, by adopting practices that discourage such a disorder among priorities, a disorder that stunts philosophical development in favor of competence at a certain type of test-taking.


Comments

Grad Applications and GRE Scores — 20 Comments

  1. This post is incredibly apt. I sent three emails out to prospective departments (that is, their respective graduate directors) concerning the status of GRE scores and admission into their grad programs with funding.

  2. I’m a graudate student who once served on a panel for prospective undergraduates about getting into grad school. During the panel, a highly respected senior philosopher at a PhD-granting school outside of Leiter’s Top 50 said that he thinks high GRE scores are a good indicator of success in graduate school, but low GRE scores are not an indicator of anything.

    Anyone care to confirm or deny this?

  3. ***Anyone*** care to confirm or deny?

    My GRE scores were very unimpressive (less than 1000!), but my papers and grades in philosophy courses, and my recommendations are absolutely stellar.

    Anyone?

  4. On tpy’s question, the only relevant information would be statistical evidence. Philosophers, even well-respected senior ones, shoot from the hip on these things a bit too much. I expect you could find general statistical data at the ETS site, but it won’t be data that goes discipline by discipline. And it certainly won’t include data about how many practice or real tests it took to get the score one got.

    Perhaps the results will mirror SAT studies, where good scores correlate well with success in college, but where the correlation pretty much disappears once grades are controlled for. That’s still a general result for all majors, and might not hold for particular disciplines.

  5. I’m an applicant this year (and thus wish to remain anonymous to all except the blog admin), and my GRE scores are less than the average for a top 10 school (but not in the 1200s). This is my second time applying and because I have a family I had the choice between doing philosophy in the summer or studying for the GRE. I chose philosophy. I think I made the right choice (my writing sample is much better than it would have been and I got two pubs in good journals).

    I think that the best solution is to get rid of the GRE requirement. But since I doubt that will happen any time soon, I think the next best thing is for schools to be very up-front about how they use the GRE part of the application. If they are one of the schools who discount the GRE quite a lot, then they should say that the GRE is by far the least important part of the app. But, if they are one of the schools who count the GRE for quite a lot, then they should say that. It’s frustrating for applicants to not know how things will be weighed–and it probably ends up forcing some applicants to waste lots of money either applying to a tremendous amount of schools or retaking the test many times.

    I also think that it would be bad to set a hard and fast cut-off score. I know of several cases of students getting into top 15 (and in some cases top 5) schools with GRE scores less than 1350. It would be *very* unfair to those students to have a hard cut-off point.

  6. What is the average cut-off point for departments of GRE scores? What scores-ranges would anyone consider poor (i.e. retake the GRE!), acceptable, or superior?

  7. Amanda,
    I am a graduate student in an M.A. program, and I am in the middle of the application process for PhDs, so I have been asking the same question. Based on anecdotal evidence (from maybe 15-20 students), the cut-off for great but not Top 10 programs seems to be around 1300. I am sure this varies, and it higher for Top 10 programs, but it seems like a good number to shoot for.

  8. A note of caution about fairness arguments, relevant to the last sentence in comment #5. It is not unfair to use an inflexible standard that treats all applicants the same, and those who don’t meet the standard have no legitimate unfairness complaint against such use of a standard. In this way, the use of inflexible GRE standards is not susceptible to the unfairness argument of the sort I used in my post. The use of an inflexible standard is unwise, imprudent, and subject to other criticisms. But it is not unfair, so far as I can see.

  9. Thanks for the response, but how much do GRE scores matter to people applying (with funding) to M.A. programs? How much do they matter there?

    Do they delimit who gets funding and who doesn’t in PhD and M.A. programs?

  10. For those wanting to know what weight is assigned to GRE scores or what scores are good enough scores, there is no known, general answer to either. Given the obvious wisdom of having no inflexible cutoff, no such information will be forthcoming from any source. What you might get from a particular grad program is advice on what that particular program looks for. It’s better to talk to the programs you are interested in than to look for general answers in places like this.

  11. Hi Jon,
    You didn’t mention another important factor in the unfairness argument: The GRE is an exam that (pace those who make the exam) on which one can improve one’s performance, not only by retaking the exam, but by taking advantage of the many prep services before hand. (I strongly recommend our majors who want to go to grad school to do this, for instance.) Those with money can afford to take the Kaplan course for instance, even multiple times, before taking the exam. That doesn’t show up at all on the exam reports at all. So even if you average or discount for multiple tries, you can’t control this other factor. So I’m not sure there’s much that one can do about the problem.
    Grad recruiters, however, can minimize impact: When I go through applications, I myself (when I’m doing this job in the department) first form an opinion without looking at the scores, based on all of the materials. Once I have a ‘no numbers’ ranking, I then go back and think about whether I want to revise my ranking in light of the numbers. (Usually, that will only happen if someone was thrown out of the running, but then was discovered to have high scores.) But then again, since we’re not a top department we have to work harder to find our recruits.

  12. Hello,

    Here are a few places to find some information for those that are interested:

    http://www.fairtest.org/facts/gre.htm
    http://www.news.cornell.edu/releases/Aug97/GRE.study.ssl.html
    http://www.highbeam.com/doc/1P1-68590244.html
    [The title of this one is a bit silly, but the references are legit]
    http://www.acm.org/ubiquity/views/pf/v6i21_orlando.pdf

    I think it is a good idea to try and find information about the GRE that is independent of ETS (this is not an easy task). Though I think ETS can be somewhat objective, taking their word about the predictive validity of the GRE feels a little like taking Bush’s word that his policies are effective.

    As a non-traditional student I am at a serious “standardized” disadvantage. Because of this I find Jon’s last point to be the most pertinent. Nevertheless, I will probably have to put philosophy aside for awhile so I can standardize myself and hopefully get a good score on the GRE. It is a real shame that after working so hard to graduate with a high GPA, developing a good writing sample, and having good letters of recommendation, I could still be perceived as inadequate based on a one-time score on some test (by inadequate here, I mean not make some good programs cut-off).

    What’s worse, is that it is not even clear that the GRE is a good predictor of rate of attrition. One would think that graduate programs would want a test that correlates with the completion of their program, not merely with first year success. I can’t help but think many people just want a quick and easy method for selecting students which is unfortunate because I think hard-working individuals deserve more than this.

  13. Another reason for grad schools to treat GRE scores with caution:

    Non-native English speakers are at a disadvantage in taking such tests.

    Talented and motivated foreign students are thus at risk of being evaluated unfairly.

  14. Robert, I like the “no numbers” approach you use better than any of the ideas I included in the post. My only worry is that the number crunchers will still revise in a problematic way on the basis of GRE info, once it is added in. Moreover, departments that get, say, 300-400 applicants won’t want to pursue a “no numbers” approach first, since it would be too time-consuming. To which I’m inclined to say, “yeah life sucks so bad when you are in a top 20 department…”

  15. I think Kvanvig’s point is well demonstrated in the comments to this post. You have a bunch of frantic students stressing about a GRE score, when they could be progressing in more discipline oriented activities. Also, it is elitist to demand higher GRE scores (at all cost) knowing that this potentially subjects you to a particular “class” of individuals who have the time and resources to do so.

    Then again, my experience of Graduate school has been a realization of what I can do without. I can do without TV, I do without… a host of things, as I focus in on my academic work. In that regard, the GRE is not a secret… you know it’s coming. So you begin early, watch TV or refresh on algebra that you had 5 years ago in High School. As a student, the GRE (in my opinion) is another leveling of desires: do I want this? or do I really want to get into a quality Ph.D. Program and do this Philosophy or Theology thing for the rest of my life.

  16. I have to agree with Jon’s concerns here about the GRE. Having served on a half-dozen or so admissions committees, I have seen a good many students whose GRE scores were not in the 1400 or 1500 range but whose written work indicated a basis for thinking that her/his future in philosophy was bright. The problem is that, at a department that gets e.g. 200 applications annually, it is hard for the committee to read each of the papers at all, let alone read each with care. (This is so even if the work is split up, so that each paper is read by only one committee member.)

    In light of this I would strongly encourage those students who have lower GREs to (1) use the (brief!) cover letter to highlight those features of their application materials that indicate their philosophical promise, and (2) consider either a short writing sample (

  17. dunno why the rest got cut but here it is:

    and (2) consider either a short writing sample (less than ten pages) or indicate to the committee where it might read if it only wants to read five pages or so of a longer paper. (There’s a nice discussion of this on Leiter’s blog.)

  18. A few comments:

    (1) I believe that there was once evidence in ETS’s own information about how to use the GRE that women with lower GRE scores do as well in their subsequent schooling as men with somewhat higher scores do. In other words, the test under-predicts the success of women applicants. I don’t have a link to the data and I may be misremembering whether this was about the GRE, the LSAT, the SAT or another exam of roughly the same sort. The Fairtest website (already mentioned by John above) says that the GRE under-predicts the performance of older women, but I thought I’d seen something broader than that.

    (2) Many departments have no consensus on the value of GREs. I generally discount them while some in my department would not go that far. And the same range of variation existed at the very good grad program I went to at the time I went there.

    Given such variation and depending on how admissions is done in a department there just may not be a fact of the matter about what the cut-off is or even if there is a cut-off. There may be vote which just weighs different people’s judgements on the totality of the evidence where each voter gives different factors different weights. Or there may be a process with different stages where different folks pass people on to the next based on their assessment of the total evidence.

    Adopting a policy about a department rule might in some of these cases actually make them count for more than they would otherwise.

  19. Thinking carefully about whether GREs have any predictive power is a good idea, but it’s hard to predict the effects of changing the role of the GRE in isolation from other changes. So far as I see it, the primary advantage of the GRE is that it provides standardization. If you de-emphasize the GRE, you could actually exacerbate the fairness concerns you raise by leading schools to place greater emphasis on academic pedigree.

    I don’t think that’s yet a good argument for using the GRE’s, mind you. Just a consideration to keep in mind.

  20. Following up comments 10 and 18:

    I’ve been on several grad admissions committees at two different PhD programs, and while GRE scores were always part of the applications, none of the committees I’ve served on have had policies on how to weigh GRE scores. Committee policies concern such things as how different applications are assigned to particular committee members for the initial evaluation, what kind of output that initial evaluation is supposed to yield (e.g., a numerical score on a scale of 1-5, a letter grade of A [should admit], B [close call], or C [should not admit]), how those initial evaluations yield some kind of short list, etc. But how to weigh GRE scores has always been left up to the individual evaluators, in my experience. So, as previous commentors have mentioned, there’s really not much that can be said about how GREs are weighed, for instance, “at Yale”: That all depends on who is assigned the application.

    Departments can keep records on the GPAs and GREs of all accepted applications, and could then at least have something to say to those trying to determine whether they have a chance by citing what kind of GREs and GPAs have gained admittance in recent past years. But that information would have to be treated with great caution, because it could all change very much from year to year.

    I can say that I don’t recall GREs ever being much of a focus of discussion when the committees I’ve served on met to discuss which of the candidates on the short list to admit. But I wouldn’t conclude from that that GREs haven’t been important: They’re probably most important in early stages, and by the time you’re at the committee meeting discussing the final stages, their importance may well have faded quite a bit. So I can’t even say how important GREs have been to admissions committees I myself have served on.

    And it’s not even easy to say how important they’ve been *to me*. There’s a bunch of fairly easily accessed bits of information that I take in all at once: GREs, GPAs, then, taking a bit more time, transcripts – where I get to see which classes the student has taken & the grade received at each – and a quick look at the personal statement. All of that goes toward an initial impression I form before turning to the parts of the application that take more work to digest – and that are more important: letters of rec. & writing sample. It’s hard for me to give any kind of quick answer that’s at all accurate to the question of how important GREs are to me. They interact with other pieces of information. When Yale students want to know what their chances are for getting into good programs, I’ll often offer to look at their transcripts – exactly as they’ll appear in their applications, complete with the course titles that will be included – together with their GREs, and I feel I can give them a good indication of what kind of position that quickly accessed information puts them in, before evaluators get to the letters & writing samples. But I am unable to describe any formula that would allow someone to run this quick check on their own.

Leave a Reply

Your email address will not be published. Required fields are marked *