Some Interview Practice Info

I just looked at the Philosophy Job Market Wiki, wondering about current interview practices. I divided departments into three groups: those doing eastern division APA interviews, those doing phone interviews, and those just bringing top candidates to campus. The numbers are as follows (as best I can tell):
APA interviews: 113
Phone interviews: 16
Only campus interviews: 32

As a defender of the idea that short interviews provide nothing more than “vivid noise” (Gil Harman’s nice phrase), I think these numbers are revealing. If we go back 25-30 years or so, I’d be shocked to see as many departments going straight to on-campus interviews. But I have no data with which to compare, so my attitude here is largely impressionistic.

If I’m right, though, perhaps the significance of Fundamental Attribution Error and the studies surrounding it are affecting interview practices in philosophy departments. To my mind, that would be a very good thing, both in terms of the quality of the hiring process itself and also in terms of the financial burden placed on graduate students to go to APA’s to interview. Lifting that financial burden will require a much larger percentage of departments bypassing APA interviews, and here’s hoping they see the light!


Comments

Some Interview Practice Info — 9 Comments

  1. There’s also great costs to the schools doing the interviewing: flying their interviewing team out to the convention & putting them up in hotels. Think what that money could be used for instead. And interviewers and candidates alike all being gone for several days over the holidays. All for the sake of something that I’m convinced results in worse hiring decisions.

    But, as I admitted to my colleagues as I tried to convince them not to use convention interviews, those interviews may have this good effect: If everyone used a more rational hiring procedure you might end up with too many places all trying to hire the same candidates. It could be messy straightening that all out. Adding a little noise into the procedure may help the profession as a whole, at least in mitigating the problem of too many places trying to hire the same candidates. However, that same benefit could be obtained much more cheaply by hiring departments rolling dice for each of their candidates, and weighing the results of those rolls into their decisions as to who to have out to campus – as a very major component of their decision. What I proposed was that we simply skip the convention interviews (and the dice: though I didn’t mention that, having not raised the possibility of using dice) and go straight to campus visits based on dossiers, and, with regard to the benefits to the profession of having some noise added to the process, I thought we should just free-ride off the irrationality off the departments doing interviews.

    With absolutely no success: My department decided to do the convention interviews.

  2. Could someone please enlighten me about what departments think they gain by doing convention interviews? Keith mentioned a potential secondary benefit of the “noise.” But what primary benefits do people think they confer? Is it that they really think they’re that good at sizing up philosophical talent (a la G.W. Bush staring into Putin’s soul)?

  3. John, yes, people think they are good at learning, from face-to-face short interviews, who’s good and who isn’t. Even those who know the experimental evidence, and are put in the same situations as the experiments, tend to think they are exempt from the problem, so it shouldn’t surprise us that philosophers think that the experiments don’t generalize to the special circumstances of APA interviews.

    So the benefit is you get to have data on a larger sample of candidates than if you just brought people to campus. Like Keith, I couldn’t convince my present department of the wiser course, though both of my prior institutions had welcomed the idea of only doing on-campus interviews. We are doing only on-campus interviews, but not for the reasons discussed here.

  4. “people think they are good at learning, from face-to-face short interviews, who’s good and who isn’t.”

    Some of us who are both aware of the (overstated by philosophers but still relevant) experimental results and still want APA interviews deserve more credit than Jon is giving us. In particular, I sure don’t go into an APA interview thinking that 45 minutes of generic philosophical conversation will tell me “who’s good and who isn’t” in some dramatic overall sense. I think instead that *buidling on what one learns from a careful reading of the file one can learn additional relevant information by focusing on specific and well chosen bits of incomplete information*. Please show me the studies that show that this can’t be done.

    I try to use whatever time I can in an interview to explore aspects of a file that further time with the file can’t help me with. So, for example, if I have a question about “research” it won’t be “tell me about your research” it’ll be about some specific claim or argument in a writing sample that wasn’t adequately explored in the writing sample. Or, if it’s desirable in making the appointment to have a teaching competency in ethics and 3 candidates all claim to have this, I’ll initiate a specific discussion of what part(s) of the field they know well enough to teach at the various levels. I don’t need 45 minutes to tell the difference between candidates who really do know their way around a field and its literature and candidates who don’t. Similarly — in a metaphysics search with two candidates with very strong writing samples on focused issues concerning the metaphysics of causation…it won’t take me 45 minutes to find out who doesn’t know much besides the causation literature and who has a strong wide ranging grasp of at least several sub-issues. Good luck doing that by re-reading a writing sample or letters. Again, what study shows that I’m unlikely to be able to tell the difference between candidates who say “I don’t really know much about the metaphysics of ‘events'” and the one who shows a strong grasp of how some issues about events matter to the causation debates? [yes, I’m caricaturing Jon’s position…or at least trying to do so – I find myself doing that after seeing the strawman position attributed to those of us who still see value in APA interviews].

    There’s more to be said about the studies Jon mentions and about healthy and unhealthy interview practices. But my current view is that there’s no slam dunk experiemental evidence showing we can’t learn much of value in a convention interview.

    Is my head in the sand Jon?

  5. Fritz, no head in the sand accusations coming from me. I’m sure you know the following, but I’ll say it anyway. There are two parts to Harman’s phrase “vivid noise.” The noise part you address here, and there certainly is information one might want and need in order to decide between candidates. Using convention interviews to elicit such information is certainly possible, and a wise use of such interviews if they are going to be done. But the other part is the vividness point, which concerns the way in which face-to-face contact swamps more reliable information, even when experimental subjects know this fact. What is stunning in some of the experiments is the way in which experimental subjects prefer first-person-contact information even when they claim to know that other data is more reliable (well, maybe “claim to know” should be replaced with weaker language, like “acknowledge” or something like that).

    So the vividness point has two aspects relevant for interviewers. It can be ignored if the interviewer is special, able to discern needed information face-to-face that can’t be found out otherwise. It can also be ignored if the philosophical context is special, or can be rendered special, so that what is usually more reliable information isn’t so, when compared with the reliability of interviewer judgments.

    On these points, there is no “slam dunk experimental evidence.” But there is general evidence that face-to-face short interviews are less reliable than other information such as can be found out from dossiers and calls to references and others who know the candidate, and there is evidence that there is a general human tendency to mistake the quality of evidence when comparing interview information with non-interview information.

    So, for convention interviews to be conducted, I’d like to be persuaded by my colleagues that they can control for these two defeaters of what they come back reporting. I think this can occur in special cases, so if you let me pick the interviewers, the information might be trustworthy (for me). But as a general practice, I’d want some information as to why a particular individual isn’t subject to the vividness worry before trusting what they report about candidates.

  6. Fritz,

    “I don’t need 45 minutes to tell the difference between candidates who really do know their way around a field and its literature and candidates who don’t.”

    I believe you when you say this. But I’m curious how you differentiate between performance error and genuine lack of competence.

    I guess even if you couldn’t do that, the interview might at least give you sufficient evidence that the one candidate does know her way around the field. The danger of a false positive seems much lower than a false negative.

  7. If depts just bring a few people to campus, you can count on this effect: the vast majority of job seekers will not get interviews at all. Not one. And that will just reinforce the unbearable elitism of an already seriously unjust profession.

  8. A question for those who are against APA interviews but for fly-outs: what makes vivid noise less of a factor for the latter? Are there empirical evidence that show the personal interactions on campus, and indeed, the job talks, do not have the same effects as personal interactions during interviews? Fly-outs may be less costly than APA interviews, but it is not clear what benefits they would bring either.

  9. Most places require on-campus visits in order for any hiring to take place, so fly-outs are not optional. Moreover, an on-campus visit involves significantly more time with candidates, and in a significantly wider variety of settings, and usually with the entire department rather than just with a few representatives. There is evidence that larger groups are better predictors than smaller ones, and it would be surprising if longer contact across different contexts wasn’t useful information as well. There will still be the vividness problem, since the on-campus information might not be as reliable as other information, but since on-campuses are typically required by upper administration, that’s an issue for departments to try to compensate for rather than eliminate.

Leave a Reply

Your email address will not be published. Required fields are marked *