“you cain’t stop what’s comin’, it ain’t all waitin’ on you.”

said by Ellis in No Country for Old Men, and brought to mind when reading Fish’s discussion here of the recommendations by the Texas Public Policy Foundation apparently being implemented at Texas A&M and being considered elsewhere in the state, as reported here. I hope to be wrong, but my pessimistic side always reminds my optimistic side of the track-record argument…

Fish takes the opportunity to whine about teaching evaluations in response, and shows a common epistemic foible in the process. Here is a well-taken point: teaching evaluations are not the Grand Unified Metric for finding out who the best teachers are, and committees and think tanks that make recommendations focusing on this area alone are doing nothing more than GUMchewing. But no one should think that student opinion has no bearing whatsoever on the question of whether quality teaching is occurring. The debate should be over the weight assigned to this one piece of evidence, not whether it is (incremental) evidence at all. (And efforts should be made to filter the raw numbers by discounting the effect of handing out cookies on the day of evaluation or by giving students reasons to expect really high grades in the course…) (One point of agreement, though: I’ve always thought the best time to give student evaluations is about 20-30 years after the course is over, qualified of course in terms of the problem of not remembering.)

But focusing on teaching evaluations, their history, and current and future role isn’t the right focus. It isn’t what brings Ellis’s remark to mind. There are deeper signs of inanity.

Here, in summary form, are the seven reforms:
1. Measure teaching efficiency and effectiveness and publicly recognize extraordinary teachers.
2. Implement a voluntary program that gives cash bonuses to instructors based on student evaluations.
3. Split teaching and research budgets.
4. Require evidence of teaching skill for tenure.
5. Use “results-based” contracts with students to measure quality.
6. Put state funding directly into the hands of students.
7. Create results-based accrediting alternatives.

Some of these are not reforms, but are already being done. For example, hardly any institutions of any quality don’t require evidence of teaching skill for tenure. Moreover, I doubt there are many universities that don’t have a bevy of prizes and public recognition for extraordinary teachers.

But then there are the details. Here’s one, quoting from the second linked source above:

It calls for most tenure appointments to be given to teachers who have taught “on average three classes per semester and thirty students per class for the seven or more years that a teacher is on the tenure track,” and for student satisfaction ratings to determine teaching effectiveness. Average teacher ratings, the reform states, must be at least a 4.5 on a 5.0 scale.

Both requirements show that the members of the Foundation in question are incompetent, from some simple points such as the number of years prior to tenure for most institutions to more substantive issues regarding class size and what normal teacher ratings look like. Imagine a change in graduation requirements, so that curving of grades was not allowed and no student could graduate without a GPA of at least 3.5 on a 4 point scale. That is an appropriate analogy for the teacher rating requirement. I’ve seen lots of university-wide numbers on teaching evaluations, and a recommendation like this, informed by the realities in question, could only be made if one is willing to assume that most university instruction is really bad. One might wish for some evidence for that assumption before endorsing the recommendation. Finally, the average class size recommendation is laughable, and ignorant almost beyond belief, apparently unaware of disciplinary differences in terms of how instruction ought to be handled and what quality instruction demands in terms of contact moments between instructors and students. To make a recommendation focusing on the minimal number tolerable is egregious.

It makes me want to start a school. Give me public funding. The school is for offspring of idiots at this public policy foundation. When they show up, I remind them that this school is excellent. We fire every teacher their kids don’t like. And, most important, each child is in a large class, which of course, is a A REALLY GOOD THING. In fact, the teachers hound me every day to give them more students:  “three figures yes!  four figures, better!”  Everything is bigger in Texas you know.


“you cain’t stop what’s comin’, it ain’t all waitin’ on you.” — 15 Comments

  1. Teacher quality is an important factor in student success in the classroom in both public and higher education. The proposal to use student evaluations as part of an effort to reform Texas higher education recognizes this fact and seeks to reward the best educators. In the past couple of days, criticism from outside the state has appeared – criticism which ignores several key parts of the proposal.

    • The program is voluntary. Faculty members are not required to participate. The plan rewards those teachers based upon evaluations and the number of students taught. This encourages faculty to teach as many students as possible.

    • Existing evaluation forms submitted at the end of the year are used to rate the teachers. These evaluations are typically conducted before final grades are awarded. Multiple studies have shown that students’ ratings are not biased by their likely grades, thus limiting teachers’ incentives to award higher grades in an effort to secure a higher evaluation and thus, a bonus. Additionally, all faculty members are encouraged to agree to limit high grades and grade inflation when first joining the program.

    • Studies show that student evaluations are effective measures of teacher performance, especially when the goals and expectations for a course are clearly laid out.

    • These bonuses would be available to all teachers, not just tenured professors. As we showed in a 2009 article, 70% of courses taught in public universities are taught by non-tenure-track faculty, including graduate teaching assistants. The average tenured professor teaches fewer than three courses per year. Non-tenured faculty normally earn far less than tenured faculty, often as little as $10 per hour. Those teachers who do the most and best work of educating our youth should be rewarded for such.

    The criticisms of student evaluations also ignore the fact that these are only one part of a much larger reform plan that encourages students, parents, and taxpayers to become more involved in improving higher education at public universities.

    The Foundation has no problem with sound academic research; after all, we are researchers ourselves. But as important as academic research is, it is secondary to the primary goal of educating students. These are public universities funded by public tax dollars established to educate our citizens.

    So while there may be value to “highly qualified scholars working on problems that may have no practical payoff except the unquantifiable payoff of advancing our understanding of something in philosophy or nature that has long been a mystery,” the public should rightly expect accountability from these institutions for the tax dollars being spent. And unlike the students who the critics say “may not realize [the value of a course] for decades, the public is pretty savvy at understanding immediately the value it is getting in return for its money.

  2. In fairness, I decided to approve the comment by Bill Peacocke, even though it is boilerplate aimed at Fish’s column and unresponsive to the specific criticisms I raised in the post here.

    Bill, if you wish to engage on the specific points raised here, I’ll approve the comments and continue the discussion, but I will treat anything else as spam.

  3. It may also be worth responding to the content in Bill’s PR attempt.
    1. Why should teaching as many students as possible be a good thing? That’s nuts. If I know I can get the same evaluation numbers from teaching 100 a semester or 10,000 a semester, why is it *in the interests of the student* for me to do the latter? If I’m a student, I’d be against this idea.

    2. The voluntariness of the program isn’t relevant. If you tell me that those who wear a green shirt more than other profs will get a bonus, but the program is voluntary, I’ll enlist. The relevance of this to anything important is yet to be demonstrated.

    3. points 3 and 4 above are truly bizarre. If you can show that evaluations are responsible measures of teaching quality, you must have some independent means of assessing teaching quality. Any responsible approach here would favor using that independent means.

    4. The last point, and follow-up paragraphs, show why the foundation in question is getting ridiculed publicly. Nearly everyone knows that universities are about educating people, but apparently too many think education involves just the transmission of known information. That’s a confusion between a teacher and instructor, on the one hand, and a professor on the other hand. Professors are supposed to not only communicate what is known (in the very loose sense of that term) but also enlarge the boundaries of what is known. The foundation’s recommendations thus show a rather simple failure to understand what is relevant in the K-12 system and what is relevant at the university level. The US higher education system got to be the premiere system in the world by recognizing and rewarding the difference between secondary and higher education. Apparently, the difference wasn’t communicated well to those working in the Texas Public Policy Foundation.

    The saddest of all is that the politicians in Texas don’t understand either. Recognizing and rewarding good teaching is always a primary goal in university settings. But it isn’t the only goal, and figuring out how to measure it takes more than a kindergarten-level-understanding of reducing it to teaching evaluations (or to the even dumber idea that it is some function of evaluation numbers plus numbers taught).

  4. I’m not a great fan of informal fallacies (by which I mean I’m not a fan of calling certain sorts of informal reasoning fallacious), but there really ought to be a name for the kind of reasoning demonstrated by Bill Peacock when he suggests that opinions from within Texas are worth more than those from without.

    Jon, on a more substantive point, people (both fans and foes) have a tendency to think of student evaluations in isolation. But, I think, one can use them in a smart way to confirm or disconfirm various hypotheses along with other evidence. And the numerical information is often less useful than the pattern of student comments one gets when students are allowed to explain what is good or bad about a course. Of course the way in which they are useful is not a way that is apt for incorporation into the sort of paint by numbers reform proposed by Mr. Peacock and his friends. If the value of a bit of evidence varies with the other evidence available it is unlikely that a bureaucrat will hit on an algorithm that captures that feature.

  5. Jon,

    I don’t think you are reading Peacock’s comment with sufficient charity. The idea that larger classes yield better educational outcomes is so manifestly absurd that it couldn’t possibly be what is meant. Given Peacock’s lack of educational background on policy matters pertaining to education, I assume his main concerns lie elsewhere. Presumably, he is one of those establishment conservatives whose goal is to dismantle all public sector goods. Since our public university system is the best in the world, and Texas’s is among the best in the nation, it would presumably be a big achievement in his world to be involved in dismantling the Texas public higher education system. By turning Texas’s public universities into enormous factories, with huge classes, entertainment style education, and driving out all the leading researchers, he would presumably accomplish this goal. All the educated parents would then send their kids to private universities, which is a win in his world.

  6. Their idea, Jason, is to quit funding universities directly, but rather to give money to students who would then choose where to spend it. They may also have the deeper agenda that you mention, but they don’t say that overtly.

    But if you are right, let’s go all the way: no more public roads or water systems or sewer services, no more food safety inspections, etc. What a vision!

  7. Jon,

    The irony is that Texas IS going that way with regard to public roads, sewers, etc. via sloughing them off on “Homeowners’ Associations” and Texas is super keen on them (many state legislators are on HOA boards too– I’m from TX). There was just an NPR story about one HOA in Texas foreclosing on a soldier deployed in Iraq because he was late on $800 in dues. So not only does the state not provide your sewer, but private individuals can foreclose on a home you own. Progress.

  8. “But as important as academic research is, it is secondary to the primary goal of educating students. These are public universities funded by public tax dollars established to educate our citizens.”

    Public money goes into educating citizens. Research is an unnecessary bonus.

    Frege, the founder of modern logic, and Cantor, the founder of modern set-theory, were both teachers. And researchers. Why should public money only fund teaching, and not research? This could have made Frege’s, Cantor’s discoveries – and indeed, the discoveries of tens of thousands of researchers – impossible. State universities should be proud of their researchers.

    By the way, if university teachers stop doing research, what will they be teaching in 50 years? The Bible?

  9. Julien, thanks for noticing that priority issue! I had let it slip past, and it needs to be confronted directly. It is simply a non sequiter to argue from the premise that these institutions are funded by tax dollars and designed to educate our citizens to the conclusion that research and teaching are not to be equally weighted at research universities, or that there shouldn’t be research universities at all. Peacocke and the Foundation show again here a failure to grasp the difference that legislatures have been wise to recognize: teachers are one thing and professors another, and it is a good thing to have higher education institutions that not only disseminate what is already known but enlarge the boundaries as well. That is integral to the idea of “higher”, as opposed to “secondary”, education, and, to me, it is stupid beyond belief to have people thinking that because the money is public and for education, it can’t be targeted equally at research and teaching.

    In place of the quote you started with, we should say: “As important as research is intrinsically, it is even more important because it and the quality of mind displayed by those engaging in it are integral to the task of educating citizens. So even though educating citizens is the primary good here, that isn’t pursued best when the importance of research is minimized nor should it be identified with what happens in classrooms. The facts about intrinsic goods and primary goals doesn’t tell us how best to pursue that goal, just as the intrinsic good of being happy doesn’t tell us how much time or money to spend on any particular activity.”

  10. MBA in Public Finance? MBAs are almost invariably in the field of business administration; often people will have a concentration of around 15 hours, but that rarely shows up on a diploma or transcript. Also, public finance sounds more like an MPA than an MBA, unless Houston has some unusual program. If the words “in public finance” aren’t on Peacock’s diploma, that could be a good case for firing him.

    Jason: that’s a charitable interpretation? I’d hate to get on your bad side. I know, you mean hermeneutic charity, trying to make (in context) the best sense of the statement. But still . . .

  11. Carl, from the UH Graduate Catalogue on MBA degrees, there is a 10-course core requirement plus 6 electives. Regarding the latter, it says,

    Electives can be selected from the following business areas: accounting and taxation, finance, general business, international business, management, marketing, MIS, and operations management.

    Website source here.

    When I look at the coursework that can be taken, say, in Finance, but at the other possibilities as well, there are no courses, except possible independent study courses, on public finance. So I suspect this is fluff, without substantiation in the coursework taken and not on the transcript or diploma either. (Though it is possible that Houston used to offer things that they no longer do…).

    On Jason’s charity, the goading was deserved. But, if we define small classes as, say, something less than 20, the recommendation about class size would have a natural effect of minimizing or eliminating small classes. Of course, it’s not required, but if my tenure decision depends on averages of 30 or more, I’d ask strongly never to be given a small class.

    Just what one would expect from a business model of higher education and certainly not what students want or should want.

  12. Guess I should make clear that I’m not calling for the guy to be fired. There was an implicit irony intended, which I don’t think came across. I read in the Chronicle that people get fired for sometimes small unclarities or omissions on their CVs, and sometimes this is just used as a pretense for getting rid of troublemakers. The intended irony was that when Peacock is talking about tenure, he’s talking about, essentially, whether other people should get fired. In that case, he should perhaps be held to standards of performance analogous to those he wants for faculty (very strict and unyielding, as well as somewhat irrational). Cut him a break on his CV, and cut the poor prof a break when he finds that ancient philosophy, or epistemology, doesn’t draw 30 students.

  13. From a recent article in Dallas morning news (here):

    “Justin Keener, vice president of policy and communications at the Texas Public Policy Foundation, said it is essential for public colleges and universities to post professors’ salaries online.

    “This is a public institution, it’s public dollars being used, and the public deserves to know how it’s being used,” Keener said. “The taxpayer deserves to see what they’re paying you.”

    If professors don’t want their salary information to be open to the public, they should work at private institutions, Kenner said.”

    So that’s how you recruit the best to public education. The Texas Public Policy Foundation is clearly on the charm offensive these days.

Leave a Reply

Your email address will not be published. Required fields are marked *