Wednesday, October 22, 2008

ACTA's Dishonest, Incompetent Survey in Missouri

The American Council of Trustees and Alumni has a new survey out of students at the University of Missouri and Missouri State University. Unfortunately, it's a piece of total garbage.

I don't mean garbage in the sense of disagreeing with it ideologically. I mean that as a scientific survey, the ACTA is utterly useless for drawing any conclusions whatsoever. It is completely worthless.

Every question asked by ACTA is similar in form to this one: “On my campus, some professors use the classroom to present their personal political views.” No competent survey researcher would ever ask these kinds of questions. Here's the problem: the questions are purely speculative. Instead of asking students for their personal experiences (or even for incidents they might have heard about), the question forced students to guess about whether there are “some” professors who do this. Needless to say, it's easy to imagine that any campus with 1000 professors has three (or “some”) who express personal views about anything in the classroom. And that's what these questions ask students to do.

We have a direct test of how useless this survey is because ACTA actually included a question where we know for certain that the correct answer at every university is “yes”: “On my campus, some panel discussions and presentations on political issues seem totally one-sided.” Every university has some “one-sided” presentations on political issues. When a liberal or a conservative speaker appears on campus, it's “one-sided.” So every student should have agreed with this question. Yet only 48 percent agreed.

What does this mean? It means nothing. The entire survey means nothing. It's so bad you can't conclude anything from it.

That doesn't stop ACTA from drawing outrageous conclusions such as “Students report that major Missouri universities do not provide an intellectual atmosphere conducive to a robust exchange of ideas.” Of course, ACTA's report only mentions the questions where they like the answers, such as the 51 percent who agreed that “On my campus, there are courses in which students feel they have to agree with the professor’s political or social views in order to get a good grade.” Omitted in the report (unless you examine the full list of questions in the appendix) is this question: “On my campus, some professors penalize students because of the student's political or social views.” Only 12 percent of students agreed. So which response is correct, since they deal with essentially the same issue? The answer is: neither one, because this entire survey is worthless.

This isn't anything new. I've criticized ACTA in the past for using the exact same biased questions as they do here. So why hasn't ACTA ever asked a real question about the real personal experiences of students? Probably because they're afraid that very few students will say that they've experienced ideological discrimination. It's a lot easier to do a bad survey with terrible questions that give you inflated numbers that can be manipulated in the press. Here, ACTA uses this worthless survey to give an “F” grade to Missouri universities for intellectual liberty.

In reality, ACTA deserves an “F” grade for their incompetent survey and their intellectually dishonest presentation of it.

1 comment:

Aerothorn said...

It's amazing how entire studies can be constructed and vetted by large teams of people, without a single person pointing out how scientifically invalid it is.