Dart-throwing chimpanzees

The expertise defense is an attempt to defend the use of intuitive judgments elicited by hypothetical cases as evidence in philosophical argumentation. Critics of experimental philosophy who appeal to the expertise defense argue that only the intuitive judgments of experts, i.e., professional philosophers, should count as evidence in philosophical arguments.

In “Are Philosophers Expert Intuiters?” (2010), Jonathan Weinberg, Chad Gonnerman, Cameron Buckner and Joshua Alexander argue that one cannot simply use the “expertise card” without the right kind of empirical evidence to back it up. Proponents of the expertise defense have to show that philosophers really do have some kind of philosophical intuitive expertise, especially in light of what the extant scientific literature says about expertise.

Now, I think that a paper of mine on arguments from expert opinion can contribute to this debate in the following way. Those who appeal to the expertise defense are effectively saying that an intuitive judgment J in response to hypothetical case C is more likely to be true when made by a professional philosopher than by a non-philosopher. More generally, J is more likely to be true when made by an expert than by a non-expert. In my paper, however, I argue that this is not the case. Studies on the accuracy of expert opinions show that, on average, expert opinions are only slightly more accurate than chance. As one researcher put it, the experts he studied did no better than “a dart-throwing chimpanzee.”

If this is correct, then, even if philosophers are expert intuiters, it would still not follow that their expert judgments are significantly more likely to be true than the judgments of non-experts. And if that’s the case, then there is no reason to give the intuitive judgments of professional philosophers more weight in philosophical arguments than to the intuitive judgments of non-philosophers.

I’m curious to know what readers make of this additional challenge to proponents of the expertise defense.

Comments

[mb]

Well there are a fairly large (and ever-growing) number problems that you have to be a philosopher (or be in a class on philosophy) to have intuitions on.

Non-philsophers don't have intuitions about e.g. whether some instance of a proposition expresses a singular thought, because they don't know either what singular thought is or why one would want to have a theory about it.

[Moti Mizrahi]

Thanks for your comment, mb. My questions to you: (a) What prevents us from asking non-philosophers, as some experimental philosophers do? And if they (the non-philosophers) don't know what X is, we can explain it to them, as we do when we teach philosophy. (b) If, on average, experts do not much better than chance, i.e., they might as well have been guessing, then why not ask non-experts? They can guess, too. To guess one does not need prior knowledge of the subject matter.

[Bijan Parsia]

Well, for some things getting up to the level of chance is very hard. So we might not prefer an expert to chance but we might prefer it to a nonexpert.

Even if the expert isn't more accurate than chance, they still might be more useful to have than those derived from a random process. For example, they might have explanatory value.

Imagine that 50% of the time an arbitrary member of our pool of experts gets the right answer for the right reason and otherwise the wrong answer. This suggests a two stage process wherein we verify the answers by some additional mechanism and they exploit the rationale provided by the experts.

Of course if the expert is no more likely to be accurate than the nonexpert and there are no other benefits then go with whomever's cheapest.

(Sorry, I've not yet had a chance to read your paper. It's certainly true that experts need to be handled with extreme care. For a variety of reasons. Self deception is a powerful force.)

[Noah]

Did you verify that non-expert opinions are also only as accurate as chance? When I see YouTube video comments or Yahoo News article comments, I get the impression that non-expert opinions are much worse than chance.

[Moti Mizrahi]

It looks like a clarification about the notion of chance is in order here. Take a simple (fair) coin toss. The probability (or chance) of getting heads is ½ or 50%, since there are two outcomes: heads or tails. Now, translate that to opinions: I assert that p. The probability (or chance) that I am right is ½ or 50%, since there are two outcomes: right or wrong. Now, it is assumed that if I am an expert on subject matter S, and p is within the domain of S, then the probability that I am right is much higher than 50%. As it turns out, however, the probability that I am right, even if I am an expert, is still very close to 50%. But I don’t need to be an expert to get a 50% chance of being right. I already have a 50% chance of being right just by guessing.

[Bijan Parsia]

That's what I understood your notion of chance to be. My two points were that chance might outperform nonexperts (thus don't use them but flip a coin) and the side products of expert judgement mind add value even if their accuracy is low.

For example, if experts have great precision then studying their judgements might reveal something about their decision mechanism which might help identify when they are significantly more accurate than chance (if they ever are).

[Moti Mizrahi]

Thanks for the follow-up, Bijan.

You write: “chance might outperform nonexperts.” If that were the case, then it would demand an explanation, in much the same way that getting 18 heads in 20 coin tosses might lead you to believe that the coin is not fair.

More importantly, you write: “if experts have great precision then studying their judgements might reveal something about their decision mechanism which might help identify when they are significantly more accurate than chance (if they ever are).” I agree that, if they were, then it would have been useful to study the “decision mechanism of expertise.” But empirical evidence shows that, on average, experts are not significantly more accurate than chance.