What Has Experimental Philosophy Discovered about Expert Intuitions?

The recent post by Moti Mizrahi brings up an important question. Much of the existing research in experimental philosophy has been concerned with the intuitions of ordinary folks, but a critic could respond to this work by saying: 'We were never really interested in people's ordinary intuitions in the first place. Our real interest was in the intuitions of trained experts -- people who have spent years thinking deeply about philosophical questions -- and the intuitions of trained experts might turn out to be radically different from those of ordinary folks.' 

This is certainly a legitimate and interesting objection. We might be able to get some insight into it by looking to very general considerations (about, say, the nature of expertise or the nature of philosophical intuition), but of course, the most direct way of going after an issue like this one is to look at experimental studies that actually examine the philosophical intuitions of professional philosophers. 

So I was thinking that it might be helpful to try to put together a list of the studies that have looked at philosophers' intuitions and to summarize their principal findings. I'll start out with a few here, but my hope is that people will chip in by putting some further ones into the comments section, so that we can eventually get a pretty complete list. (Please feel free to include studies that have not yet been published.)

With that as a prelude, here are a few studies on the intuitions of philosophers:

Eric Schwitzgebel & Fiery Cushman (2012). Expertise in Moral Reasoning? Order Effects on Moral Judgment in Professional Philosophers and Non-Philosophers. Mind & Language 27:135-153.

• The intuitions of ordinary folks showed an order effect. The intuitions of philosophers showed exactly the same order effect, but unlike ordinary folks, philosophers were able to develop principles that justified whichever pattern of intuitions they held. 

Knobe, J. & Samuels, R. (2013). Thinking Like a Scientist: Innateness as a Case Study. Cognition. 126: 72-86.

• Ordinary folks gave different answers depending on whether they are presented with each vignette separately or given both vignettes together. Philosophers made exactly the same error. 

Sytsma, J. & Machery, E. (2010). Two Conceptions of Subjective Experience. Philosophical Studies, 151(2): 299-327. 

• Philosophers were asked a question about consciousness and also asked how ordinary folks would respond. Philosophers gave different answers from ordinary folks but also mistakenly attributed to the folk a view that was more like their own. 

Dunnaway, B., Edmonds, A., and Manley, D. (forthcoming). The Folk Probably Do Think What You Think They Think. Australasian Journal of Philosophy.

• Philosophers were asked to predict the results of experimental philosophy studies in which the outcome had been alleged to be 'surprising.' Philosophers were highly accurate in making these predictions. 

Machery, E. (2012). Expertise and intuitions about reference. Theoria.

• Results  tentatively suggest that linguists' and philosophers' intuitions about reference are influenced by their disciplinary background.

Stotz, K., Linquist, S., Griffiths, P., and Machery, E. (ms). Expert representations of innateness. (Order of the authors to be determined.)

• Intuitions about innateness were not influenced by disciplinary affiliation across a wide range of scientific fields.

Tobia, K., Buckwalter, W., & Stich, S. (Forthcoming). Moral intuitions: Are philosophers experts? Philosophical Psychology.

• Both non-philosophers and philosophers show an actor/observer bias.

Kuntz, J.R. & Kuntz, J.R.C. (2011). Surveying philosophers about philosophical intuition. Review of Philosophy and Psychology 2: 643-665.

• Philosophers were asked about their conception of intuitions and about the role of intuitions in philosophy. The results indicated that philosophers did not agree about what what intuitions were but did show a tendency to think of intuitions as paying a role in discovery rather than in justification.

Livengood, J., Sytsma, J., Feltz, A., Scheines, R. & Machery, E. (2009). Philosophical Temperament. Philosophical Psychology, 23: 313-330.

• Philosophers showed higher levels of reflectiveness (as measured by the Cognitive Reflection Test) even controlling for education level. 

Schulz, E., Cokely, E.T., & Feltz, A. (2011). Persistent bias in expert judgments about free will and moral responsibility: A test of the Expertise Defense. Consciousness and Cognition, 20, 1722-1731. 

• Extraversion continues to predict compatibilist free will judgments even after controlling for free will expertise using a verified test of free will expertise.

Vaesen, K. & Peterson, M. The Reliability of Armchair Intuitions. Unpublished draft.

• Philosophers were asked a series of questions about knowledge. The results showed that philosophers gave different answers depending on their native language.

Comments

[Andrew]

Dunnaway, B., Edmonds, A., and Manley, D. (forthcoming). "The Folk Probably Do Think What You Think They Think". AJP.

• Much of contemporary experimental philosophy involves taking surveys of ‘folk’ subjects to test their intuitions involving philosophically relevant concepts. The results of these surveys are often claimed to be surprising, and treated as evidence that the relevant folk intuitions cannot be predicted from the ‘armchair’. We conducted an experiment to test these claims, and found that a solid majority of philosophers could predict even results that were claimed to be surprising in the literature. We discuss some methodological implications as well as some possible explanations for the common surprisingness claim

Link: http://www-personal.umich.edu/~dmanley/Site/Papers,_etc._files/MetaXPhi%20with%20final%20fixes.pdf

[Eric Schwitzgebel]

Cool idea, Josh! You might also like Michiru Nagatsu's paper "Experimental Philosophy of Economics", forthcoming in a special issue of Economics & Philosophy edited by James Konow and me. A draft is available here:
http://www.academia.edu/2450096/Experimental_Philosophy_of_Economics

No data, but an interesting angle on the potential value of exploring experts' opinions about theoretical concepts in economics.

[Joshua Knobe]

Thanks! Both of these papers look very helpful, and I've now included the one with data on philosophical intuitions in the post itself.

[Edouard Machery]

Machery, E. 2012. Expertise and intuitions about reference. Theoria,

Accessible there: http://www.ehu.es/ojs/index.php/THEORIA/article/view/3482

(M. Devitt has a response in the same issue).

I present some data that tentatively suggest that linguists' and philosophers' intuitions about reference are influenced by their disciplinary background.

[Edouard Machery]

Stotz, K., Linquist, S., Griffiths, P., and Machery, E. (ms). Expert representations of innateness. (Order of the authors to be determined.)

We provide evidence that the concept of innateness is not influenced by the disciplinary affiliations of scientists across a range of areas.

[Edouard Machery]

Tobia, K., Buckwalter, W., & Stich, S. (Forthcoming). Moral intuitions: Are philosophers experts? Philosophical Psychology.

This article shows that philosophers suffer from the actor-observor bias

[Joshua Knobe]

Thanks once again! I have now included all of these references in the main post.

(Quick note: I have not read all of these papers myself and am in some cases simply relying on the summaries given in the comments above.)

[Eric Schwitzgebel]

Kuntz JR & Kuntz JRC (2011). Surveying philosophers about philosophical intuition. RPP 2: 643-665.

[Joshua Knobe]

Thanks Eric -- I've now added that one in as well.

[Jonathan Livengood]

I'm not sure whether this is on target or not, but Justin Sytsma, Adam Feltz, Richard Scheines, Edouard Machery, and I have a paper on philosophical temperament, in which we observe that philosophical training is positively associated with CRT score, even after controlling for education level.

We could not determine on the basis of the data whether increased philosophical training causes increased CRT scores or whether there is a selection effect (or a bit of both). Hence, the effect of philosophical training on cognitive reflection is still very much an open question.

[Jonathan Livengood]

Here is a link:

http://philsci-archive.pitt.edu/4972/

(I thought that html flags worked, but apparently, they don't.)

[Adam Feltz]

Schulz, E., Cokely, E.T., & Feltz, A. (2011). Persistent bias in expert judgments about free will and moral responsibility: A test of the Expertise Defense. Consciousness and Cognition, 20, 1722-1731. (http://www.sciencedirect.com/science/article/pii/S1053810011001231)

We find that extraversion continues to predict compatibilist free will judgments even after controlling for free will expertise using a verified test of free will expertise.

[Kevin Tobia]

This is an interesting draft by Krist Vaesen and Martin Peterson: The Reliability of Armchair Intuitions. http://home.ieis.tue.nl/kvaesen/reliability_of_armchair_intuition_online.pdf

They find that philosophers' intuitions about knowledge vary with respect to linguistic background (English vs. Dutch, German, or Swedish).