What Experimental Philosophy Might Be

What is "experimental philosophy"? There is, I think, some tension in how philosophers use the term -- tension between a narrow and a wide conception.

Most of the work canonically identified as "experimental philosophy" surveys ordinary people's judgments (or "intuitions") about philosophical concepts, and it does so by soliciting people's responses to questions about hypothetical scenarios. (See, e.g., Knobe's chairman study regarding intentional action, Swain et al.'s study of order effects on knowledge judgments, and Machery et al.'s study of cultural variation in the perceived reference of proper names.) Thus, philosophers sometimes think of "experimental philosophy" as the enterprise of running these sorts of studies. Joshua Alexander, for example, in his 2012 book portrays experimental philosophy as the study of people's philosophical intuitions, as does Timothy Williamson, in his forthcoming critique of the "Experimental Philosophy revolution".

Conceived narrowly in this way, experimental philosophy is a coherent and (mostly) recent movement, with a distinctive methodology and an interrelated network of results. It is possible to discuss and critique it as a unified body.

However, there also seems to be a broader conception of experimental philosophy -- a conception that has never, I think, been adequately articulated, a conception that Josh Knobe and Shaun Nichols, for example, gesture toward in their "Experimental Philosophy Manifesto", before they shift their focus to experimental philosophy in the narrow sense. In this broad sense, philosophers who do empirical work aimed at addressing traditionally philosophical questions are also experimental philosophers, even if they don't survey people about their intuitions. In practice, the narrow/broad distinction hasn't meant too much because almost all of the empirical work of almost all of the people who are canonically recognized as "experimental philosophers" fits within both the narrow and the broad conceptualizations.

My own work, however, tends to fit only within the broad conception, not the narrow (with a couple of recent exceptions). So the difference has personal importance to me, affecting both my and others' conceptualization of my role in the "x-phi" community.

How to articulate the broad conception? Experimental work done by people in philosophy departments seems an odd category, since similar work can be done by people in different departments. Experimental work that addresses traditionally philosophical issues, however, seems far too broad, sweeping in too much research that we would not ordinary conceptualize as philosophical, including physics experiments on the structure of space and time, biological research on the origins of our social roles, and psychological work on the origins of our concepts, since these issues are at the root of philosophy of physics, philosophy of biology, and philosophy of psychology; and sometimes what the most theoretically ambitious physicists, biologists, and psychologists say is not so different from what philosophers of those disciplines say; if by adding experimentation to that, we get experimental philosophers, then most experimental philosophers are employed by science departments.

Here's a thought that is perhaps more satisfactory: Experimental philosophy in the broad sense is empirical research that is thoroughly contextualized within an intimate knowledge of the philosophical literature on which it bears, and which is presented, primarily, as advancing that philosophical literature. (For present purposes, we can define "philosophical literature" sociologically as what is published in philosophy journals and in books classified as philosophy books by academic presses.) This will omit the typical developmental psychologist working on conceptual categories, but probably allow as marginal cases of "experimental philosophers" the most philosophically informed developmental psychologists (such as Alison Gopnik and Susan Carey). It will allow in, as central, work by Shaun Nichols that is not typical intuition-polling x-phi (such as his work on disgust norms and on quantitative history of philosophy). Maybe some of Elisabeth Lloyd's empirical work on the female orgasm will also qualify.

And of course (my main, not-so-secret intention in developing this account) so also will all of my own empirical work. Included, for example, will be my work on the moral behavior of ethics professors, which does tend to be recognized as "experimental philosophy" by the x-phi crowd, despite not fitting the narrow conception; and also my empirical research on consciousness (e.g., here and chapters 1 and 6 here), which is less often mentioned as x-phi.

But more than that, I think this conception explicitly encourages the thought that a broad range of traditionally philosophical issues can be illuminated by empirical studies, even though they have not yet commonly been approached with the tools of empirical science. For example, empirical research might help illuminate the question of whether philosophical study, over the centuries, tends to progress toward the truth. We might be able to take an experimental approach to the question of whether the external world exists. Questions in the history of philosophy might be approachable by means of the systematic study of citation patterns in philosophical venues. And so forth.

Experimental philosophers, break free! There is so much more to do than just surveying intuitions!

(No accident maybe, that I was a student of Gopnik and Lloyd?)

[Cross-posted at The Splintered Mind.]

Comments

[Brian Robinson]

Thank you for this post. This is a great articulation of how I've long thought of x-phi. You're absolutely right that x-phi includes much more than surveys of intuitions. I expect that as the x-phi movement grows, we'll see more and more philosophers working in x-phi in your broader sense.

[jonathan weinberg]

This sounds very good to me! I have always opposed attempts to define x-phi in terms of "surveys of intuitions", both because x-phi needn't just use surveys, and needn't only study intuitions. The way I have always formulated the spirit of x-phi to myself is in terms of philosophers taking responsibility for the empirical commitments of our theories & methods, where that sometimes means engaging with existing or ongoing empirical work, and where that sometimes also means going out and doing such work oneself, if no scientists are already doing it for us. I take your formulation here to be an excellent way of putting flesh on that spirit.

I do expect that intuitions (or philosophical judgments, or whatever one chooses to call them) will, and should, remain _an_ important topic within x-phi, simply because it's an area where there is a great amount of psychological rubber meeting the philosophical road. But one can only applaud attempts to bring in a wider array of ways to bring empirical results into contact with philosophical disputes.

[Kevin Reuter]

I do agree with both the distinction you draw and your appeal for philosophers to 'go wide'.

There also seems to me to be a larger issue in the background which is this:
Should philosophers - not just X-philosophers - think harder about whether and how aspects of their philosophical claims can be empirically falsified? If yes, are philosophers responsible to at least suggest experiments that can be conducted (maybe in collaboration with other departments) to either falsify a position or to lend support to a certain view.

Certainly, many aspects of philosophical theories are not open to empirical falsification. However, many parts are. And, in my opinion, way too little effort goes into thinking about what would falsify certain claims. If it were common practice to consider the empirically-falsifiable aspects of one's theory, I believe your hope for X-phi 'going wide' would follow automatically.

[Eddy Nahmias]

A few sentences from the opening paragraph of Thomas and my "The Past and Future of Experimental Philosophy":

"Experimental philosophy is the name for a recent movement whose participants use the methods of experimental psychology to probe the way people make judgments that bear on debates in philosophy.... All of this work shares a commitment to using controlled and systematic experiments to explore people’s intuitions and conceptual usage and to examining how the results of such experiments bear on traditional philosophical debates. But, as we will explain below, while some experimental philosophers use data about ordinary intuitions to support philosophical theories, others use such data to better understand the psychological mechanisms that generate such intuitions, while still others gather such data to show that some intuitions may be too unreliable to support philosophical theories in the first place."

I still think these descriptions demarcate the boundaries and the various projects about right. I like Eric's description too but I worry it sounds a bit forced. Does our description exclude or include projects that should count as x-phi?

I may post something on this eventually, but it is relevant to ask in this thread: Is there a history of the term "psychology of philosophy"? And do people think this term would be a good name for a sub-branch of x-phi?

[Eric Schwitzgebel]

Thanks for the interesting comments, folks!

@ Jonathan: I find myself sympathetic with your remarks along these lines. One issue your phrasing above raises is, what is a "philosopher"? Do Joshua Greene, Fiery Cushman, and/or Alison Gopnik count? Brian Greene? Piaget? Dan Ariely? At some point it becomes a reach. Once that is settled, the rest seems just right. I recommend that we focus on breadth and depth of explicit connection to the sociologically-defined existing philosophical literature.

@ Kevin: Yes, I am completely on board with that! That is pretty much how I have approached philosophy since my first day in Alison Gopnik's lab in 1993 (even if it's not always obvious on the surface that that is what I've been doing).

@ Eddy: I would consider that to be an excellent articulation of the narrow view. Now there's definitely something to be said for the narrow view. As I mentioned in my post, it makes x-phi a coherent movement with a related set of methods and results that can readily be discussed as a whole. But my guess is that most x-phi-ers would prefer a broader vision of the enterprise, as do I, even at the cost of making it less of an internally coherent enterprise.

I myself like the label "the psychology of philosophy". In fact, that's the title of a talk I'll be giving next week, which will include the ideas of this post. I have some older programmatic remarks on how that label might be applied in my comments on Shaun Nichols for the 2nd Online Philosophy Conference:
http://experimentalphilosophy.typepad.com/2nd_annual_online_philoso/files/comments_on_nichols.pdf

[Mark Phelan]

Along the lines of the post, a non-academic article I wrote about a year ago for Philosophy Now, just came out this week. It explores various misconceptions of experimental philosophy, including the one being targeted here, the narrow conception, that experimental philosophy investigates people's intuitions about philosophical concepts. The full article is available here (http://philosophynow.org/issues/92/Experimental_Philosophy_As_An_Elephant). But here's the relevant section:

Each of the articles so far cited involve a reassessment of intuitive evidence. The negative project described in the first section urges a wholesale revision of the use of intuition in philosophy on the basis of experimental research by broadening the sample class of those whose intuitions matter. On the other hand, the experimental projects discussed in the second section embrace the standard evidential role of intuition in philosophical practice, but attempt to reassess specific intuitions. Thus, despite their divergent aims, the papers cited so far give the strong impression that experimental philosophers are intuition-obsessed, inviting the retort that experimentalism does not reach beyond one limited source of philosophical evidence. Once again, this characterization is due to a failure to survey the full field of experimental philosophical projects.

Unfortunately, this ‘intuitive’ characterization of experimental philosophy has been aided and abetted by some prominent early descriptions of the movement by experimentalists themselves. For instance, in ‘The Past and Future of Experimental Philosophy’ (Philosophical Explorations, 2007), Thomas Nadelhoffer and Eddy Nahmias claim that all work in experimental philosophy “shares a commitment to using controlled and systematic experiments to explore people’s intuitions and conceptual usage.” Self-reflective characterizations of a new movement are necessary (I self-reflectively write); but if the movement is still developing, as experimental philosophy has been, they run the risk of being outpaced by new developments and becoming prominent mischaracterizations of it.

It is no longer correct (if ever it was) to say that all experimental philosophy projects involve the empirical study of intuitive assessments of particular cases – the ‘thought experiments’ (or ‘intuition pumps’) of traditional philosophy. Just as philosophers have traditionally appealed to a disparate assortment of evidence, so experimentalists have begun to systematically assess not only intuitive judgments, but other sources of evidence as well. For example, a team led by Eric Schwitzgebel has attempted to gain insight into the ancient and oft-asserted claim that philosophical reflection on ethical issues can improve one’s behavior, by examining how professional ethicists behave in a variety of circumstances. At least at face value, their results suggest that moral reasoning has little effect on moral behavior. Schwitzgebel and colleagues found that professional ethicists are no more likely to vote, or respond to student emails, than are non-ethicist philosophers and professors. Audiences in ethics sessions at philosophical conferences are generally just as likely to behave discourteously as audiences in non-ethics sessions. And ethics books (even obscure ones) are more likely to be missing from library collections than are books from other philosophical sub-disciplines. If, as Schwitzgebel et al argue, this is important evidence regarding the behavioral efficacy of ethical thinking, it is not evidence from intuition.

In other work, Joshua Knobe and Jesse Prinz examined how frequently people use different kinds of mental state ascriptions on the internet, and argued from their findings that people are ordinarily willing to ascribe beliefs and desires, but not experiences or emotions, to disembodied or distributed entities like corporations. No intuitions are invoked in these findings. And in my own experimental work, by systematically comparing how people paraphrase metaphorical and literal utterances, I have found evidence against the claim that it is distinctively difficult to express what a metaphor means.

Each of the approaches I have just described does something other than survey intuitive reactions to thought experiments. In these studies, both thought experiments and intuitive assessments are absent, as in many other recent projects in experimental philosophy. So if one characterizes experimental philosophy as the systematic investigation of intuitions about particular thought experiments, one has got hold of only part of the elephant.

[jonathan weinberg]

"One issue your phrasing above raises is, what is a "philosopher"? Do Joshua Greene, Fiery Cushman, and/or Alison Gopnik count? Brian Greene? Piaget? Dan Ariely? At some point it becomes a reach."

Seems to me that if a proposed account of "experimental philosophy" ends up with the same vast untractable swathes of vagueness as the term "philosophy" by itself, then that's a mark in favor of the account... :-)

[Eric Schwitzgebel]

Right, Jonathan! I wouldn't want to disagree with that!

[Eric Schwitzgebel]

Thanks, Mark!

[Justin Sytsma]

Wholeheartedly agree with the broad conception. And the first thought I had in reading Williamson's review is that it left out a large chunk of what I would consider to be experimental philosophy.

Here it is worth adding that a good deal of work that could be described as investigating intuitions doesn't fit neatly with the program that Williamson describes either (for example, isn't primarily concerned with the use of intuitions as evidence).

Sadly, this debate seems to be shaped more by the image of the burning armchair than the actual corpus.

[jonathan weinberg]

@Justin - I don't think he's even read but the tiniest sliver of the part of x-phi he _does_ mean to be targeting. Like, there's a part where he suggests that x-philosophers should ground their researches in work in cognitive psychology. But basically every single paper out there already does that, and is very explicit about it!

[Florian Cova]

Hi,

I could't agree more with you. In fact, we have a french anthology of X-Phi forthcoming (co-edited with Julien Duant, Edouard Machery, Joshua Knobe, Shaun Nichols and Eddy Nahmias), and in the introduction, we make a similar distinction. Here is a translation of the passage:

"In a first approach, one can define experimental philosophy as the use of experimental methods to make progress on particular philosophical questions. However, this definition is still too broad, because one can distinguish experimental philosophy in the broad sense from experimental philosophy in the narrow sense : while, in the broad sense, every philosopher using experimental methods to adress a philosophical question does exepimental philosophy, an experimental philosoper in the narrow sense conduct experiments in order to find out what are our intuitions (those of laypeople as well as those of philosophers) and study the nature of the psychological mechanisms behind them (more on that later). One can find in the litterature examples of experimental philosophy in the broad sense that do not count as experimental philosophy in the narrow sense (the reverse being logically impossible). First example : seeking whether the practice of moral philosophy (and thus, of moral reasoning) improves one's moral behavior, philosopher Eric Schwitzgebel ran a series of studies about the behavior of ethicists (studies that suggest that ethicists do not behave better than other academics, whether in philosophy or other disciplines). Other example : seeking what "skills" distinguished expert philosophers from other experts, Jonathan Livengood and his colleagues collected data suggesting that expert philosophers are more "reflexive" than collegues from other disciplines, meaning that they would have a greater tendency to defy themselves from "self-evident" answers. Such studies can contribute to make progress on philosophical questions (e.g. can being better at reasoning make us act better?) but are not concerned with common sense's intuitions. They do not consist in asking people whether, according to them, better reasoning leads to better behavior, or whether philospers are more reflexive than average."

The fun fact is that the main motivation for this distinction was to integrate Eric's work in the list.

However, the text is more than one year-old and I think some things could be changed. For example, in the definition of the broad version of experimental philosophy, I would not restrict experimental philosophy to studies made only by philosophers, shifting the burden on what is pursued (i.e. making progress on a "philosophical question", with the hope that one can recognize one when he sees one).

Also, I would not be so sure that all experimental philosophy in the narrow sense is also experimental philosophy in the broad sense : Knobe & Prinz studies on consciousness attribution are (i) experimental philosophy, (ii) study intuitions but (iii) are not aimed to solve a philosophical question (from their own admission).

Also, in touch with Eddy's proposal, I once defined (in my book) X-Phi as "philosophical psychology", with the hope of synthetizing the broad and narrow views. However, I don't think it can integrate Eric's work on introspection, so I'm not very sure about this definition any more.

[Brian Robinson]

A somewhat tongue-in-cheek thought:

To determine what is and isn't X-Phi, let's do a survey of philosophers (X-phi'ers and non) to get their intuitions on the subject

[Kareem Khalifa]

Hi All,

As I'm relatively new to this blog, forgive me if I'm regurgitating some older ideas (and forgive me twice over if the regurgitation turns out to be protracted!)

I start with three observations. First, in many empirical fields, it is commonplace to reserve the word "experimental" for those empirical studies in which some sort of intervention or manipulation occurs. Second, Eric's original dichotomy between narrow and broad x-phi does not seem to invoke this distinction. He defines narrow x-phi as "survey[ing] ordinary people's judgments (or 'intuitions') about philosophical concepts, and [that] does so by soliciting people's responses to questions about hypothetical scenarios." By itself, a survey does not entail an intervention. A fortiori, Eric's broader definition of x-phi also cannot invoke the concept of manipulation/intervention. Third, paradigmatic works in x-phi *do* involve an intervention (e.g. the Knobe effect is inferred from an experiment in which the help/harm variable is intervened upon).

Hence, I would suggest the following tripartite distinction:

(1) We follow the jargon of other fields and restrict the term "experimental philosophy" to only those philosophical studies in which a philosophically relevant conclusion is inferred directly from empirical studies involving an intervention.

(2) "Empirical philosophy" would then apply to those philosophical studies that:
(a) are experimental philosophy, OR
(b) in which the inference from an intervention is mediated (to some large degree) by several auxiliary claims, OR
(c) where the evidence is empirical, but does not involve an intervention.

(3) "Non-empirical/armchair philosophy" would then refer to philosophical studies in which the philosophically relevant conclusion is not drawn from any empirical evidence.

This still leaves quite a bit unsettled. For instance, we'd need to flesh out what counts as "philosophically relevant" or how to measure the mediation of an inference with respect to auxiliary claims). Moreover, we'd arguably want to add some rider about authorial intent. (For instance, the fact that Philosopher X subjects Philosopher Y's claim to an experimental test does not seem to make Philosopher Y an x-phi'er.)

These caveats notwithstanding, it seems to me that this classification has some virtues:

(i) It does not entail that x-phi is/needs to be "intuition-based." In principle, results from CERN could make for some interesting experimental metaphysics/philosophy of physics;

(ii) For similar reasons, x-phi can overlap with any experimental field (not just cog psych, as is sometimes suggested);

(iii) "Empirical philosophy" seems to have all of the virtues of Eric's "broad conception" of x-phi, but would more accurately describe when philosophers use empirical, non-experimental evidence, e.g. case studies from (the history of) science, politics, law, etc.; ethnographies; descriptive stats; census data; citation data...

[Eric Schwitzgebel]

Thanks for the continuing comments, folks!

@ Florian: Very cool! I do think the challenge -- as with Jonathan Weinberg's remarks above, and as with most broad characterizations of x-phi -- is how to avoid the problem of either defining "philosophy" or "philosopher" too narrowly in terms of departmental affiliation, without overcompensating by sweeping in far more scientists than the label really ought to include -- since there are plenty of scientists who address issues with philosophical import or who are in part reacting to (sometimes cartoon versions of) Hume, Kant, etc. It was really my thought about how to find a possibly satisfactory compromise that triggered the post.

@ Brian: I trust this will be your next project!

@ Kareem: You are right about how "experimental" is used in other disciplines. (And your suspicion that this fact has been noticed before is correct.) However, the label seems to have sunk in, so we need to deal with it. In my view, for most x-phi purposes it's probably not worth drawing a sharp, subdiscipline-defining distinction between empirical studies that involve experimental manipulations and those that don't. Thus, in my own characterization above, I was careful to use "empirical" rather than "experimental" in my own preferred characterization of the field.

[Justin Sytsma]

@ Brian -- Actually, Jonathan Livengood and I conducted a survey looking at how people understand "experimental philosophy" for a textbook that we're writing. (The text treats experimental philosophy broadly.) A description of some of the results will hopefully be forthcoming in the near future.

In brief, though, both experimental philosophers and philosophers more generally are divided on whether they think of experimental philosophy as being restricted to the study of intuitions. The average response for each group was just on the "broad" side of neutral (with the responses of self-identified experimental philosophers slightly more broad than those who didn't self identify).

[Florian Cova]

A good idea would be to list papers belonging only to the broad conception of X-Phi, so that people would get a better idea.

So, I cited Eric's work on ethicist, and Livengood et al.'s study on Philosophical Temperament. We can add Eric's work on instrospection, and I think that Adam Feltz and Edward Cokely have fun stuff on first-person intentionality ascriptions. What else ?

[Sebastian Tor]

My understanding of the term 'exmperimental philosophy' is limited. However I do attend a festival every year which has had world renowned philosophers every single time I've attended - what they do appears somewhat on the cusp: http://www.iai.tv/

[Mark Phelan]

I like Florian's proposal (to enumerate studies that count as broad), particularly because I think Florian and I have different conceptions of narrow vs. broad X-phi. I think this comes out in relation to Knobe and Prinz's studies. Florian has characterized these studies as narrow philosophy above. But, at least the key study in that paper, which elicits people's appropriateness assessments for group-state sentences, and uses that data to draw a conclusion about folk metaphysics of mind, does not fit the narrow conception as I'm thinking of it. Nor does it fit Eric's conception in the original post. According to that conception, narrow x-phi "surveys ordinary people's judgments (or "intuitions") about philosophical concepts, and it does so by soliciting people's responses to questions about hypothetical scenarios." K&Ps study does not fit this conception because, on the one hand, it does not solicit people's responses to questions about hypothetical scenarios. And, on the other and more importantly, it does not survey ordinary people's judgments about philosophical concepts. I do not mean this in the sense in which Florian mentions above, that the target of the paper (folk metaphysics of mind) may not be a canonical philosophical question. Rather, I mean the target of the paper and the target of people's judgments come apart. The target of the paper is theoretically interesting (what is the folk theory of mind). The target of people’s judgments is not (how natural sounding are some sentences). I think we can point to a real and potentially interesting difference between experimental studies where the data that constitutes the relevant evidence are people’s judgments about the philosophical concept that is the topic of the paper (e.g., Knobe’s original intentionality studies) and studies where this is not the case (e.g., the aforementioned K&P study). I’m skeptical of the distinction Florian mentions above, which characterizes the narrow sense as conducting experiments in order to find out what are our intuitions and study the nature of the psychological mechanisms behind them.

[Eric Schwitzgebel]

Thanks for the continuing comments, folks!

In thinking further about these issues, I'm inclined to think the "narrow" sense might be something like an exemplar concept, where there are a range of studies that have some features like the canonical studies but also differ in some important ways. Knobe and Prinz do a variety of things, for example, to get at folk conceptions of group minds (including counting Google hits) -- but, in a way, because they are still focusing on exploring the folk conception there is something importantly similar to the exemplar cases. Some of my other cases mentioned above diverge away from the exemplars even farther.