I just found this critical notice by Janet Levin, which came out in the October issue of Analysis:
She discusses nearly every chapter in the collection by Knobe and Nichols, Experimental Philosophy, and ultimately argues that:
the results of the most methodologically sound and philosophically relevant studies discussed in this volume could have been obtained from the armchair, and thus that experimental philosophy may not present a serious challenge to the traditional methods of analytic philosophy. (p. 761)
However, I think this statement of her conclusion is a bit misleading. Sometimes she does, as this quote indicates, simply want to reject the revisionist claim that x-phi should replace the armchair entirely (or pretty much entirely). But at other times she seems to be targeting the more modest claim that x-phi is a more promising methodology for some projects in philosophy. For example, she says "it seems that traditional analytic methods may well suffice for identifying the judgements that provide the right sorts of data for philosophical theories" (p. 767). Here she's not just saying x-phi doesn't undermine the armchair entirely; she's suggesting the armchair is as good as x-phi at gathering the relevant information about our concepts and so on.
Anywho, I thought readers of this blog might like to take a look at her piece if they haven't already.