Legal Decision Theory: A Cautionary Tale

In a series of well-written and well-researched articles, Gregory Mitchell--the Sheila McDevit professor of Law at Florida State University--has launched an assualt on the behavioral law and economics movement (or as he calls it, "legal decision theory"). Legal decision theorists typcially rely on empirical research from social psychology--especially the research on heuristics and biases--to undermine some of the foundational assumptions of traditional law and economics (especially the rational actor models of human psychology and decision-making so prevalent among economists). According to Mitchel, legal decision theorists all-too-often make sweeping claims about human rationality (or lack thereof) that often go well beyond the data that have been collected. On his view, much more caution is in order. Conceding that many of the recent developments in social psychology give us reason for being suspicious of many of the main tenents of law and economics, Mitchell nevertheless thinks that legal decision theorists overstate their case--a trend that he believes may unfortunately threaten to undermine the long-term credibility of empirical research among legal theorists. Mitchell points out a number of problems with the research that is relied on by legal decision theorists--many of which are relevant to the work being done in experimental philosophy. For example, small sample sizes, the near exclusive reliance on between-subject studies, using the rational or right choice as the null-hypothesis, the overstating of the significance of "statistical significance," the dearth of meta-analyses, making inferences about individual differences based on group differences, etc. Nearly all of the worries that Mitchell expresses about legal decision theory are worries that apply equally to the kinds of studies that we experimental philosophers have relied on so far. As such, I think we would all do well to pay attention to Mitchell's important work in this area.

Comments

[jonathan weinberg]

I'm not sure that I've seen the X-phi crowd make many of the kind of mistakes that Mitchell is mostly talking about, and I'm also not sure that everything he argues is mistaken is in fact such. For example, I think he misreads the logic of many between-subjects designs, and what it means to say that subjects are "inconsistent" in such a design.

His point about ignoring individual differences may be more on the mark for some recent X-phi research, though.

[Matt James]

While Mitchell's criticisms may apply to the behavioral law and economics movement, I agree with Johnathan that the specific flaws in experimental procedure that Mitchell mentions, such as ignoring individual differences in "cognitive capacity" or rationality, either are not generally committed by, or are usually not relevant to, the X-phi work. I find it hard to believe that someone would react to something like the trolley problem in a deontological or utilitarian manner depending on whether the person was in a bad mood that day or didn't score high enough on the SAT. (However, if that could be shown, it would potentially be a very devastating finding for the practice of philosophy...) Given the intense and partisan debate over such cases by professional philosophers who are (usually) educated and rational, hopefully those concerns don't apply.

However, to follow up on Johnathan's thought, there is a tendency to ignore individual differences in thinking in the X-phi research. Behavioral law and economics can somewhat get away with a focus on the majority, as the behavior of most people is their primary explanandum. Yet as philosophers, we should be interested in the 25% or so who don't answer with the majority in experiments, such as in the surveys on free will and intentional action done by contributors to this blog and others, why they answer in the way that they do, (and maybe background features such as education just to be sure). The X-phi stuff gets a pass for a while because there is not yet an established research program to define the phenomena that we want to explain, but I think the next generation of X-phi work should probe further into reasoning and background factors responsible for certain intuitions if we want to be taken seriously as a legitimate and established research method in philosophy in the long term. Mitchell's work may not highlight problems specific to X-phi, but it does implicitly reveal more areas where X-phi research needs to be performed.

[Jonathan Weinberg]

I find it hard to believe that someone would react to something like the trolley problem in a deontological or utilitarian manner depending on whether the person was in a bad mood that day or didn't score high enough on the SAT. (However, if that could be shown, it would potentially be a very devastating finding for the practice of philosophy...)"

I want to register my complete agreement with the second half of Matt's comment, and take his point in the bit I just quoted a bit further: there's an important difference between X-phi in its 'destructive' and 'constructive' forms. When it's just being destructive -- i.e., making trouble for traditional analytic philosophy -- there's no dialectical worry about whether one has controlled for various sorts of odd confounds. Since analytic philosophical practice doesn't control for such factors itself, discovering that mood, time of day, etc. had a substantial impact on intuitions would only serve to undermine that practice as well.

But when X-phi wants to be constructive -- i.e., to make positive claims about what the intuitions of various groups 'really' are -- then such issues might, over time, become relevant.

[ZAZ]

I personally tend to agree with some “conclusions” of Behavioral Law and Economics but: they are not still capable (and I am afraid they will never be working that way) to offer a different substantial and useful theory of human decisions. Besides, we all know the rational assumption of classical economics is not a description of “reality”. They would better try to undermine the prediction capacity of the analytical power of the model instead of showing the way the model (something we all know) does not explain completely “reality”. But of course, the BL and E project, in my opinion has some useful insights.