The Pervasive Impact of Moral Judgment

It is now widely agreed that people's moral judgments can affect their intutions about whether or not a behavior was performed intentionally, but considerable disagreement remains about what exactly this effect might be telling us.  Some researchers have suggested that the effect observed for intuitions about intentional action is pointing us toward some more general truth about the fundamental relationship between folk psychology and moral judgment, while others argue that the effect just happens to arise because of some quirky feature of the concept of intentional action in particular. 

In hopes of making progress on this question,  Dean Pettit and I conducted a series of new studies.  In essence, we checked to see whether the effect observed for attributions of intentional action would also arise for other concepts.  The answer is that it does.  In fact, the very same patterns observed for the concept intentionally also arise for the concepts desire, deciding, intending, in favor of, opposed to, and advocating. 

For example, suppose that a person decides to implement a policy.  He is trying to attain a particular goal, but he knows that the policy will also bring about a certain side-effect.  Did he decide to bring about the side-effect?  The typical answer: he did decide if the side-effect was a bad one, but he did not decide if the side-effect was a good one. 

Or suppose that a person makes a speech in favor of a policy.  He says that we should adopt the policy because it will help us to attain a particular goal, but he also mentions that it will bring about a side-effect.  Did he advocate bringing about the side-effect?  Once again,  people's intuitions depend on the status of the side-effect itself.  If the side-effect is bad, he advocated it;  if good, he did not. 

At this point, Dean and I are thinking that it might not be helpful to continue trying to understand this effect in terms of something very specific about the concept of intentional action in particular.  What is needed, it seems, is a deeper and more general theory about the relationship between moral judgment and the rest of cognition. 

[For further details, you can download the full paper.]

Comments

[Eric Schwitzgebel]

Neat essay. I wonder if this principle might help explain your effects:

People are keener to blame than to praise (maybe because detecting scalawags and cheaters has always been particularly important).

[Joshua Knobe]

Eric,

That sounds like a promising approach, but I'd be interested to hear more about how exactly it would work. If we give you the claim that people are more willing to blame than to praise, how do you get from there to the asymmetries in judgments of deciding, advocating, and so forth?

[Eric Schwitzgebel]

My thought is that it is more blameworthy to decide to harm the environment than to decide to help it, more blameworthy to advocate harming the environment than to advocate helping it, more blameworthy to intend to harm the environment than to intend to help it, etc., and that the first sort of judgment might (ceteris paribus) be reached more easily and on the basis of thinner evidence than the latter sort of judgment -- partly, perhaps, because of the importance to us of detecting cheaters and creeps. Thus, in intermediate or blurry cases we might find more people ascribing intentions, advocacy, desires, beliefs, etc., when those mental states are blameworthy than they would in exactly parallel cases where the mental states are praiseworthy.

It seems to me that this principle might explain all the examples you describe in your essay. Or am I missing something obvious? (Quite possible!)

[steven]

Suppose that people agree that the main "intended" course of action is desirable. Then, if the side-effect is also desirable, it doesn't change the positive value-judgment of the action, and some principle of evaluatory parsimony might deny the utility of awarding double credit by saying that the good side-effect was intended too.

If the side effect is undesirable, though, that changes or to some extent mitigates the value judgment of the action as a whole, and so it might make more sense to people engaged in the judgment then to ascribe the side-effect to intention too, so that they can keep in mind a picture of one person evaluating outcomes when coming to form a moral evaluation of the person's decision.

[TGGP]

You might be interested in Kinsella's "Causality and Aggression". Another paper I like that takes a very different attitude is Greene & Cohen's "For the law, neuroscience changes nothing and everything".

[Thomas Wood]

I agree w/ Schwitzgebel with a twist. Why do we attribute intention to blameworthy acts more than to praiseworthy acts? Why do we vest more responsibility in agent-acts which cause harm than in agent-acts which cause help? An analysis of harm and help should indicate that blaming harm is more important to the survival of the group than praising help. Keeping a tight lid on harm would seem crucial in a way that making people feel responsible for help is not. Well, consider that people are quick to attribute responsibility to themselves for their [praiseworthy] helping anyway. They are all too ready to take credit, right? So its functionally redundant to attribute intention for what they are already taking credit for. However, people are much more likely to deny responsibility for [blameworthy] harming whenever there is the thinnest excuse available. It's much more important to challenge this habit of rationalizing by saying, "you did that intentionally and therefore you are blameworthy." It's not the case that an agent looks for excuses to get out of praising himself is it? There is a basic asymmetry here in our self-attributions that the asymmetry in other-attributions is sensitive to.

So E.S. seems right to suggest that keeping a lid on scalawags, cheaters, and creeps explains the asymmetry. But I would propose it's also because we make the opposite assignments of responsibility to ourselves when we try to weasel out of blame and yet credit ourselves with every remotely possible attribution of praise. All of this happens unconsciously and has been understood by social psychologists for 30 or so years. It's consistent with cognitive dissonance theory.

[Joshua Knobe]

So sorry to be this late in replying to people's comments!

Eric,

This looks like a very promising approach to the problem, but I'm not sure if it's right to suggest that there is something about *blame* in particular that explains the asymmetries in people's intuitions.

After all, one gets exactly the same effect when one compares neutral behaviors to morally good behaviors. That is, people are more inclined to regard neutral side-effects as intentional than they are to regard morally good side-effects as intentional. Yet it seems that there is no blame at all in either neutral or morally good cases.

Can the hypotheis you are suggesting explain these further effects?

Steven,

This comment opens up another interesting avenue, but I wonder if the idea here can account for people's intuitions in cases in which *both* the side-effect *and* the main effect are morally bad. Ron Mallon has shown that people regard the side-effect in such cases as intentional. Can the hypothesis here account for that finding?

[Joshua Knobe]

Thomas,

Yes, it is puzzling the way people are so willing to assign blame for bad side-effects but do not assign praise for good side-effects. One possible explanation of this phenomenon is the one you provide here -- that people just show a quite general, across-the-board tendency to be more inclined to blame than to praise. This is definitely a promising approach, but it does face the problem that there also seem to be cases in which people are more willing to praise than to blame. For example, people are more willing to praise than to blame in cases where the agent acts out of an irresistible emotional impulse, and they are more willing to praise than to blame in cases where the agent suffers from moral ignorance and does not know the moral status of the act he or she is performing...

Of course, that does not necessarily mean that your hypothesis is mistaken. It could be that people show a general tendency to be more willing to blame than to praise but that there is something overriding that general tendency in the specific cases I've described here.

[Scott Hughes]

I think people simply tend to be quicker to blame someone (or something) for bad effects than to give them credit for good ones.

[Thomas Wood]

Joshua,

Thank you for your reply. Yes, truly it’s never as simple as we at first suppose. Let the ad-hoc adjustments begin! Think of our moral intuitions (of intention, blame, praise, whatever) as a kind of emergent wisdom that natural selection carved into us over eons. With that in mind it’s not hard to imagine that the asymmetry you mention (blame>praise vs. praise>blame) is context dependent. Normally when the agent is perceived by all concerned as having voluntary control over her bx my first-post analysis follows and blame for harm is emphasized over praise for helping (because, as I described earlier, agent rationalizations exculpating one's self from blame for harm have to be challenged but agent rationalizations exculpating one's self from praise for help are exceedingly rare and therefore require no challenge). But the asymmetry should reverse in the context of involuntary [control of] behavior. It would be socially unproductive or dysfunctional to attribute intention to behavior or decisions emerging from irresistible emotions and impulses. Such impulsive motives do not square with common sense notions of voluntary behavior or 'free-will'. So it's hard to actually blame someone for it. And it makes no sense to instill a sense of responsibility for bx which the agent has little or perhaps no potential voluntary control over. Better to identify the psychological processes at work in such cases and address them directly (by keeping the alcoholic out of the bar, for instance). But conversely, there is no harm and arguably a bunch of good done by praising pro-social bx even when it was done involuntarily under irresistible emotional impulse, as it were. You want the agent to identify with the pro-social bx and to potentiate its future performance so you praise the bx and perhaps the agent or others will increasingly reproduce the bx in the future.

I don't want to sound like I've got an evo-psych just-so story for every scenario but I think all of these cases can be seen as examples of "what attribution would do the most good and minimize the most harm for the group?" So it's not a simple asymmetry of blame and praise, as it were, but one that is modulated/determined by the degree to which the agent has some kind of higher order influence in preventing harm (by knowing he'll get blamed) or contributing something helpful (knowing that if he fails to do the good thing he’ll get blamed) or, conversely, is at the mercy of mental illness, emotions run amok, extremely difficult circumstances, etc., and has little volitional control. When the subject cannot be seen as having had much of an alternative given the situation we don't blame him, however, we may praise him in like circumstances as it’s generally a good idea to promote pro-social bx. We do this all the time when we praise people for doing the right thing accidentally. It’s a form of persuasion and is the basis of animal training. It doesn't threaten the force or institution of intentionality attributions to praise an involuntary good bx because people are not loath to take credit for helping even if it was not performed under voluntary control. But they are loath to take responsibility for harms done involuntarily.

When it comes to moral ignorance praise makes sense because it clears up the ignorance and reinforces good bx. For moral ignorance and blame, I'm less sanguine with my evo-psych explanation. Surely the agent has to learn that you only get a free pass once for moral ignorance, so the failure to blame doesn't mean that the agent isn't well informed that they screwed up and shouldn't do it again. Think of the times that you've been ready to blame only to learn that the agent was ignorant of the relevant facts. You don't like the outcome but you cease to blame him (unless you are really invested in the blame already...). These are interesting cases and can all be approached from a functional perspective. The moral ignorance case seems pretty analogous to the irresistible emotional impulse though. They both hang on common sense free-will attributions. We all recognize that bx done at the tip of a gun or done w/o awareness of its moral salience has a different status than bx performed w/ no obvious exculpatory features and, I would argue, that blaming such involuntary bx only serves to weaken the force and institution of responsibility attributions over the long haul. A similar analysis holds at the praise end of the spectrum. You don't want to over-praise someone for what they likely will take credit for already and for what doesn't need social approbation, yet you do want to praise good bx when it isn't recognized as volitional and praiseworthy by the agent (so that he may take credit for it and be motivated by it in the future). Praising current involuntary good bx may potentiate future voluntary good bx.

Lastly, I'd like to suggest that distinctions between attributing intention, blame/praise, and volition be kept in mind. It's a tricky business sorting out all of these attributions and they surely do not all fall into place in a simple order. Thus not all instances of praise or blame involve attributions of intention. There are plenty of times when we are praising or blaming the bx more than the agent. These may function as more nakedly instrumental acts of persuasion or coercion. Likewise there is a difference between intention and volition. Attributions of volition may precede attributions of blame or praise (as when we note the influence of strong emotion or impulse or simple ignorance) while praise or blame may precede attributions of intention (as you have so ably demonstrated). There is a lot to tease apart experimentally here.

[Eric Schwitzgebel]

Thanks for the replies Joshua and Thomas!

I agree with just about everything Thomas has said to refine the hypothesis I suggested in my initial comments.

It seems that a large swath of examples can be neatly and simply explained by the general view, which I think has a priori appeal, that people are generally quick to blame and reluctant to praise in ambiguous circumstances (the reverse holding for themselves, as Thomas suggests). Furthermore, it appears many of the exceptions can be explained not entirely implausibly by the same sort of ev-bio/social-utility maneuver (i.e., not in a completely ad hoc manner). If all that holds, I'd say that's pretty good for a simple psychological hypothesis about this complicated brain of ours!

[Joshua Knobe]

Thomas's suggestions here sound very reasonable, but I wouldn't say that they are best understood as an attempt to flesh out the basic idea that people show a quite general, across-the-board tendency to be more willing to blame than to praise. Instead, the suggestion seems to be that people are more willing to blame in cases of side-effects but that they are more willing to praise in cases of irresistible emotions or moral ignorance.

The question now is whether Eric is right to suggest that this asymmetry between blame and praise explains the asymmetries we find in people's intuitions about intending, being in favor, advocating, etc.

It seems like maybe one way to test this hypothesis would be to check to see whether one gets corresponding asymmetries in cases of irresistible emotions. Thus, suppose that the agent either (a) does something good because he is overcome with compassion or (b) does something bad because he is overcome with anger. Studies show that, in cases like this, one gets the opposite sort of asymmetry from the one we find with side-effects. Specifically, people are more willing to say that the agent is praiseworthy in case (a) than they are to say that the agent is blameworthy in case (b).

Now comes the critical test. If you think that the intuitions about intending we find in side-effect cases are driven by the asymmetry in blame, would you expect to find the opposite asymmetry here? That is, would you expect people to be more likely to say that the agent specifically intended to do the good thing in (a) than to say that the agent specifically intended to do the bad thing in (b)?

[Thomas Wood]

Josh and Eric,

Thanks for responding. Josh, you are correct to interpret my comments with “…the suggestion seems to be that people are more willing to blame in cases of side-effects but that they are more willing to praise in cases of irresistible emotions or moral ignorance.” Now the question becomes, where do attributions of intention enter into these two different kinds of cases? As you said, “The question now is whether Eric is right to suggest that this asymmetry between blame and praise explains the asymmetries we find in people's intuitions about intending, being in favor, advocating, etc.” My answer is yes and no; yes in the original side-effect effect, perhaps no (or maybe) in the cases involving irresistible emotions or moral ignorance.

Josh asked, “If you think that the intuitions about intending we find in side-effect cases are driven by the asymmetry in blame, would you expect to find the opposite asymmetry here? That is, would you expect people to be more likely to say that the agent specifically intended to do the good thing in (a [overcome by compassion]) than to say that the agent specifically intended to do the bad thing in (b [overcome by anger])?” Well, as you’ve found experimentally Josh, overcome by compassion cases generate strong praise while overcome by anger cases do not generate a corresponding degree of blame. This asymmetry in praise and blame accords with the comments I made previously. But do they similarly track attributions of intention? Here I have had to ride my armchair pretty hard to intuit that… I’m not sure… maybe… sort of…

I think that something like an intentionality attribution is being made in the strong praise of overwhelming compassion. This is because I think it would be hard not to identify the compassion as reflecting the true nature of the agent. The compassion leaked out, as it were, and betrayed the compassionate nature of the agent. From a functional perspective, we want to credit people for good deeds when they are done involuntarily or through ignorance because it signals to them that their bx is valued positively, sets up expectations that the bx should be repeated in similar circumstances, and rewards the agent. Now is identifying the good nature of the agent the same as saying that the good done was intentional? Sort of. It is identifying the good with the agent in a strong way. But it is identifying the good with the nature of the agent and not so much the voluntary decision enacted by the agent. The strong emotion of compassion is hard to square with voluntary bx. It’s as if the agent couldn’t help but act compassionately. But this is no less praiseworthy. Were that we all suffered such compassion! However, as I mentioned it’s not praising the act or finding the act intentional but perhaps the agent’s disposition intentional. That sounds a little incoherent, I realize. But coherence isn’t necessary for moral intuitions to function effectively. It’s a tricky distinction but illustrates how intention and volition do not mean the same thing and that a third factor, disposition or nature (or 'soul?') might be in play as well. So I don’t know how your subjects, Josh, would respond to the intention question. I think they would be torn between yes and no; yes in their identification of the agent’s dispositions or nature, and no in their identification of the involuntary nature of the agent’s overwhelming emotion. This dissociation in two possible senses of intention should be kept in mind and may help when interpreting survey responses.

This analysis leaves unexamined many assumptions which I didn’t want to complicate my comment with. There are assumptions about personhood and agency which I think are contributing to the intention attributions. What exactly does it mean to say that an agent intended to do something, or is in favor of something, or approves of something? What is doing the intending, favoring, approving? There is a presupposed theory of agency implied and involved here. Is it the case that the agent is making decisions independent of her dispositions? Or do her decisions always reflect her dispositions? Does this concept of agency change to match context? Ultimately, morality works (for the most part). Our job is to figure out how.

I’ll leave it at that. Nice show Josh on bloggingheads.tv!

PS. A good heuristic for the side-effect effect is that there exists an implicit obligation to prevent harm (within reason). When an agent fails to abide this obligation he has in effect meant it to happen, that is, intended it to happen.

Hi All,

I have a very general methodological point to make about the discussion so far.

Thomas and Eric, you've been taking an interestingly different (and potentially fruitful) approach to the side-effect phenomenon from the approach we took in the paper. Our approach was to think about the underlying psychological mechanism of the effect. You have been focusing on the utility of such a mechanism from an evolutionary perspective. This is an issue we don't consider in the paper.

There is a familiar worry about evo psych explanations: they don't lend themselves to being tested in the standard ways evolutionary theories get tested, e.g., because the mind doesn't fossilize.

However, it seems to me that the value of your approach is that it generates hypotheses about the underlying psychological mechanisms, which is something we can test using the methods of experimental philosophy/psychology, rather than the methods of evolutionary biology. Even if we cannot test the evo psych conjecture that generates the psychological hypothesis, we can test the psychological hypothesis itself. The utility of the evo psych reasoning lies its having generated a psychological hypothesis that might not have otherwise occurred to us.

If this is the right methodological strategy, we should be thinking about how the conjectured selectional pressures are supposed to be psychologically realized. The initial thought was that a conjectured evolutionary pressure to expose cheats is manifested as a disproportionately greater disposition to blame than to praise. However, there does not seem to be an across-the-board disposition of this sort. Other factors enter in to the ascription of praise and blame, such as whether the action was due to an irresistible impulse.

The revised proposal seems to be that we have a disproportionate impulse to blame rather than praise, but this can be overridden by such competing factors. The kind of issue I want to bring into focus is: how exactly does the override work? How exactly does the underlying computational mechanism go? Does the fact that the action was impulse-driven always trump the disposition to blame? Or do the strength of the impulse and the degree of harm somehow get weighed against each other? What is the underlying calculation?

This would give us a testable hypothesis about the underlying psychological mechanism.

[Dean Pettit]

Sorry, that last post was by Dean Pettit. Somehow my name didn't show up.

[Thomas Wood]

Dean,

Yeah, I think that’s definitely the idea with the adaptationist approach. But I’ve been purposefully vague in my appeal to functional explanations. Surely we’ve evolved moral intuitions in some sense, but I don’t think such evolved structures need to be appealed to when functionally analyzing the work that our moral intuitions do for us right here and now. Our moral intuitions still play their role, still function. Whether their presence is ultimately owed to the eons spent on the savannah or to our capacities as social creatures to learn cultural/moral norms is somewhat immaterial. I’m sure both are involved and that they are actually inseparable, but I don’t think that that issue has to be decided to make use of a functionalist approach. So whatever their ultimate cause, functional explanations can be used to explain both our moral intuitions and Knobe’s own findings. Then these explanations can be used to generate additional testable hypotheses which, in turn, will further refine our functionalist theories and eventually mechanistic theories of the mind. It’s a ratcheting affair that it’s hoped will converge on an understanding of the psychological mechanisms guiding our moral intuitions.

My best bet is that it will converge on the kind of computational mechanisms being discussed by John Mikhail right now. If so, a broad question then becomes whither the conceptual analysis approach that Josh seems wedded to? Although he has left behind the armchair as the fountainhead of philosophical insight, from what I can tell Josh hasn’t relinquished conceptual analysis (albeit an empirically informed conceptual analysis) as the font of philosophical investigation. I think that the best explanations for his findings are going to require giving up on the idea that intention is a single, unitary concept or that moral judgments influence attributions of intention as if one actually precedes the other in time (or at least that the two are conceptually discrete). I think that we are going to converge on an understanding of mental computations that track the context bound behaviors of agents to render [the folk] attributions of intention, belief, desire, praise, blame, hope, guilt, innocence, culpability, and even goodness and wickedness, etc. There won’t be a neat visio diagram to illustrate separate moral judgments and folk attributions [of intention, etc.]. These things will arise together as a single involved and complex computation of the social interactions of agents. The moral disapprobations and the attributions of intent will be found to be inseparable and constitutive of each other. The diagrams will look like those Mikhail himself has generated in his papers. Where does conceptual analysis fit into such an approach? I wonder.

As for some testable hypotheses--how about manipulating the kind of irresistible impulses that seem to be flipping the asymmetry of blame and praise? Some impulses like hallucinations and delusions are pretty easy to identify as exculpating factors while strong emotions like anger and jealousy seem more like intentional states. I think you should be able to manipulate the degree to which the side-effect gets flipped by manipulating the degree to which the agent seems to be the victim of impulse and irresistible emotion. Crimes of passion, while mitigated, are still prosecuted, after all. But someone like Hinckley gets a different kind of dispensation for the delusions he suffered.

[Dean Pettit]

Thomas,

I think you are right that conceptual analysis is not promising as a strategy to explain the effect. In fact, we argue that this explanatory strategy is hopeless in light of the pervasiveness of the effect. It seems unlikely that we can analyze the concepts of intending, deciding, desiring, advocating, opposing, and so on in terms of some common folk psychological concept.

We argue that the pervasiveness of the effect is to be explained by postulating a common psychological mechanism at work in the application of each of these concepts. The particular mechanism we propose is what we call "moral calibration". Each concept involves a positive/negative attitude and is applied against a scale that represents the degree of positivity/negativity, relative to a baseline level. The concepts are all influenced by moral judgments, we suggest, because the baseline level of positivity/negativity is the level we morally expect people to have.

We are less wedded to this particular proposal than to the general explanatory strategy. The pervasiveness is to be explained by a common psychological mechanism underlying the application of each of the folk psychological concepts that exhibit the effect, not by a concept that is common to the analysis of each.

[Nain]

I find this study fascinating, as blame plays a significant role in coloring social perceptions. One pertinent element in this study is the term "people" refers only to those in the western world. These results might differ greatly with global samplings.

In their paper, CULTURE, DIALECTICS, AND REASONING ABOUT CONTRADICTION, Peng and Neisbitt, found fundamental differences in how those from western and the eastern cultures assign blame. In fact, Western subjects were far more likely to find blame relevant and assignable to a specific individual than were the Chinese subjects.

[Joshua Knobe]

Nain,

This comment raises some really interesting issues. As it happens, a number of researchers actually have run cross-cultural studies on this phenomenon. Chongliang Li has shown that the phenomenon arises for Chinese subjects in Beijing, and Arudra Burra has shown that it arises for Indian people living in the United States.

Still, I think there is something very promising in your suggestion that Americans seem especially focused on the idea that blame should be assigned to *individuals*. Suppose that the study was changed so that, instead of individuals performing certain actions, there were *groups* performing those actions. (For example, there could be a corporation that advocates a particular policy on the grounds that it will increase profits.) It would be interesting to see whether there might be an effect such that people from Asian cultures were more willing to say that the group as a whole was 'deciding' or 'advocating' or 'in favor of' some bad effect.