Intentionality and Implicit Moral Judgment

Joshua Knobe, Paul Bloom, and I have just written a short empirical piece using the 'side-effect effect' as a measure of 'implicit' moral judgments, entitled 'College students implicitly judge interracial sex and gay sex to be morally wrong.' In short, we demonstrate that while inidividuals explicitly report that certain acts are morally permissible (gay kissing and interracial sex), they still report these acts as more intentional.

Any comments would be welcome! Any errors of the analytical or otherwise philosophical sort should be directed solely to Joshua Knobe. Paul Bloom should receive complaints about psychological matters or quantitative errors. Which leaves me alone to respond to any positive comments.

And thanks to Thomas for granting access to a social psychologist...

-David

Comments

[Neil]

I haven't read the paper (yet). That doesn't stop me commenting on it! Commenting on Knobe and Roedder's paper in the OPC, I pointed out that departures from norms get treated systematically differently. There were some hypotheses suggested by commentators as to why this might be the case. See thread here:

http://experimentalphilosophy.typepad.com/online_philosophy_confere/2006/05/joshua_knobe_an.html#comments

[Edouard Machery]

Very clever paper, as usual.

- You should give the probes that you used, or at least some of them.

- I wonder why you compared interracial sex with office gardening and not with sex. It is unclear whether people judge the side-effect increasing the frequency of interracial sex to be intentional because it involves *sex* or because it involves *interracial* sex.

- You might want to see whether scores at the IAT correlate with the ascription of intentionality in the interracial case.

I suppose that there are also measures of people's attitudes toward homosexuality that are similar to the IAT.

- I think you need to argue more cogently that people judge the side-effects increasing the frequency of gay kissing in public and increasing the frequency of interracial sex to be intentional because they make a *moral* judgment about these side-effects. It is now pretty clear now that the asymmetry found by Josh in moral contexts shows up in numerous non-moral contexts and is not always driven by moral judgments (on this topic, see Knobe and Mendlow, the first 3 experiments in Phelan and Sarkissian and my paper , the last 2 papers being posted on the blog).

[Tamler Sommers]

I agree, it's very clever but I'm skeptical. An alternate hypothesis would be that students attribute some degree of activism to a director whose video causes an increase in gay kissing. (I'd like to see the vignette too....does the video feature gay men kissing--that would make the alternate hypothesis even stronger.) They think he is trying to make some kind of statement--whereas the director who has heterosexual kissing is only trying to make a popular video. Same with the ad campaign. Can anyone think the people who created those old Benetton ads weren't intentionally trying to increase tolerance (even if the their main goal was to make money)? Or that the diretor of 'Cherry Bomb' wasn't trying to make a statement about race? (I'm dating myself here...) Well, a cynic could, but it seems plausible that students would find more intentionality in cases like this without judging the acts to be morally wrong.

[hagop sarkissian]

Social psychologists? Okay, there goes the blog...

;)

On a serious note, Tamler's suggestion seems plausible. I'll add another one - the hypothesis offered by Edouard's paper (posted just before this one): "People have asymmetric intuitions when they read the harm case and the help case (as well as similar cases), because they conceptualize the harm case, but not the help case, as involving a trade-off between a foreseen cost and a foreseen benefit and because they believe that costs are intentionally incurred in order to reap benefits." I wonder if this might be parlayed into an explanation of the judgments in this study as well; the students see the director as going after a benefit (fame, publicity) while risking a cost (controversy, backlash in the bible belt?), and thus think he is producing the effect intentionally. But I, too, would like more details about the study.

"A substantial minority (27%) of the subjects judged the gay kissing
scenario to be wrong, but even more (45%) judged the heterosexual kissing story to be wrong." David, the lower numbers for homosexual french-kissing (i.e. its greater permissibility) caught my attention, and brought to mind the 'trolley problem' studies you presented at the UNC moral psych workshop. If I remember correctly, subjects were presented one of two vignettes that went something like either: a) killing Leroy Jackson to save the Manhattan Wind Ensemble; or b) killing Louis Winthorpe III to save the Harlem Jazz Ensemble, with subjects finding the latter more permissible. Do you think this is another instance of the same basic effect?

[David Pizarro]

Wow- thanks for the comments to all. In such a short paper it's hard to communicate all our ideas about what's going on, but let me try to respond to some of your comments. I don't speak for Joshua and Paul, but I think we're on the same page here... I will post the scenarios in a second comment.

Control Condition:
Edouard and Adam Arico (in an email) ask about the weird Gardenia control condition. I admit that it's not ideal, but when we were constructing the scenarios we were afraid that using another sex act as a comparison might elicit higher disapproval (or implicit disapproval--if we're interpreting this correctly) than a completely benign behavior. In fact, this happened with a relatively benign control condition for a 3rd vignette that we mention in the footnote. Now, if we want to be certain about the claim that it's interracial sex and not sex in general, I think you are right--we just went for the most potentially strong example to try to nail down the effect.

Implicit Morality?:
I agree that our interpretation does turn on the claim that these intentionality effects are driven by moral evaluations of some sort. If it turns out that these judgments are tracking some OTHER feature, then we rest on much less solid ground. I say this not having read Edouard's papers yet. It is also perfectly reasonable to suspect that these judgments are tracking more than one feature, or some combination of features. It's true, as Edouard points out, that some sort of convergent validity check would be useful here--an IAT for gay/straight or black/white. I think this would be a good next step for us.

Adam Arico asked me if this isn't evidence to contradict Knobe's interpretation--after all, people are reporting that it's not wrong, so it seems reasonable to take this as evidence that these intentionality judgments are tracking something else. I think that Joshua is comfortable saying that these intentionality judgments are tracking an evaluation that may not be explicit, so he would take this as evidence for a revision of his claim. (he can correct me if I'm wrong)

Tamler's "activism" suggestion is also an intriguing possibility, although if I understand it correctly it would still require perceiving the action as something that was a candidate for activism, and in this case it may be that frowned-upon sexual behaviors are perceived as such precisely because they seem disagreeable. For instance, individuals are resisting the "proselytization" of sexual behaviors they implicitly evaluate negatively. One way to test this would be to do an experiment using an "activism" scenario with a behavior we KNOW the participants are in favor of, such as a pro-choice video (or whatever-you get my point).

BUT, I want to point to the possibility that the correlation we got with disgust-sensitivity and intentionality judgments are in fact evidence in favor of an implicit moral evaluation story, and would make less sense with the alternative stories. Why would disgust track something like norm-deviation (for example)? In my opinion, given the link between disgust and moral evaluation (see, for instance, the paper by Haidt & Wheatley), this is good evidence in favor of our position...

Finally, Hagop--yes, the flipped percentages of the kissing story are pretty funny. I think it's exactly the same as the Tyrone/Chip effect--people are pretty careful about being politically correct! I have a draft of a paper which reports the effects I talked about that day, if any are interesed it can be found here:
http://peezer.net/pubs/putd_motivated.pdf

[David Pizarro]

Here are our "transgression" vignettes:
*******************
1) Gay Kissing: (heterosexual kissing was identical, it just said "encouraging couples to kiss in public).

A director was working on a music video. His assistant said: ‘I took a look at the first cut of your video, and it looks to me like some of the images in it will encourage homosexual men to French kiss in public.’

The director said: ‘Look, I know that it will be encouraging homosexual men to French kiss in public, but I don’t care at all about that. I just want to make a video that will increase sales of the album.’

He included the images in the video. Sure enough, it encouraged homosexual men to French kiss in public.

2) Interracial Sex (control was encouraging gardenias in office)

The vice-president of advertising developed an ad campaign for a new product. He asked the head of marketing to test its effectiveness.

The head of marketing said ‘This campaign is indeed effective, but our marketing tests showed that it will also encourage interracial sex between Black men and white women.’

The vice-president of advertising said, ‘Look, I know that we’ll be encouraging interracial sex, but I don’t care at all about that. I just want to increase sales of the product. So let’s go ahead and launch that campaign!’

They launched the new campaign. Sure enough, it ended up encouraging interracial sex.

[jonathan weinberg]

I wonder whether the disgust-intentionality correlation suggests that a version of Edouard's hypothesis, in which the accepted cost is that the agent is doing something that leads to more disgustingness in the world. Your subjects are probably all high-SES, so the disgust-morality connection wouldn't be especially tight, anyway.

Has anyone done a nonmoral-disgusting side effect case yet? That would be an ideal sort of comparison.

[James Gambrell]

It could be that the transgression vignettes affect subjects by making them put their PC hats on. Evaluation of these hot-button type issues may cause changes in intuitions directly (some kind of priming effect) or may cause second-guessing behavior. I'm sure if you measured response times you would find delayed responses in judgments for the transgression vignettes.

Perhaps being put into a PC frame of mind just makes people more easily disgusted? It often seems to make them more easily offended, and disgust and offense are somewhat similar emotions. Call this the state sensitivity hypothesis. If the PC frame makes people more "sensitive" in some sense, then it might make them more sensitive to intentionality judgments as well. In my experience, sensitive people tend to read more intentions into other's actions than stoic people.

To test this you could invade subject's "personal space" right after they finish judging a vignette. Have the experimenter rather obviously get up close and personal while collecting the survey. Subjects in the transgression/PC activation condition should be more offended (they should back away more, or more often, for example). Just make sure it isn't a black person doing the invading.

You could also simply give these tests to a population outside the reach of political-corretness, or liberal ideas generally.

Maybe this sensitivity concept isn't the right way to put it, but the basic idea I want to offer is that subjects are being put into a temporary "PC" state; which is somehow affecting their intentionality judgments and many other judgments. The more general idea is that folk concepts are modified by mood states to a larger extent than is typically supposed. While most people would accept the idea that concepts are used a bit differently when people are drunk, or angry; it is more interesting to suggest that concepts are altered when something like considering difficult or controversial questions comes up.

I have no idea what to make of the gardenia correlation. Perhaps the people who thought the ads promoting office gardenias were run intentionally were put into a flowery, high-disgust mood?

[Tamler Sommers]

Couple quick things:

In activism story, the students only have to perceive that the general public finds gay sex to be morally wrong (hard to miss in an election year). But they don't have to consider the act to be wrong themselves. On the contrary, they would likely see the activism as necessary to change these attitudes.

Second, following up on Jonathan's point, isn't Haidt's line that liberals don't use the 'disgust module' as a means of moral evaluation--that liberals only consider fairness and harm issues? (I second Jonathan's request for non-moral disgusting side effect case.) If the disgust/morality connection isn't there then the results could only show that many liberal college students find gay sex disgusting. And many of them would likely admit to that anyway.

(Although I doubt they'd admit that in the case of interracial sex.)

[David Pizarro]

Jonathan- your comments about being put in a more sensitive state piqued my curiousity so I just took a look at our data to see if the folks who got the "transgression" vignettes reported more disgust-sensitivity. There's no significant effect, and the means are in the wrong direction slightly (neutral condition reports a touch more DS, p=.12). This doesn't defeat your hypothesis (a null result could occur for a variety of reasons, after all), but it would've been nice to see it, as I kind of like your sensitivite-state hypothesis.

Tamler, I agree that it's not necessary to perceive "badness" to perceive "activism". But, I still think it would be a stronger effect if it was activism of the sort that you disagreed with. It's an empirical question I guess.

[Chris]

I'd use Evaluative Movement Assessment, rather than IAT. EMA has a meaningful score of zero, since it uses a netural reference point, and while it's still hard to say that the difference between a score of 3 and 6 is the same as a score between 6 and 9, it's at least more likely to be the case in EMA than IAT.

[Tanasije Gjorgoski]

I got here through the Mixing Memory blog, and while I commented there, I guess this is the better place to put the comment. It is about an interpretation of why do you get those results.

The stories which are used in the test usually fall under this general case: If person P does A with intention I, it would also cause R, and person A knows it.
(e.g. P="Director" A="makes a video" with intention I="to increase the sale of album" or P="Vice-President" A="starting a new program" with intention I="to make money")

Now, if someone thinks that R gives a reason to person P not to do A; he will have to conclude that person P (aware of that reason) intentionally ignored that reason as such.
E.G. If we consider that "hurting the environment" might be reason not to "start new program", "Vice-president" will intentionally ignore the reason, hence intentionaly hurt the environment

If the person on other side thinks that R doesn't give any reason to A not to do A... well it doesn't affect his intentional action in any way.
E.G. "helping the environment" is not giving any reason to start or not to "start the program". "Vice-president" doesn't have anything to consider except his original intention ("to make money"). One could also say "it will leave the environment intact" or "will change the place of two grains of sand in desert". Those are irrelevant and don't give any reason not to do what is intended to do.

So, by this interpretation people will tend to give positive answers to the question if someone produced the consequence intentionally, if they consider that the consequence should have been seen as a reason not to do the intended act. This I think explains also Edouard's results, as reason not to act might be not just moral reason, but also economical reason, aesthetical reason, and so on...

[Edouard Machery]

Tansije,

It seems unlikely that 95% of my subjects thought that paying an extra-dollar was a reason NOT to get the milk-shake in the extra-dollar case in my paper.

Edouard

[Jonathan Weinberg]

Why not, Edouard? Isn't a higher cost always _a_ reason not to do something, even if only rarely a conclusive reason? Indeed, it seems to me that Tansije's proposal might just be a notational variant of your proposal: "doing phi while having a reason not to phi" seems like an example of something involving a trade-off, doesn't it?

[Edouard Machery]

Jonathan,

You are right--having to pay an extra-dollar is indeed *a* reason not to phi, if not *a conclusive* reason.

Edouard

[David Pizarro]

Chris-Thanks for pointing me to the EMA, I hadn't read about it. For those who are interested, I found the article describing this method here:
http://www.psy.utexas.edu/psy/FACULTY/Markman/JESP04.pdf

[Chris]

David, welcome. You can get the software from Miguel Brendl, and probably from Art (at least, he'd send you to Miguel).

[Benjamin Nelson]

-How was the "liberalness" of the subjects tested? How do the experimenters understand the term, "liberal"?

- Were all the subjects heterosexual?

-On the heels of Jon Weinberg's comment: This study seems to presuppose the notion that a moral judgment is identical to a value-judgment. What I mean is, there seems to be, on the face of it, evidence for a value-judgment here. But I can detect no moral judgments involved.

Let's say a person thinks some act is morally permissible, but personally wants nothing to do with it because it flat counters their personal preferences. Is this a plausible distinction? Are scenarios where this may occur imaginable? I think so, in both cases. I cringe when I see people eat mayonnaise. But so far as I can see, the conclusions of this study can only be cashed out if there were no such distinction: my disgust for mayonnaise would have to automatically imply that I morally condemn mayonnaise-eaters. You would also have to show that, when push came to shove, the hypothetical disgust for the act would lead to actual condemnation, either tacit or explicit. This would require a more controlled experiment.

- Also, more seriously, I think the argument in this study can't get off the ground in principle. For, due to what's called the "publicity condition", moral judgments cannot be conceived as implicit. All moral judgments must be cashed out in terms of the disposition toward certain other-affecting behavior, speech, etc. Morality is essentially about social guidance, not just self-direction.

- To take a different tack: if I were Larry Kohlberg, I might go on to say that not even the publicity condition would be enough. Rather, you'd also need to show moral reasoning.

- Many of these comments are interesting (esp. James's). One possible explanation for the 27-45 difference would be that people wanted to give the impression of tolerance in the 27 case when focusing on the *character* of the act, and then focusing on the ad-man activist *intentions* behind the act in the 45 case (i.e., of a sleazy ad agent manipulating others). The desire to make a PC interpretation, in other words, may eclipse the desire to condemn ad-man activist intentions (as one seems more important than the other, either morally or socially).

[G]

I agree with Tamler for the most part.
The thesis might be a little shaky because it assumes that Knobe's phenomenon is correctly defined. Has his thesis been tested against things that society things are bad but they don't? Or there could be an even more complex relationship.

and also see something in the argument

"liberals don't use the 'disgust module' as a means of moral evaluation--that liberals only consider fairness and harm issues?"

is this what defines a person as a liberal as opposed to a conservative? (Otherwise maybe there are a few conservatives that reject disgust in favor of some sort of a "conformity is good" model or "history should be honored".

[David Pizarro]

I just wanted to make a quick point about what Benjamin said--

"I think the argument in this study can't get off the ground in principle. For, due to what's called the 'publicity condition', moral judgments cannot be conceived as implicit..."

I think that you're certainly right about morality being about social guidance, so it's a little weird to speak of our finding this way. So perhaps you're right that we're misguided in using the term "implicit moral judgment". However, I think that we might be tapping into something psychologically real and interesting about the nature of moral cognition. Maybe there's an implicit component that contributes to the psyhological process we normally refer to as "moral judgment." Perhaps this is just the psychologist in me, but I'm not sure I would want to constrain the concept of "moral judgment" so as to preclude the possibility of an underlying implicit process.

Also, a quick note for "G" about disgust: In my lab we have some findings on the relation between disgust sensitivity and political conservatism (positive correlation in a number of different sample). When we write it up I'll post it here to see what you guys think!

-david