Knowledge, Stakes, and Mistakes

Do stakes impact knowledge ascription?  According to a prominent claim in recent epistemology, people are generally less likely to ascribe knowledge to a high stakes subject for whom the practical consequences of error are severe, than to a low stakes subject for whom the practical consequences of error are slight, even when “traditional” epistemic factors like evidence and belief are held fixed (So for example, people are less likely to agree that Sarah knows that she parked her car on the upper level, if an armed robber is lurking on the lower level).  

The claim that stakes impact knowledge ascriptions holds intrinsic interest to anyone interested in the psychological basis of knowledge ascriptions. It holds further interest to epistemologists because it features as the lead premise in an abductive argument to the radical claim—defended by Fantl & McGrath (2002), Hawthorne (2004), Stanley (2005)—that knowledge itself is sensitive to stakes. And, it holds a different sort of interest to experimental philosophers as an empirical claim made from the armchair.

Given all of this interest, it is perhaps unsurprisingly that so many different philosophers over the last few years have beugn studying the behavioral role that stakes could be playing in people's knowledge judments. There was a "first wave" of empirical studies—due to Feltz & Zarpentine (2010), May et al (2010), and Buckwalter (2010)—which casted doubt on folk stakes sensitivity.  Then, there was a "second wave" of empirical studies—due to Pinillos (2012) and Sripada & Stanley (2012)—said to vindicate it.  (And see great posts summarzing the major findings of the second wave experimental philsophers here on xphi and over on certain doubts by N. Angel Pinillos and by Jason Stanley.)

For those interested in the latest on stakes, or are maybe just getting into this debate for the first time, Jonathan Schaffer and I have recently finished a new paper offering an opinionated discussion of the “state of the art” of this research.  In our paper, we review the first and second wave results above, as well as present a few new studies of our own.  We conclude that the balance of evidence to date still best supports the folk stakes insensitivity thesis--or the claim that all else equal, people are equally likely to ascribe knowledge to a high stakes subject as to a low stakes subject.

Comments

[Dustin Locke]

Very interesting paper. It's excellent to have such a nice summary of all this work in one place. But I would like to offer a few criticisms, all of which concern your discussion of Pinillos's work in section 2. I'll simply cut and paste what I wrote on Stanley's FB post.

(1) p. 11 plausibly involves an equivocation on the word 'guess'. On one reading, 'guess' means something like 'shot in the dark'. That's the reading where there is plausibly no guessing-knowledge principle. But on another reading 'to guess' means something like 'to reckon' or 'to figure', and this plausibly requires belief. That's the reading most naturally suggested by the prompt, and on that reading there is perhaps a guessing-knowledge principle.

(2) The alternative account of Pinillos's results offered in section 2.3 doesn't seem to handle Pinillos original results--the results concerning the prompts that use the word 'know'. The authors say that all of Pinillos's results can be accounted for by the hypothesis that subjects are treating 'has' as a deontic modal. They claim that subjects are responding to the fact that stakes affect how much evidence one must have before it is reasonable/rational/etc to form a given cognitive attitude. But the cognitive attitude expressed by "knows that" does not seem to be subject to any such principle. How much evidence must you have before it is reasonable to start to knowing that p? That question hardly makes any sense.

(3) On p. 15 the authors introduce a new study about which they claim that Pinillos's account predicts that there will be a much stronger tendency of subjects to agree with the knowledge-ascription in the low-stakes prompt than in the high-stakes prompt. That seems false. Pnillos's account is entirely compatible with the claim that, despite the higher stakes in the high-stakes prompt, the subject still has sufficient evidence for subjects to attribute knowledge (which is exactly what the authors find). Indeed, this seems plausible given the prompt, which asks us to suppose that Peter is "naturally a pretty good speller" and that he has used a "dictionary" to "carefully check his paper twiceover". (Note that Pinillos's original prompts include nothing about Peter being a good speller, nor anything about a dictionary, nor anything about him "carefully checking" his paper. They simply ask about "proofreading", without any explanation of how good Peter is at proofreading or what method he will use to do it.)

[Jonathan Schaffer]

Thanks Dustin! We appreciate the thoughtful criticisms. Here are some replies:

On (1): I doubt that there is any second sense of ‘guess’ that means what ‘reckon’ means, and which is governed by a knowledge norm (and which all of our participants used). I think that at most ‘guess’ can be used figuratively as an understatement for a more reasoned process of reckoning. If ‘guess’ did have the sense you suggest it has then it should be perfectly smooth to say things like: ‘Ann came to know that Albany is the capital of New York through a process of guessing,’ and (as an expression of a knowledge norm): ‘You should only guess it if you know it.’ Do these claims sound ok to you? At least in my idiolect, these claims simply do not have plausible readings.

Anyway, as you’ll recall, our main point was not merely that we could replicate all of Pinillos’s results substituting ‘guess’ for ‘know,’ but that we could replicate all of Pinillos’s results (without statistically significant difference) substituting any of ‘belief’ or ‘guess’ or ‘hope’ for ‘know’ (p. 13). We thought that was a pretty good reason to suspect that ‘know’ wasn’t doing any real work in generating the results! To block this argument you’d also need to argue that there was a sense of ‘hope’ that was governed by a knowledge norm, for which ‘You should only hope it if you know it’ was a plausible norm, etc. Are you thinking that there is also a sense of ‘hope’ which means what ‘reckon’ means?

On (2): You are certainly right that the question ‘How much evidence must you have before it is reasonable to start to knowing that p?’ hardly makes sense. But that was your question, not ours! We think that knowledge is an attitude, and that attitudes can be formed (of course for some attitudes like knowledge, the world must cooperate for successful knowledge formation). I would have thought that this was fairly uncontroversial. So the relevant question is, how much evidence must you have before it is reasonable to form the attitude?

Perhaps what the agent does in forming the attitude of knowledge is to come to a certain belief, on the basis of certain evidence, in apt circumstances (where the proposition is true and the evidence is indicative etc.) Of course any articulation of this will be controversial. But just pick your favorite story of what the agent does to form knowledge. Now ask: how much evidence do you need before it is reasonable to do that? I hope that is a perfectly sensible question! That’s what we are suggesting participants are responding to in Pinillos’s studies.

On (3): Let me try to explain the context of this study in more detail. Apologies if it was not clear from the paper. First of all, Pinillos’s original prompts do indeed include the claim “Peter is a pretty good speller” and that “Peter has a dictionary with him” (see p.7 of our paper). What Pinillos then claims in his studies is that, in a low stakes scenario, participants tend to say that Peter needs to proofread his paper (with the dictionary) TWO times in order to know, but that in the high stakes scenario Peter needs to proofread his paper (with the dictionary) something like FIVE times in order to know. That’s the background.

So we wondered: what would happen if we told participants that Peter had in fact proofread his paper (with the dictionary) exactly TWO times, while varying the stakes? By Pinillos’s interpretation of his results, Peter should have done just enough (by the lights of the typical participant) to know if the stakes are low, but he should not have done enough (by the lights of the typical participant) to know if the stakes are high. Make sense?

Yet people did NOT respond in the way. People tended to say (very strongly) that Peter knows after two proofreads, even when the stakes were high. Indeed stakes made NO statistically significant difference to knowledge ascription. So we took this to suggest that, whatever is going on in Pinillos’s original studies (which involve complex probes with multiple elements), stakes do not impact direct knowledge ascription. This should not be surprising. Indeed this is just what the "first wave" studies tended to suggest.

[Dustin Locke]

Thanks for the reply, Jonathan. Some thoughts in turn.

(1) My point here was minor, and was intended to apply only to 'guess' (and not to 'hope'). For my part, I don't think that belief is governed by a knowledge norm, but I'm thinking that people who *do* think that it is will also find it plausible that one sense of 'guess' is also governed by such a norm. Here's the sense I have in mind. "Is John going to the store? I think so." "Yeah, I guess he is going to the store." That sense. But you have to read the sentence right: put the emphasis on 'is' and not on 'guess'.

(2) Let x be the cognitive attitude part of knowing. You say that participants are interpreting the question "How many times does Peter have to proofread his paper before knowing that p?" as "How many times does Peter need to proofread his paper before it is reasonable to form the attitude x that p?" Either x is equivalent to knowing or it isn't. If it is, then you're claiming that they are interpreting the question as a question that doesn't make any sense. If x is not equivalent to knowing, then you are claiming that *not only* are participants interpreting the 'has' as an epistemic modal, but *also* that they are interpreting 'know' as meaning something other than what 'know' means--i.e., they are misinterpreting 'know'. Maybe. But all things being equal that's less likely than the hypothesis that they are interpreting it correctly.

(3) My apologies for misrepresenting Pinillos's original prompts, and thereby your work. You are absolutely right that they do include what I said they don't include. But here's an interesting fact: they include the information about being a good speller and having a dictionary towards the beginning of a somewhat lengthy prompt. Hence it's reasonable that participants are forgetting this information. Case in point: I forgot it! Moreover, Pinillos's prompts don't say that Peter will be using the dictionary, nor do they say he "carefully checks" when he proofreads. For my part, I would like all of these differences controlled for. Oh, one other thing I forgot to mention last time: Pinillos's prompts also don't say that in fact, there are no typos, while yours do. (Or am I am mistaken again?)

[Jonathan Schaffer]

Hi again Dustin,

(1) Ok, as long as you agree that this does not extend to 'hope,' we can still make our main point on this basis. But for the record I hear your "I guess he is going to the store" (after hearing testimony to that effect) as understatement, along the lines of someone who steps outside into a rainstorm and says "I guess it's raining."

(2) I am happy to say that x is knowledge. The question is then "How many times does Peter have to proofread his paper before it is reasonable for him to form the attitude of knowledge to the claim that there are no typos?" Maybe I'm missing something, but I don't see anything problematic about that question. Do we agree that there is an attitude of knowledge? Do we agree that people can form this attitude? If so what's wrong with asking whether someone has enough evidence to form this attitude??

(3) I think we may be talking past each other. We aren't presenting this case as a minimal pair contrast with Pinillos's case! We are running a very different experimental design here, on which we fix evidence (two proofreads) and truth (no typos), and then ask people to make a simple knowledge ascription.

The point of this study is that Pinillos's probes are not simple knowledge ascriptions but complex sentences with lots of other elements tossed in (incl. deontic modals). So we were trying to cook up a different but related study that worked directly with simple knowledge ascription probes. And we then found no statistically significant effect of stakes -- people tend to think that Peter knows on the basis of two proofreads, and stakes just don't seem to impact this judgment at all, at least in any way we could detect statistically.

[Wesley Buckwalter]

Hey Dustin,

Thanks for these follow-up comments. Just on the (3) front, and perhaps Angel can correct me here, I thought some of the original studies lacked the truth confirmation, and then some subsequent evidence collection studies (like the one designed to check to see if the original stakes effect was due to this objection raised previously by myself and by Jennifer Nagel that the agent needs more evidence before forming the belief) included it. Then Angel was able to get effects in both circumstances, though slightly smaller effects in the latter. Hard to know if the slightly smaller effect was actually due to truth information (and what we should make of that if so), there was something to the belief objection we had all been raising after all, simply noise, or some completely different unaccounted for thing between them. But I think interesting questions in their own right.

Regarding the differences in language in Angel’s case (“Even though Peter is a pretty good speller, he has a dictionary with him that he can use to check and make sure there are no typos”) and the experiment we ran specifying the evidence that Peter did in fact have (“Peter is naturally a pretty good speller, plus he has a dictionary with him which he has already used to check the paper carefully, twice over”), fair point that maybe the ‘carefully’ is adding a bit extra evidence into the two-checks from the original...such that perhaps one would worry it rises above the threshold where you would get the effect. (I’m skeptical that it would be an addition of the equivalent evidence gotten from the 3 extra checks Angel has demonstrated people want for High in the original, but we could try this!) Also I think it's important to bear in mind that, as Jonathan mentioned, there’s obviously going to be some structural variation (for example the “can use to check” and “has already used to check”) difficult to control for just in virtue that Angel’s experiment is made to measure the evidence collection necessary for knowledge and ours is a very different type of experiment, made to measure people's agreement with knowledge statements when we specify it. That said, I think it would be totally interesting to play around with the base amount of evidence (e.g. just one check, check no dictionary, etc) to see how this might influence agreement.

[Dustin Locke]

Thanks Jonathan and Wesley! I'll just comment once more in regards to (3). Jonathan, I understand that you aren't presenting your case as a minimal pair contrast with Pinillos's case. Rather, you are claiming that Pinillos's account of the results of his own study predicts a certain result in your study--namely, that there will be a much stronger tendency of subjects to agree with the knowledge-ascription in your low-stakes prompt than in your high-stakes prompt. I'm saying that Pinillos's account of the results in his own study doesn't predict what you say it predicts about your study. This is because your study includes and makes salient certain information that was not included/made salient in Pnillos's study, and this information is likely to make it so that even in the high-stakes prompt subjects will think that Peter has enough evidence to count as knowing.

I like Wesley's suggestion to play around with the base amount of evidence. What Pnillos's account does predict is that as you start lowering the base amount of evidence at *some* point you will start to see a difference between strength of agreement in low-stakes and strength of agreement in high-stakes. My claim about the study you've thus far ran is that you may simply have not lowered the base amount of evidence far enough yet. In fact, it's pretty plausible that you haven't: you've got a subject who is "naturally a pretty good speller" and who has "twice" "used a dictionary" to "carefully check" a paper that "in fact contains no typos".

[Wesley Buckwalter]

Wait but he's "a pretty good speller" who can/does "use a dictionary" in both experiments, right? Then, in the original, people say he has to check twice (median score) in Low. So in our study he just simply goes ahead and does those very two checks.

Entirely willing to grant that it's possible the evidence isn't lowered far enough here *independently* of what we might have predicted on the basis of the original study (though at some point that's going to become uninteresting when it drops below the threshold for knowing entirely)...but I'm not really sure I get how it's so plausible to you that we didn't lower the threshold enough *by* the lights of the previous study...I would have thought the most plausible prediction here would be to expect that since people think five (median score) checks were needed for High, actually doing only the two would be totally insufficient!

[Dustin Locke]

"I would have thought the most plausible prediction here would be to expect that since people think five (median score) checks were needed for High, actually doing only the two would be totally insufficient!"

What I'm saying is this: "two checks" relative to your prompts is not equivalent to "two proofreads" relative to Pinillos's prompts. Your checks are "careful" and it has been made explicit and salient that Peter is using a dictionary to perform the checks, that he is a good speller, and that in fact the paper contains no typos. It Pinillos's prompts it was made explicit (much earlier in the prompt) that Peter "has" a dictionary that he "can" use for proofreading and that he is a good speller. But these facts were not made salient at the time the question was asked--that is, they don't occur at the end of the prompt. Moreover, in Pinillos's prompt there is no mention whatsoever of Peter "carefully" checking, nor is there any mention of whether he actually uses the dictionary, nor is there any mention of whether in fact the paper contains typos.

Moreover, and here's a new point, even leaving aside all of the above differences, one "check" for typos is not necessary equivalent to one "proofread". If you tell me you've "checked" for typos, I'll assume that you've done whatever such a check involves, where this very well may involve *more than one* proofread.

[Dustin Locke]

What I am saying is that I would like to see the results using the following high-stakes prompt and analogous low-stakes prompt, rather than the prompts you in fact used:

"Peter, a good college student, has just finished writing a two-page paper for an English class. The paper is due tomorrow. Even though Peter is a pretty good speller, he has a dictionary with him that he can use to check and make sure there are no typos. There is a lot at stake. The teacher is a stickler and guarantees that no one will get an A for the paper if there is a
typo. Peter needs an A on the paper to get an A for the class, and he needs an A for the class to keep his scholarship. If he loses the scholarship he will have to leave school, which would be devastating for him. So it is extremely important for Peter that there are no typos in the paper. Peter has proofread his paper twice."

And then ask your question:

To what extent would you agree or disagree with the following claim: “Peter knows that there are no typos in his paper.”

If you use these prompts, I think we'll have to accept your claim that Pinillos's account predicts much stronger agreement in the low-stakes case than in the high-stakes case. As things stand, I think that claim should be resisted.

For what it's worth, let me not resist the opportunity to put some money down from the armchair: I predict that you will find that there is stronger agreement in the low-stakes case than in the high-stakes case, just as Pinillos's account predicts.

[Jonathan Schaffer]

Hi Dustin,

A bet! Wow! You are confident! There is a small problem with your vignette though, which is that you are not controlling for other factors such as truth (whether there are in fact any typos). Nor is this an idle concern: participants in the high stakes prompt, out of more concern for Peter, may very well be more worried about undetected typos. This can be fixed though, simply by adding on (as we did in our studies): "In fact there are no typos in his paper."

So -- just for fun -- I actually went ahead and ran a pilot on your vignette (with this one final sentence added to control for truth). I used MTurk, and had 30 respondents in High and 30 respondents in a standard Low variant. Following Sripada & Stanley's technique I for asked participants for an assessment of quality of evidence, and then asked the knowledge question.

Guess what? Induction holds. No stakes effect was detected, just as in most of the other studies. I'll report the means for the evidence and knowledge questions.

Locke's vignette, high stakes
Evidence: 5.63
Knowledge: 5.5

Locke's vignette, low stakes
Evidence: 5.73
Knowledge: 5.43

Convinced now?

[Dustin Locke]

Very interesting, and thanks for running the pilot!

Unfortunately, no, I'm not yet convinced. I understand that you want to control for truth, but there's a problem with your new vignettes. As I understand what you've just said, the vignettes you gave now end with the following two sentences.

"Peter has proofread his paper twice. In fact there are no typos."

Non-philosophers notoriously treat talk of facts in a much more epistemic way that philosophers do. I direct your attention to a google search for "definition of fact". It's quite possible that subjects are reading the last two sentences here as "Peter has proofread his paper twice and discovered that there are no typos." Since to "discover that p" plausibly entails "came to know that p", subjects may simply be taking the vignette as saying that Peter knows that there are no typos.

I'm not sure how you're going to control for truth without implicitly suggesting knowledge. You could try:

"Peter has proofread his paper twice. There are no typos."

But subjects might take this to imply a connection between the two claims--i.e., by proofreading, Peter has discovered that there are no typos. How about putting "There are no typos in his paper" towards the beginning of the prompt and in such a way that it does not suggest to readers that this is something that Peter knows?

By the way, are you using a comprehension check on your MTurk survey?

[Dustin Locke]

By the way, how can you conclude anything about my confidence level from the fact that I offered a bet? Don't you need to know the stakes? I was assuming that you were going to give me odds. :)

[Jonathan Schaffer]

Dustin,

I think we have been extremely fair with you. I took the time and trouble to run your vignette *verbatim*, with one added sentence needed to control for truth. I got the same sort of results we always seem to see: yet again, there was no detectable difference due to stakes. This still doesn’t move you?

It is always possible to speculate about possible misreadings. Personally I don’t find it at all plausible that participants are reading “In fact there are no typos” as “Peter knows that there are no typos,” and think you would owe some serious empirical evidence on that score, instead of one google search and pure speculation. (By the way I did your google search and found a definition of fact as “something known to exist.” But that does not imply that *Peter* knows.)

Indeed there is an obvious reason to think that participants are not reading “In fact there are no typos” as “Peter knows that there are no typos.” The reason is this. If participants were reading “In fact there are no typos” as “Peter knows that there are no typos,” then they ought to completely agree that Peter knows that there are no typos (e.g. rate this a 7). The fact is that the means on this question were around 5.5 (with lots of 4s, 5s, and 6s). So I would say that the data already speaks fairly directly against your speculation.

PS We are indeed using various comprehension checks and screens on MTurk.

[Wesley Buckwalter]

Hey Dustin,

Double team, sorry! I have to admit, when I compare up the actual materials from the original and subsequent experiments, it really seems like you might be exaggerating the evidential differences between the various cases. So I would be totally curious to hear what other readers think about this! And to help with that, I’ll just reprint (though bear in mind there are also other variations):

Pinillos 2012 evidence collection:
John, a good college student has just finished writing a two-page paper for an English class. The paper is due tomorrow. Even though John is a pretty good speller, he has a dictionary with him that he can use to check and make sure there are no typos. There is a lot at stake. The teacher is a stickler and guarantees that no one will get an A for the paper if it has a typo. He demands perfection. John, however, finds himself in an unusual circumstance. He needs an A for this paper to get an A in the class. And he needs an A in the class to keep his scholarship. Without the scholarship, he can’t stay in school. Leaving college would be devastating for John and his family who have sacrificed a lot to help John through school. So it turns out that it is extremely important for John that there are no typos in this paper. And he is well aware of this. How many times do you think John has to proofread his paper before he knows that there are no typos? ____ times.”

Buckwalter & Schaffer evidence specified:
Peter is a good college student who has just written a two-page paper for an English class. The paper is due tomorrow. Peter would like his paper to be free of typos, and there is a lot at stake. The teacher is a stickler and guarantees that no one will get an A for the paper if there is a typo. Peter needs an A on the paper to get an A for the class, and he needs an A for the class to keep his scholarship. Otherwise he will have to leave school, which would be devastating for him. Peter is naturally a pretty good speller, plus he has a dictionary with him which he has already used to check the paper carefully, twice over. In fact there are no typos in his paper.“ Peter knows that there are no typos in his paper.” [Agree/Disagree]

The only thing that makes me think we’ve added evidence here is *maybe* the ‘carefully’ or perhaps the truth element verification as you mentioned. But indeed, I think Jonathan's pilot tells against the former...and for the latter, we simply have prior data from a subsequent variation from Pinillos showing the effect including truth verification. (If anything, I would suggest we offer *less* evidence since in the original the DV involves 'proofreading' and our case only mentions two 'checks', but maybe I'm wrong that proofing is better than checking!) So I'm not too worried about those two things, but perhaps there are others we have yet to consider other bloggers could help point out. And of course, as we've been discussing, perhaps it would be fun to run more variations with lower evidence independently of what we predicted given prior findings.

Lastly, all this said, I also think it's important to bear in mind that this is just one (negative) part of our argument (effect disappears when ‘know’ is retained but the modal removed). When presenting this work, I usually tend to highlight the positive result we found (exactly the same stakes effect persists when ‘know’ is removed but the modal retained for a series of diverse verbs).

[Dustin Locke]

Sorry, I'm not trying to be obstinate. I do think you've been more than fair to me and I appreciate your taking the time to run a study that was near verbatim to the one I suggested. It just so happens that I don't think it was near enough.

Also, why are these captchas so difficult to read? Haha.

[Angel Pinillos]

Hi guys,

I really liked this paper, the data and the analysis. Your work has prompted me to design some further experiments (with Shawn Simpson). But until then I have a few comments.

(1) Concerning the experiments where you guys replaced ‘knows’ with ‘believes’, ‘guesses’ and ‘hopes’ in the evidence-seeking experiments-- “how many times do you think Peter has to proofread his paper before he knows/believes/guesses/hopes that there are no typos?". There are two interesting but distinct results which you guys take to support the thesis that the results in my paper don’t have much to do with knowledge. (a) There seems to be stake sensitivity for those attitudes. (b) Responses to the high stakes versions of those verbs are the same across all attitudes---and the same goes for low stakes. So the first point (a) that there is stake sensitivity for “hopes”, “guesses”, and “believes” is not necessarily something that is very problematic for me, at least prima facie. IRI is neutral about that. Your point, however, is that we don’t expect there to be stake-sensitivity on the literal reading of those sentences (or at least the reading required for the experiment I ran). So there must be some other reading at play. And presumably this other reading has little to do with knowledge or the attitude at issue. Point (b) is related, presumably we don’t expect all of the attitude prompts to be given the same response. So again, the reading, must be something different from the intended one. Now, I think that (b) is not very strong evidence since it is a null result, so I don’t put too much weight on it--though it is still informative and helpful. About (a), I find it very plausible that there are uses of “guesses” which exhibit stake sensitivity (I also agree with Dustin Locke that "guesses" has the "reckon" reading). Guesses are commitments, after all. We often make informed guesses and not all guesses are random stabs in the dark. So it is natural to think that we will gather more information before we guess when there is a lot at stake. So it is not surprising that "guess" is stake sensitive. But what about ‘hope’? the prompt is this: “how many times do you think Peter has to proofread his paper before he hopes there are no typos?”. Even after reading this a couple of times, I am not sure I really understand what it means. It seems like, in general, proofreading more does not increase the likelihood that you hope there are no typos. It seems like you probably were hoping right after the first proofread. In fact, it might even be that, in some cases, the more you count, the less you are likely to *say* you hope there are no typos: "I don't hope there are no typos, I *know* there at no typos!!!"--so this makes the question puzzling. At any rate, due to the oddness of the question, I’m thinking here that participants are just reinterpreting the question as the closest question that the experimenter could have posed: “how many times must he check for typos?” and the answer here should be the same as the knowledge prompt according to IRI. This is just the result that you guys got.

(b) This is about the experiment where you stipulate that Peter has counted twice (in high and low stakes) and ask people their level of agreement concerning the sentence “Peter knows there are no typos”. The result is that people tended to agree with the statement. And this is presumably something I would not predict since in the high stakes evidence-seeking probes, people say Peter has to count 5 times before he knows. So in the prompt it says Peter counted twice. I detect an implicature that he is done counting. If people assume Peter is done counting, then they would simply trust that Peter has done enough to know there are no typos given his high stakes situation (he must have proofread extra-carefully). So I am not surprised by this result. This is a tricky experiment to run for the reason I just mentioned. I have ran similar ones (with Shawn Simpson). The difference for ours was that we had an interlocutor confront the protagonist with something like "hey, you haven't counted enough times, this is a high stakes situation". I think this modified prompt makes us trust the protagonist less and leaves it open that Peter will count more times. For these experiments we got the result in favor of IRI for 2 out of 3 cases. If you are curious, these results are in a paper “Experimental Evidence in Support of Anti-Intellectualism About Knowledge” in my website pages 16-19 (summary on page 19) (please note: the version on that website is an old draft). This paper is going on a forthcoming volume.

[order]

Waoo this article is really good for learning :)

[Jonathan Schaffer]

Thanks Angel, for the thoughtful and detailed comments. You’ve given us a lot of excellent points to think about!

(1) On the studies where we replaced ‘knows’ with ‘believes’ and ‘guesses’ and ‘hopes’ and got the same results:

Overall it seems to us that there is a real concern with your probes, in that you do not use simple knowledge ascriptions (‘Peter knows…’), but more complex probes that include lots of other material (‘Peter has to proofread his paper ___ times before he knows…’) We were claiming—by way of reconciling your results with the other results detecting no stakes effect on simple knowledge ascriptions—that your stakes effect was coming from some other element in your probe (in particular the deontic modal element in ‘has to’). It still seems to me like a very fair, natural, and straightforward test of this claim is to see what happens if we replace ‘know’ with other attitudes (e.g. ‘believe,’ ‘guess,’ and ‘hope’). We found that it made no detectable difference whatsoever, either to the presence or the magnitude of the effect. That seems like a pretty good reason to think that ‘know’ is not triggering the effect, no?

Now there are things you can say about each study. It is possible—indeed not implausible—that belief is governed by a knowledge norm. And it is possible—though I do not find this very plausible, that there is a reading of ‘guess’ that is actually governed by a knowledge norm, and which was elicited in all of our participants. And it is possible—though I do not find this very plausible either—that all of our participants engaged in some reconstructive reading of the probe with ‘hope’ as an action claim governed by a knowledge norm. It just seems to me still that the overall must plausible interpretation of the data is that your stakes effect is coming from the deontic modal and not from ‘know.’ It’s not just that this interpretation fits the first wave data detecting no stakes effect on knowledge ascription, and it’s not just that this interpretation fits our data showing that your effect survives the replacement of ‘know’ with all sorts of other attitude verbs. It’s also that your probe clearly contains a deontic modal, and that deontic modals are clearly stakes sensitive (by everyone's lights).

(2) On your new knowledge ascription studies:

First and foremost I’m excited that you are doing these new studies! I went ahead and looked at your (very cool) paper with Shawn Simpson. That said I’m not convinced that there is any implicature in our studies of the sort you are concerned with. Perhaps participants assume that Peter is done with his proofreading. Still I don’t see how that carries any implicature that Peter knows that there are no typos (is there any general implicature from ‘x is done’ to ‘x knows she did a perfect job’??) Moreover, the fact that we’re immediately asking participants whether or not Peter knows ought to cancel any such “implicature” by conveying that there is a question of whether Peter knows.

Secondly, I’m concerned about the methodology you and Shawn use. Essentially you add a character who comes in and confronts Peter with something like: “You only checked X times, even if you are right you don’t know that you are right. You should check again.” It seems to me that this ‘should’ claim is true in the high stakes case but false in the low stakes case (for reasons purely involving the modal ‘should’), in which case you’ve introduced a second point of difference between the cases, which then no longer form minimal pairs. Moreover, it seems to me that this speech very plausibly functions in the high stakes case to make the possibility of a mistake more salient (for whatever it is worth, in the high stakes version my mind immediately goes to ways Peter could potentially have miscounted). Are you doing anything to control for a salience interpretation of your data? Recall that our criticism of Chandra & Jason’s studies is exactly that they do not control for salience (and that doing so seems to eliminate their effect). I note that your effect in these cases seems quite small, not on the order of the large effect you were finding on your original studies, but very much like the small effect Chandra & Jason find. So, if I might offer a suggestion, it might be worthwhile to try out variants designed to control for salience, perhaps using some of the techniques Wesley and I used for keeping salience fixedly high. I'd be really interested to see if you could still detect any stakes effect with salience held fixed.

[Angel Pinillos]

Hi Jonathan,

I was really impressed by your (and Wesley’s) experimental design where you manage to distinguish saliency from stakes by mentioning both the dangerous and innocuous allergy possibilities in a single probe only one of which applies to the protagonist (from Sripada and Stanley). I think I will try to use it in some of my work. I do have a couple of concerns regarding your analysis of these data. First, if you think the effect that Sripada and Stanley found was due to saliency, then you should be able to get a statistical difference between low stakes (regular) and low stakes plus (salient). right? Instead, you just show that there is a null effect between high stakes plus and low stakes plus. I don’t think that is quite enough.
The second worry is a general one about design (it also applies to some of my work). So in a lot of your experiments using Likert, your mean results are close to the neutral point in the scale. One worry that I have is that this might just mean that people aren't really sure what to say: This could just be a hard question. It would be nice to get some clear cases of knowledge and clear cases of non-knowledge where people agree. This would give us a base comparison point. A related point is this: In the salience high stakes case most if not all philosophers agree the protagonist has no knowledge, and if people aren’t saying that, then maybe there is something seriously wrong with the experimental design.

Finally, just to clarify my comment about your experiment where Peter counts twice. I am suggesting that there is an implicature from the claim that Peter has counted twice, that Peter has stopped counting after counting twice, and so he is satisfied with the proofreading. Now I think that people are inferring from this that Peter knows there are no typos (since Peter is in the best position to know if he has gathered the appropriate amount of evidence--he was extra careful). But this inferred proposition is not implicated. It is just inferred using one’s theory of mind. This should get around your objection, I think.

[Mikkel Gerken]

Hi Jonathan and Wesley,

Lots of interesting stuff here! Just one point for now:

On p. 22ff you argue that the alleged stakes effect is, in effect, a salience effect. Your point is that the Sripada/Stanley study is prone to the confound that heightened anxiety produced by the high stakes case will “intensify the generation of alternative hypotheses." The upshot, then, is that the Sripada/Stanley vignette fails to hold salient alternatives fixed across the High/Low stakes cases.

You cite the Knobe/Schaffer finding of a salience effect as “independent evidence” for this assumption. This struck me as odd insofar as one might equally well argue that the Knobe/Schaffer vignette fails to hold stakes fixed across the base cases and the salient alternative case. The latter features, whereas the former does not, talk of “getting in trouble” and “imagining how frustrating” it would be to add on a false assumption. These are matters concerning practical interests – including the practical interests of the subjects. (For what it’s worth, I found, independently of this dialectic, the point sufficiently important to make it in print. See p. 4 of my forthcoming AJP piece and p. 141 in my chapter of the Brown/Gerken volume. I am not sure whether others have made the point in print but I discussed it with quite a few people before I did. So, I don’t think it is idiosyncratic.)

In any case, the dialectical worry is this: If the Knobe/Schaffer study itself confounds stakes and salient alternatives, it can hardly be cited as “independent evidence” for assuming that since the Sripada/Stanley study confounds stakes and salient alternatives, their “moderate effect” should be explained by salience alternatives, rather than stakes. So, this bit of the overall argument did not seem cogent to me.

I think that I have something more – and deeper – to say (and will write you iff I get there). But I just thought that this point was something that you should address head-on.

Best,
Mikkel