Experiment Month

Tamar Gendler, Joshua Knobe, and I are putting together a project to help philosophers interested in applying experimental methods to philosophical topics. We are calling our project, “Experiment Month”. In brief, we will encourage philosophers to submit proposed experiments in the Fall of 2010. A team of qualified volunteers will select the most interesting and viable proposals for inclusion in the Experiment Month. They will provide helpful comments on selected submissions and assign “experiment buddies” to help researchers further refine their proposed experiments. In March of 2011 (the Experiment Month), revised versions of the selected experimental studies will be made available on the project website, where we will encourage large-scale experimental participation from the philosophical community and from the broader public.

 

As we move forward with this project, we would like to get your input on the attached proposal. The proposal was submitted to the American Philosophical Assoniation on June 30th, in time to be considered for a portion of the grant money the Eastern Division is making available to such projects. We will be making further revisions to this proposal and plan to submit it for an NEH Digital Humanities Start-Up Grant in April of 2010. Suggestions from the experimental philosophy community would be much appreciated as we move forward with this project.

 

 

Download Experiment_Month_Final

Comments

[Josh May]

This is a great idea! Good luck on the proposal. I sure hope you get the funding for it. This could be extremely helpful for many people. I'm especially glad to see students will be eligible to participate. That was one of the major downsides of the recent NEH Summer Institute on Experimental Philosophy. I wanted to go but couldn't because it was restricted to faculty.

[Mark Phelan]

Thanks Josh. And sorry about the NEH thing. I barely made the cut myself! For what it's worth, it was NEH policy, and that policy will be changing next year (according to the NEH rep who spoke the last day). She also said that the NEH expects to have lots more money to fund institutes, etc, given the new administration. So maybe you won't have to wait long to get another chance to attend an X-phi institute!

[Eddy Nahmias]

Mark (and Tamar and Josh), this is a fantastic initiative and I hope it gets funded. A few things I noticed:
1. it wasn't clear enough to me that you will be designing a website that can run the surveys for people (I'm assuming that is part of the plan--if not, should it be?). That way they don't have to create their own or learn to use an online source like questionpro. Alternatively, you could offer them assistance to learn how to use such a source and then link to it through your website.
2. Though it is great that you will be recruiting subjects for the researchers, you may want to say that they will also be taught how to recruit their own subjects, in parks perhaps, but mainly through their university (some psych depts need more credits for their students and are happy to let you use their system and I've always had success using big intro philosophy classes). I understand that the recruitment online will draw a more diverse set of subjects, but especially for initiating future studies, researchers may need to learn other ways to get subjects.
3. it is not clear whether the statistical analysts you hire will actually help the researchers learn the stats. If so, that should be highlighted. If not, you should try to explain how the researchers will gain this crucial experience, presumably from their "buddies." To me, the main attraction of your initiative is to help people learn how to do good experiments so they can keep doing them.
4. Indeed, I think you could emphasize more that funding for this project could set up the foundation for a long-term project (an experiment month each year), since you'll have the website and resources in place.
5. Finally, depending on who judges these proposals, it might help to offer a bit more info about what sort of experiments have been done in the past and what sorts you are expecting to happen under this grant (though you can emphasize that part of your goal is to get people thinking about new ways of doing exp phil).
Anyway, thanks for getting this going. I hope I'll have a student or two applying.

[Mark Phelan]

Hey Eddy,

Thanks for the close read, and the excellent suggestions! We'll take these into account as we move forward with the NEH proposal. To clarify matters a bit, I'll say something about each:

1. We will be creating a website to host the surveys, and we plan to hire an administrator to run it. Our plan is to eventually turn this resource over to the philosophy community at large. (That bears on the long-term aim of the project that you bring up at 4.)

2. As you no doubt noticed, we plan to have tutorials on the website. It seems like "subject recruitment" would be an appropriate topic for one of these tutorials. It would certainly help facilitate future research, as you point out.

3. Adina Roskies also brought up this issue just before I turned in the proposal, but I didn't have time to address it before the deadline. I agree that it's probably better to teach people to fish than to give the fish to them. We plan to have tutorials on statistical analysis, and also to help people perform their own statistical analyses through the buddy system. But perhaps we should arrange for statistical consultation, as opposed to statistical analysis.

4. Duly noted. That was our plan, but we should certainly emphasize it to a greater extent than we currently do.

5. I think it will be particularly important to do this for the NEH. The grant process there is much more indepth. (Our proposal to the APA was already pushing the upper limit of length.)

[Josh May]

I second Eddy's points.

Also, since you'll plan on turning this site over to the philosophical community (which I think is a great idea, BTW!), I have some first hand experience to share on that front. I'm working on a study right now and tried to get it up on the Philosophical Personality website that Justin Systma and others have set up. But my IRB for human subjects approval was *significantly* more demanding than theirs. As a result, my IRB wouldn't let me run my surveys on there.

It mostly had to do with not having enough privacy measures (like securing the transmissions with SSL) and issues with the consent form. On the latter, it posed a problem that we couldn't get the consent forms to match up with the questions subjects received randomly. That is, multiple researchers are offering up probes on the site at once, yet my IRB wanted my consent form (which differed from the others') and contact information to go with my vignettes. This could be tricky to do on your site for Experiment Month. It's certainly not impossible, though, especially if the site is initially designed with this issue in mind. So I just thought I'd throw it out there early on as something to think about when you get to the website phase.