Photo credit: Timothy Epp, Shutterstock
Four years ago, Neil Levy gave the concluding lecture at the first Brain Matters conference in Halifax. He alerted the audience of neuroethicists to the fact that the field of philosophy was undergoing a revolution – rather than muse from their armchairs in the ivory tower, a group of renegade philosophers were carrying out real experiments, asking people what their intuitions were about central issues in philosophy. Dubbed experimental philosophy, the new initiative was met with more than passing resistance from traditional philosophers. The apostate experimental philosophers responded by developing a logo of a burning armchair.
The landmark experiment was carried out by Josh Knobe, and its findings subsequently became known as the Knobe effect (you can watch a great recreation of the phenomenon in this YouTube video). Essentially, what Josh did was repurpose an old method from social psychology called the contrastive vignette technique (CVT) [1]. At its simplest, the CVT involves designing a pair of vignettes that carefully describe a particular situation (in the case of experimental philosophy, one that is often morally charged) but crucially differ in one detail, hence the term contrastive. Respondents see one and only one version of the vignette, and are then asked questions about what they have just read, with responses commonly recorded as a numerical rating on a Likert scale. By comparing the averaged responses between separate groups of people who have read the vignettes, the experimenter can systematically investigate the effects of small changes (of which the respondents are entirely unaware) upon attitudes towards nearly any topic. The experimental philosophers tend to use the technique to explore the meaning of concepts. Neil Levy pointed out that this same approach could, in principle, be applied to the full range of issues in neuroethics.
Neil’s presentation struck me like a thunderbolt. I had come to the field of neuroethics with a background in cellular and molecular biology, and had spent much of my career as a card-carrying reductionist: as a graduate student in the 1980’s, I championing the then-novel technique of recording from single neurons in freely moving animals, and as a postdoc I moved on to the better controlled (if less naturalistic) technique of patch clamp analysis of identified neurons in slices of brains. My subsequent rise through the ranks of academia was one in which I applied quantitative rigor to every question that I asked, and in the circles in which I traveled, this was lauded as the ultimate way to provide reproducible (and by inference, meaningful) results. I saw at once that the CVT opened the door towards doing something similar in the field of neuroethics.
My research group at the National Core for Neuroethics has embraced the use of contrastive vignettes wholeheartedly, and with a nod to the experimental philosophy camp, we call the approach Experimental Neuroethics. The team is applying the technique to a range of issues in contemporary neuroethics, probably best exemplified by our recent publications exploring public attitudes towards cognitive enhancement [2] as well as the acceptability of overt and covert nudges [3].
If the vignettes appear simple, I can assure you that properly crafting them is hard work. We begin with a carefully considered hypothesis and regularly find that the hypothesis morphs substantially (usually into something much more insightful) as the process unfolds. We then compose two or more contrastive vignettes, working hard to have the vignettes as minimally contrastive as possible (one word differences between vignettes is the ultimate goal, but this is often not feasible). Finally, we develop questions; we like to have the wording of the questions always be identical irrespective of the contrastive nature of the vignette.
Then the real fun begins. After a day or two, we assemble as a team and attack our previous work. Inevitably, we find it wanting in some respect. Sometimes, embarrassingly so. We find it best to begin by asking whether the vignette and the questions directly address the hypothesis. Sometimes that means that the hypothesis changes. Nearly always, that means that the vignette changes. This process is repeated again and again, over days and weeks and sometimes months (yes, and even sometimes years!) until we have a set of vignettes that get to the heart of the matter.
At some point late in the process we carry out cognitive pre-testing. This involves sharing the vignette and the questions with someone who has no particular expert knowledge (friends of friends are likely culprits), and debriefing them about what they read. We are sometimes amazed to find that what we intended for people to glean from a vignette is at odds with their reading of the vignette. That sends us back to the drawing board.
We also run some metrics to determine whether the words we have used are understandable by a general audience. We use online readability tests such as this one to establish the educational level required for understanding the vignette; our goal is that no more than a high school education is required. Finally, we launch the survey, recruiting respondents from amongst the thousands of people who have signed up on Amazon’s Mechanical Turk – they’re more representative of the real population and aren’t as blatantly WEIRD as typical undergraduate samples. And then we hold our breath.
Once the data is analyzed, we get mired once again in deep discussion. For it is not just the quantitative aspect of Experimental Neuroethics that it satisfying (to me), but also that the data gives us an entirely new benchmark for engaging in the process of wide reflective equilibrium. Throughout this process we remain aware that an ought can not derive from is, but having the data at hand, our version of ought is very much informed by the is. Ultimately, our data emerge in concert with our normative insights, and then one more advantage of Experimental Neuroethics is realized: it is easy for others to replicate our experiments, or even to improve them by taking our vignettes and modifying them to further test their own. This iterative process of replication, critique, and systematic modification has proven to be a robust strategy for advancing insights into the nature of biological and physical phenomena. Only time will tell whether Experimental Neuroethics catches fire in our discipline as it has in the field of philosophy (where it remains controversial). If it does, we can trace it back to Neil’s presentation in Halifax….
[Cross posted at the Neuroethics Blog]
[1] Burstin K, Doughtie E, Raphaeli A. Contrastive Vignette Technique: An indirect Methodology Designed to Address Reactive Social Attitude Measurement1. Journal of Applied Social Psychology. 1980;10(2):147–65.
[2] Fitz NS, Nadler R, Manogaran P, Chong EWJ, Reiner PB. Public attitudes toward cognitive enhancement. Neuroethics. 2013 doi: 10.1007/s12152-013-9190-z.
[3] Felsen G, Castelo N, Reiner PB. Decisional enhancement and autonomy: public attitudes towards overt and covert nudges. Judgment and Decision Making. 2013;8(3):202–13.
Neuroethics at the Core 