PAPER: An Argument For Proper Multiple Comparisons Correction
It has been a long road, but our multiple comparisons paper including the salmon has been published. See below for more details, including the abstract and a link to the download page of the journal. If you have any questions or comments please post them below or send me an email directly.
Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Proper Multiple Comparisons Correction
Craig M. Bennett(1), Abigail A. Baird(2), Michael B. Miller(1) and George L. Wolford(3)
1)Department of Psychology, University of California at Santa Barbara, Santa Barbara, CA 93106
2)Department of Psychology, Blodgett Hall, Vassar College, Poughkeepsie, NY 12604
3)Department of Psychological and Brain Sciences, Moore Hall, Dartmouth College, Hanover, NH 03755
Journal of Serendipitous and Unexpected Results, 2010. 1(1):1-5
Early Access: Oct 20, 2010
With the extreme dimensionality of functional neuroimaging data comes extreme risk for false positives. Across the 130,000 voxels in a typical fMRI volume the probability of at least one false positive is almost certain. Proper correction for multiple comparisons should be completed during the analysis of these datasets, but is often ignored by investigators. To highlight the danger of this practice we completed an fMRI scanning session with a post-mortem Atlantic Salmon as the subject. The salmon was shown the same social perspective-taking task that was later administered to a group of human subjects. Statistics that were uncorrected for multiple comparisons showed active voxel clusters in the salmon’s brain cavity and spinal column. Statistics controlling for the family-wise error rate (FWER) and false discovery rate (FDR) both indicated that no active voxels were present, even at relaxed statistical thresholds. We argue that relying on standard statistical thresholds (p < 0.001) and low minimum cluster sizes (k > 8) is an ineffective control for multiple comparisons. We further argue that the vast majority of fMRI studies should be utilizing proper multiple comparisons correction as standard practice when thresholding their data.
See the JSUR early access page to download the article and supplementary material.
http://www.jsur.org/v1n1p1
Riverside Presentation Slides
Just wanted to take a second to thank the kind folks in the Psychology Department at UC Riverside for hosting me this afternoon. I gave a neuroimaging stats talk for their cognitive brown bag series, and it was a really great time!
For anyone who is interested a copy of the slides from my presentation can be downloaded at the link below. If you have any questions or comments feel free to email me – I would love to chat more. Take care UCR!
http://prefrontal.org/files/presentations/Bennett-Riverside-2010.pdf
Spring 2010 Conference Posters
I have been remiss in uploading copies of my spring conference posters. October seems like a fine month to rectify that. Below are links to the research I presented at the Cognitive Neuroscience Society meeting in Montreal and at the Organization for Human Brain Mapping meeting in Barcelona. Both meetings were fantastic – I got to meet a lot of new people and experience all the awesomeness that Montreal and Barcelona have to offer.
* How reliable are the results from fMRI?
Conference Poster: [PDF] [JPEG]
* A device for the parametric application of thermal and
tactile stimulation during fMRI
Conference Poster: [PDF] [JPEG]
“MATLAB code practices that make me cry”
Great post from Doug Hull at MathWorks on the Top 10 MATLAB programming practices that only lead to consternation and the regret.
It is a fun read:
http://blogs.mathworks.com/videos/2010/03/08/
top-10-matlab-code-practices-that-make-me-cry/
Personally, I have often been guilty of #9 and #3…
APS Conference – Presentation Slides
I have wanted to attend the Association for Psychological Science annual convention for a number of years, but I was always frustrated by the number of other conferences I had to attend during the spring. All that changed early this year when I was offered the opportunity to give a presentation on interoceptive development. I suddenly had a very good reason to free up some time and hop on a plane!
I want to thank everyone who attended my address this morning. After untold amounts of airline trouble getting to Boston it was a real pleasure to have the chance to talk about the insula and interoceptive development.
If you are interested you can download a copy of my presentation slides here. Send me an email if you have any questions or comments. Thanks!
PAPER: How reliable are the results from functional magnetic resonance imaging?
- Current Citation:
Bennett CM, Miller MB. (in press). How reliable are the results from functional magnetic resonance imaging? Annals of the New York Academy of Sciences.
- Abstract:
Functional magnetic resonance imaging is one of the most important methods for in vivo investigation of cognitive processes in the human brain. Within the last two decades an explosion of research has emerged using fMRI, revealing the underpinnings of everything from motor and sensory processes to the foundations of social cognition. While these results have revealed the potential of neuroimaging, important questions regarding the reliability of these results remain unanswered. In this chapter we take a close look at what is currently known about the reliability of fMRI findings. First, we examine the many factors that influence the quality of acquired fMRI data. We also conduct a review of the existing literature to determine if some measure of agreement has emerged regarding the reliability of fMRI. Finally, we provide commentary on ways to improve fMRI reliability and what questions remain unanswered. Reliability is the foundation on which scientific investigation is based. How reliable are the results from fMRI?
- Downloadable Versions:
[Manuscript PDF]
[Link to Journal PDF]
LOVE Conference Wrapup
The Lake Ontario Visionary Establishment (LOVE) conference just wrapped up and, I have to say, it was a genuinely fantastic experience. I gave a lighthearted presentation on Type I error and reliability in functional imaging, which hopefully made the message a bit easier to swallow. I also got the chance to catch up with longtime friends while making some new aquaintances. Thanks to the organizers Daniel Ansari and Jonathan Fugelsang for having me up to present.
For all those who are interested: you can download a copy of my presentation slides here.
Send me an email if you have any questions or comments. Thanks!
Spring/Summer 2010 Conference Schedule
It is going to be a busy conference season again this spring. I will be at the following professional gatherings over the next few months – send me an email if you will be attending as well and would like to meet up. I’ll buy the first round and we can talk shop.
Lake Ontario Visionary Establishment Conference [LOVE]
February 11-12, Niagra Falls, NY Ontario, Canada
Cognitive Neuroscience Society Conference [CNS]
April 17-20, Montreal, Canada
Association for Psychological Science Convention [APS]
May 27-30, Boston, MA
Organization for Human Brain Mapping Conference [HBM]
June 6-10, Barcelona, Spain
Applied Human Factors and Ergonomics Conference [AHFE]
July 17-20, Miami, FL
Here is some of what I will be presenting:
[LOVE]
• Special topic talk: ‘Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: addressing the multiple comparisons problem in fMRI.’
[APS]
• Invited talk: ‘The development of interoceptive information processing across adolescence.’
[CNS] [HBM]
• Poster: ‘How reliable are the results from fMRI?’
Bennett CM, Guerin SA, Donovan CL, Miller MB
[HBM]
• Poster: ‘A device for simultaneous thermal and tactile stimulation in an MR environment.’
Bennett CM
Quote of the Week – Pashler
“It’s hellishly complicated, this data analysis, and that creates great opportunity for inadvertent mischief.” – Hal Pashler (As seen in Science News)
PAPER: The Principled Control of False Positives in Neuroimaging
- Current Citation:
Bennett CM, Wolford GL, Miller MB. (in press). The Principled Control of False Positives in Neuroimaging. Social Cognitive and Affective Neuroscience.
- Abstract:
An incredible amount of data is generated in the course of a functional neuroimaging experiment. The quantity of data gives us improved temporal and spatial resolution with which to evaluate our results. It also creates a staggering multiple testing problem. A number of methods have been created that address the multiple testing problem in neuroimaging in a principled fashion. These methods place limits on either the familywise error rate (FWER) or the false discovery rate (FDR) of the results. These principled approaches are well established in the literature and are known to properly limit the amount of false positives across the whole brain. However, a minority of papers are still published every month using methods that are improperly corrected for the number of tests conducted. These latter methods place limits on the voxelwise probability of a false positive and yield no information on the global rate of false positives in the results. In this commentary we argue in favor of a principled approach to the multiple testing problem – one that places appropriate limits on the rate of false positives across the whole brain and gives the reader the information they need to properly evaluate the results.
- Downloadable Versions:
[Manuscript PDF]
[Link to Journal PDF]