Skip to main content
Move the World.
experiment design

Lead Image © Andrew Brumagen

Listen up, whitecoats!

McGill University researchers Jay A. Olson and Amir Raz have a suggestion for your next experiment design: improve your results by improving your deception.

The former professional magicians, writing in the Journal of Experimental Social Psychology, know that some studies necessitate a little snowing of their subjects to get the best data possible. 

"Social psychologists, placebo scientists, and consumer researchers often require deception in their studies, yet they receive little training on how to deceive effectively," they write. 

For example, a social psychologist may want to study how someone reacts to criticism. They give them a quiz, and tell them they scored poorly. The subject's test performance, of course, never mattered; they're going to be told they suck no matter what.

"Ineffective deception, however, can lead to suspicion and compromise the validity of research. The field of magic offers a potential solution; magicians have deceived audiences for millennia using a variety of robust techniques."

Not only have magicians been availing themselves to the highly enjoyable art of deception since, well, "magic" has existed, but they do so in some of the most hostile climes imaginable: in front of suspicious, aware, heckling, maybe drunk, definitely hoping to catch them out, audiences. 

Compared to these on-their-guard marks, the authors theorize, how hard can it be to put one over on a research subject in a staid scientific study? 

A Few Subtle Cues > One Big Lie

Olson and Raz pull from their hat a proposal called the Swiss cheese model. You may have heard of this in the context of overlapping or redundant safety measures — say in a commercial jet or rocket, or in the context of COVID-19

The idea is simple: any given slice of Swiss cheese is going to have big holes in it, right? Holes you can see right through. But stack layers of Swiss cheese, and the holes are very unlikely to line up.

So, while a mask, social distancing, and being outside as much as possible all aren't guaranteed to keep you COVID-free, when you add them all up you're pretty well covered — and more so than if you just did any one of those things on their own.

The same principle applies to a good magic trick. Olson and Raz say that most experiment design deceptions are going about it all wrong. They use a few "thick" layers of deception — a cover story, or a plant in cahoots with the researchers — rather than a thin, beautiful tapestry of deceit.

Most experiment design deceptions go about it all wrong, using a few "thick" layers of deception rather than a thin, beautiful tapestry of deceit.

Compared to science, "magic generally uses many thin layers, providing subtle yet elaborate and effective deception," they write.

Which isn't to say the scientists never get it right. Consider the tricks used in one placebo study of alcohol.

To find out how much alcohol itself alters people's behaviors, and how much is just people's expectations about alcohol, researchers needed a way to trick people into thinking they were drinking, without actually giving them any alcohol. 

Rather than just telling people in the study that their cocktails were alcoholic, the researchers applied the Swiss cheese model: "confederates made off-hand comments about friends who got drunk while previously completing the study, the researchers sprayed the room with an alcohol scent, and the (non-alcoholic) drinks had real alcohol rubbed along the rim for subtle taste cues."

I mean, the attention to deceptive detail ... that's good experiment design, right there.

Fake Mistakes, Real Results

A commonly used magic trick that is not often considered in experiment design, Olson and Raz write, is the "fake mistake."

Raz offers up an example from his own career.

He pulls an audience member onstage and, shockingly, reads her mind. The trick here is dead simple: the audience member is a plant. Any skeptic can see through that. 

But in the midst of the mindreading, another audience member stands up and shouts "I was here last week and he chose the same woman. She's a stooge!"

No one suspects that such a calamitous occurrence as someone blowing up your whole act would ever be done on purpose. 

Raz, nonplussed, calls up this interloper — and proceeds to read his mind, as well. The audience is effectively deceived. The secret? The loudmouth is in on it, as well. (In a final punchline, he explains to the first confederate that he always could read minds, but it was just tough on him, which is why he hired her. So he deceived someone in on the act! Fabulous.)

"Magicians often use such elaborate forms of deception," they write. "Audiences may suspect stooges in a magic show, but they are less likely to suspect one stooge to cover up another."

The trick works because no one suspects that such a calamitous occurrence as someone blowing up your whole act would ever be done on purpose. 

A Scientists Always Reveals Their Secrets

Those subjects who you've so elaborately deceived, unlike a magician's audience, do, of course, deserve to have the wool over their eyes shorn away. (It goes beyond being the right thing to do: Internal Review Boards tend to only approve deception when said deception is revealed to volunteers in the end, Olson and Raz note.)

"We recommend extensive in-person debriefing to explain and justify the various layers of deception, giving participants a 'behind-the-scenes' view of the study," they write. 

"It may be particularly important to stress that being deceived says little about the participant; the procedure could be compared to a magic performance that fools almost everyone."

In other words, the subjects need to leave knowing they were deceived, why that deception was crucial to the experiment's design, and not feel like a pigeon for it. 

"...magicians seeking deception may be helpful to scientists seeking truth."

Olson and Raz

Olson and Raz argue that serving up the Swiss cheese model can provide better data by ensuring whatever deception you need actually works — be it a convincing placebo, an unsuspecting social relationship, or a hidden motive masked by one you need to feel real.

"Applying the principles of the Swiss cheese model with subtle yet elaborate layers of deception can improve the quality of experimental protocols and the validity of their resulting conclusions," the former pros write. 

"In an unlikely collaboration, magicians seeking deception may be helpful to scientists seeking truth."

P.S.: That paper contains a wonderful table full of deception methods and tips on deploying them, for any up-and-coming magicians — or whoever else needs such fun but dark arts — in the Freethink audience.

Up Next

DIY Science
Hackers Build a DIY Satellite Tracker to Eavesdrop on Space
satellite tracker
DIY Science
Hackers Build a DIY Satellite Tracker to Eavesdrop on Space
The NyanSat guided challenge asks hackers to build a satellite tracker, making low-earth-orbit satellites more accessible.

The NyanSat guided challenge asks hackers to build a satellite tracker, making low-earth-orbit satellites more accessible.

Future of Medicine
Every Significant Breakthrough That’s Stemmed from the Pandemic
timeline of the coronavirus
Future of Medicine
Every Significant Breakthrough That’s Stemmed from the Pandemic
A timeline of coronavirus solutions from the people on the frontlines of the fight against COVID-19.

A timeline of coronavirus solutions from the people on the frontlines of the fight against COVID-19.

AI & Privacy
Hey Alexa, Quit Eavesdropping!
audio jammer
AI & Privacy
Hey Alexa, Quit Eavesdropping!
Researchers at the University of Chicago have developed an ultrasonic audio jammer that doubles as a fashion accessory.

Researchers at the University of Chicago have developed an ultrasonic audio jammer that doubles as a fashion accessory.

Drones of the Future
This Nail Gun Wielding Drone Looks Terrifying, But Could Save Lives
This Nail Gun Wielding Drone Looks Terrifying, But Could Save Lives
Drones of the Future
This Nail Gun Wielding Drone Looks Terrifying, But Could Save Lives
According to OSHA, one in five workplace deaths in 2017 were in construction. Can nail gun drones help save lives?

According to OSHA, one in five workplace deaths in 2017 were in construction. Can nail gun drones help save lives?

Superhuman
Reprogramming Your Immune System to Fight Cancer
Reprogramming Your Immune System to Fight Cancer
Watch Now
Superhuman
Reprogramming Your Immune System to Fight Cancer
Your T cells already know how to kill cancer. These doctors can train them to hunt it down.
Watch Now

Josh Feldman was on his honeymoon when he felt a lump on his neck. Returning home after the best month of his life, his doctor gave him the news: non-Hodgkin's lymphoma. There was no cure, and it was about to get much worse. After multiple rounds of chemotherapy failed to stop his tumors from growing, Josh went to see Dr. John Timmerman, an oncologist at UCLA who is trying something different, known as immunotherapy. This...

Superhuman
These Gloves Can Teach You to Play the Piano. And Maybe Heal Your Brain.
These Gloves Can Teach You to Play the Piano. And Maybe Heal Your Brain.
Watch Now
Superhuman
These Gloves Can Teach You to Play the Piano. And Maybe Heal Your Brain.
Through "passive haptic learning", these gloves can teach you how to play the piano in an hour. Braille in four hours. Now researchers want to see if victims of traumatic brain injuries can use these gloves to re-learn critical skills.
Watch Now

Georgia Tech researchers Thad Starner and Caitlyn Seim have developed a pair of gloves for playing piano that can magically get you up to speed in just an hour. They've also taught blind people to read braille in four hours, a process that usually takes up to four months. The gloves work through a process called passive haptic learning, and is another great discovery from Georgia Tech researchers. Basically, they vibrate in...

The New Space Race
Here's What Happens to the Human Body in Outer Space
Here's What Happens to the Human Body in Outer Space
The New Space Race
Here's What Happens to the Human Body in Outer Space
As the idea of colonizing space becomes mainstream, it’s important to keep in mind that traveling in outer space...
By Mike Riggs

As the idea of colonizing space becomes mainstream, it’s important to keep in mind that traveling in outer space does some crazy stuff to our bodies.

This week in ideas: How VR changes our dreams, a stem cell miracle, and the shoes...
This week in ideas: How VR changes our dreams, a stem cell miracle, and the shoes of the future
This week in ideas: How VR changes our dreams, a stem cell miracle, and the shoes...
Virtual reality users experience more lucid dreams, a paralyzed man gets movement back, and self-lacing shoes....
By Mike Riggs

Virtual reality users experience more lucid dreams, a paralyzed man gets movement back, and self-lacing shoes. These are our favorite stories this week.