Imagine a scene in a movie in which two people are having a conversation. First you see one person talking, and then the other. You see a close-up of some detail, and then a far-away view of the whole room. These rapid shifts in perspective don't happen in real life, yet our eyes and brains seem to have no problem keeping up. How can this be true? Jeff Zacks, author of Flicker: Your Brain on Movies, again joins Hold That Thought to discuss how our brains react to film.
Transcript:
Claire Navarro: Hello listeners! Thanks for tuning in to Hold That Thought. I’m Claire Navarro. In this week’s podcast, we return to our conversation with Jeff Zacks, author of Flicker: Your Brain on Movies. Zacks is associate chair of the department of psychological and brain sciences here at Washington University in St. Louis. Previously on Hold That Thought, he explained some of the reasons why movies are so good at making us laugh and cry. But when it comes to how our brains react to film, there’s a lot more to talk about. In his book, Zacks also explores how we see movies – how our eyes and brains work together to understand all the movie magic flashing before us at 24 frames per second. Here’s Dr. Zacks to explain.
Jeff Zacks: One thing that totally freaked me out when I first started thinking about this is that eyes have been around on this planet for about four hundred million years. They’ve evolved multiple times. We have pretty sophisticated eyes that can register all these fancy changes in motion and lightness and darkness and color, but never in the history of that evolutionary trajectory was it the case that what was sitting in front of those eyes was ripped away and instantaneously replaced with something different.
CN: Yet whenever there’s a cut during a movie – when you suddenly see a different angle or perspective – that’s exactly what happens. Think about something like a car chase scene, and just try to count all of the cuts. In a matter of seconds you see a close up of an actor, then another actor, then tires screeching, then a shot out the front windshield, a shot out the back windshield, an overhead look at cars speeding down the road, and so on and so on.
JZ: I just didn’t understand why our heads didn’t explode when that happened, and worse yet, we’re not even that sensitive to cuts.
CN: This is especially true during what film people call continuity edits, when multiple cameras film the same scene. Think of a conversation between two characters in a movie. First you see the face of one person speaking, then the other, then back again.
JZ: And you can do that every second or so. People don’t experience that as jarring or fast or anything. How could that be? The answer, as far as I can tell, turns out to be that although our brains and our eyes didn’t evolve for dealing with cuts, they did evolve for dealing with a bunch of other visual discontinuities that we experience in real life.
CN: These everyday visual discontinuities start with our eyes. It’s tempting to think about eyeballs like their own little movie cameras, taking in one shot of a scene, getting the whole picture from one angle. In reality, however, our eyes are kind of making their own little continuity edits all the time. We move our eyes – often without realizing it.
JZ: Most of the eye movements that we make are ballistic movements called saccades, where you fixate on one thing and then you shift your eyes rapidly.
CN: Really rapidly. Like, within a hundred milliseconds. It’s one of the fastest things our bodies can do.
JZ: During that saccade, you are effectively blind—the information that’s coming in through your eyes is garbage, and your brain has this nifty mechanism for shutting down those inputs. Those saccades happen three times a second or so.
CN: Three times, every second. Then, on top of that, every few seconds you blink. With every blink, once again you’re momentarily blind, this time for 2-300 milliseconds. And all this time adds up.
JZ: A colleague of mine did a calculation and came up with a scary statistic that between blinks and saccades, we are functionally blind a third of our waking lives.
CN: Luckily, human brains are really good at working around these kinds of visual disruptions. This is especially true when, like with continuity edits in a single movie scene, all of the information seems to hang together as part of a single story or event.
JZ: What we are used to doing in the real world is taking these disjoint samples of the world that we get from these different glances and gluing them together into a common event model that helps us keep track of whose there and what they are doing and what objects they’re acting on. As long as we are within an event, we ought to be engaged in that stitching together and not too bothered about the fact we are stitching together from discontinuous slices.
CN: The event models that Zacks just mentioned are a really important part of how our eyes and brains deal with movies, and also an important part of his overall research, even not dealing with movies. Basically, an event model is the representation in your head of what’s happening in a story, whether that story is real or fictional. We build these models all the time.
JZ: Intuitive folk psychology tells us that we have perception and imagination or imagery and memory and planning and that these are all different things. The view that I’ve increasingly come to is that those are all different input processes to constructing a common event model. I can construct an event model from what’s in front of e coming into my eyeballs; we’d call that perception. I can construct an event model by someone giving me a verbal description, saying, “Imagine this.” We call that imagery. I can construct an event model by retrieving patterns that were stored in my brain on a previous occasion, and then we’d call that memory retrieval. But the result of all those mechanisms we think is a common representational format.
CN: So, event models help us make sense of what’s happening in any given situation. They help us think about what might be about to happen. They’re how we remember what’s happened to us in the past. And they also help us piece together visual information – in real life, and in movies.
JZ: We actually did some research in the lab. My colleague Joe Magliano and I looked at fMRI data that we had from people watching commercial films, and we divided the cuts in the film up into continuity edits (where they ought to be stitching across the boundaries) and scene changes (where they ought to be rebooting their event models and forming a new one). What we found is there are higher-level parts of our visual system that are doing special work at those continuity edits. It’s all under the hood; we’re not aware of it. We think that computation is what makes it nice and fluid and smooth for us even though the actual low-level visual stimulus is horribly discontinuous.
CN: Here’s where things can get fun for filmmakers. Knowing that viewers are good at smoothly stitching together cuts, especially from within a single event, moviemakers can start playing with those expectations to do the unexpected.
JZ: One of the really nifty things that artists, including filmmakers, can do is exploit facts about the visual system to create that work because they violate the ways things usually go in nature. A simple example that you used to see all the time in the 80s music videos when I was growing up is the jump cut. A jump cut is when you change the camera angle just enough so that you break the sense of smooth motion that you get in the normal run of the camera. But you don’t break it so much that the visual stem can’t find a correspondence between objects pre-cut and post-cut. One easy way to do that is to have a sequence where something is moving and you delete three or four or five frames from that sequence. If you have the band walking towards you and then you keep deleting a quarter second or so, you get this effect where they are jumping forward and forward and forward. That was a staple of heavy metal hairband videos from the 80s.
CN: Whether viewing rapid jump cuts in an 80s music video or smooth transitions in an engrossing Oscar-nominated film, the work that our eyes and brains do to process movies – and really, to process everything we see – is way more complex and amazing than most of us ever really think about. In many ways, the most magical thing about movie magic happens inside our brains.
JZ: The intuitive theory that most of us have about how our visual system works is that the eye is kind of like a camera and it projects to some screen in our heads and that representation on that screen is the thing that underlies our vision. Really, what it is more like is our eye is like a suite of test instruments and then there is a bunch of specialists who are looking at the results of those tests and pulling out information that they are specialized to deal with. Just like if you are sick, you might get an opinion from a radiologist and a cardiologist and a neuroendocrinologist. Visual information is getting processed by one part of the brain that specializes in figuring out what’s moving and where, and another part of the brain that’s specialized for figuring out what colors things are. Another part of the brain is specialized for figuring out how you are moving. Anther part of the part of the brain is specialized for recognizing the objects that are present in the scene. Those are all coordinating with each other, and to some extent there is a kind of primary care physician who is playing a central coordinative role, but its not like there is one place in the brain where this all comes together and boom, a miracle occurs and that’s what produces visual experience.
CN: So next time you sit down to watch a movie, say a quick thank-you to the suite of instruments in your eyes, and the brain that makes sense of it all. Many thanks once again to Jeff Zacks for joining Hold that Thought. You can learn much m ore in his book Flicker: Your Brain on Movies. For many more ideas to explore, please visit us online at holdthatthought.wustl.edu. Or join the conversation on Facebook or Twitter. Thanks for listening.
Credits:
Free Music Archive: Chris Zabriskie, Crossworlds, Jahzzar, Podington Bear, Reed Blue, Transient
