Watching the Human Brain Process Information

Marcel Just is the director of the Center for Cognitive Brain Imaging at Carnegie Mellon University, where he and his team use functional Magnetic Resonance Imaging (fMRI) scans to examine brain activation as people perform various high-level tasks such as spatial thinking, problem solving, multitasking, and real-time, dynamic decision making. In a conversation with Nieman Reports editor Melissa Ludtke, Just describes what imaging has revealed—and what it one day might reveal—about how the human brain processes information. Just begins by talking about how the brain “codes” concepts. Edited excerpts follow:

Melissa Ludtke: When you talk about trying to unravel or study the code of the mind, what do you mean by “code”?

Marcel Just: Most people have seen the illustrations in magazines and newspapers of hot spots in the brain when a person is thinking. Those accurately depict where the activity is and tells which brain areas are in play. For the first time we’ve been able to identify what concept a person is thinking about from their brain activity. We use machine-learning algorithms to put together what is being coded in various places in the brain so that we can determine what concept the person is thinking about. In effect, we can read their mind.

The second finding is that the pattern of brain activity for one of these relatively concrete concepts is common across people. If we train the computer program to recognize the patterns for 60 words (concepts such as apple, shoe, tomato) on one group of people and then a new person comes in who has never been encountered before, the computer can tell what the new person is thinking fairly well.

It’s the first time we’ve been able to answer the question that philosophers have been asking: Is your concept of something the same as my concept of it? For these concrete nouns it is. And if you train the computer to recognize those concepts in one language, it can then identify what the bilingual person is thinking about when he or she encounters those concepts in another language. So the brain representations are not only common across people, but they’re common across languages for translation equivalents.

Ludtke: I’m wondering how you see the notion of multitasking in terms of brain function and how the brain copes with distraction.

Just: In many of our studies we are interested in finding out the upper limits of our thinking capability. We often put the brain on a mental treadmill and just see how high we can put it. With multitasking we ask people to do two relatively high-level tasks simultaneously. We’ve aimed at a real-world issue: using a cell phone while driving and we have measured the brain activity with people who are using a driving simulator while also listening to someone talk at the same time.

We find there’s an upper limit to how much information a person can process per unit time. If you’re working on two tasks simultaneously, you can do that, but there will be less activity allocated to each task. For example, when we compare how much brain resource allocation there is to just driving while listening to someone speak, we find that while listening to someone speak the amount of brain resources allocated to the driving goes down by 37 percent. It’s an enormous difference, and it shows up in the driving simulator in how well you maintain your lane, whether or not you hit the side of the road. It’s possible to drive while listening to someone talk. We all do it and sometimes using a cell phone. However, there’s absolutely no question that it takes away from the quality of the driving.

You ask about distractions. Particularly with respect to automaticity, which we study, that’s a very interesting question. There are drivers who say, “Oh, well, when the driving gets tough, I’ll start ignoring my cell phone partner or my passenger or my talk radio and I’ll just focus on the driving.” We ran a study in which we told experienced drivers that they are going to hear someone speak but to just ignore them and do the main spatial task. But we found that they couldn’t ignore the speaking because the processing of spoken language is so automatic that you can’t turn it off. You can’t will yourself not to understand a speaker’s next sentence. It just gets in.

In watching the brain’s activity, we can see this. When the next sentence starts, even though our subjects are poised to ignore it, the activation starts up in the language areas and the activation in the other areas of the brain—for the other task—goes down. There is no blocking out someone talking to you. It goes in and it consumes brain resources.

Ludtke: I’ve been reading about how people aren’t processing things as effectively as they think they are when they’re multitasking.

Just: There are dozens of cases where we overestimate our thinking ability. And it never goes in the other direction. It’s always “Oh, yes, of course I can do this.” However, in perceptual judgments and cognitive judgments people always—probably 98 percent of the time—overestimate their abilities. It’s not cheating, it’s not bragging. It’s an honest intuition that’s wrong.

Ludtke: Does the brain reach a point where it’s simply too stressed in ways that make it more difficult to take in and process information?

Just: In brain imaging we’re actually measuring electrochemical activity. Like all biological functions it needs an energy source. We measure the amount of brain activity while somebody’s doing something. You can’t generate more activity beyond a certain point. There’s an upper limit. You can sort of see it yourself when you’re trying to do a digit-span task. Somebody asks you to repeat back five digits. It’s not very hard. When they start going to eight or 12 digits, there’s no more room for improvement and you just give up at eight or 10, or whatever your digit span is.

RELATED ARTICLES
“Digital Demands: The Challenges of Constant Connectivity”
“Generational Divide: Digital Technology’s Paradoxical Message”
-Interviews with Sherry TurkleLudtke: In speaking with Sherry Turkle who teaches at Massachusetts Institute of Technology and studies identity and self as it relates to digital media, we found out that she’s not allowing the use of computers in her class. Students were trying to do e-mail and Twitter and Facebook and also listen and participate in class while occasionally looking down at their cell phone when it vibrated. She felt that all of this was leading to a diminution in terms of what she was able to teach and what they were able to take in.

Just: Yes, I think the research supports what Turkle is saying. When I was a student, the problem was people reading the newspaper at the back of the room. Competition for a student’s attention is an old problem, but now the media are so portable and interesting and flexible that it’s even more tempting.

Ludtke: We’ve been talking about measuring and locating brain activity, about upper limits of the brain’s capacity and distractions. All of this relates to the tasks confronting journalists as they compete for attention in a broader media universe.

Just: That’s right. Now each of us is like an organization in that we have to prioritize and decide what would be the most beneficial way to spend our thinking time in the next five or 30 minutes. Because of the competition and the availability of various sources, I know that I have to be much more thoughtful about what would be the best use of my time, and such choices come up many times an hour. So we need to be more strategic, constantly deciding whether to click on this next thing or go back to what we were reading. It’s wonderful to have so much information, but choosing and allocating our resources is a tough thing. The limiting resource now isn’t the information; it’s our time.

Ludtke: But it’s also our brain, isn’t it?

Just: Oh, yes, that’s what I meant: our brain time.

Ludtke: There is a notion that in this time of information overload—of everyone trying to grab people’s attention—that one thing that the brain is very pumped for is to take in emotionally charged information. Do you have any thoughts on this dynamic?

Just: Yes. One of the biggest contributions of brain imaging is to reveal how intensely social and emotional the human brain is. To me it was a very big surprise. Ask people to read some innocuous little narrative and the brain activity shows that they’re computing things like the character’s intention and motivation. I think there is a constant tendency to be processing social and emotional information. It’s there and it’s ubiquitous.

Ludtke: As journalism moved onto television, news began to be conveyed in visual ways and this often led to what is referred to as a “if it bleeds, it leads” style of reporting.

Just: Processing print isn’t something the human brain was built for. The printed word is a human artifact. It’s very convenient and it’s worked very well for us for 5,000 years, but it’s an invention of human beings. By contrast Mother Nature has built into our brain our ability to see the visual world and interpret it. Even the spoken language is much more a given biologically than reading written language.

Ludtke: Does this mean that as we move out of the era of print and paper and into the digital era with more visual media, it’s going to be a more natural environment for humans to take in information than when it was the printed word?

Just: Yes, and it can be informative in a visual way. Now you can circumvent written language to a large extent. A lot of printed words are there to describe things that occur spatially. In many cases a picture is worth a thousand words. Now we can generate these pictures and graphics and we can convey them to other people very easily. I think it’s inevitable that visual media are going to become more important in conveying ideas and not just about raging fires.

Ludtke: Ideas?

Just: Ideas of physics and biology and politics and so on. Now I think there’s a role for the printed word. I don’t think it’s going to go away.

Ludtke: With children gaining a facility with digital media that many in their parents’ generation don’t have, would you expect that years from now brain imaging is going to show the brain functioning in different ways because of this orientation?

Just: Yes, I think that’s very plausible. Nobody has done that yet. But let me give you an analogy done without imaging. In the 1970’s there was a psychologist who studied people who were illiterate in Portugal. He found a group of people who had never learned how to use written language. He compared them to a control group who could read. He found that they processed things differently just as a function of having learned to read. I think that’s a counterpart to your question.

Nobody’s done the brain research yet because it’s hard to get a control group to really study this. But I think that learning about the organization of information in a particular way is extremely likely to affect the organization of the brain processing. Some things are going to stay the same, such as the coding of concrete concepts that we talked about earlier. But the notion of hyperlinks, of temporary diversions to get more information is, I think, a slightly different way of thinking that’s likely to have an impact on the brain.

We’re not going to change the brain, of course, just as reading doesn’t really change the fundamentals of the brain. But you can repurpose some areas to do some things that nobody’s done before.

Ludtke: Do you suspect that we’re going to be able to push that upper limit that we talked about in terms of the multitasking?

Just: No. You can get more proficient with it, but we’re never going to change the biological limits. However, if you practice a task, you can do it better and faster.

RELATED ARTICLE
“Understanding the iGeneration—Before the Next Mini-Generation Arrives”
– Larry RosenLudtke: Larry Rosen, a psychologist who is the author of “Rewired: Understanding the iGeneration and the Way They Learn,” has done some studies at the University of California, Dominguez Hills, in which he’s found that mini-generational divides are determined, in part, by the amount of multitasking that young people say they are able to do; with each new mini-generation the number of simultaneous tasks increases.

Just: The possibility exists. Everybody sees young people with iPod earphones walking around. Does that decrease their capability to do other things? Some of my staff—very talented smart people—sometimes have an iPod bud in their ear while doing scientific work at a computer. I don’t know whether this degrades their work.

Ludtke: These are questions that people like you are going to be looking at?

Just: Yes. There are a number of studies starting. It’s hard to get a really well-controlled study of this stuff, but it’s very plausible we’ll find changes in people’s brains. Sixty years ago nobody knew how to do computer programming and now there are hundreds of thousands of people who can program. They had to have learned to think in a slightly different way.

People ask what’s special about the human brain. It’s that we can invent new things and learn to think about them, like computer programs and digital media devices and printed language. That’s part of the genius of the human brain: new injections and this intellectual bootstrapping. We’re obviously on this accelerated path upward. I can’t guess exactly where it’s going to take us, but almost certainly it’s going to take us upward into a different place.

Ludtke: That’s what we’re exploring. I appreciate you bringing us along on your journey.