Science Journalism
Those who report on science have never been better prepared to do so, according to Los Angeles Times science and technology writer Robert Lee Hotz, whose insights open our section on science journalism. But as Hotz also observes, the challenges these reporters confront have never been larger: Newsroom cutbacks mean the reporters “are stretched to cover increasingly complex science stories ….” And their task is made harder by the dearth of impartial sources, forcing them “to look as hard at the scientists as we look at the science itself.” – Melissa Ludtke, Editor
On January 22, 1997, USA Today ran its customary Snapshot in the lower-left corner of Page One. Given intense feelings about the killing of animals to make fur coats, the paper’s lead graphic of the day qualified as news. According to Responsive Management, cited as the source, a whopping 86 percent of adults either strongly or moderately agreed that “people should be free to choose to wear fur.” To get that many Americans to agree about anything is to tap into sentiments akin to those for the flag.
Perhaps to convey that spirit, USA Today’s graphic artists created a nifty, attention-grabbing pie chart superimposed on Davy Crockett’s all-American coonskin cap. The tag line was “I’m OK, your fur’s OK.”
What USA Today’s Snapshot did not report was that the Fur Information Council of America (FICA) paid Responsive Management to do this poll. Had they known that, busy readers might have stopped briefly to question the data. Was the survey conducted in a way that made its results completely unbiased? Or were the numbers accurate but ambiguous? Were Americans saying it was okay to wear fur? Or were they responding that freedom to choose was an inalienable right? If the latter, wasn’t it more interesting that 11 percent would limit fellow Americans’ free economic choice in order to protect furry creatures? That, too, would be news, but would lend itself to quite a different backdrop, a pie chart superimposed on a splayed Bambi.
Not so long ago, newspapers had just about as much interest in dressing up as the Amish. But fear of losing readers to TV and more recently to the Internet, and the advent of new graphics technology, has changed that. Editors are investing in presses that print brilliant colors, in photographic equipment that gets the most out of images, and bigger layout and design staffs.
USA Today has been a leader in this graphics revolution. Along with its widely emulated weather map, Snapshots are a signature item. On the five days it publishes each week, the newspaper runs Snapshots on Page One of each of its four sections. “Snapshots are part of our strategy to establish a consistent identity for the paper, one that sets USA Today apart from other newspapers,” Richard Curtis, managing editor for graphics and photography, wrote several years ago. “They are a clear (some would say persistent) signal that USA Today is a visual newspaper.” Proud of its work, the newspaper uses Snapshots from the previous year in a calendar it gives to advertisers and others.
Several years ago, curious how graphics measure up as sound journalism, we looked in detail at the Snapshots appearing on USA Today’s front page during January 1997, a month chosen at random. Intrigued by the journalistic failings we found, we later looked at two other randomly selected months: April 2001 and January 2002. In our entire investigation—in which we checked the accuracy, clarity and sourcing of each Snapshot published—about one-third of them fell short of established journalism standards.
We started our analysis by assessing the fundamental verity of journalism: getting facts right. Each of the months we scrutinized had 22 Snapshots. Three were inaccurate in January 1997, five in April 2001, and two in January 2002. A few errors were small: A January 1997 graph said losses to private insurers due to highway crashes were $82.76 billion a year; in fact, they were $82.215 billion. An April 2001 Snapshot reported that Williston, North Dakota had a mean temperature of 40.1 degrees. The source for the graphic put the mean temperature at 40.8 degrees.
Other mistakes were more substantial. What follows are some examples:
Accuracy was only one problem we looked at in our analysis. Even though Snapshots, which are essentially standalone news stories, offer an efficient, interesting way to give readers information, it is, unfortunately, difficult to provide balance and context in one-source news stories. Moreover, reliance on a single, compelling graphic image—Davy Crockett’s coonskin cap is a good example—can result in misleading oversimplification. Typical of these problems is a 1997 graphic about attitudes toward travel. It shows an airplane swerving out of control and offers this tag line: “Just plane scared.” The rationale: 22 percent of adults are afraid to fly. With 78 percent not afraid to fly, though, the tag line might as well as have been “Just plane safe.”
An April 2001 Snapshot, which oversimplifies a complex calculation, reported the number of firearm deaths during a 20-year period. After examining their files, USA Today staff believes that the figure of 670,000 is a “conservative” estimate provided by Handgun Control, a nonprofit organization whose name is indicative of its agenda. Handgun Control based its 20-year estimate on 19 years of hard data from the Centers for Disease Control (CDC). CDC and Handgun Control are each cited in the Snapshot, but the reader is given no idea how the estimate was arrived at. In fact, the graphic does not acknowledge that it is an estimate.
The desire to not clutter up the graphics with lots of text also results in Snapshots like this one from April 2001: A survey asked respondents how much they would like to hear people like Bill Clinton and Colin Powell speak. The categories were “very interested,” “somewhat interested,” “not too interested,” “not at all interested,” and “no opinion.” The Snapshot combined “very interested” and “somewhat interested” in a new, nebulous category called “most interested.”
Yet other examples of the perils of simplification are two Snapshots reporting survey results in January 2002. Neither indicates if respondents chose from a list of alternatives in expressing their opinions or if they answered open-ended questions.
Editors at USA Today are candid about the entertainment value of Snapshots. These graphics are supposed to add a spot of fun to the news. They draw people into the paper. The newspaper’s surveys show that graphics have higher readership than news stories, although editors don’t expect readers to spend a lot of time on them. “It is a snapshot,” explained Fred Meier, a USA Today editor. “If you have to sit and stew over it, it probably hasn’t been done well.”
Perhaps that is true. But Snapshots should hold up if someone does bother to think about them. Too often, however, under a little scrutiny they lose credibility. What should someone make of the Snapshot that appeared in January 2002 with the headline “Most Native Hawaiians, Pacific Islanders live in the West?” And what about the finding that 41 percent of the public say that “making the world a better place” is “always on their mind?” Can anything always be on anybody’s mind? Does anyone never think about making the world a better place?
USA Today has an elaborate review process for Snapshots. By Richard Curtis’s count, staff members “touch” each one at least seven times and very often much more than that because of the frequency with which senior editors check work. Ideas for Snapshots originate with news researchers, graphics editors, and reporters. Many ideas, Curtis said, do not make it past the news editor for graphics or Bob Reynolds, who is the graphics director for the graphics and photography department. The staff is expected to check and double-check facts. They also are supposed to contact unfamiliar sources to ensure that the data are legitimate.
Checking out sources is a crucial part of reporting, particularly when only one source is used in a story. Accordingly, we tried to contact each source used during the three months of Snapshots that we examined. Although employees at these “source” organizations frequently said that no one from USA Today had been in contact, this is not particularly meaningful. These organizations have many employees as well as personnel turnover. What we can assert is that USA Today staff sometimes does not probe sources deeply.
The Fur Information Council of America study is the most dramatic example, but it is not the only one of its kind. In determining the accuracy of a January 2002 Snapshot on the number of police officers shot in the line of duty in the previous year, we found that USA Today staff initially worked off a press release from the National Law Enforcement Officers Memorial Fund. The fund’s PR person subsequently e-mailed at least two updates with new numbers. The fluidity of numbers might have suggested to the staff that they get the report and look at it—or wait until the numbers were no longer preliminary (a caveat not noted in the Snapshot). But they never looked at the original report.
Failure to get original reports also led to mistakes in an April 2001 Snapshot about the percentage of women applying to law school. USA Today accurately reported the data published in JD Jungle magazine. If USA Today staff had checked with JD Jungle, however, they would have learned that the data came from the Law School Admissions Council (LSAC). And if they had checked with the LSAC, they would have found that JD Jungle had misreported the findings.
To be fair, we detected improvement in the Snapshots during our final look in January 2002. Furthermore, we learned from Curtis that the April 2001 time frame that we chose at random to investigate happened to coincide with the illness of a key staffer. We were impressed by the attitude of the editors with whom we spoke. “Our goal is zero errors,” Reynolds said. “All errors are indefensible.” After he had initially heard of our findings, Curtis arranged a lunch so that one of us could talk with staff, including one member who works with the newspaper’s accuracy task force. Staff members said that they do get many calls asking for guidance on how to reach Snapshot sources, but very few about inaccuracies. Further improvements might come as a result of editor Karen Jurgensen’s error reduction program for the newspaper.
In their early phase of development, journalism graphics have been akin to the time 150 years ago when outrageous sensationalism by the penny press ultimately led to the higher journalism standards widely accepted today. USA Today has the glory and burden of being a pioneer in this aspect of contemporary journalism. It produces more than 1,000 Snapshots annually, and that is only a fraction of the newspaper’s total graphics output. How it deals with this challenge of conveying information graphically but also accurately has significance for journalism generally.
Newspapers are grappling with two serious problems. One is to attract readers. The other is to maintain credibility. The worst solution—graphics long on looks and short on substance—accentuates both problems over the long run. And when one section of the paper shows by its actions that it has neglected such basics as accuracy, who is to say that other sections won’t follow? And even if they don’t, lessening of standards in one section will lead readers to doubt the paper’s overall reliability.
Anyone inclined to dismiss this as a pedantic concern should consider how seriously Responsive Management treated the Snapshot from the Fur Information Council study. In its promotional literature, it boasted that its work has appeared in USA Today.
John Maxwell Hamilton, a former journalist and coauthor of “Hold the Press: The Inside Story on Newspapers,” is dean of the Manship School of Mass Communication at Louisiana State University. David D. Perlmutter, author of “Photojournalism and Foreign Policy,” is on the Manship faculty, and Emily Arnette Vines, a former graduate student, works at Shoot magazine. The authors acknowledge the help of doctoral student Mohamed el-Bendary.
Perhaps to convey that spirit, USA Today’s graphic artists created a nifty, attention-grabbing pie chart superimposed on Davy Crockett’s all-American coonskin cap. The tag line was “I’m OK, your fur’s OK.”
What USA Today’s Snapshot did not report was that the Fur Information Council of America (FICA) paid Responsive Management to do this poll. Had they known that, busy readers might have stopped briefly to question the data. Was the survey conducted in a way that made its results completely unbiased? Or were the numbers accurate but ambiguous? Were Americans saying it was okay to wear fur? Or were they responding that freedom to choose was an inalienable right? If the latter, wasn’t it more interesting that 11 percent would limit fellow Americans’ free economic choice in order to protect furry creatures? That, too, would be news, but would lend itself to quite a different backdrop, a pie chart superimposed on a splayed Bambi.
Not so long ago, newspapers had just about as much interest in dressing up as the Amish. But fear of losing readers to TV and more recently to the Internet, and the advent of new graphics technology, has changed that. Editors are investing in presses that print brilliant colors, in photographic equipment that gets the most out of images, and bigger layout and design staffs.
USA Today has been a leader in this graphics revolution. Along with its widely emulated weather map, Snapshots are a signature item. On the five days it publishes each week, the newspaper runs Snapshots on Page One of each of its four sections. “Snapshots are part of our strategy to establish a consistent identity for the paper, one that sets USA Today apart from other newspapers,” Richard Curtis, managing editor for graphics and photography, wrote several years ago. “They are a clear (some would say persistent) signal that USA Today is a visual newspaper.” Proud of its work, the newspaper uses Snapshots from the previous year in a calendar it gives to advertisers and others.
Several years ago, curious how graphics measure up as sound journalism, we looked in detail at the Snapshots appearing on USA Today’s front page during January 1997, a month chosen at random. Intrigued by the journalistic failings we found, we later looked at two other randomly selected months: April 2001 and January 2002. In our entire investigation—in which we checked the accuracy, clarity and sourcing of each Snapshot published—about one-third of them fell short of established journalism standards.
We started our analysis by assessing the fundamental verity of journalism: getting facts right. Each of the months we scrutinized had 22 Snapshots. Three were inaccurate in January 1997, five in April 2001, and two in January 2002. A few errors were small: A January 1997 graph said losses to private insurers due to highway crashes were $82.76 billion a year; in fact, they were $82.215 billion. An April 2001 Snapshot reported that Williston, North Dakota had a mean temperature of 40.1 degrees. The source for the graphic put the mean temperature at 40.8 degrees.
Other mistakes were more substantial. What follows are some examples:
- A Yankelovich survey found that 51 percent of respondents did not want a genetic test to warn them that they were susceptible to certain diseases; 46 percent said they wanted to know. A 1997 Snapshot reversed the figures, changing the message.
- On April 6, 2001, USA Today presented a Snapshot titled “Americans fear school shootings.” No problem. The Snapshot accurately presented data from Gallup, which has a partnership with the newspaper. But not even three weeks later, on April 24, the newspaper used the same data again in a Snapshot with the headline “Some Americans fear school shootings in their community.” Worse, the second time around it mixed up the numbers. It reported that 31 percent of Americans (instead of 13 percent) thought it “very unlikely” that a school shooting would occur in their community and 13 percent (instead of 31 percent) considered it “very likely.”
- A January 2002 graphic reported that 73 percent of Americans support the idea of women serving in special military operations behind enemy lines and 63 percent support them serving on submarines. In fact, the numbers once again should have been reversed.
Accuracy was only one problem we looked at in our analysis. Even though Snapshots, which are essentially standalone news stories, offer an efficient, interesting way to give readers information, it is, unfortunately, difficult to provide balance and context in one-source news stories. Moreover, reliance on a single, compelling graphic image—Davy Crockett’s coonskin cap is a good example—can result in misleading oversimplification. Typical of these problems is a 1997 graphic about attitudes toward travel. It shows an airplane swerving out of control and offers this tag line: “Just plane scared.” The rationale: 22 percent of adults are afraid to fly. With 78 percent not afraid to fly, though, the tag line might as well as have been “Just plane safe.”
An April 2001 Snapshot, which oversimplifies a complex calculation, reported the number of firearm deaths during a 20-year period. After examining their files, USA Today staff believes that the figure of 670,000 is a “conservative” estimate provided by Handgun Control, a nonprofit organization whose name is indicative of its agenda. Handgun Control based its 20-year estimate on 19 years of hard data from the Centers for Disease Control (CDC). CDC and Handgun Control are each cited in the Snapshot, but the reader is given no idea how the estimate was arrived at. In fact, the graphic does not acknowledge that it is an estimate.
The desire to not clutter up the graphics with lots of text also results in Snapshots like this one from April 2001: A survey asked respondents how much they would like to hear people like Bill Clinton and Colin Powell speak. The categories were “very interested,” “somewhat interested,” “not too interested,” “not at all interested,” and “no opinion.” The Snapshot combined “very interested” and “somewhat interested” in a new, nebulous category called “most interested.”
Yet other examples of the perils of simplification are two Snapshots reporting survey results in January 2002. Neither indicates if respondents chose from a list of alternatives in expressing their opinions or if they answered open-ended questions.
Editors at USA Today are candid about the entertainment value of Snapshots. These graphics are supposed to add a spot of fun to the news. They draw people into the paper. The newspaper’s surveys show that graphics have higher readership than news stories, although editors don’t expect readers to spend a lot of time on them. “It is a snapshot,” explained Fred Meier, a USA Today editor. “If you have to sit and stew over it, it probably hasn’t been done well.”
Perhaps that is true. But Snapshots should hold up if someone does bother to think about them. Too often, however, under a little scrutiny they lose credibility. What should someone make of the Snapshot that appeared in January 2002 with the headline “Most Native Hawaiians, Pacific Islanders live in the West?” And what about the finding that 41 percent of the public say that “making the world a better place” is “always on their mind?” Can anything always be on anybody’s mind? Does anyone never think about making the world a better place?
USA Today has an elaborate review process for Snapshots. By Richard Curtis’s count, staff members “touch” each one at least seven times and very often much more than that because of the frequency with which senior editors check work. Ideas for Snapshots originate with news researchers, graphics editors, and reporters. Many ideas, Curtis said, do not make it past the news editor for graphics or Bob Reynolds, who is the graphics director for the graphics and photography department. The staff is expected to check and double-check facts. They also are supposed to contact unfamiliar sources to ensure that the data are legitimate.
Checking out sources is a crucial part of reporting, particularly when only one source is used in a story. Accordingly, we tried to contact each source used during the three months of Snapshots that we examined. Although employees at these “source” organizations frequently said that no one from USA Today had been in contact, this is not particularly meaningful. These organizations have many employees as well as personnel turnover. What we can assert is that USA Today staff sometimes does not probe sources deeply.
The Fur Information Council of America study is the most dramatic example, but it is not the only one of its kind. In determining the accuracy of a January 2002 Snapshot on the number of police officers shot in the line of duty in the previous year, we found that USA Today staff initially worked off a press release from the National Law Enforcement Officers Memorial Fund. The fund’s PR person subsequently e-mailed at least two updates with new numbers. The fluidity of numbers might have suggested to the staff that they get the report and look at it—or wait until the numbers were no longer preliminary (a caveat not noted in the Snapshot). But they never looked at the original report.
Failure to get original reports also led to mistakes in an April 2001 Snapshot about the percentage of women applying to law school. USA Today accurately reported the data published in JD Jungle magazine. If USA Today staff had checked with JD Jungle, however, they would have learned that the data came from the Law School Admissions Council (LSAC). And if they had checked with the LSAC, they would have found that JD Jungle had misreported the findings.
To be fair, we detected improvement in the Snapshots during our final look in January 2002. Furthermore, we learned from Curtis that the April 2001 time frame that we chose at random to investigate happened to coincide with the illness of a key staffer. We were impressed by the attitude of the editors with whom we spoke. “Our goal is zero errors,” Reynolds said. “All errors are indefensible.” After he had initially heard of our findings, Curtis arranged a lunch so that one of us could talk with staff, including one member who works with the newspaper’s accuracy task force. Staff members said that they do get many calls asking for guidance on how to reach Snapshot sources, but very few about inaccuracies. Further improvements might come as a result of editor Karen Jurgensen’s error reduction program for the newspaper.
In their early phase of development, journalism graphics have been akin to the time 150 years ago when outrageous sensationalism by the penny press ultimately led to the higher journalism standards widely accepted today. USA Today has the glory and burden of being a pioneer in this aspect of contemporary journalism. It produces more than 1,000 Snapshots annually, and that is only a fraction of the newspaper’s total graphics output. How it deals with this challenge of conveying information graphically but also accurately has significance for journalism generally.
Newspapers are grappling with two serious problems. One is to attract readers. The other is to maintain credibility. The worst solution—graphics long on looks and short on substance—accentuates both problems over the long run. And when one section of the paper shows by its actions that it has neglected such basics as accuracy, who is to say that other sections won’t follow? And even if they don’t, lessening of standards in one section will lead readers to doubt the paper’s overall reliability.
Anyone inclined to dismiss this as a pedantic concern should consider how seriously Responsive Management treated the Snapshot from the Fur Information Council study. In its promotional literature, it boasted that its work has appeared in USA Today.
John Maxwell Hamilton, a former journalist and coauthor of “Hold the Press: The Inside Story on Newspapers,” is dean of the Manship School of Mass Communication at Louisiana State University. David D. Perlmutter, author of “Photojournalism and Foreign Policy,” is on the Manship faculty, and Emily Arnette Vines, a former graduate student, works at Shoot magazine. The authors acknowledge the help of doctoral student Mohamed el-Bendary.
