“It was a success in every dimension except the one we thought it would be.”
That’s Daniel Jacobson’s tweet-length summary of his experience, in 2008, opening up a hefty selection of NPR’s vast resources to any software developer anywhere in the world who had a good—or bad—idea for an app that could use it. His experience mirrors that of his colleagues at The New York Times and The Guardian who shared his vision.
Yet all three of these organizations remain decisively enthusiastic about the technical architecture they’d developed to enable this access. It turns out that it’s excellent for news organizations that publish their news in multiple formats, are engaged in strategic partnerships, and want their infrastructure to be robust even as inputs and workflows change. In short, it’s an excellent architecture for almost any news organization.
So, why did the first round of this infrastructure fail to stimulate a wave of innovative apps by external developers? Why have these three major news organizations stuck with it for internal use? And how might we still get to the near-utopia in which reliable news is on tap for any site, service, or user?
The open platforms that NPR, the Times, and The Guardian built are all based around an abstraction layer called an API, or application programming interface. Evan Sandhaus, director of search, archives, and semantics at the Times, uses the metaphor of the shared language of a menu to explain it. Without that vocabulary, if you want to order an omelet, you’ll have to say something like, Please ask the chef to heat up a 9-inch pan, crack two eggs, add some milk, etc. With a menu, you can just say, “Omelet, please.” So long as you get what you ordered, you don’t know or care which chef makes it, whether it’s cooked in a non-stick pan, or they used a gas or electric burner. In fact, it won’t matter to you if the restaurant has switched out everything from its egg supplier to the chef’s toque.
If you’re an in-house developer, the API is your menu. If your new application needs to check if a user is allowed to read an article, you can just pass the user’s ID and password to the API, which will then route your request—your menu choice—to the log-in service your organization has created. As Rajiv Pant, the Times’s chief technology officer, points out, this is not only far more efficient, it also means that if the Times has to update the log-in service—perhaps it wants to address a new security threat—the changes can be made without having to change any of the products that use that service.
The same is true of content. For example, when NPR wanted to enable its member stations to include highly customized content on their websites, the API did the trick. An API can be queried the way you query a database. Ask it only for stories tagged with “Minneapolis” and “music,” and you’ll get the appropriate content. Ask it for the most recent stories that have an ISBN number attached to them and you’ll get the new stories about books.
All three news organizations point to an additional and crucial benefit. In fact, NPR and The Guardian tell almost exactly the same story. In 2008, they both were given just a few weeks to develop an app to demo on the iPad at its launch event. Without an API this would have been a non-starter. With an API they could focus all of their attention on designing the user interface, knowing that populating it with content would be a snap. Both organizations succeeded.
The iPad apps were developed internally, but modern APIs are accessible via the Web, which means anyone can use them as easily as an in-house developer does unless some simple steps are taken to limit access. In 2008, NPR, the Times, and The Guardian left their APIs open, limiting what could be requested to information about the stories and snippets. Open APIs seemed like an easy way to participate in the open culture of the Net. Javaun Moradi, a product manager at NPR at the time, recalls: “The 2008 conventional wisdom on APIs was that it would let a thousand flowers bloom. People in our community would rise up and build mash-ups that we couldn’t even imagine.”
There were some blossoms. Moradi remembers a visual app that displayed a “globe that spun that popped up stories from around the world.” Matt McAlister, The Guardian’s general manager of new digital business, recalls that there were about 50 apps from independent developers in that newspaper’s gallery, including one that used The Guardian’s political data “to create an index to understand how powerful your vote is.” Joe Fiore, manager of technology at the Times, recalls an app a high school history teacher wrote that’s like a fantasy football league except you draft countries, not players.
Then, after an initial spurt, growth flattened. Development of the public APIs and of the supporting documentation and tools slowed at all three organizations.
The three organizations have more than enough reasons why. McAlister says they should have helped external developers make money. Fiore points out that for a developer to invest in building a serious app, she or he has to be not only confident in the API, but be willing to bet that that API will be maintained in the long term. Graham Tackley, director of architecture at The Guardian, says that it was harder than they thought to give developers perfect legal clarity about the status of the intellectual property (IP) they were being invited to use. Plus, the IP issues meant a continuing commitment to resolving problems that might arise. And there were cultural issues; McAlister says, “A lot of institutions that had created open APIs became scared of what they’d unleashed.”
Yet, all three organizations swear by APIs for internal use:
API-based platforms are solidly ensconced in these news media for totally non-altruistic reasons. But there are some paths that might lead to the sort of utopia that motivated the early adopters in 2008.
Enabling external developers and sites to access some level of a news organization’s content and metadata could become more attractive with the growth of the economic imperative that news content become an easily accessible and deeply embedded resource on the Internet.
A news organization might pioneer a business model that provides sufficient incentives for opening up its API. It might create an ad network with revenue sharing, or it might be the sort of unpredictable innovation characteristic of the Net’s history. You never know.
News organizations might start adopting a yet-to-be-written open source news API that brings them internal benefit and that can easily be configured to provide access to some content and metadata. If multiple organizations started using the same API, news could be more easily shared, compared, and re-used.
There are ways we could move closer to the utopian open ecosystem of news even if news creators don’t create public APIs. For example, news aggregators might create their own APIs to make their collected snippets accessible to developers. Or, even without APIs, more news organizations could start supporting Schema.org, a way of embedding metadata into content. Schema.org is being supported by the major search engines because it makes it easier for them to identify the information the text expresses. It has the side effect of making content more re-usable.
[sidebar style="right"]David Weinberger wrote this as a Fellow at the Shorenstein Center on Media, Politics and Public Policy. He is a senior researcher at Harvard’s Berkman Center for Internet & Society. His full report on open APIs for news can be found at the Shorenstein Center.[/sidebar]
So, the dream is not yet over. But the first priority is getting news organizations to use APIs simply because they’re a good way to run a complex, information-driven business. That’s the one sure lesson to be learned from the early, public-spirited attempts by NPR, The New York Times, and The Guardian to get news pumping freely through the Internet’s veins.
That’s Daniel Jacobson’s tweet-length summary of his experience, in 2008, opening up a hefty selection of NPR’s vast resources to any software developer anywhere in the world who had a good—or bad—idea for an app that could use it. His experience mirrors that of his colleagues at The New York Times and The Guardian who shared his vision.
Yet all three of these organizations remain decisively enthusiastic about the technical architecture they’d developed to enable this access. It turns out that it’s excellent for news organizations that publish their news in multiple formats, are engaged in strategic partnerships, and want their infrastructure to be robust even as inputs and workflows change. In short, it’s an excellent architecture for almost any news organization.
So, why did the first round of this infrastructure fail to stimulate a wave of innovative apps by external developers? Why have these three major news organizations stuck with it for internal use? And how might we still get to the near-utopia in which reliable news is on tap for any site, service, or user?
The open platforms that NPR, the Times, and The Guardian built are all based around an abstraction layer called an API, or application programming interface. Evan Sandhaus, director of search, archives, and semantics at the Times, uses the metaphor of the shared language of a menu to explain it. Without that vocabulary, if you want to order an omelet, you’ll have to say something like, Please ask the chef to heat up a 9-inch pan, crack two eggs, add some milk, etc. With a menu, you can just say, “Omelet, please.” So long as you get what you ordered, you don’t know or care which chef makes it, whether it’s cooked in a non-stick pan, or they used a gas or electric burner. In fact, it won’t matter to you if the restaurant has switched out everything from its egg supplier to the chef’s toque.
If you’re an in-house developer, the API is your menu. If your new application needs to check if a user is allowed to read an article, you can just pass the user’s ID and password to the API, which will then route your request—your menu choice—to the log-in service your organization has created. As Rajiv Pant, the Times’s chief technology officer, points out, this is not only far more efficient, it also means that if the Times has to update the log-in service—perhaps it wants to address a new security threat—the changes can be made without having to change any of the products that use that service.
The same is true of content. For example, when NPR wanted to enable its member stations to include highly customized content on their websites, the API did the trick. An API can be queried the way you query a database. Ask it only for stories tagged with “Minneapolis” and “music,” and you’ll get the appropriate content. Ask it for the most recent stories that have an ISBN number attached to them and you’ll get the new stories about books.
All three news organizations point to an additional and crucial benefit. In fact, NPR and The Guardian tell almost exactly the same story. In 2008, they both were given just a few weeks to develop an app to demo on the iPad at its launch event. Without an API this would have been a non-starter. With an API they could focus all of their attention on designing the user interface, knowing that populating it with content would be a snap. Both organizations succeeded.
The iPad apps were developed internally, but modern APIs are accessible via the Web, which means anyone can use them as easily as an in-house developer does unless some simple steps are taken to limit access. In 2008, NPR, the Times, and The Guardian left their APIs open, limiting what could be requested to information about the stories and snippets. Open APIs seemed like an easy way to participate in the open culture of the Net. Javaun Moradi, a product manager at NPR at the time, recalls: “The 2008 conventional wisdom on APIs was that it would let a thousand flowers bloom. People in our community would rise up and build mash-ups that we couldn’t even imagine.”
There were some blossoms. Moradi remembers a visual app that displayed a “globe that spun that popped up stories from around the world.” Matt McAlister, The Guardian’s general manager of new digital business, recalls that there were about 50 apps from independent developers in that newspaper’s gallery, including one that used The Guardian’s political data “to create an index to understand how powerful your vote is.” Joe Fiore, manager of technology at the Times, recalls an app a high school history teacher wrote that’s like a fantasy football league except you draft countries, not players.
Then, after an initial spurt, growth flattened. Development of the public APIs and of the supporting documentation and tools slowed at all three organizations.
The three organizations have more than enough reasons why. McAlister says they should have helped external developers make money. Fiore points out that for a developer to invest in building a serious app, she or he has to be not only confident in the API, but be willing to bet that that API will be maintained in the long term. Graham Tackley, director of architecture at The Guardian, says that it was harder than they thought to give developers perfect legal clarity about the status of the intellectual property (IP) they were being invited to use. Plus, the IP issues meant a continuing commitment to resolving problems that might arise. And there were cultural issues; McAlister says, “A lot of institutions that had created open APIs became scared of what they’d unleashed.”
Yet, all three organizations swear by APIs for internal use:
- “It’s completely revolutionized how we publish content,” says The Guardian’s Tackley. The API provides a stable abstraction layer from which content and services can be easily requested.
- News organizations are under near-constant pressure to keep up with the stream of new devices and services to support. An API makes this much faster and less expensive to develop.
- It’s far easier to improve user-facing services because the front-end team can work independently of the back-end folks.
- The hurdle to creating and fulfilling strategic partnerships is lowered. For example, NPR was able to supply content to Ford’s in-car entertainment system as easily as it does to its member stations.
- The back-end infrastructure can be updated, altered, or entirely swapped out without disrupting the services users rely on. As NPR’s Moradi puts it, “It’s as close as life ever gets to future-proof.”
- It lowers the cost and risk of experimentation.
API-based platforms are solidly ensconced in these news media for totally non-altruistic reasons. But there are some paths that might lead to the sort of utopia that motivated the early adopters in 2008.
Enabling external developers and sites to access some level of a news organization’s content and metadata could become more attractive with the growth of the economic imperative that news content become an easily accessible and deeply embedded resource on the Internet.
A news organization might pioneer a business model that provides sufficient incentives for opening up its API. It might create an ad network with revenue sharing, or it might be the sort of unpredictable innovation characteristic of the Net’s history. You never know.
News organizations might start adopting a yet-to-be-written open source news API that brings them internal benefit and that can easily be configured to provide access to some content and metadata. If multiple organizations started using the same API, news could be more easily shared, compared, and re-used.
There are ways we could move closer to the utopian open ecosystem of news even if news creators don’t create public APIs. For example, news aggregators might create their own APIs to make their collected snippets accessible to developers. Or, even without APIs, more news organizations could start supporting Schema.org, a way of embedding metadata into content. Schema.org is being supported by the major search engines because it makes it easier for them to identify the information the text expresses. It has the side effect of making content more re-usable.
[sidebar style="right"]David Weinberger wrote this as a Fellow at the Shorenstein Center on Media, Politics and Public Policy. He is a senior researcher at Harvard’s Berkman Center for Internet & Society. His full report on open APIs for news can be found at the Shorenstein Center.[/sidebar]
So, the dream is not yet over. But the first priority is getting news organizations to use APIs simply because they’re a good way to run a complex, information-driven business. That’s the one sure lesson to be learned from the early, public-spirited attempts by NPR, The New York Times, and The Guardian to get news pumping freely through the Internet’s veins.
