We have reached a point in time where it is no longer a question as to whether or not journalists have embraced open-source investigation methods. From covering the Israel-Hamas war to exposing extremism networks in sub-Saharan Africa, more and more newsrooms are successfully conducting investigative research based on publicly available online sources, often in combination with traditional reporting techniques.
However, the increasing numbers of journalists who comb through social media posts or analyze satellite imagery not only experience the benefits, but also confront the struggles that can come with this type of research. Among them: feeling overwhelmed by the task of finding and using open-source research tools.
This is why I spent my 2024 fellowship year at Harvard — as a joint fellow at the Nieman Foundation for Journalism and the Berkman Klein Center for Internet and Society — trying to find ways to make it easier for journalists to navigate the open-source landscape by developing a resource they could use to learn about these powerful investigative tools. I used data I had collected in a series of surveys, as well as interviews with 40 users, to figure out how to design a toolkit in a way that would be most useful.
My interviewees knew exactly what they wanted: First, they said, any information provided by such a site needed to be continuously kept up to date. This is a particular challenge, since tools can change or disappear at any point, and what worked today might not exist tomorrow. Examples include the discontinuation of the Twitter scraper Twint and Meta’s social media monitoring tool Crowdtangle, both of which created a gap in many open-source journalists’ workflows.
My interviewees also said they wanted the toolkit to include guides on how to use the tools, a description of each tool’s limitations, and information on pricing. Finally, they pointed out the need for collaboration to keep the toolkit’s information up to date.
It quickly became clear to me that this was not something one person, or even an organization like Bellingcat — a global investigative nonprofit based in Amsterdam that focuses on open-source research — can do alone. To test a collaborative approach, I invited members of Bellingcat’s volunteer community, people from various backgrounds who are enthusiastic about open-source investigations, to create this toolkit with me.
Each volunteer chose several open-source research tools to test and analyze. I provided them with a list of categories, such as “description” or “cost” to use to evaluate each tool. We had already set up a platform where they submitted their findings.
The results were promising. Initially, I had envisioned a toolkit that would only offer short paragraphs of key information about each tool. However, many of our volunteers delved deeper than I expected into understanding and testing their tools. As a result, their entries became guide- or review-like. When potential users said they preferred the longer reviews, I changed course and encouraged volunteers to write longer texts. This was one of several examples where our volunteers drastically changed how the final product would look.
A crucial task for us was to create documentation for our tools — describing in simple language what a tool does and how it can be used. It’s often the least favorite part of a software developer’s job, and some tool providers produce insufficient documentation. In addition, open-source journalists often use tools in a different way than they were intended, so documentation targeted at a typical user might not meet their particular needs. For instance, bird identification apps or a tool like PeakVisor, which was built to provide orientation for mountain hikers, are used by open-source journalists for geolocation purposes.
Luckily, our volunteers understood the importance of providing clear and accurate documentation. While many providers use marketing language to present their tools in a solely positive light, we made sure to identify a tool’s potential weaknesses and limitations in addition to its usefulness.
After several months of experimenting and adding more reviews, we launched Bellingcat’s Online Investigation Toolkit in fall 2024. In the year since, the article announcing the toolkit’s launch has been the most viewed page on Bellingcat’s website. More than 1,000 unique visitors per day are currently using the toolkit, and the numbers are growing.
Here are some key things I learned from building this collaborative toolkit from scratch:
Know your users and focus on iterative product development
Product managers know this very well: The best way to build a tool, or any other newsroom product, is by learning about your users’ needs, building a prototype, and incorporating feedback to improve it over time. While some of the needs of open-source journalists were known to me from the start, others arose only after the toolkit had launched.
For example, user feedback suggested we should have a stronger focus on comparing tools that solve similar research tasks, so we introduced a “similar tools” section. Users also asked for the ability to download all the content from the toolkit. At first, I did not understand why, but it turned out that many journalists wanted to create their own versions of the toolkit to which they could add other tools they discovered. In this way, we are continuously tweaking the toolkit to make it better, based on user feedback and what we learn.
Don’t shy away from collaborating with people who are not journalists
Most of the volunteers collaborating with me are not journalists. This is where the need for clear communication comes in. Some concepts related to journalism and research seemed so normal to me that I did not realize at first that I had to explain them. This included volunteers’ questions like, “What are the lines between marketing and journalism?” or “Why, and when, do we have to link out to sources from a tool review?” I learned that misunderstandings are often a consequence of my failing to explain things to people who work outside my field.
The key for collaborating successfully is to not expect everyone to have the same understanding or interpretation of things, but to communicate in a way that can be universally understood. If this is kept in mind, collaborating with nonjournalists can open up a whole new avenue of ideas and thoughts, which has the potential to improve the quality of products for journalists.
Volunteers need both freedom … and structure
Volunteers are dedicating their free time to building our toolkit, and they should have fun doing so. I therefore try to give them as much freedom as possible. They can choose the tools they want to test and decide how much responsibility they want to take on within the project. At the same time, it is important that we provide enough structure to make the toolkit project — which has had more than 70 contributors so far — work.
This has meant learning to become more strict with our volunteers, to make sure we are all heading in the same direction. We have deadlines for submitting and updating tool reviews, and we require the reviews to be written according to a clearly defined set of rules aimed at achieving consistency. People who are not able to commit to following these guidelines are removed from the project.
Structure also means that the contribution of our volunteers is woven into our organizational goals. Volunteers have a good sense of whether a project is seen as important and valued by the organization that is offering it. In our case, we provide a structure for them to make an impact in the field of open-source investigation, and we do so by helping journalists discover tools they need to conduct investigations in the public interest.
Allow yourself to set new standards
Some journalistic standards are simply nonnegotiable. One of them is accuracy. Before publishing any new tool review, Bellingcat staff check everything that goes online. Other rules that are considered normal in the world of journalism, however, were simply not suitable for this project and we decided to let them go.
The toolkit is a functional resource which acts as a living document for sharing information about tools. It is not about beautiful writing, and many of those who are contributing to or using the toolkit are not native English speakers. We felt it was therefore OK that some of the tool reviews contained a few grammatical errors, or sentence structures that might sound unusual to a native speaker’s ears. Perfect language standards were simply not needed for this specific project, so we decided to prioritize diversity of expression.
While this text only reflects my personal experiences building Bellingcat’s Online Investigation Toolkit, I encourage other media organizations and tool builders to try to build products and tools for journalists by looking outside the world of journalism, too. You might be surprised at what you learn.
