What we are learning from studying key metrics for tiny newsrooms
Tiny News Collective aims to build a metrics dashboard for members to help them track what matters most
By Madison Karas and Pilar de Haro
The explosive growth in small, independent news organizations is a major bright spot in the journalism field as new systems emerge. Individuals nationwide are raising their hands to be publishers of local and topical outlets to help inform and connect their communities. As they create and launch tiny news organizations — especially one- or two-person shops — inevitably, the question of “what does success look like?” comes up. We see success in many ways in our Collective – a publisher receiving a major grant, being quoted in a larger publisher’s story, or joining a training cohort. But to communicate and measure these organizations’ work, more tactical measurements are needed. That leads to the question of what are the essential data points to collect and the metrics to track in order to gauge progress.
For the last decade, larger, traditional newsrooms have focused a lot of time and energy on becoming more data-focused in all of their operations, from the journalism they produce to their revenue streams. Extensive dashboards and detailed reports are now the norm. The choices of what to collect abound: page views, unique visitors, social followers, newsletter subscribers, newsletter open rates, paying subscribers or members, number of grants received, frequency of publishing, engagement, time on site, etc. But for small publishers, all of that data can just be too much overwhelming noise. More importantly, these publishers need to know what these metrics add up to and how these relate to the impact that they want to make in their communities.
Inside these organizations, we saw that founders need tools, training and support to understand which data and metrics are most useful to them at each stage of their development. And we recognized the role we could play to build the infrastructure to collect, synthesize and provide insights for their operations. A program that could accomplish things like that for early-stage news founders could be a game changer for individual organizations — and also could be powerful in the aggregate, as Tiny News Collective is uniquely positioned to collect data from our members' systems to understand better what works and what doesn’t for early-stage news founders on the whole.
Thanks to funding and support from the Patrick J. McGovern Foundation, our ambition to start building a data and metrics program began this spring. Our team planned our program development into three steps: 1) researching important metrics for small startup newsrooms, 2) tooling and infrastructure development, and 3) implementation and training. We have concluded our research phase and want to share what we’ve found with the journalism field, in the hopes that it will be helpful not just for our members but for other founders in the ecosystem. We pursued our research with the goal of identifying a set of metrics, sourced from community, audience, revenue and product data, that could be used as indicators and boosters for just-launched and very-tiny organizations beginning their publisher journeys.
TNC
From raw data to routine
To start, we surveyed our publishers, other news publishers and experts in media analytics and technology for small newsrooms. These surveys focused on the what and how of practicing metrics: collecting and analyzing important metrics, creating good user experiences for tiny publishers to view their data, and helping publishers establish routines to review their metrics regularly. We also conducted interviews with experts and referenced materials from others in the field who inspire us.
As we wrap up this work, here’s what we’ve learned:
Everything should tie back to the mission and audience of a publication.
Many of the conversations we had in our research brought us back to the same point: We’ll never be able to prescribe one definitive set of metrics to all tiny news organizations, as news organizations need to tie their metrics to their mission and audience. That underscores the need for thorough pre-launch audience research. When startup news organizations clearly assert and define their audiences’ needs before launch, much of the “homework” of narrowing important audience metrics to track after launch is already done. Tying metrics to organizations’ goals also requires interpreting raw data. As Sophie Ho from the News Revenue Hub told us, “You can collect a lot of data, but if you don't have the time to analyze it, then it’s pointless. With metrics, lead with your mission and main objective.”
Selecting focus metrics for just-launched newsrooms is both easy and difficult at the same time.
When we asked folks which metrics to prioritize for tiny news organizations, some points of clear consensus and some points of divergence emerged. Some felt obvious, like being aware of your organization's “core vitals,” such as total revenue and staff size. Others were more disputed, as we heard mixed opinions on the importance of page views as an audience metric, for example. Also, many of the people we spoke to dismissed the importance of social media followers as an indicator of success for early-stage tiny news organizations.
The top choices that emerged for our focus metrics include both “internal” and “external” metrics:
External metrics, or metrics sourced from the locality, community or industry the organization is within, include:
- Total addressable market: the organization’s largest audience opportunity
- Reader impacts: reader responses and community actions that follow reporting
- Media mentions: story pickups, citations and attributions to the organization in other media
- Event appearances: when the organization’s journalists or staff are invited as speakers or participants in community events
Internal metrics, or metrics in which the data is sourced from within the news organization measuring them and are customized to the organization’s products and revenue models, include:
- Total supporters: That accounts for the organization’s broadest audience that is not yet financially supporting them, such as total newsletter sign-ups or total registered users.
- Financial supporters: That reflects the organization’s broadest audience that is financially sustaining them, such as recurring donors or paid subscribers.
- Total revenue: Depending on the organization’s revenue model at launch, that could be the number of grants or total earned revenue.
- Unique visitors to the organization’s website or primary platform
- Frequency of publishing new primary content
This list of metrics allows insight into an organization’s relationship with their community through both monetary and non-financial support. These metrics can also help publishers sharpen their organization’s initial “funnel journey” — the path that an audience takes from awareness of an outlet to becoming a financial supporter of it. We plan to use this list as a base set of metrics that can be customized depending on an organization’s products, revenue stream, and of course, audience and mission.
Publishers need context for their data.
Another point repeated in our research is to identify not just the metrics themselves but also the strategy for analyzing them. The people we spoke to emphasized the need to create simple tools to help tell the story of the metrics. That simplicity is needed both in how we display the data in the tool we develop and in the supplementary training that we create for our members to learn about the data. For example, when viewing a number, publishers need to know the source of it, how it relates to their other metrics and how they may investigate it further or respond to it.
Surveys of our Collective publishers echoed that, too. A few publishers shared that as they grow, they increasingly want to use metrics to guide how they allocate their time and where they focus their efforts as individuals and teams with limited capacity. We plan to take this sentiment into consideration as we develop a tool and training for our members. Clear and thorough guides and explainers will help members learn and retain the information better as they build their metrics routines.
Where we’ll go next
Throughout our work so far, we’ve been met with enthusiasm for this project’s goal because of the focus on small news organizations, where data and metrics practices have been less developed than in larger, more established news organizations. As more small news organizations proliferate to fill news and information gaps across the country, the need for well-defined data approaches and practices rises. Even the launch of Press Forward’s first open call aimed at small news organizations stressed the need for better clarity around small startup newsrooms’ data, as newsroom entrepreneurs need that information to help make their case to potential financial supporters of all kinds.
We will spend the next few months planning, prototyping and developing a tool to be used by our members and the Collective as a whole to track the focus metrics we identified from the research in real-time. As TNC grows, we believe this approach will be more sustainable for us to measure progress consistently for and alongside our members than continuously requesting their data and trying to respond to it retroactively. Taking a cue from the experts we spoke to, we aim to make this tool simple to use and a companion for goal setting.
We also plan to continue sharing our progress and what we learn from our tool and training development for member newsrooms. Now that we know better what we’ll track, our next phase will be focused on how we’ll do it. Stay tuned here for more.
Pilar de Haro is the support engineer for the Tiny News Collective. Madison Karas is a project manager for R&D working with Tiny News Collective on this project.
Many thanks to the individuals who spent time sharing their experiences with us for this work: Lindsay Green-Barber from Impact Architects, Eli Flournoy from Media Growth Partners, sarah day owen wiskirchen from minerva media co., Emily Roseman from Institute for Nonprofit News, Chloe Kizer from LION Publishers, Sophie Ho from News Revenue Hub, Ishena Robinson from URL Media, Nicolas Rios from Documented, Ned Berke from BlueLena, Liz Worthington from American Press Institute, Tyler Fisher from The 19th, Kara Meyberg Guzman from Santa Cruz Local, Becky Pallack from Arizona Luminaria, Styli Charalambous from The Daily Maverick and Nahima Ahmed from Nahima Ahmed Consulting.
A special shout-out to the Collective members who shared their time and thoughts with us: Jeannette Andruss of Spotlight Schools, Teri Mott and Emily Christensen of The Shout, Cara Kuhlman of Future Tides, Amy Bushatz of Mat-Su Sentinel and Latasha Eley Kelly of The Millennial Black Professor.