The long road ahead
A few weeks ago in our Manifesto for the Data Informed, one of the five beliefs presented was Company-wide familiarity with metrics rather than outsourcing to ‘data people.’ one change here Immediately, we were pummeled with questions: Does this really matter? Is this a realistic expectation? How can an organization achieve this? So for our next few posts, we’ll deep dive into how to make this lofty aspiration practical. As with anything else in life — whether people, languages or customs — there is no shortcut to gaining familiarity; the only way is through direct and frequent exposure. With data, most teams inherently understand this — that is why dashboards are built and links are passed around and we are all reminded to “please bookmark it and check it often.” Unfortunately, unless your job title includes the word data, the practice of loading said bookmark does not frequently arise to the top of your to-do list, even if you really truly do think data is important! Thus begins the great death spiral of dashboards — because they go unused, they become unmaintained. Because they are unmaintained, when you finally have a need to look at them, they’re broken and useless. This is why data-informed teams rely on practices other than sheer will to create data familiarity. The big three are 1. weekly metrics reviews, 2. weekly insight reports, and 3. insights reviews. In this installment, we’ll tackle one of the single most impactful practices of building a data-informed team: the weekly metrics review. What is a Weekly Metrics Review? A weekly metrics review is a synchronous team meeting to review the key metrics for a scaling, post-PMF product with all functional team members present — ie, PM, engineering, design, operations. This type of review can (and should!) happen at the executive level, with the CEO and C-level executives, and recurse down to individual product teams. A weekly metrics review should be short and sweet (think 5–15 minutes, typically at the start of a regular team meeting) and led by the data person who walks the group through the key metrics for your collective area of work (e.g. new user growth, revenue, conversion rates, tickets resolved). The group should examine how key metrics have progressed over the past few weeks, ideally by looking at a series of time-series line charts. The presenter can also prepare a few key segments to review, for example if a certain type of user, platform, or market is strategically important to the team, or if the team has launched something that impacts a particular segment (like a new feature in a test market). It’s best to keep the meeting lightweight. Preparation should be easy, ideally no more than 30 minutes. Many great metrics reviews simply start with screenshots of dashboards. The data person shouldn’t have to have all the answers at their finger tips (why did active users spike two weeks ago?). It’s fine to circle back with an answer later.
