Metric selection

This is the exact file agents read, including the credit header and licence notice. Save it as metric-selection/SKILL.md.

---
name: metric-selection
description: >
  Choose the few metrics worth tracking before anyone builds a dashboard or report: define the decision each metric serves, test candidates to separate actionable from vanity metrics, pick one primary metric with three to five inputs and guardrails that stop gaming, map leading against lagging, set an honest baseline and target, and document each metric. Covers the north-star-and-inputs structure, AARRR, Goodhart's law, counter-metrics and ratios over raw counts. Use it when someone says "which metrics should we track", "pick our KPIs", "define our north star metric", "what should we measure", "our metrics are vanity metrics", or "set a target for X". Do NOT use it to lay out a dashboard (see dashboard-design), pick a chart type (see chart-selection), write a narrative report (see data-storytelling), design an experiment (see statistics-and-experimentation), or plan event tracking (see analytics-and-tracking).
license: Free to use and adapt under CC BY 4.0, with credit to Afeez Adebayo. Link afeezadebayo.com, https://www.linkedin.com/in/afeez-adebayo/ and this skill's page at https://afeezadebayo.com/skills-md/data/dashboards-reporting/metric-selection/.
---

# Metric selection

## When to use this, and when not to

Use this skill when the job is to decide what to measure: which metrics a team, product, campaign or report should track, which one leads, and which are noise. The triggers are a request to pick KPIs, define a north star, choose success metrics, set a target for a number, or a complaint that the current metrics are vanity figures, that there are too many of them, or that nobody can say which one matters most.

Do not use it to arrange metrics on a screen once they are chosen, which is `dashboard-design`, or to choose the chart for a single number, which is `chart-selection`. Writing a report or presentation that argues from the numbers is `data-storytelling`. Designing a test to prove a metric moved is the statistics-and-experimentation skills, and planning the events and properties that produce the raw data is the analytics-and-tracking skills. This skill is the step before all of those: it decides which handful of numbers are worth the attention, so the later work is spent on the right things.

The core idea: a metric earns its place only if a plausible change in it would change what someone does. Most numbers a tool can produce fail that test. The job is not to collect every measurable thing, it is to find the one metric that captures the value being created, the few inputs that move it, and the guardrails that stop it being gamed, then to leave everything else out. Measuring more is not measuring better; a shorter list that drives action beats a long list that drives meetings.

## Inputs the agent must have before starting

1. **The decision or behaviour the metric is meant to drive.** A metric exists to change an action: what someone ships, funds, stops, or prioritises. Ask what decision this number will inform and who makes it. "We want the growth team to know whether activation is improving week to week so they can prioritise onboarding work" is enough; "we want to track engagement" is not.
2. **The audience and the altitude.** A board, an executive, a product team and an operations shift need metrics at different altitudes. Ask who reads this and how often, because that sets whether you need one strategic outcome or a set of operational drivers.
3. **What value the product or team delivers to its customer.** The primary metric should reflect value received by the user, not activity logged by the company. Ask, in plain words, what good thing happens for the customer when the product works.
4. **The candidate metrics already in use or proposed.** The list someone wants to track, so the skill can test and prune rather than invent in a vacuum.
5. **The data that actually exists, and how fresh it is.** A perfect metric you cannot measure is useless. Ask what is instrumented today, what would need new tracking, and how current the data is.
6. **Any target, benchmark or constraint already set.** An existing goal, a contractual SLA, or a known industry benchmark, so the target-setting step is grounded rather than guessed.

If the decision the metric serves is unknown, stop and ask. Selecting metrics without a decision to serve produces a tidy list that no one uses, which is the most common failure in this work.

## The workflow, step by step

### 1. Write the decision the metric serves, in one sentence

State: "[audience] will use [metric] to decide [action], reviewed [cadence]." If you cannot fill the "decide" slot with a real action, the metric is decoration. This sentence is the test every later choice is measured against (Croll and Yoskovitz, *Lean Analytics*, the "one metric that matters"; Kaushik, "Four simple rules for a good metric", on a metric being instantly useful).

### 2. Separate the metric altitudes, and pick the layer you are choosing for

Metrics live at three altitudes, and mixing them is why lists sprawl:

- **Primary outcome (north star):** the single number that best captures value delivered, reviewed weekly or monthly.
- **Inputs:** the three to five levers a team can move through its work, which together drive the outcome.
- **Guardrails and diagnostics:** the counter-metrics that must not get worse, and the detail used to explain a move.

Decide which layer this request is about before generating candidates. A board wants the outcome and one or two guardrails; a product team wants the inputs. Choosing for the wrong altitude is why executives get buried in operational detail and teams get a vague outcome they cannot act on.

### 3. Generate candidates from value and from a funnel, not from the tool's menu

Produce candidates two ways and merge them:

- **From value:** complete the sentence "the customer got value when they ___", and turn that event into a metric (a booking taken, a question answered, a track played to the end). This is where a strong primary metric comes from (Amplitude, *North Star Playbook*).
- **From the funnel:** walk the AARRR stages, acquisition, activation, retention, revenue, referral, and name the one metric that best represents each stage for this product (McClure, "Startup Metrics for Pirates"). This surfaces the inputs.

Do not start from the list of everything the analytics tool can export. A menu-driven list is how dashboards fill with page views and raw totals that no one chose on purpose.

### 4. Test every candidate, and cut the ones that fail

Run each candidate through the five-part test. A metric should be:

- **Understandable:** a non-specialist can say what it means in one sentence. A metric people cannot explain gets ignored (Croll and Yoskovitz).
- **A ratio or rate, not a raw count where a rate is available:** rates compare and normalise, counts only accumulate. "Activation rate" beats "number activated"; "revenue per active account" beats "revenue" for steering (Croll and Yoskovitz; Kaushik on uncomplex, comparable metrics).
- **Comparative:** it can be set against a target, a prior period, a segment or a benchmark, or it means nothing on its own (Kaushik).
- **Actionable:** someone can name the work that would move it. Kaushik's rule is that "no action" is acceptable only as a deliberate decision, not as the usual outcome.
- **Behaviour-changing:** the golden rule, if a plausible change in the metric would not change what anyone does, it is a vanity metric, so drop it (Croll and Yoskovitz, "if a metric won't change how you behave, it's a bad metric").

Decision rule: a candidate that fails two or more of these is cut, not shrunk onto the list "for reference". Total registered users, cumulative sign-ups, page views, raw impressions, social followers and headline revenue are the usual casualties, large and reassuring but disconnected from any lever (basedash, "Vanity metrics vs actionable metrics"). Convert each survivor to its best rate-or-segment form before moving on.

### 5. Choose one primary metric, and test it hard

From the survivors, choose a single primary metric. One, not three: a team with three top metrics has no top metric. Test the candidate against three checks (Amplitude, "What makes a good vs bad north star metric"):

- **Value:** it represents the value users get from the product, not activity the company logs.
- **Influence:** it sits within the team's sphere of influence through normal work.
- **Leads revenue:** it is a leading indicator of revenue, not revenue itself.

Then apply the direct-control test (John Cutler, via Amplitude): "if you can move your [primary metric] directly, it's probably not a good one." A good primary metric is an outcome you move through inputs, not a dial you set. Revenue, MRR and ARR fail as primaries because they are lagging: once recorded they cannot be changed, and a team cannot act on them directly. Keep revenue related to the primary metric, not as the primary metric itself (Amplitude). Validate the choice against your own data: does the candidate, measured historically, actually track retention and revenue? If it does not, it is the wrong primary (Amplitude).

### 6. Pick three to five inputs that move the primary

Choose the inputs the team can shape week to week, which together drive the primary metric. Three to five is the working range: fewer and the model is thin, more and no one can hold it (Amplitude, *North Star Playbook*). Each input should pass the same five-part test as the primary and should answer "if we move this, the primary should move". Write the causal link out loud, "more onboarding completion drives more weekly active teams drives retention", so a broken assumption is visible and testable later.

### 7. Add guardrails and counter-metrics so the primary cannot be gamed

Every metric optimised alone invites a shortcut that harms something unmeasured. Goodhart's law, as sharpened by Marilyn Strathern, is "when a measure becomes a target, it ceases to be a good measure." Protect against it by pairing the primary and each input with a guardrail that must not degrade:

- Speed of response paired with quality or reopen rate.
- Sign-up rate paired with day-30 retention of those sign-ups.
- Average revenue per user paired with refund or churn rate.
- Output volume paired with an error or complaint rate.

Decision rule: if you cannot name the shortcut that would juice a metric at someone's expense, you have not found its guardrail yet. Name one guardrail per primary metric at minimum before shipping the set.

### 8. Map leading against lagging, so the set can both steer and confirm

Label each chosen metric as leading (it predicts and can be influenced now) or lagging (it confirms what already happened, like MRR, ARPU or churn) (Amplitude, "Leading vs lagging indicators"). A healthy set has both: leading metrics to steer by, lagging metrics to confirm the steering worked. Decision rule: if every chosen metric is lagging, the team can only watch outcomes arrive, not change them, so add at least one leading input. If every metric is leading, add a lagging metric to confirm the leading ones actually pay off.

### 9. Set a baseline and an honest target for each

A metric with no comparator cannot be read. For each metric:

- Establish the baseline: the current value and its normal range or volatility, measured over enough history to know what "normal" looks like.
- Set a target that is a stretch from the baseline, time-boxed, and derived from something, a prior trend, a benchmark, or a modelled input, not invented to look ambitious.
- Prefer a target expressed as a direction and threshold over a single number when the metric is noisy ("hold activation above 40 percent" rather than "hit 41.3 percent").

Never fabricate a benchmark or a target to fill the slot. If no credible basis exists, say so and mark the target provisional pending a baseline period. Any external benchmark carries a "verify before relying on this" note, because benchmarks age and vary by definition.

### 10. Document each metric so it is trusted and consistent

An undefined metric is a distrusted metric, and two teams computing "active user" differently is how reporting arguments start. For each chosen metric, record a one-line definition in plain words, the exact calculation, the data source, the owner, the cadence, the segment it applies to, and whether it is leading or guardrail. This is the metric's single source of truth. Where a metrics layer exists (dbt, LookML, a semantic model), define it there once so every report inherits the same number (see the analytics-and-tracking skills for instrumentation).

### 11. Verify the set against the quality bar

Run the chosen set against the quality bar below before handing it over. Every item is checkable. Hand the arrangement of these metrics to `dashboard-design`, the chart per metric to `chart-selection`, and the narrative to `data-storytelling`.

## Worked example, end to end

A subscription language-learning app asks, "what KPIs should we track? Can you set us up a metrics list?" The reflex is a list of twenty: downloads, total users, daily active users, lessons started, time in app, social followers, app-store rating, revenue, and so on. It looks thorough and steers nothing.

Working from the decision instead: "The product team will use one metric weekly to decide where to focus onboarding and retention work." That is an input-and-outcome request for a product team, not a board scorecard.

Value sentence: "the learner got value when they finished a lesson and came back the next day." From the funnel, the stages suggest candidates: installs (acquisition), first lesson completed (activation), weekly learning days (retention), subscription (revenue), invites (referral).

Testing candidates: downloads and total users fail understandable-as-a-rate, comparative and behaviour-changing, so they are cut. App-store rating is a guardrail, not a driver. "Daily active users" survives but is weak as a primary because a user can open the app without learning anything.

Primary metric: "weekly active learners who completed at least three lessons." It represents value (actual learning, not just opening the app), sits within the team's influence, and leads revenue (people who learn renew). It passes the direct-control test, the team cannot set it directly, only move it through onboarding and content. Revenue stays related but is not the primary, because it is lagging.

Inputs (three to five the team moves): first-lesson completion rate for new users, week-one return rate, lessons completed per active learner, and streak-resumption rate after a missed day. Each has a plausible causal line to the primary.

Guardrails: subscription refund rate (so growth is not bought with mis-sold annual plans) and a content-difficulty complaint rate (so "more lessons completed" is not achieved by making lessons trivial). Leading-versus-lagging: the four inputs are leading; net revenue retention is added as the lagging confirmation.

Targets: baseline first-lesson completion is 54 percent over the last quarter, so the target is "lift first-lesson completion from 54 to 62 percent this quarter", derived from the gap top-quartile cohorts already reach, not from a round number. The deliverable is one primary metric, four inputs, two guardrails and one lagging confirmation, each with a definition, an owner and a baseline. Sixteen of the original twenty numbers are gone.

## Templates you can use verbatim

**Metric set spec (fill this in before building any dashboard or report):**

```
Decision served:   who decides what, how often
Primary metric:    the one outcome (value-based, leading, influenceable)
  definition:      plain words + exact calculation
  baseline/target: current value, normal range, time-boxed target + basis
Inputs (3-5):      the levers the team moves, each with its causal link to the primary
Guardrails (1+):   counter-metrics that must not degrade
Lagging confirms:  the metric(s) that prove the leading ones paid off
Owner + source:    who maintains each number and where it comes from
```

**Candidate test table (one row per candidate metric):**

| Metric | Understandable | Ratio/rate | Comparative | Actionable | Behaviour-changing | Verdict |
|---|---|---|---|---|---|---|
| Weekly active learners (3+ lessons) | yes | yes | yes (target) | yes | yes | primary |
| Total registered users | yes | no (raw count) | weak | no | no | vanity: drop |
| App-store rating | yes | yes | yes | partly | as a limit | guardrail |

**Primary-metric checklist:** reflects customer value; within the team's influence; leads revenue; fails the direct-control test (an outcome, not a dial); tracks retention and revenue in historical data; stated as a rate where possible.

## Failure modes and anti-patterns

1. **The metric buffet.** Twenty numbers tracked because they were measurable, with none named as primary. Spot it when no one can answer "which one matters most?" Fix: run every number through the five-part test, pick one primary, demote or drop the rest (Croll and Yoskovitz; basedash).
2. **A lagging metric as the primary.** Revenue, MRR or ARR set as the headline the team is told to move, though they cannot act on it directly. Spot it with the direct-control test and the leading-versus-lagging label. Fix: make the primary a leading, value-based metric and keep revenue as a related lagging confirm (Amplitude).
3. **Vanity metrics as headlines.** Cumulative totals and follower counts shown as the big number because they only ever rise. Spot them with the behaviour-changing test. Fix: convert to a rate or segment the team controls, and demote the cumulative total (basedash; Kaushik).
4. **No guardrail, so the metric gets gamed.** A single metric optimised until a shortcut harms something unmeasured, support speed up and quality down, sign-ups up and retention down. Spot it by asking "what is the cheapest way to juice this?" Fix: pair every primary and input with a counter-metric that must not degrade (Goodhart; Strathern).
5. **Undefined or inconsistently defined metrics.** "Active user" or "conversion" tracked with no written definition, computed differently by two teams. Spot it when two reports disagree on the same name. Fix: write one definition, calculation, source and owner per metric, and define it once in the metrics layer (Kaushik on relevance and clarity).

## Quality bar

- Every chosen metric traces to a stated decision and audience; a metric that serves no decision is removed.
- There is exactly one primary metric, and it is value-based, leading, within the team's influence, and fails the direct-control test.
- The primary has three to five inputs, each with a written causal link to it, and each input passes the five-part test.
- At least one guardrail or counter-metric protects the primary against gaming, with the shortcut it guards named.
- The set contains both leading metrics (to steer) and at least one lagging metric (to confirm).
- Every metric is a ratio or rate where one is available, not a bare cumulative count.
- Each metric has a plain definition, an exact calculation, a source, an owner, a baseline and a time-boxed target with a stated basis.
- No target or benchmark is invented; provisional targets are labelled, and external benchmarks carry a "verify before relying on this" note.

## Data handling and safety

Metric definitions often reference personal or commercially sensitive fields: individual revenue, health outcomes, salaries, named accounts. Define metrics over aggregates and respect the access controls on the source; do not pull raw personal rows into a metric's working notes, and never paste real customer records or credentials into prompts while selecting metrics. Changing the definition of a live KPI, or publishing a new target to a team, is a decision made on other people's behalf: propose it as a draft and confirm before it is adopted, because people plan and are judged against these numbers. Never fabricate a baseline, a benchmark or a target to make a set look complete; an honest "no baseline yet, measure for four weeks first" is better than a confident invented figure. When a metric drives decisions about money, health, legal standing, hiring or someone's pay or performance, state plainly that the metric is an input to a human decision, not the decision, and add a review step, because optimising a single number against people's interests is how measurement causes harm.

## Appendix: tool-specific notes (verify before relying on these)

- **Amplitude and Mixpanel:** both are built around the north-star-and-inputs idea; define the primary as an event-based metric and the inputs as the events that precede it, and use their retention and funnel reports to validate that the primary tracks retention before committing to it. Confirm current event-definition and formula behaviour in the product, as these change.
- **GA4:** "key events" (formerly conversions) are the nearest native concept to a chosen metric; prefer a rate built from an event and a denominator over a raw event count, and beware that default engagement metrics are activity, not value.
- **dbt / LookML / semantic layer:** define each chosen metric once in the metrics or semantic layer so every downstream dashboard and report inherits the same calculation; this is the single most effective guard against two teams disagreeing on a number. Verify the metric-layer syntax for your version.
- **Spreadsheets:** fine for a small KPI set; keep one tab that is the metric dictionary (definition, calculation, source, owner, target) and reference it from every sheet so the definitions do not drift.

## Sources

- Amplitude, "What makes a good vs bad north star metric" and the *North Star Playbook*: a good primary reflects user value, sits within the team's influence, and is a leading indicator of revenue; the direct-control test (if you can move it directly it is probably not a good north star); three to five inputs the team can shape; revenue and MRR are lagging and weak as primaries; validate a candidate against retention and revenue in your own data. Verified 3 November 2026.
- Amplitude, "Leading vs lagging indicators: how to identify and map your metrics": leading indicators predict and can be influenced now, lagging indicators confirm what already happened; a healthy set holds both. Verified 3 November 2026.
- Alistair Croll and Benjamin Yoskovitz, *Lean Analytics*: the One Metric That Matters, and the tests that a good metric is understandable, comparative, a ratio or rate, and behaviour-changing, with the golden rule that a metric which will not change how you behave is a bad metric. Verified 3 November 2026 (via the published book summary and the awesome-pm-skills and pm-skills metrics implementations that encode it).
- Avinash Kaushik, "Four simple rules for identifying a good metric" (Occam's Razor, via Analytics Demystified): a good metric is uncomplex, relevant, timely and instantly useful, and "no action" is valid only as a conscious decision. Verified 3 November 2026.
- Dave McClure, "Startup Metrics for Pirates" (AARRR): acquisition, activation, retention, revenue and referral as the funnel stages to name one metric for each. Verified 3 November 2026.
- basedash, "Vanity metrics vs actionable metrics: how to tell the difference": cumulative totals, page views and follower counts are vanity metrics; convert to rates and segments the team controls. Verified 3 November 2026.
- Charles Goodhart and Marilyn Strathern (Goodhart's law): "when a measure becomes a target, it ceases to be a good measure", the basis for pairing every metric with a guardrail against gaming. Verified 3 November 2026.