Growth Experiment Metrics: Frequently Asked Questions Answered
Answers to the most common questions about choosing and using metrics in growth experiments — primary metrics, guardrails, proxy metrics, and North Star alignment.
Metric selection is where most growth experiments go wrong before any data is collected. Teams pick metrics that are easy to measure rather than meaningful, or they choose so many that no single result can drive a decision. These are the questions teams ask most often about experiment metrics — with direct answers.
Common Questions About Growth Experiment Metrics
How do I choose a primary metric for an experiment?
Start with the behavior change your experiment is designed to produce. If you are changing the onboarding flow, the primary metric should be something in the onboarding funnel — not a downstream revenue metric you cannot attribute to this experiment in a meaningful timeframe.
A good primary metric is:
- Directly influenced by the change you are making
- Measurable within your experiment window (if it takes 90 days to observe, your test needs to run 90 days)
- Actionable regardless of direction — you should be willing to ship or discard based on this metric alone
- Sensitive enough to move with the effect sizes you can realistically produce
If you cannot identify one metric that satisfies all four, your experiment scope may be too broad.
Can I have more than one primary metric?
No. The moment you have two primary metrics, you have a decision problem: what do you ship if one goes up and the other goes down? Define one primary metric and treat everything else as secondary guardrails.
Secondary guardrails are real — track error rates, page load time, downstream funnel steps, and other metrics you would not want to harm — but they do not define the outcome. They define the floor below which you would not ship even a positive primary result.
What is the difference between a primary metric and a guardrail metric?
The primary metric defines success. If it moves in the right direction at statistical significance, and guardrails are not violated, you ship.
A guardrail metric defines harm avoidance. It catches cases where your variant improved the primary metric by degrading something else — faster checkout conversion that happens because you removed required address validation, for example. Guardrails should be defined before launch and should have explicit thresholds that trigger a "do not ship" decision.
Should I use business metrics or product metrics?
Both are useful, but they serve different purposes. Business metrics (revenue, retention, LTV) are the ultimate measure of success, but they require large sample sizes and long test windows to move reliably. Product metrics (activation rate, feature adoption, session depth) are more sensitive and faster to move.
The practical approach: use a product metric as your primary if it has a validated relationship to business outcomes, and confirm the business metric direction over a longer observation period after shipping the winner. This gives you speed without losing accountability to business outcomes.
What is a proxy metric and when should I trust one?
A proxy metric is a faster-moving leading indicator used as a substitute for a harder-to-measure business outcome. Activation within the first session as a proxy for 30-day retention, for example.
Trust a proxy metric when:
- You have validated the historical correlation between the proxy and the underlying business metric
- The correlation is stable across user cohorts and time periods
- The change you are making affects the mechanism the proxy is measuring — not just the proxy itself
A proxy that is manipulated directly without improving the underlying outcome (more notifications sent as a proxy for engagement, for example) is not a useful signal. Validate proxy-to-outcome relationships before relying on them for decisions.
How do I align experiment metrics with the North Star metric?
Map your experiment surface to the funnel position that connects to the North Star. If the North Star is monthly active users, and your experiment affects the onboarding flow, map the onboarding activation rate to MAU using historical cohort data. Experiments that move onboarding activation should move MAU by a predictable ratio.
This mapping serves two purposes: it lets you estimate the business impact of a given lift before the experiment, and it lets you audit whether the metric hierarchy is consistent after the experiment.
When experiment metrics cannot be connected to the North Star through a plausible causal chain, the experiment may not be optimizing for anything the business actually needs.
How many metrics should I track in an experiment?
Track as many as are relevant to monitoring for harm and understanding what happened. But make decisions based on one. The number of tracked metrics does not limit analysis — it limits decision-making clarity. A result dashboard with 40 metrics and no declared primary invites cherry-picking.
Define before launch: primary metric (one), guardrail metrics (three to six), and exploratory metrics (anything else you want to observe but will not act on directly). Make this categorization explicit and visible.
When should I consider changing a metric mid-experiment?
In almost no circumstance. Changing the primary metric after results are visible is p-hacking, regardless of the stated reason. If you genuinely believe the pre-specified metric was wrong before you look at results, stop the experiment, document the issue, redesign, and restart.
The only legitimate mid-experiment metric change is correcting an instrumentation bug that caused the metric to not measure what you intended — and this requires transparent documentation of what changed and when.
How do I handle metrics that go in opposite directions?
Scenario: your primary metric goes up, but an important secondary metric goes down. The first step is to check whether the secondary result is statistically significant. Many secondary movements are noise.
If both are significant, you have a genuine trade-off. This requires a business judgment call — quantify the magnitude of each effect, estimate the business impact of each direction, and decide with the relevant stakeholders. There is no formula for this. It is a product decision, not a statistics decision.
Document the trade-off analysis explicitly so the reasoning is available for future reference.
If your team is building the metrics framework and instrumentation that makes rigorous experimentation possible, Clixo can help design and implement the right system.