# Minimum Detectable Effect in A/B Testing: What It Is and Why It Matters

> Understand minimum detectable effect (MDE) in A/B testing — how to calculate it, choose the right value, and why most teams set it wrong and run useless experiments.

- **Published:** 2025-08-13
- **Author:** Clixo
- **Reading time:** 5 min read
- **Tags:** a/b-testing, statistics, minimum-detectable-effect, growth-engineering
- **Canonical URL:** https://clixo.sh/blog/minimum-detectable-effect-ab-testing-explained

Teams new to experimentation often skip straight to picking a sample size tool, plugging in numbers, and launching a test. What they skip is the question that makes every other number meaningful: what is the smallest lift that would actually change what you ship?

This is the minimum detectable effect (MDE), and setting it incorrectly is one of the most common causes of wasted experimentation cycles.

## What Is Minimum Detectable Effect?

The minimum detectable effect is the smallest relative (or absolute) change in your primary metric that your experiment is designed to reliably detect, given your chosen statistical power and significance threshold.

It is not the effect you expect to see. It is the floor below which your experiment cannot distinguish signal from noise.

If you set your MDE to 5% and your variant produces a real 3% lift, your experiment will not detect it. The test will come back inconclusive — not because nothing happened, but because you designed the experiment to be blind to changes that small.

## Why MDE Is a Business Decision, Not a Statistics Decision

Many teams treat MDE as a technical input they need to feed into a calculator. It is actually the reverse. MDE is the output of a business question:

> "How large does this effect need to be to justify shipping it?"

Consider your context:

- If your current checkout conversion rate is 4%, a 5% relative lift is 0.2 percentage points. If your volume is high enough, this is substantial. If your volume is low, this may not move a business metric you can measure.
- If a variant requires significant ongoing engineering maintenance, you may only want to ship it if the lift is large enough to justify the overhead.
- If a variant is zero marginal cost to ship (a copy change, a color, a headline), you might be willing to ship a 2% lift.

The MDE should reflect the minimum effect that makes the decision economically meaningful. Do not choose it by asking "what can we detect?" — choose it by asking "what would change our decision to ship?"

## How MDE Connects to Sample Size and Test Duration

MDE, sample size, statistical power, and significance threshold are mathematically linked. Given any three, you can solve for the fourth.

The practical relationship:

- Smaller MDE requires a larger sample size to detect
- Larger MDE requires a smaller sample size
- Higher power (e.g., 90% vs 80%) requires a larger sample size
- Lower significance threshold (e.g., 99% vs 95%) requires a larger sample size

This means that choosing an aggressive MDE (wanting to detect small effects) is only feasible if you have the traffic to support the required sample size within a reasonable time window.

A team with 1,000 daily visitors to a funnel step cannot reliably detect a 2% lift in a four-week experiment window. The math will not allow it. Setting an MDE you cannot support is not ambitious — it is a guarantee of an underpowered test.

## Common MDE Mistakes

**Setting MDE too small without checking traffic.** Teams want to detect every possible improvement, so they set MDE at 1-2%. For most products at early or mid-stage scale, this requires sample sizes that would take months to accumulate. The test either never reaches significance or gets stopped early.

**Setting MDE to whatever size the calculator accepts.** Some teams work backward from traffic to find the MDE they can detect and treat that as the answer. This inverts the decision: you should set the MDE based on business value, then decide if you have enough traffic to run the test.

**Using the same MDE for every experiment.** A pricing page experiment and an error message experiment have completely different business implications. The minimum lift worth shipping is not a constant across your product.

**Confusing absolute and relative MDE.** A 5% relative lift on a 10% conversion rate is 10.5% — an absolute change of 0.5 percentage points. A 5% absolute change on a 10% baseline is 15%. Make sure your calculator and your team are using the same definition.

## Choosing an MDE in Practice

```mermaid
flowchart TD
  A["Estimate business impact if change succeeds"] --> B["Set MDE at minimum meaningful lift"]
  B --> C["Calculate required sample size"]
  C --> D{"Traffic supports 2-4 week window?"}
  D -- Yes --> E["Run the experiment"]
  D -- No --> F{"Accept longer window?"}
  F -- Yes --> G["Plan extended run explicitly"]
  F -- No --> H["Increase MDE and recalculate"]
  H --> C
```

A reasonable starting approach:

1. Look at the expected business impact of the change if it succeeds. What is a 10% lift worth in revenue or activation over the next year?
2. Set your MDE at the level where the business impact justifies the engineering and analysis cost of the experiment.
3. Calculate the required sample size. Check whether your traffic can support it in a 2-4 week window.
4. If it cannot, either increase the MDE or accept that you will run this experiment over a longer window — and plan for that explicitly.

The goal is not to run experiments that detect everything. It is to run experiments that reliably answer the questions that matter to the business.

If you want help designing an experimentation system built around the metrics that actually drive your product's growth, [talk to the team at Clixo](https://clixo.sh/#contact).

---

Clixo · 1141 W Bryn Mawr Ave, Itasca, IL 60143, US · [hello@clixo.sh](mailto:hello@clixo.sh)
[Start a build](https://clixo.sh/#contact) · [All services](https://clixo.sh/services) · [Agent guide (llms.txt)](https://clixo.sh/llms.txt)