A/B experiments
Bayesian sequential tests on paywalls and onboarding — server-assigned, evaluated continuously, resolved with an alert instead of a spreadsheet.
Dashboard → Experiments. An experiment pits variants of a paywall or onboarding flow against each other on one placement. Assignment happens server-side on the config fetch — the SDK just renders what it's told, so old builds participate correctly and allocation changes don't need a release.
- Sequential by design — evaluation runs every 15 minutes rather than waiting for a fixed sample size. You can look at the numbers any time without invalidating them; that's the point of the Bayesian machinery.
- Holdouts are honest — users assigned to a variant keep it; the live config serves everyone else.
- Resolution alerts —
experiment_winner_shippedandexperiment_leader_foundfire through Slack, email, and your webhook the moment the test resolves.
What to test first
Price presentation beats palette. The levers with real effect sizes in locked templates are which screens run and what the offer says — and because a variant is just config, testing them never risks a broken layout.
Measurement is the same funnel everything else uses: paywall_shown → paywall_cta_tapped → purchase_completed, per variant, with revenue joined from the billing webhooks.