Fictura
Experiments

How assignment works

Server-side, deterministic, sticky per device — decided before a config fetch even needs to know the answer.

Dashboard → Experiments. An experiment tests paywalls or onboarding flows: two to four arms, each pointing at one you've already built. experiment_type (price point, free trial offer, and so on) is a label for your own records; every arm still just serves the config you picked.

New experiments default to new users only and notify me (nothing ships until you roll it out). Existing subscribers bring their old subscription's renewals with them, which skews every money number, so enroll them only on purpose.

Assignment

A device is bucketed by sha256("<experiment_id>:<bucket_key>"), where bucket_key is whichever identifier arrives first — device ID, RevenueCat ID, or account ID. Weighting is relative, not literal percentages: two arms at 50/50 behave identically to two arms at 30/30.

Assignment is keyed on the device, not the person

This is deliberate — it's what makes the very first launch stable, before any event has created a user record for that person yet. The tradeoff: the same person on two devices can see different variants until identity merge unifies them. Analysis still attributes them to whichever variant they hit first, so the numbers stay correct even when the serving experience briefly diverges.

Before the split, three gates run in order: enrollment (a new-only experiment skips anyone who existed before it started), targeting rules (malformed rules fail closed — an evaluation error means not enrolled, never a guess), and audience % (hashed in its own separate domain, so widening from 50% to 100% only adds users and never reshuffles who's already assigned).

One test per user, in priority order

Like RevenueCat, a user is in at most one experiment at a time. Running experiments are tried in priority order (lower first, then the older test). A user joins the first one whose gates they pass and no other; someone who fails a test's targeting simply falls through to the next. Users already assigned keep their arm. Experiments that belong to a campaign audience sit outside this order.

Pausing

Pause stops new users from joining while everyone already in the test keeps their arm, so results keep maturing (trials convert, renewals land). Resume opens enrollment again. A paused test never ships itself. Stop is final: everyone goes back to the live config; use Run again to start a copy.

Holdouts

One arm can be marked a holdout — it serves the untouched live config while still recording an assignment, which is what makes "did the new paywall actually beat doing nothing" measurable rather than assumed. A holdout still counts toward the exposure floor before a decision can be made.

Backfill

A device can be served a variant before it has a user record to attach an assignment to — the SDK stamps the experiment and variant IDs on the exposure event itself, and a backfill pass reconciles it once the user record exists. This is what keeps a brand-new install's assignment from being lost.

Turning it off

Resolution is skipped entirely — no assignment, no config change — when the ab_testing tool is off, checked both server-side and by the SDK. Previewing your own paywall never counts as an exposure.

On this page