Per-link conversion experiments for creators

The brief

Stan creators range from a designer selling a $15 e-book to a coach selling $5,000 cohorts. They share one obsession: conversion. They are not, however, growth engineers. The tooling that growth teams at SaaS companies use to A/B test product pages is wildly out of reach for someone running a storefront from their phone between content recordings.

The bet: build experimentation that is one tap to start, one tap to ship the winner, and let every Stan creator operate with the same conversion superpowers as a small e-commerce team.

What we built

Per-product, per-page A/B experiments with two-tap setup inside the Stan creator app. Variant routing happens at the edge — every visitor gets assigned to a stable variant inside the same response that serves the page, adding less than 5 milliseconds of latency. The analytics pipeline lands in ClickHouse and surfaces results to creators in near-real-time with conservative statistical significance reporting.

The unlock that mattered: once a winning variant emerges with enough data, a single tap promotes it to be the default and archives the losing one. Creators do not need to understand p-values. The product does that work for them.

We built a guardrails layer too — minimum sample sizes, multiple-testing correction, and a hard limit on simultaneous experiments per store so creators do not shred their conversion data by testing six things at once.

Outcome

Across opted-in stores, median click-to-purchase conversion lifted ~12% within 30 days of running their first experiment. The system runs 40,000+ active experiments a week at the edge, with the latency overhead never materializing as a customer-experience cost.

Setup time, end to end, sits under 30 seconds. The feature is now a default-on part of every new Stan creator's onboarding.

"Conversion experiments used to be a thing only big SaaS companies could afford. Now every creator on Stan has one."