Most companies say they’re data-driven. Few have the infrastructure to actually be. At Preply, I led the team building our experimentation platform — the system that lets product teams run rigorous A/B tests without needing a data scientist in the room for every decision.
The hardest part wasn’t the statistics. It was the organisational work: getting teams to trust the platform, defining what “statistical significance” actually meant in our context, and building self-serve tooling that didn’t sacrifice rigour for convenience.
One of the most impactful things we shipped was a product-level holdout group — a small percentage of users who never see any product changes, giving us a clean causal measurement of how much value the product org is generating. That single feature changed how leadership thought about product investment.
The technical architecture matters, but the cultural architecture matters more. An experimentation platform that nobody uses is just infrastructure debt with good documentation.