CRO OS is being built to separate three questions most experiment reports blur together: was the test delivered correctly, what effect is supported by the data, and what should the business do next? The first commercial job is simpler: bring an existing experiment and find out whether its evidence is fit to interpret.
Missing assignment/exposure evidence blocks a supported-effect claim.
Effect is not shown as decision-grade until the input contract is satisfied.
Adopt / reject / continue / investigate stays separate from statistical output.
A test can run for weeks, report a lift and still be operationally invalid: the split was wrong, some visitors crossed arms, outcomes disappeared on one side, the assignment unit changed, or exposure was never actually verified.
The configured allocation is not necessarily the allocation visitors actually experienced.
Missing, duplicated or late events can move the reported result while leaving a clean-looking chart.
A low p-value alone does not tell the business whether the result is meaningful, mature or safe to ship.
CRO OS starts one step earlier than “which version won?” The first question is whether the experiment was delivered and measured well enough to interpret at all.
Hypothesis, eligible population, randomization unit, allocation, baseline, minimum worthwhile effect, sample-size calculation, outcome window, missing-data rules, guardrails and decision owner.
Those are different events. CRO OS is designed not to treat “assigned to B” as proof that B was delivered or seen.
Allocation mismatch, duplicate assignment, cross-arm exposure, event loss, cache leakage and outcome maturity are diagnostics, not afterthoughts.
The data snapshot hash, analysis version, plan, effect estimate, uncertainty, guardrails and owner decision stay linked for review.
Tests whether observed allocation is consistent with the configured split for the initial two-arm case.
A versioned deterministic assignment foundation exists for the native engine.
The design decides whether the visitor is eligible before randomization; later consent does not silently reshape the denominator.
The null experiment path exists so the machinery can be tested before real conversion decisions depend on it.
Inherited enterprise primitives provide isolation and governance, but still require CRO-specific product verification.
The first external customer job: bring allocation, exposure and outcome data and expose imbalance, duplicates, cross-arm delivery and missing outcomes.
Integrity state, effect estimate with interval, practical relevance and owner decision stay separate.
Traffic feasibility, minimum worthwhile effect, sample size, stopping rule and missing-data policy are locked before interpretation.
Serving variants arrives only after integrity review, Gate B safety work and A/A validation are ready.
Enterprise metric semantics, guardrails, rollout flags and broader experimentation depth remain roadmap work.
Passed · Failed · Unverified.
Incomplete allocation, exposure or outcome data can block a supported-effect claim.
Effect size and uncertainty at the registered horizon, with practical size shown alongside statistical evidence.
Adopt, reject, continue or investigate, recorded separately so a statistical result never impersonates a business judgment.
No automatic winners. No repeated significance peeking in the initial engine. No guarantee of conversion lift. A well-run experiment can be inconclusive and still be useful.
Owns allocation rules, metric definitions, integrity interpretation and the reproducible readout.
Provides independent browser evidence when a variant damages a protected journey such as signup, login or checkout.
When both are installed, the reviewed consent state can determine whether a persistent assignment method is permitted. Neither product requires the other.
Shared engine: evidence, approvals, release identity, incidents, tenancy, audit and recovery are inherited family primitives. CRO-specific assignment and statistics remain owned by CRO OS.
Against the narrower v4 CRO gates, the product is about 80% built. The hard foundations exist, but CRO OS does not yet serve native variants, import and reconcile a real external experiment, or produce a complete supported-effect readout.
Best fit: a web team or agency already running experiments and willing to supply the experiment definition plus the allocation, exposure and outcome data available from the current platform.
It creates customer value before CRO OS serves a single variant. It also forces the integrity engine to prove itself against messy real-world experiment data before native delivery raises the blast radius.
A test with incomplete data can remain unverified. The review will not manufacture an effect estimate just to make the report feel complete.
No. Native variant serving and the complete readout are not finished. The first external product is an experiment-integrity review of an existing test.
Sample-ratio mismatch is treated as a diagnostic that requires investigation. CRO OS should not claim a specific root cause from the mismatch alone.
No. Integrity, effect estimate and business decision remain separate. A low p-value alone is not a business decision.
Because first-party, pseudonymous or cookieless does not automatically mean permitted. The experiment must use an assignment method compatible with the reviewed privacy policy.
Imported-experiment integrity proof, Gate B safety tests, deterministic assignment tests, analytical reference cases, repeated null simulations, and a meaningful A/A run before a powered customer experiment.
CRO OS is early by design. The fastest way to make it useful is to expose bad experiment evidence before native variant delivery adds more complexity.