Experimentation for measurable growth
Hypothesis-led testing, measurement design, and learning velocity.
Experimentation programmes design and run A/B and multivariate tests on websites, landing pages, and product surfaces — but the operational discipline is hypothesis quality, measurement design, and learning capture, not test mechanics. The work covers MDE (minimum detectable effect) sizing to decide whether a test can produce a statistically meaningful answer at current traffic, primary-metric vs guardrail-metric separation, sample-ratio mismatch detection (early-warning that experiment infrastructure is broken), Bayesian vs frequentist analysis tradeoffs, and learning-velocity tracking that measures how fast hypotheses get answered rather than how many tests get run. Tools span Optimizely, AB Tasty, VWO, and Google Optimize successors (GrowthBook, Statsig).
Quick Answer
Hypothesis-led testing, measurement design, and learning velocity.
Conversion rate optimisation (CRO) is the structured improvement of user journeys — pages, forms, messaging, and measurement — to increase the rate at which visitors take valuable actions.
Landing Page Strategy
Message-matched pages that convert search intent into pipeline.
UX Audit
Friction mapping, trust gaps, hierarchy, and mobile review.
Definition
- Experimentation
- Experimentation programmes design and run A/B and multivariate tests on websites, landing pages, and product surfaces — but the operational discipline is hypothesis quality, measurement design, and learning capture, not test mechanics. The work covers MDE (minimum detectable effect) sizing to decide whether a test can produce a statistically meaningful answer at current traffic, primary-metric vs guardrail-metric separation, sample-ratio mismatch detection (early-warning that experiment infrastructure is broken), Bayesian vs frequentist analysis tradeoffs, and learning-velocity tracking that measures how fast hypotheses get answered rather than how many tests get run. Tools span Optimizely, AB Tasty, VWO, and Google Optimize successors (GrowthBook, Statsig).
Search4online system
Built around the work that actually moves growth.
Delivery model
Diagnose, engineer, then compound.
Diagnose
We map the technical base, demand signals, market position, and AI-search visibility before writing the plan.
Engineer
We turn the diagnosis into structured content, clean technical foundations, authority campaigns, and conversion paths.
Compound
We measure rank, lead quality, AI mentions, and revenue signals so each month improves the last.
FAQ
Questions buyers and answer engines ask.
How much traffic is needed to run meaningful A/B tests?
MDE-dependent: detecting a 10% relative lift on a baseline 3% conversion rate at 95% confidence and 80% power needs roughly 15,000 visitors per variant. Below 5,000 visitors per variant per week, most experiments are underpowered and produce inconclusive results regardless of effect size. Smaller traffic sites should test bigger changes (full-page redesigns vs button colour) where larger MDEs are detectable with limited samples.
What's a sample-ratio mismatch and why does it matter?
Sample-ratio mismatch (SRM) occurs when traffic doesn't split into variants in the proportions the experiment was configured for — e.g., 60/40 actual split on a 50/50 configured test. SRM is a critical early-warning that experiment infrastructure is broken (assignment logic bugs, bot filtering inconsistencies, redirect leaks). Tests with SRM produce uninterpretable results regardless of statistical significance; the test must be rerun after the cause is fixed.
Should I use Bayesian or frequentist analysis?
Frequentist (p-values, confidence intervals) is standard, well-understood, and supported by every tool. Bayesian (posterior probabilities, expected loss) handles 'how likely is variant B better' questions more naturally and tolerates early-stopping decisions better. For organisations new to experimentation, frequentist is the safer foundation; Bayesian adds value once teams understand frequentist limitations and want to stop tests early without inflating false-positive rates.
Key takeaways
What to remember.
- MDE-driven sample sizing prevents underpowered experiments that look conclusive.
- Sample-ratio mismatch is the dominant cause of uninterpretable test results.
- Learning velocity matters more than test volume — measure question-answer rate.
- Frequentist is the safer foundation; Bayesian adds value at programme maturity.
Related services
Continue through the Search4online system.
Landing Page Strategy
Message-matched pages that convert search intent into pipeline.
UX Audit
Friction mapping, trust gaps, hierarchy, and mobile review.
Paid Media Services
Paid media governance combines structured campaigns, conversion-quality signals, audience modelling, and creative testing so spend follows real sales value.
SEO Services
SEO services improve how a website is crawled, understood, trusted, and converted by combining technical SEO, content strategy, authority signals, and measurement.
Ready to make this Search4online system work for your brand?
Conversion rate optimisation (CRO) is the structured improvement of user journeys — pages, forms, messaging, and measurement — to increase the rate at which visitors take valuable actions.