For years, A/B testing has been the backbone of website optimisation. The logic is simple: show half your visitors version A, half version B, and let the numbers decide which performs better.

It feels scientific and certain - and for companies with millions of visitors, it can work.

But for most businesses, A/B testing is painfully slow, requires too much traffic, and wastes valuable visitors on losing versions. Meanwhile, the world around it has changed - dramatically.

The Promise of A/B Testing

Marketers embraced A/B testing because it seemed like the perfect solution.

• Clear: run a test, declare a winner.
• Data-driven: no gut feelings, just numbers.
• Proven: the Amazons and Bookings of the world built entire growth machines on top of it.

But what works at giant scale doesn’t translate to everyone else.

The Reality: Slow, Expensive, and Wasteful

Here’s what most teams discover when they try to run A/B tests:

• They take forever. To reach statistical significance, you need massive sample sizes. If your site gets 5,000–10,000 visits a month, a single test might run for months.
• They waste traffic. Half of your visitors are forced through a losing version until the test ends. That’s wasted revenue.
• They’re resource-heavy. Each test means new copy, new design, developer time, QA. 
• They're too simplistic. A/B tests track one metric while ignoring rich behavioral signals like scroll depth, engagement patterns, and click behavior that could provide faster insights into what's working.
• They treat everyone the same. The large sample sizes required force you to find one "average winner" rather than adapting to different visitor types and contexts.

And here’s the kicker: while your website is frozen in a months-long test, your ad platforms are running thousands of micro-tests every single day. Google and Meta automatically adjust bids, audiences, and creatives in real time — that’s why ads get smarter and more efficient over time.

But when those paid clicks finally land on your website, the optimisation stops. That’s where businesses bleed conversions.

What’s Changed: AI, Bandits, and Personalisation

The good news: we’re no longer stuck with the old way.

• AI has removed design and dev bottlenecks. In the past, optimisation was limited by resources as much as traffic. Every new test meant writing copy, designing creatives, coding changes, shipping updates. Today, well-engineered AI can generate, implement, and launch variations instantly. The limit is no longer human bandwidth.
• Machine learning unlocks behavioral data insights. Modern algorithms can use behavioral signals like engagement patterns and scroll behavior alongside conversions to identify winning variations faster and understand what drives performance.
• Multi-Armed Bandits eliminate waste. Instead of splitting traffic 50/50 until the bitter end, bandit algorithms dynamically reallocate visitors toward winning variations as the test runs. Every visitor contributes to learning, but more see the better option sooner.
• Personalisation breaks the “one winner” assumption. A/B testing assumes there’s a single best version for everyone. But is the best headline for a Google search click the same as for a Facebook ad click? Probably not. Personalisation adapts in real time so each visitor sees the version most likely to convert for them.

Continuous Optimisation in Practice

Here’s what this shift looks like in reality:

• You install one line of code.
• From the very next visit, your site is already experimenting and adapting.
• Headlines, buttons, layouts — all optimised automatically.
• Visitors from different sources see different experiences that match their intent.
  
  

And crucially, it’s still data-driven. The biggest promise of A/B testing was always measurement. But measurement doesn’t have to mean waiting.

With continuous optimisation, you’re not choosing between “scientific but slow” and “fast but fuzzy.” You’re combining both: constant change and constant measurement. Every visitor makes your website smarter, and you can see the impact in real time.

A Note on Personalisation

We’ll go deeper into this in a future post, but personalisation deserves a quick mention here.

One of the biggest flaws of A/B testing is that it treats all visitors the same. Yet the truth is obvious: people behave differently depending on where they came from, what device they’re on, or whether they’ve seen you before.

A single “winning” version often isn’t truly winning - it’s just the average. Personalisation changes the math. By adapting to context, you don’t need giant sample sizes to find a universal winner. Instead, you serve each visitor the experience that fits them best.

Recommendation engines solve a similar challenge with radically different methods. When Netflix suggests your next show or Amazon recommends products, they're working with sample sizes of one - your individual behavior. They use dozens of signals (viewing history, time spent, clicks, skips) to predict what you'll engage with, updating in real time. Your website optimization shouldn't be less sophisticated than your product recommendations

Why Self-Improving Websites Are the Future

A/B testing gave us the structure to think about optimisation scientifically. But the internet has moved on.

• AI removes the production bottlenecks.
• Bandit algorithms eliminate wasted traffic.
• Personalisation makes “one winner” obsolete.
• Continuous optimisation makes websites truly data-driven — in real time.

The result is a new category: self-improving websites.

Websites that test, learn, and adapt continuously.
Websites that personalise for every visitor.
Websites that grow conversions on their own.

That’s what Dalton makes possible.

‍