From A/B testing to A-Z testing
Most CTV testing still relies on running one or two creative variants against a broad audience and waiting weeks for a readable result. That's fine when your audience is one segment. It falls apart when the audience you're actually trying to reach is dozens of segments, each responding to a different message, language cue, or cultural reference. This is where AI changes the math: instead of testing creative A against creative B and waiting to see which wins, you can test A through Z at once, across every audience segment, while the campaign is still live. This case study looks at what happened when a streaming brand made that shift, and validated the results independently before trusting them.
Philo needed to grow subscribers without slowing down to find out what worked
Philo, a live TV streaming service built around affordable entertainment, was competing for subscribers in a crowded field. The team wanted to grow both trial sign-ups and paid conversions, and to finally understand which creative approaches were actually resonating with which audiences, without waiting on traditional, sequential A/B tests to get there.
Real-time AI optimization, checked against a live control group
Claritas deployed AI Optimization (AIO) across Philo's :30 CTV campaigns, testing multiple creative and audience combinations simultaneously instead of one at a time. A control group ran with AI disabled so the lift could be measured directly. Philo then ran its own independent holdout group on top of that, and the two measurements agreed.
Philo landed a 13.5% lift in trial subscribers and a 66.7% lift in paid subscribers. Here's what's still locked: how.
The numbers above are real. What the full case study unlocks is the mechanics behind them: the audience-and-creative combinations the AI actually surfaced, how that lift held up against the AI-disabled control group, and the specific insights Philo is now using to plan its next round of testing.
- The audience-and-creative combinations that actually drove the 13.5% and 66.7% lift
- How that lift was measured against the AI-disabled control group, and confirmed by Philo's own holdout test
- What the AI surfaced about how audience response varies by creative message
- Where Philo is taking its testing strategy next
The real constraint on reaching multicultural audiences usually isn't insight, it's testing speed
Most multicultural marketers already know that creative resonance shifts by audience: language, cultural cues, and even pacing can change whether a message lands. The hard part has always been proving it fast enough, and at enough scale, to act on it before a media flight ends. This is the practical case for moving from A/B testing to A-Z testing: instead of proving out one segment at a time, AI lets you test every audience-and-creative combination at once and surfaces the winners while there's still time to act on them. What this case study shows is a model for doing exactly that, with the result checked against an independent control so it's trustworthy, not just fast. That's the piece that tends to be missing when "test more segments" turns into an operational bottleneck.
About This Case Study
"Philo Turns AI Into Subscriber Growth" is a Claritas case study documenting a live CTV deployment of Claritas AI Optimization (AIO), independently validated by the client's own holdout methodology, part of Claritas's ongoing work helping brands reach audiences authentically and at scale.