Dominion Akinrotimi
Back to Projects
PythonA/B testingStatisticsMarketing analytics

Did the Ads Work? A/B Test of a Marketing Campaign

The problem: a company showed ads to most users and a public service announcement to a control group. Did the ads actually lift purchases, or would those customers have bought anyway?

What I did: analysed 518,565 users with a two-proportion z-test, then looked at how conversion changes with the number of ads seen.

View the code on GitHub

Result

GroupConversion rate
Saw the ads2.81%
Saw the PSA (control)2.25%

That's a lift of 0.56 percentage points, about 25% relative, with z = 4.28 (p < 0.001). Users were randomly assigned to the groups, so the lift can be credited to the ads.

More ads, more sales?

Conversion rate rising with the number of ads seen

Conversion by number of ads seen, ad group only.

Read with care: conversion climbs from 0.4% (0 to 10 ads) to about 17% (100 to 200 ads), but exposure wasn't randomised. People who browse more both see more ads and buy more. This curve shows association, not the effect of adding ads.

Recommendations

Have data you cannot make sense of yet?

I turn raw data into dashboards, reports and decisions your team can act on.