Evaluating the Impact of New Traffic Sources through A/B Experimentation
Problem:
A digital product needs to evaluate whether adding new traffic sources improves conversion-funnel performance and, in particular, the key payment event. The goal is to make an informed decision on whether to scale these sources without compromising profitability or user experience.
What do I do in this project?
– Build and analyze the event funnel
– Validate the experimental design using A/A
– Run A/B across multiple funnel events
– Rigorous statistical evaluation (including multiple comparisons)
– Interpret findings and final recommendation
Value added:
I don’t just report metrics: I translate statistical results into actionable decisions (when to scale an initiative, when to stop it, and what additional changes could create more impact). This reduces the risk of investing resources without proven benefit and strengthens evidence-based decision-making.
Key metrics:
- Evaluation of over 4,900 users distributed in three experimental groups.
- Behavior analysis across 5 key events conversion funnel events.
- Observed differences were not statistically significant at α = 0.05 nor under the multiple-testing adjusted level.