All Segments
The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
2 segments available
The full pod with Ronny Kohavi drops tomorrow, Thursday, 27 July! #startup #founder #FAANG #productmanager #softwareengineer best tech podcasts, big tech podcast, a/b test tips, how to do a/b tests, a/b testing and experimentation
Segments Timeline
The Power of A/B Testing
Ronny Kohavi highlights the immense value of A/B testing, sharing a success story where a tiny change generated a hundred million dollars in revenue for Bing. He emphasizes that while ideas may seem promising, the true test lies in controlled experiments, which serve as the ultimate arbiter of success.
"this thing was worth a hundred million dollars a tiny change that was the best Revenue generating idea in Bing's history former vice president from Microsoft and B helped Airbnb reach 2.24 0 in Revenu..."
Expecting Failure in Experiments
Kohavi discusses the reality of A/B testing, revealing that a significant percentage of ideas fail. He cites statistics from Microsoft and Amazon, where about 66% and 85% of ideas, respectively, did not succeed. This segment prepares listeners for the likelihood of failure while underscoring the potential for breakthroughs when experiments do succeed.
"be excited about things you might believe it's a good idea but ultimately the Arbiter the Oracle is the controlled experiment what should people expect when they're running experiments what percentage..."