All Segments
The million-dollar question!
2 segments available
You can listen to the full pod with Ronny Kohavi here: https://www.youtube.com/watch?v=hEzpiDuYFoE #startup #founder #softwareengineer #sofwaredeveloper #vctok when to a/b test, a/b testing for startups, a/b testing basics
Segments Timeline
When to Start A/B Testing
In this segment, Ronny Kohavi addresses a common question among startups: when should they begin A/B testing? He suggests a heuristic of starting to build a testing culture and platform when reaching around 200,000 users, emphasizing the importance of having at least tens of thousands of users to ensure statistical validity in testing. Kohavi advises startups to focus on detecting significant changes, ideally around five percent, rather than minor adjustments.
"and this is a constant question for startups is when should we start running a b tests do you have a heuristic or rule of thumb up here is a time you should really start thinking about the million dol..."
The Magic Number: 200,000 Users
Ronny Kohavi elaborates on the significance of reaching 200,000 users for effective A/B testing. He explains that at this scale, startups can start testing more rigorously and ensure they are not degrading their user experience while extracting valuable insights. This segment highlights the critical threshold for startups to transition from foundational testing culture to advanced experimentation.
"unless you have at least tens of thousands of users the math the statistics just don't run out for most of the metrics that you're interested in startups should focus on trying to detect changes that ..."