All Segments
![[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments](https://img.youtube.com/vi/0APJlxMjPw4/mqdefault.jpg)
[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments
26 segments available
If at first you don't succeed, test test again! Brian Balfour, VP of Growth at HubSpot, shares his framework for tracking & analyzing rapid experiments. See the full slide deck here: http://s.500.co/Wk7n302WsFg Want MOAR growth? Attend WMD 2016 conference in San Francisco on Nov. 4, 2016 http://wmd.co Ready to grow even more? Apply for our next accelerator round today: http://s.500.co/vVRv302Wt4H -~-~~-~~~-~~-~- Please watch: "Growing with Pinterest - Pinterest Marketing for Startups" ➨ https://www.youtube.com/watch?v=spjaZRokMVo -~-~~-~~~-~~-~-
Segments Timeline
Building on Foundations
Brian Balfour introduces his framework for growth experiments, emphasizing that his approach builds on previous insights shared by James. He highlights that the principles discussed apply not only to viral experiments but also to various growth strategies at HubSpot, including content and retention experiments.
"see here um so uh a little embarrassing the first part of my deck actually uh repeats a lot of what James said uh which totally was not on purpose um but uh it's actually really good because uh uh wha..."
The Tactics vs. Process Debate
Balfour addresses a common question he receives about growth tactics, stating that he cannot provide a specific answer without understanding the company and its audience. He emphasizes that successful growth is less about tactics and more about establishing a solid process tailored to each unique business.
"virality experiments on some of our new products but content experiments Facebook ads experiments things that we work on with retention everything that we do um so uh similar James the the number one ..."
Four Reasons to Focus on Process
In this segment, Balfour outlines four key reasons why focusing on process is crucial before diving into tactics. He explains that what works for others may not work for you, and stresses the importance of having a method to discover what will be effective for your specific business context.
"met you I know nothing about your company I know nothing about your audience I know nothing about your product and typically the reason I get this question is because it comes from a sense of they're ..."
The Sum of Small Parts
Balfour discusses how growth often results from many small actions rather than a single breakthrough. He encourages listeners to analyze the incremental steps that lead to significant growth, reinforcing that chasing quick fixes is often unproductive.
"first reason is a lot of what James says what works for others is not going to work for you your product your audience is different your product is different your business model is different your cust..."
Adapting to Rapid Changes
Balfour highlights the accelerating rate of change in acquisition channels and the necessity for businesses to adapt continuously. He provides examples of how SEO and social media landscapes are evolving, emphasizing the need for a flexible approach to growth strategies.
"a lot of cases ends up being the sum of a lot of small parts I'm sure a lot of us every week are probably reading Tech crunch Venture beat one of these Tech uh Publications and we see some company wit..."
Creating a Scalable Growth Machine
In this segment, Balfour explains the concept of building a scalable growth machine. He defines a successful growth process as one that is predictable and repeatable, which ultimately produces effective tactics for business growth.
"you will certainly have experiments and tactics that you do that will end up being outliers but chasing silver bullets is kind of like chasing unicorns it's kind of a waste of your time you want to be..."
Goals of the Growth Process
Balfour outlines the four primary goals of the growth process at HubSpot: establishing a rhythm of experimentation, fostering continuous learning, and ensuring autonomy and accountability within the growth team. He emphasizes the importance of these elements in driving successful growth initiatives.
"introducing content freshness social sharing they're massively Google's massively changing the search result format display continues to be disrupted by rtb uh browsers are taken away cookies and of c..."
Setting Effective Growth Objectives
Balfour introduces the OKR framework for setting growth objectives, explaining how to identify impactful areas for improvement. He discusses the importance of time frames and measurable key results to ensure that growth goals are both ambitious and achievable.
"finding the new things uh that are going to continue to accelerate the growth of your business and the fourth reason about process first uh before tactics is that what you're really trying to go for i..."
Measuring Success with Key Results
In this segment, Balfour elaborates on how to set key results that measure the success of growth objectives. He categorizes these results by difficulty and explains the importance of hitting these targets to gauge the effectiveness of growth strategies.
"you want to establish a NeverEnding uh regular Cadence of experimentation to basically fight through the failures to find the successes and really find that momentum that keeps you keeps carrying you ..."
The Importance of Iteration
Balfour emphasizes the need for rapid iteration in the growth process. He discusses how teams should quickly move from setting objectives to executing experiments, ensuring that they learn and adapt based on the outcomes of their efforts.
"I'm not providing as as the team leader I'm not providing a very specific set of directions but with autonomy comes accountability and so uh um people on the growth team they don't have to be right al..."
Identifying High-Impact Growth Areas
Brian Balfour emphasizes the importance of identifying the most impactful area for growth before diving into experiments. He discusses setting a clear timeframe of 30 to 90 days for achieving objectives and establishing three key results (KRs) that measure success. This segment highlights the strategic approach to prioritizing growth initiatives.
"the question what is the one area that we can work on or the one thing that we can achieve that drives the biggest impact on our growth curve right now um and taking this step back and really understa..."
Setting Key Results for Success
In this segment, Balfour explains how to set key results (KRs) in order of difficulty to gauge success in growth experiments. He outlines the expectations for achieving KRs, emphasizing the balance between ambition and realism in goal-setting. This approach ensures that teams remain motivated while pushing for significant growth.
"basically uh in order of difficulty and so kr1 is basically like hey we did a good job this isn't sandbagging it this is basically we did a really good job we tend to hit our K1 or we we try to set th..."
The Experimentation Cycle
Brian Balfour introduces the experimentation cycle that follows the setting of objectives and key results. He outlines the steps involved: brainstorming, prioritizing, designing tests, implementing, analyzing, and systemizing successful tests. This structured approach is crucial for continuous growth and learning.
"happy medium so after we T sort of take this step back and and really think about what is the most impactful thing that we can do and basically set some goals against it we really try to dive into thi..."
Essential Documents for Growth Experiments
Balfour discusses four key documents that facilitate the growth experimentation process: a backlog for idea collection, a pipeline for tracking experiments, experiment docs for detailed planning, and playbooks for systemizing successful strategies. These documents ensure transparency and collaboration within the team.
"macro level so the steps the the constant steps in the cycle that we go through after we set these okrs is first as we brainstorm uh then we'll prioritize then we'll design tests around the things tha..."
Brainstorming Growth Ideas
In this segment, Balfour emphasizes the importance of brainstorming inputs rather than outputs when generating growth ideas. He suggests breaking down complex processes into smaller components to facilitate targeted brainstorming, which leads to more actionable insights and improvements.
"we can come back to later the second is the pipeline this is basically a ledger of experiments it's all of the experiments that we've ran previously as well as the ones that are currently running and ..."
Four Techniques for Generating Growth Ideas
Balfour shares four techniques for generating growth ideas: observing competitors, questioning assumptions, associating unrelated concepts, and leveraging networks. He highlights the value of inspiration over prescription, encouraging teams to adapt successful strategies from various sources to their unique context.
"the entire process and we'll we'll take a look at one specifically and the fourth is playbooks so basically anything that we find that's successful we first try to productize it with software and Engi..."
Networking for Growth Insights
In this final segment, Balfour discusses the importance of networking with other growth professionals. He shares his experiences of exchanging ideas and strategies with peers, emphasizing that collaboration and communication are vital for discovering what works and what doesn't in the growth landscape.
"output and so there's really four ways uh that we use to generate growth ideas and um to be honest I blatantly stole this from a book called the innovator solution I recommend everybody to read it but..."
Networking for Growth Insights
Balfour emphasizes the value of networking with other growth professionals. He shares how exchanging ideas and experiences with peers can provide inspiration and practical insights that feed into the growth experimentation process, enhancing the overall effectiveness of strategies.
"the phone probably almost every day with a lot of people who are going to be here at this conference today and exchanging ideas talking about the things that we're trying what worked what didn't work ..."
Prioritizing Growth Experiments
This segment focuses on the critical process of prioritizing growth experiments. Balfour outlines how to assess ideas based on their probability of success, potential impact, and resource requirements. He stresses the importance of being honest in evaluations to ensure effective experimentation.
"important to find a way and force yourself to be brutally honest uh on these three elements and so we look at our top ideas in the backlog and we always compare them on three uh components the probabi..."
Formulating Hypotheses for Experiments
Balfour discusses the necessity of formulating clear hypotheses for each experiment. He explains how to structure hypotheses to reveal underlying assumptions and the importance of using data to support predictions. This approach helps teams clarify their objectives and expectations.
"places that we look to justify our assumptions are quantitative qualitative and secondary data quantitative data we typically get from all of the data collection around our product um we'll look at a ..."
Designing Minimum Viable Tests
In this segment, Balfour introduces the concept of minimum viable tests as a way to validate hypotheses without extensive resource investment. He provides an example of optimizing a referral program through simple changes to a landing page, demonstrating how quick tests can yield valuable insights.
"out of thin air and so we always try to prioritize once again against these three things and a lot of times um if you know if we're weighing between things that have very you know a few things that ha..."
Creating Effective Experiment Documentation
Balfour highlights the importance of documenting experiments thoroughly. He explains how to outline objectives, hypotheses, and assumptions in an experiment document to facilitate understanding and replication. This practice helps teams avoid common pitfalls and ensures valid results.
"do we expect out of this one compared to this one how do we feel about these assumptions it allows other people on the team allows me to challenge the person's assumptions and hopefully get more and m..."
Extracting Learnings for Future Growth
Balfour discusses how to extract actionable insights from experiments, including identifying successful variations and integrating them into future processes. He stresses the importance of continuously refining predictions based on recent learnings to enhance growth strategies.
"or why did it fail why were we close or way off of our uh of of our hypothesis digging into the question why will help you once again go back and start understanding things about your customer your ch..."
Systemizing Successful Strategies
This segment focuses on the need to systemize successful strategies through technology and playbooks. Balfour explains how documenting processes helps scale teams effectively, ensuring that knowledge gained from experiments is retained and easily accessible.
"of your next ones and and adjust your predictions based off of your most recent learnings and then of course step six is systemize so once you find things that are successful we first once again we tr..."
The Experimentation Cycle
Balfour outlines the weekly cycle of experimentation, including team meetings to share learnings and review progress against objectives. He emphasizes the importance of continuously running experiments and adjusting strategies based on insights gained to drive growth.
"knowledge and so what a typical week looks like for one of our teams is that on Monday we have a 60 to 90 minute meeting it is the only meeting we have all week the things that we go through is we sta..."
Quarterly Optimization Review
In this concluding segment, Balfour discusses the importance of quarterly reviews to assess the effectiveness of the experimentation process. He highlights key metrics such as success rates, hypothesis accuracy, and throughput, advocating for ongoing optimization of processes and tools.
"possible and so after you go through a few of these Cycles we'll take a step back we'll look at the macro optimization and we'll look at three things we'll look at our batting average so how many succ..."