today we have Dan Hakan Meyer and Gustav all strimer so Dan was the founder advise investor and advisor at basis one which is growth strategy consulting and previous to that you were a director of growth marketing at thumbtack and Gustav's a partner at YC and also was a
product lead for growth that Airbnb so Gustav why don't you start it off sure so I was thinking about getting it right into it so Dan you advice a lot of startup on growth and you kind of get into there they're the teams and work directly with them on growth what is the
most unpopular advice that you give the founders of the companies that you work with so I say nine times out of ten we are talking them out of doing things that they want to do I think you know as you know there's this huge power law to growth I think most the gains come from
you know one channel a handful of tactics and most early teams are really good at coming up with good ideas that they want to try and an often that has diminishing returns and so we're often talking them out of things I think the mike second class of things would be
actually that the company may not be ready for a traditional growth effort and so I think it's really important to think about where they are in terms of product market fit and when the kind of the right types of tactics should be a should be used this is yeah in terms of
being specific around that like what are the most common things that people are trying to jump on to soon so I think really heavyweight acquisition particularly if it's unsustainable prior to having the right kind of retention and so like you don't have to have
best-in-class retention numbers but you need to be seeing some cohorts of people for which retention truly levels out and they use the product very long term and if you don't see that you're you're not ready and particularly if you're doing
things like paid marketing or others which can have big burn implications you can get to that - too fast that could be really problematic so let's say I run a series B company it's a consumer company and I just hired you to help me figure things out what are the questions are
you going to ask me and whether the data you want from me so I think it's a really healthy exercise to actually start with a pretty traditional spreadsheet model you know what are the kind of channels of acquisition how does how does traffic convert activate retain
what your different cohorts of retention look like and how does all this link up now how to like you're existing users then start referring other users how does monetization fuel your paid growth strategy and when you build something like this you can like play
around and say what happens if our activation rate goes up by 5% usually that's really powerful much more powerful than a 20% increase next acquisition and so I think about think about like the kind of full stack of things and how it all links together and it's a really good starting point for
where there's leverage so if I am that founder and I couldn't hire you I can afford you're like who is the person in my team that should kind of make that spreadsheet like yeah like where do I start if I don't hire you so I'm a big fan of smart analytical generalist
actually and it depends a little bit on on you know whether you need more of a marketing business type person or more of a technical person but somebody who one has kind of low pride of ownership and just wants to move fast and test things and and figure out what works and
is deeply analytical can often get a lot of that a lot of that work done I mean I think you know we can talk about the composition of a like large scale growth team but that's a luxury that many don't have and so finding somebody who can do lots of those things before you know
what's really gonna work is really valuable and so what signals during an interview would you look for so I think the big thing is is can they hold the equation of your business in their head so like do they know how a change is going to impact the overall output of
what what your business is optimizing for whether that's weekly active users or transactions in a marketplace and do they have an intuitive sense for where the kind of important things to work on are and then the other type of stuff like learning channels you know most
people can get up to speed on that pretty quickly it's much harder to teach that kind of like strategic kind of understanding of where to focus mmm did you find that true at everybody as well yeah I would say so that's probably true okay I'll be curious if you have any
kind of counter points or areas where you needed more specialty earlier on so I would say that the story of everybody was basically we had probably one of the best part Market fits that I've seen but not necessarily thanks to you like an amazing user flow and amazing onboarding
or even amazing exhibition channel it's just like a really good experience so if you think about the early days of every me half the price the most unique experiences that you can imagine was actually very compelling and you can like think about the neighborhood it's
like if you want to live in New York your options were to live in a hotel in midtown versus living somewhere in a real neighborhood no there's there are real people and you can go down to the corner and and grab a coffee just like normal citizens that is a great kind of
like product experience and it sort of sells itself and I give you an early example like before we had we call instant book the way that you would book in there baby is that you would send her a quest to someone and that person have to then request a book and that person
has it and confirm that booking before you actually got the booking in many cases actually the majority of cases in the other days the request was not confirmed so you want to book something and sorry it didn't work out and that was her like if you think of the idea of
Airbnb which is book any house or room in the world as he says booking hotel wasn't really true so thing of that kind of product experience there's a lot of friction in the early days that sort of sort of didn't it mattered but it didn't also didn't really matter for this
success of the company and when I joined MB we were like wow this is really working but what are all the things that we can optimize to be optimized sort of like that I'm boarding flow we optimize all the different oxygen channels SEO paid marketing
referrals we've tried looking at retention all of those different things but fundamentally what it worked for me was this is a great product experience and we just want to make it more available to the world right so this is a key question and then perhaps a
dichotomy between growth and product right like the classic YC devices build something people want it's like where do you guys fall in this debate like how much of it is just building a great product and how much of it is like being great at growth so I think like that may
be the better way to frame this is not what which is more important because we can debate that but like I think rather obviously both play it play a role long-term it's more how the two are tied to one another and so I think like you think in particular on the consumer side
but it's true for b2b as well to get to 100 million in revenue typically 70 80 % of all new customer acquisition is coming from one channel and so like the best products are the ones that are built to optimize for that channel so if it's paid marketing you need to monetize
quickly and deeply if it's SEO you to have a really good way to generate and curate content or else you just can't compete because the other products are doing that and so if you know I get a lot of questions around like how do we kind of like add some SEO on top of this
product reality and it's just that's just never the answer because you need to be asking that question two years ago you know a year ago and so you can build for it and so I think that piece is really important I do think there are examples where a product is so unique
that it allows them to compete on kind of like a different vector you know and so like like zooms s1 that everybody's looking at I mean like the metrics are incredible and one of the reasons is they like came in the back door basically they have like a product that
people love and becomes viral and they're already pre-qualified before sales gets to them and everybody all their competitors are knocking on the front door and it's like much less efficient way and so that was a case where the product let them do something
different on the distribution side that nobody else could do like square being able to go out for SMBs with a automated underwriting products is a similar example and so like yes product like blew those markets wide open but it's still not the same as like saying you
don't have to go build the distribution like I'm still you still need to do the work on the growth side I would say yeah we have this a YouTube video when I talk about growth from startups cool and there's the first couple slides to talk about so like the evolution of a
Facebook and their growth I would actually argue basically that a bad product will not grow a good product will go to some troll like general natural adoption curve and that curve could be fast in the beginning but eventually is slowed down a lot and the role of the growth team for a good
product is to make sure that doesn't slow down kind of built on top of multiple layers of new of new channels or new onboarding or like a better conversion and they were all the growth team it's just like identify all those things and all those opportunities and
just do it mm-hmm so I agree with Dan like a great product is going to have it as sense of natural adoption it's just going to go not towards its full potential so so there's really a lot of things that great pride can do and I use a lot of great products on my my phone my
computer and I know that they're not sort of the we should their potential in terms of adoption because many basic things are not optimized I'll give you a very concrete example so at MMB we have this team called authentication mmm they were working on sign up and log
in very basic things a lot of startup founders things that those are just two things to check out the box I build sign up in a big the log in and then I am done it is actually incredibly difficult to make a hundred percent perfect disciple logging experience
which meant that he mixes for several years and because the people who are signing up and log me into air maybe had a very clear goal like booking or leaving a review or add something along those lines and were actually very very valuable for us to fail on that was
complete like so like that was like not acceptable yeah so we just kept optimizing and optimizing optimizing and I'll give you a specific example in the early days of MB we used to automatically log you out after like I think two days and we were like why we
do that well security reasons that makes sense so then we would run an experiment we we set the kind of expiration date of that cookie to seven days and then 30 days and then unlimited and I think we lifted that experience the alchemy experience over 1% increase in revenue for the
whole company just by extending the session length and make me a little bit easier to log in and the security aspects that we actually solved in a different way we made it kind of like Amazon where if you want to do something sensitive of the site you'd have to go
and log in again or author if you were gonna book then you have to put in your past yeah but that's just like the amount of code required now to make a change was like minutes or hours but the impact on the company was crazy yeah but I think what that tells the story is
that there are these things that that that really makes it easier for user to connect access that really good product but as a founder you don't really think about it unless you actually make the spreadsheet actually know what's happening the product and go deep on it
mmm-hmm yeah absolutely do ya do you think Facebook recognized that they weren't hitting their full potential when they started their growth team it's good question I wasn't there I think I think probably they did understand that they had this amazing thing you know that and you start to
realize that if you accelerate growth in very small percentages you get something really really good I actually think like the legacy of that growth team and then like Twitter and LinkedIn and the others that followed is that she may be problematic when we think
about some of the applications of growth later on because they were applied in this case of like extreme product market fit and the things you do in that scenario are slightly different than when your earlier State and trying to figure out how how it works and so I
think like out of that we got a little bit of this kind of growth hacking mentality just like throw things at the wall and see what fits and don't think about product market fit because it's assumed that that is solved for which in many businesses it's not and so so I
think like we have been recovering from that legacy in growth for a while and that just like in a practical terms means like just don't throw gasoline on every fire like what do you mean by that specifically yeah I think it means think about the thing about the metrics you're
optimizing for making sure they're long term making sure they're sustainable I also think like one of the mistakes that companies make a lot is this this idea that speed of testing or velocity of testing is the most important thing it is important to move fast because it is
quite hard to have good hypotheses about what's going to work like I am constantly Pro honk try to predict the results of experiments and so you do need to just test a lot of things however if you just get if you optimize for speed at all costs
you know you should be instead thinking about things like how do we test really disparate things to make sure that we are really testing at the ends of the experience to understand you know what's going to to drive so instead of like moving minor things around the
conversion flow test an entirely different flow or a very different experience not only will you learn more you know based on kind of where the result Nets out but you actually will get results back faster because sample size is a product of your effect size
and so like driving a higher effect it's more important and I think the other thing is if you look at a backlog of 80 or 100 experiments often there's only four or six hypotheses underneath that and you should really be like thinking about which hypotheses you're testing
rather than to try to run through all 80 tests and so like an example of thumb tackling or optimizing conversion we may think they need more education as they're moving in we may think there's a trust issue we may think we actually don't even have the right types of users
in the flow overall there's different sets of experiments we'll test those things do they like the one or two right things against these each hypothesis validate or invalidate and then and so I think like just a more thoughtful slightly slower actually approach can sometimes work really well
I would love to dive into that so let's talk about a be testing and experimentation so this is something that I tend used to say that like if you ask any product manager at Airbnb what's the most useful tool that that you ever used there and they would say it's the
experiment framework the ability to run a B test and easily measure the impacts of new changes when you meet with startups how far along are they and sort like in terms of the ability to run experiments and what are the type of advice you give them I'll give you a
common question again in the early days were like what tools should I use but I'm not sure if that really applies at your stage but I love to hear what you hear what you think yeah so we're typically working with companies that are a little bit later so kind of series
being on often so they have some of this earlier stage I think there's like a first-order question ahead of experimentation which is like is the site instrumented in the right way are you updating that instrumentation as you change features is all of this kind of
query balled sequel and serviceable to the company all that stuff has to happen first I think so you can like really you have kind of the integrity of data but then building a thoughtful framework on top of that an experiment framework I think increasingly the the third-party
tools to do this the full stack tools are pretty good actually and so even for quite talented teams the answer is usually not to build it yourself right away you know I'd love to hear if you have a rule of thumb on this I think generally if you can be comfortable with
the fact that an entire engineer is going to be maintaining that infrastructure in perpetuity then maybe that's okay if that feels like too much then you probably should be thinking more about a third party type infrastructure prior to that yeah I am I
used to recommend building own own tool because that sort like my my primary experience there may be I think we started with it with an external tool but then we built our own own thing and I described off and how incredibly simple the first tool was it was
basically a feature flagging tool that would like you can show things to some users and other things you were not visible to some users and then you would basically just sort of like run that 50 50 to 50 feature finding tool and then you run queries on your database and
that's what we get started like the first year or two I remember all a B tests was basically in Excel like output which some some some my calculation on the Cisco significance then I mean 2014-2015 we build it's like really an elaborate tool that would do all things automatically
and it's just like magic but that wasn't how he got started and we gave early I started feature flagging tool and there those exist ass tools so you can use and as long as you like you said you can query the data you would notice or like look pretty comfortable about about sort
of like how certain new feature wouldn't perform the one thing I would say is the thing you want to decide early on is who sort of like what is that the thing that you're controlling the experiment on and I would recommend that it's going to be user accounts you're doing it on
visitors or something like that is gonna be typically much harder but you're on user accounts it's like relatively hard to fail even you'd like them the market level experiments or search experiments those are difficult those are like difficult enough the data
scientist MB would make mistakes or knew early on and kind of have to throw things away but if you just do a simple user account experiments is pretty straightforward yeah how else did you break apart metrics like you you have all this tooling but then how do you start
thinking about like okay these are the things that matter these are the things we're gonna start tracking so in the first week of YC we had the growth bootcamp and the first talk and give it growth bootcamp based on retention so basically walk through the kind of the
connection between a product value and retention and how a great product oftens have is completely reflected in that retention graph and I asked everyone are you measuring any of these things that that basically what is the metric that measured the Pronk value if you're
measuring that over time and then if you do those things you kind of have your attention graph so in many products b2b and consumer this applies and there's something that companies should be doing and you don't need an enormous sample size to to get useful data from that so
that's typically what I start the second thing is or like worse users coming from some basic Google Analytics or basic just like source tracking is a good place to start after that I like to have a survey asking people sort of like how they heard about the product because
some things can be hidden in the source tracking sometimes you have this survey on checkout or over email or oh I'd sign up like like places like for example if you come in through Google brand or good like for can you come in through Google
yeah you kind of have to determine is that someone who know about your product before not know before if they know about a product how do they hear about it well that Google isn't going to tell you that or the app store isn't going to tell you that you're gonna have to find out from the
users and it would it's sometimes useful to just break it down a little bit deeper intro like how do people hear what the product in the first time is that their friends is it through workplace or whatever that I think those things are important and then after that
it's the end-to-end funnel this is sort of like what are the steps from the first time I heard buy a product to I get to that product value there's lots of things you can measure there yeah I think I think that's a great overview one thing I would add is is if you are
working on retention or trying to optimize a retention metric very often the way to do that is actually quite early in the product experience the types of ways that you set a user out for a good retention and it could be the first session first week with a product
and so the next set of metrics you need after a retention is one of the early proxies for retention so you can experiment on that and so this is like a slightly harder question that we work with companies on but I think you know there's one level which is just what are
all the things that correlate to retention or satisfaction and then you've got to go start experimenting to understand causality but that set of metrics and having really good metrics for what is healthy activation look like what is when do we know a user is on the
right path that stuff is a really kind of important add to the model once you've got the basics built what was an example of the thumbtack so in this case and in general for marketplaces you know the kind of North Star or the top metric is not something like week of active
users because you need something that combines both sides in the marketplace so in our case it was transactions or requests place probably quite similar for it really we you know real world product the thing we were optimizing for is hires happening in the real world so
it professional getting hired by a consumer we didn't actually have full data transparency into what was happening off the platform and so we had to look for upstream proxies for when that was going to happen and the main thing was when a consumer
got a certain number of quotes from a pro we had they then believe that we did the job for them we gave them options they could go choose their higher rate was much higher there their NPS would actually like you could watch it spy get a certain number of quotes and so the
metric we were going on was the number of customer requests that had that many quotes and actually our liquidity metric in each market when he looked at general condors in New York the we measure the health of that market was what percentage of consumer requests
were getting a sufficient number of quotes and that's what set us up on a kind of a healthy trajectory in that market so I want to go back to an earlier question we were talking about experimentation well one of the experiments experiences I've had is I've kind of like start talking to later
stage companies maybe Sears B C or C and I'm like do your own experiments and they like yes we do so how many have you read in the last twelve months it's like twelve and that would be on a large engineering team so I often find that people have the people understand the
idea of an a/b test they understand that sort of like what it could be used for but they haven't fully immersed into doing it and they haven't fully kind of got the entire company rallied around it as a way to validate sort of decisions I'm curious to see like what have you
seen if that was the case you came at a company and I said I've run 12 experiments in last year I have 50 engineers like what would you tell me and how to change that so I think the big thing here is like growth increasingly I think is less of a like
team or atomic unit and it's just like the way of running a business using data and experimenting and I think that is quite hard to just talk somebody into yeah you know it's like a philosophy for how you run a business yeah and so I think the most successful way to do that
is kind of like bottoms up and organically which is why I think you sometimes see these like small SWAT team like growth teams that start to optimize and get quick wins on one part of the product and like just showing it through numbers is often the very best way to to
to get to that answer and so like one of my favorite examples here is the team the early team that SurveyMonkey did a great job with this like there's like one page and a product which is the end of the the flow when you take a survey you have like millions of people hitting
this page that could ultimately some at some point become survey creators as well and they had done no testing on that page and so the growth team basically just said we're just gonna make it our job to optimize this specific page and started running tests
there and the results became so evident that the kind of influence of the growth team expanded over time and so I think this kind of like I'm just not sure something you can talk somebody into I think you got like a bit how do you deal with the founder who's like well I both
made a great product it were now I have lots of users and I raise lots of venture capital but I'm not running experiments so like how do you change sort like I understand you can't talk me into it but like I'm the interested in the idea of making progress but I'm not interested in
having my ego sort of like hurt by having data tell me whether this is actually better or worse I mean I think yeah it's great but I think it becomes the the culture around the way that you think about experiments you know I think one of the most powerful things of
experimentation is or elements of it is that you when it's done you send it out to the whole company and talk about why I want or lost you can even get into a culture of like you know like unveiling the results of experiment having people guess which one kind of went went well
or not well and I think that becomes something that people really like and it's like you realize that it is teaching you a lot about your users and the business that would be hard to get otherwise and so I think this like building this kind of culture around is
really important what's in your Experian is a great way to get ideas for a B tests to run they're so like let's say okay I have my a B mic from an frame we'll figure it out like I know I don't experiments now lots of ideas and things to test how do you how do you think your
expands go by that so ideally they come from everywhere we actually like basically kind of like open this out pretty pretty broad funnel so certainly everybody on the growth team but even others outside of that or basically all adding stuff in a brainstorm or a Google
Doc or whatever it was you you do have to have somebody synthesizing that that is not in a crowdsourced function but I think like many of our best ideas came from engineers and other parts of the company who just had some insight into a part of the product or a user experience
that that nobody else had and so I think like basically you know broadening the scope and then having some kind of you know synthesis yeah and then and who decides what to work on first is he had 15 ideas of things to fix in the onboarding flow who decides what's
working first and what's a good framework you experience - yeah so I think there's a kind of a central point which is a product manager thinking about that point and engineering lead thinking about that that part of the product I do come back to this like
hypothesis driven approach so what do you believe you were testing and what is the best way to test that thing I think it's a really good way to to force that that conversation I think it's it's hard I do think it's hard to be very collaborative at that end stage when
you're deciding what to what to choose people of different roles that make sense you yeah so as an employee so Gustav you're talking about advising founders of these later stage companies but then you mentioned these bottom-up approaches what are other good examples of
employees at companies being like hey I want to take a growth driven approach here and then like getting buy-in across the whole company like as we mostly talk about advice in the context of founders but like let's give some advice for employees who like really want to get a
growth program going yeah absolutely so I think there are certain classes of things that are easier to get off the ground and test and so actually one of the reasons that that paid marketing is so common early in companies that just affect very quickly quickly got off the
ground and you can validate that and so I think finding those types of things where you can really prove something with a small budget or small team or super important and and very often you know testing a new channel in that way can be a way to to really prove
something you don't have to go get permission to do that in many cases and so I think there's like many examples in the acquisition area I've just go going and finding this thing that may work mmm cool you do great if you have other no I'm trying to think of good examples
here but I would say would you set around publicizing what is the goal I'm publicizing any effort towards that goal towards the company is it great what you're doing it so I really like when you have a weekly email of all the metrics in your company that goes out to
everybody now that will anchor people to say oh it's bookings that's the thing that we going for is bookings okay so then the next natural question would be what are the drivers and bookings and well that's do you funnel and then if you are any of those teams they're the
driver of the funnel of the metric bookings now you need to improve that metric and if you run an a/b test you can actually prove that you improve that metric so what we did there baby was effectively we would have an top-line metric which was bookings and then we
would say to a bunch of different teams Croteam marketplace teams and others that you guys are in charge of driving incrementality to that metric and as an engineer or anyone on any of these teams you just come up with the best idea that you can impact that bookings metric in
some way and then you start running experiments and they could be like I really love this idea of allow everyone to come up with ideas on what to fix because the more kind of ingrained you are with a certain part of the funnel let's say you are working on
sign up for example or say say you're working on even something deeper referral sign up so I never be there would be tens of thousands of people that would sign up from referral link every day they have high intent to become users and maybe eventually spend
that money but if that signup flow is not perfect well there's a lot of things you can do there and there's some engineer who knows that flow really well who's gonna know that well this is like five examples of things are not working if I fix them then that's the kind of
thing you want to share to the company so the people start understanding didn't commute I love these small things if I'm an general engineer I would say try to think about orient yourself around what's high intent this is different for different companies but in case of a and
B the early winds are often in the high intent flows so like with people actually thinking about booking or actually about to book so search conversion is important one sign of conversion is important one Sora like online marketing traffic was an important moment for us versus for
example homepage ha is super confusing and there's many companies that have this experience for like people coming to your homepage is a combination of people that really want to do the thing that you offer people just looking around people accidentially and antonio
page it's just like a random it's just a hard place to start yeah versus like if you were optimizing the landing page for google conversion well that's gonna be a lot of people that want to do exactly what you offer right yeah it's a great point you're probably trading off intent and scale
cuz you don't wanna go so deep that you're dealing with 10% of the population yeah but - hi funnel doesn't work that was the mistake that we made who actually made the mistake of going all focusing on the onboarding for the homepage because they're like well
that's has the biggest scale by a longshot but turned out that only like 10 percent of the people come to the homepage actually have a booking in mind right and everybody had something else in mind when they come to it there and be homepage for the first time that's
crazy so guys when it comes to advising companies what are the main differences in growth between a B & B b2b company and consumer when you're advising them so I think consumer has been kind of leading the chart on on what to do for the last like ten years and I think many
of these tactic some strategies are now being adopted by b2b and I one thing that I if you see YC companies coming in and doing YC in the early stage program now the most common advices were like build a framework to figure out how to do more sales faster and like that's a
funnel and we kind of give them advice or so like start in the end and say if I need to sell ten to ten customers every week work your way backwards from how many people need to pick up the phone or I'm people to have to show a demo to how
many people don't have to email or or some other way reach to get to that demo and just like fill the funnel and soar like stop that way and then now you know what sort of like the early days their their early part of that funnel look like so there are many of the tactics
they're using consumer that now being applied to b2b but it came a little later the benefit of b2b is you typically have much higher LTV so you can spend you can be a lot more creative on online marketing you can you you often like that the first question I ask
companies to sort like can you make a list of all the potential customers in the world hopefully if that list is sort of like in that hundreds of thousands or smaller make that list like find a way to make that list and then like take that those people and put it through all
the different go channels that exist whether it's online marketing whether its sales automated sales just like go after that list if your product is so mainstream that it's in the millions and millions then you're gonna act like a consumer company because you're gonna
you're gonna go after the same kind of strategy so the consumer company will go after but but what if you went after a big client in the wrong way people talk about fundraising in this context like maybe you shouldn't try your pitch on Sequoia the first time for example right
so how do you think about that there's a distinct difference to me between SMB and enterprise and they often have very different sort of like closing closing kind of cycles if you mean if you want like worse is SMB company often what you do is you have this top of the funnel on
this on your sales funnel and then you have a relatively automated way to sign them up so that maybe is a one video demo and then you have a customer and this is typical for SAS companies I think is that you have some way of scaling up the ability to sign
these customers in the enterprise world which I have much less experience the end closing event there's going to be through real traditional sales in some way now you can build up the excitement through content marketing that can even be targeted content marketing I'm trying
to figure out who's the decision-maker and talking about those people that lots of things you can do in many cases social proof it's a super critical thing here because it's relatively easy to understand how companies make decisions of a new product and you kind of have to
just like deeply understand that and then mimic your growth strategy towards how that is happen and I think in the SAS SMB kind of group there's just a lot more distributed how people make those decisions and I was one of the first users of slack inside I haven't be I
certainly wasn't the buyers up coming but I was me and a few other people be like hey we should try his new thing it seems awesome you know like a year later everyone's in slack so that was like it if you're in that category you can go grow much more bottom-up versus if you
if you enterprise category you have to identify that list of those people first and be like these are people and going after right yeah I was just curious about like potentially putting your highest value potential customer in some kind of weird experiment and like
churning them out by like oh oops maybe we should have changed the color to purple or whatever yeah I mean I do think so one of the things that has I've learned working with more and more companies is that more types of decisions or type two decisions than you
think the type of decision you can actually roll back and get another shot at it's maybe we don't work with companies that are dealing with large enterprises typically more SMB and so you maybe have a little bit more leverage there but even things like
pricing features packaging those types of things that feel very final you often get leverage to keep testing and so in general I think testing some of those things is okay particularly early on while you're while you're finding the right the right mix of things and so I
wouldn't worry too much about like burning burning that audience of course you know the reputation the community those types of things is important but as long as you go into experimentations with good and think about the end-use I think you're okay okay I'm curious David be a bit
into that you have long experience with pricing experiments or pricing some some experience yeah so high a new experience so like what's the the best advice to a series a funded company or is he'd funded company we're like I have a price for my product is somehow is correlate
to LTV I want to figure out that's the right price not like what would you start yeah yeah so I think we're like backing up one step and saying are we growing you know virally and is basically therefore it's it's all about removing friction and price is a good
way to remove friction or lowering price or are we going to need to fund this through paid marketing and other things in which case you actually need a significant a higher margin so like what's the starting point yeah I think is a really important piece and then the
second one is price and anything related monetization is really tough because it's basically system of incentives and it will change people's behavior and so you have to observe these things over very long periods of time to make sure you're getting the right answer and so I
like a big fan of running a/b tests for sufficient amount of time and then flipping it to you know maybe 5% remaining in the original test for a very long term to observe the thing that thought happened continues to happen and validating that but I think if you start
from the place of where do we want our Mars to be and how's that feeling or growth strategy and then thinking about long term metrics now you use pricing to to influence that you lay in this way I run a subscription product then you bump the price from 10 to $15 I should hold a
whole lot group for long period of time on the 10 that $10 bucket I think often that gives you a much richer kind of insight into how it changed people's behavior god I got it I see cool so when it comes to paid how do you guys feel about paid marketing especially in the
context of like the dynamic shifting over the past five years say what do you think about paid marketing I would say that the big untold story of growth is that we've shifted from from free channels to pay channels and paid channels our pain marketing referrals to
some extent but but pain marketing is the one that is the most obvious one I think the reason that happened is because the platform shifted from offering free traffic to you offering paid traffic and there is benefits with pay traffic because it's highly targeted like it
much easier to target the campaigns you're running towards specific users or is in the free channels targeting is not that easy so I think that's really the general trend and which means to me that it's not easier it's more is equally competitive if you are a subscription
service or an e-commerce service that acquiring people through paid that's very competitive because there's lots of other companies doing the same thing coming after the same customers so you have to learn how to be they are driven creative and just really treating all my
marketing ass as something that you constantly had to innovate on versus something that is just sort of like set and forget here's here's my my online marketing team and then they just try to hit their targets it's something they've constantly invest in and if you I'm not
sure if this is good advice for seeds felt funded companies because look at the evolution of the online marketing he may even be it went from like one or two engineers to when I left I think we were 15 or 20 engineers on all my marketing
and they were building everything from attribution system to automated bidding to all these different things and I think that general trend for online marketing is is going towards more data driven more automation and just less you elevating the the humans to sort of like
making more high-level decisions absolutely totally agree I mean just add on this I think it's becoming more common and it's there's an important place for pay marketing its vast to get off the ground it's relatively easy to test if it becomes your primary Channel
it's a very hard thing to sustain because it's expensive but also because it degrades pretty quickly actually so like you know if you look at the d2c companies or blue apron I think you know they grew really quickly on the back of paid marketing and then they realized
later cohorts got more expensive they retained less well and so this kind of like has this compounding negative effect and it can be a scary place to be and so I think if you're thinking about paid marketing more as an accelerant rather than the growth strategy it's a
much healthier place to be and so like you know a couple examples of a place I think it's used very effectively is to seed a new market you might be launching for example or at thumbtack or other market places very often used to help balance the market place because it can
be much more surgical than you organic or SEO or something like that and so you know we knew maybe we needed more general contractor in New York but not you know plumbers in Phoenix and so we were willing to pay a lot for one area not the other and it was helping
play this function but the core of the growth was still organic and SEO which is more sustainable over time what do I do if I have that so let's say I'm 70% I'm on yeah you second coming from Instagram ads yeah like is that a prop is that problem for the growth team of
the all-american team is a product problem like like where should I start even have that situation so I think it is is primarily a product problem if you haven't if you have tried other things that they haven't worked you you obviously need you typically need to do
core product work to enable yourself to go into these other other types of channels I think the other thing is is these are all like we were just talking earlier business model competitions where like the best companies that monetizing we're gonna win and paid
marketing and so if your product is better at monetizing it allows you to then go do do more of that and so I think it's like it's very hard to get in this place where you were just spending more and more into some of these core channels and watching that degrade and
so thinking about yeah what are the enablers of that so how do you think about that when you don't necessarily have all the data so you're talking about like say you're entering New York City right and you're like alright we're gonna do a lot of paid to enter New York
City but you don't necessarily know how long you can maintain that and at what rate the ratio will shift and you start getting organic growth you know what I'm saying so how do you even think about that in the early days of a new market I think it's really hard I mean you
certainly will be spending into negative ROI to start in any of these cases you hopefully have built a model over time as you launch markets where you can understand what that curve looks like and figure out where you are on that maturation curve but I think
understanding the shape of that curve is the most important thing you just have to test your way into it for the first few yeah okay so Dan one of the kind of experiences that I've seen now becoming a pattern is I worked with this YC company they get funded they get serious
a funding and then maybe even B funding and then somewhere in that line they get a board member and that board member said oh you need a VP of Marketing and either head of product marketing and and you already have maybe a growth person or old a marketing person and now you have
this like multiple roles or like going after the same thing and that time too can create friction so experience I am curious is to learn well you you have experienced here and so like what can advice do you give founders yeah absolutely I mean think so first of all
I think most companies can go longer without a VP of Marketing than they think they can as a general rule of thumb I think you know one of the problems we have in the market is how many really good experienced team builders who also no growth marketing and fit this VP Marketing profile are
there there's like there's not that many of them and so like hiring the perfect person but that's very tough but if you kind of like disaggregate the things that that person can be asked to do it's you know growth marketing which which often should be closer to product
actually or more like a real growth effort messaging or like product marketing that those kinds of things that can often be product marketing and it may live in product or design or something like that and then sometimes PR is also put into this which can also
be a distinct thing and so I think like are you just solving you do you want all these things and you're just like trying to find the person can do all those a way to solve your individual challenges with a with individual people that may be part of existing teams I think one of
the symptoms of bad organizational friction is when you have multiple teams working on the same metric without drawing accountability it's like very classically a retention metric will get split between in product gross stuff and a marketing team owning email and push
notifications and it's really hard if they you you get a disjointed consumer experience cuz two teams are thinking about the same problem and it's much better if you can combine that into one and adding somebody like a VP Marketing the mix can make that an even harder
problem because both these leaders think that they own that did you make all these decisions like how would you organize that you have email push notification like the classical sort CRM marketing yeah that often comes out in a box from some other company yeah and
then you have the retention product team like if you sort of like make these decisions like what would you decide so I think it depends a little bit on the company's DNA so if they are at a place where they really understand and analytical experimentation based
approaches then maybe it's easier to just build it as part of the product team if you need to like prove that this works and evangelize it you might want to stand alone team where you go get a growth person so I think that's probably like the right starting point I also
think different areas of the funnel lend themself to different type of approaches so acquisition is a little bit easier to aggregate disaggregate because they're unique channels where you know some are more marketing heavy like paid acquisition which becomes very
engineering heavy over time some start very engineering and product heavy from the start conversion and retention those types of things and so I think like thinking about the type of problem you're trying to solve but specifically for this retention question I think it
is typically healthier for that all to be within product pretty early on and do you think that every single person that you mentioned now should have a way to measure that ability to sort drive incremental growth for the company like whether you're in brand marketing or PR
or online marketing or product you - all of them sort of share this incrementality metric or are they kind of going after slightly different things slightly didn't think so so I think the the growth stuff that the growth marketing stuff certainly should be all
kind of contributing to this northstar metric in a very clean kind of tree I think where you have a challenge when you have these like very hard to measure outside inputs like PR and brand and those types of things I think you know brand is this like topic that we get all spun up on but
really like the output of brand like trust and reputation and things you care about that is obviously very important but what you can do with brand marketing is like actually a small piece of that it's like the awareness piece it's it's like maybe make wrapping in in like a
hook that people can remember it by that's not brand that's that's way it will way to accentuate brand and so actually like if you have a great product that people love brand is just accumulated customer goodwill so maybe actually the best way to grow your brand
is just to get more people to your product and that's a growth team yeah you know and and so like I think very often we'll talk to a team that they have like you know they're kind of paid marketing stuff going over here that's very performance focus and they for just
some reason have this like other budget over here that doesn't have any metrics tied to it and they're willing to just spend on quote unquote brand and very often that budgets wasted it so I'm much more kind of amenable to as much as much about kind of measurable core
growth activities that ladder up to a metric it's an important for start-up like even in the later stages to measure brand awareness and brain consideration like the traditional brand brand marketing metrics like is that useful the the sentiment is very important how
many people what your MPs is how many people would be disappointed if you went away those kinds of things are really good signal around fit but raw awareness and benchmarking its competitors and those types of things I have rarely seen that become actionable in a in a product
maybe with the exception of knowing when you've kind of tapped out a core market but that's a late late stage problem hmm so related to this Tony asks about Airbnb so you could talk about Airbnb in the context of wow they have such a great brand but on the other hand they
have a great product so Tony asks how can every bean be a startup like Airbnb grow so quickly was the need was the need of customers was that marketing strategies was that the idea of just travel in general in the dream of vacations like what do you think I think
that would make everybody unique on the guest side so in me this is two products of the host part on the guest part on the guest side it was the viral nature of this unique experience so if you go to New York and you stayed in an apartment that was in a local
neighborhood and you just had a very unique experience that you never had before it is fundamentally something you talk about when you come home and we kind of studied this we studied when do people talk about travel well before you go about to go to travel you ask your
friends where to go when you're traveling you post lots of photos and stuff online it's actually the point of the Prime things we do when we travel is to share information to our friends about it and then when you come back and tell your fans how great it was so this
is natural things that people talk about and similarly to how oberon lyft is sort of viral in their nature because it's a new experience and it's something that we often do together with friends so I think that was the primary driver of this new really great product in terms
of tactically what we did like I've seen graphs on the incrementality on online marketing the early days of MMB and it was very important like had we not down online marketing early on we were certainly not being being a spark today so that was critical that was sort of pretty proven stuff it
was Google people googling for things that relate to Airbnb or Delhi to staying in homes or apartments and we just bought those keywords and that helped us drive growth like Dan said that was a very good way to be strategic about where we wanted to grow so like
maybe it wasn't super useful to buy these keywords in San Francisco but if you were in Europe or in Asia or in or in Russia or somewhere it certainly was very important to kickstart those market mm-hmm and then after that it was the word of mouth this sold like drove most
of it but if you like the composition of growth where B word of mouth is by far the biggest driver so I think it was like way over 50% on the guest side and way over like 70% on the host side of people that heard about me for the first
time through a friend after that it was online marketing referrals in SEO those are the three sort of like team sang channels that we had and remains very important all those three three channels and at the scale it got to the point where like yeah every single small tenth
of a percent improvement matters a lot so that's sort of like when you get to that scale it's not like you spin up in your channel and now you have lots of new growth in that channel it doesn't really work that way there's this like pretty expected ways that people will
discover a product like every movie and we just made sure we optimize them like crazy yeah well are there any examples of a already big company doing something and it's like a step function improvement in retention or conversion yeah and that's actually the example I
give on the Facebook graph on my on my YouTube video from syrup school there were more product changes than there were anything else but when Facebook figured out to translate in national lysa site was very important for the growth obviously when they grew out of
made it available out of a set of Eastern University is a really big one but then mobile was a huge one so I don't know if we've seen a platform shift like that in the last five or seven years but when we saw mobile happening if you were a desktop app then
you were only speaking to a very small audience compared to what was going to happen so rearranging Facebook for mobile if you look at both Instagram and whatsapp are mobile products so so that was a really big step function for them and the last thing was making facebook available in sort of
like very data constrained and and hardware constrained markets so making like messenger lie and so like even acquiring whatsapp and making Facebook available in places where like the native Facebook app that we used to hear wasn't actually a great user experience
no it's too heavy yeah and I think those are all product images but there are step functions for a company like that so another international question michaelsavage asks it would be great to discuss growth into new regions for example Africa and UAE what would their approach be how
does it differ from region to region culture to culture we when we grew Airbnb we actually divided international growth into two different buckets one will be calls for like product limitations and one that we called sort of like cultural impacts and there's not
the right words but effectively trying to say everyone said oh people in Germany is so different people in France are different people in Japan are so different and that is in the hypothesis often coming from people who live in those countries saying that we are
different so therefore we need a different experience and it's a very common thing that's hard to prove which is why everyone a lot of people share that opinion now we need to prove those things to be able to actually build a team around making the product different
like cultures in my experience we we basically started that process by asking around and we didn't find a lot of lot of massive cultural differences like there the stack was basically mobile phones iOS and Android desktop Chrome Facebook Google this is true for most
markets in most countries like the exceptions are China South Korea Russia maybe a handful other come to countries those are the exceptions but everyone else kind of like the stack that described is sort of how people use your product so within that it was very hard
for us to prove material changes and people culturally use products now the other thing that on the product side when for example if you don't have this stack then that's the difference for sure if you have slow internet or slow phones or any of those examples or like
that will materially impact sort of like how you use your product the thing about the top of the funnel in some of these countries is for example Google is not available in the sense that like it's not a search engine in in Russia or in Korea so or in China so you
have to do something else there in the Middle East and many of the more emerging markets the ads are actually quite quite a good arbitrage for some of these countries companies because there are not a lot of premium services to charge for things and a lot of the
brands have actually not gone on to these platforms yet which means there's a good arbitrage moment for startups ads anywhere ads on ads on Facebook or ants on Instagram or as on Google ok because the price is so low and there aren't like a lot of well developed companies
that are actually sort of likely able to take advantage of it and there's just as much usage on these platforms and these countries that are here if not more next question Justin larosa asks what are some of the most common drivers of viral growth so I would say my early part of
my experience my o my career were still working on on a messaging and growing services like boxer through viral growth it changed a lot so the viral growth dependent on certain platforms typically platforms that have some sort of like list of contacts and a list of friends
that you grow grew through obviously Facebook's did a different example of that all these platforms have changed in different ways and we're less susceptible to that today there are pockets of interest I think there's like the follow growth inside corporations
like that is a really interesting one and I see a lot of companies well utilizing that in different way whether that is through your let's say there's a wise company going through teachers nailed it really well there's another one who's growing through sort of like
the Google Apps and and sort of like using the social proof via co-workers using products that's a really strong one and I think you see these more new niche variations of viral growth then you see the generals or like enviable growth it's just much more difficult
these days to do it and the channels aren't generally available in the same way so the thing to see still seems to be working pretty well is referrals like incentive iced viral growth you just have to make sure you can measure the incrementality of that because you
are giving away money when you're putting users but but III often see YC companies sort of like taking some of their referrals advice from ABB and then coming back and say actually that was really meaningful for us and we're going faster so I I have seen that continue to
work versus the general vile channels have been in many cases saturated and not not a good way for startups to go anywhere hmm yeah viral brow is much harder I totally agree it's I get much more kind of like closed networks rather than open networks the other thing is is
referrals are potentially more like paid marketing than they are like for Alby on some way stick and so thinking about the it in that way and payback and the other things is really important and so when we talk about the growth of referrals we're talking about another form of
growth of paid marketing it's just different you know and in person who's getting paid but but yeah I think that that it's until we see a big platform shift realities is harder and harder what do you think about that in the context of a bootstrap company how should they think
about growth in that way well one advantage of referrals is you have control over the the levers such that you can make it ry+ or you can you can make it work for you so I think we're come please get over there skis is when they're not thoughtful about
payback on these these things I just think thinking about it as it's free growth it's not kind of the right but a right the right way to approach it okay yeah AWS is so cheap whatever cool so after you left thumbtack you now start your own growth agency basis one sort of like what is
the if I am a CEO to be founded out there looking to get help with these things or like should I go in and high-growth agency so like why should I do that or just build my own team just a little bit about that okay so the first answer is is you should not hire
consultants in many cases I think you know for for almost all things related to growth paid marketing not only do you want to build that muscle yourself but it's like the front line you're dealing with your customers and the feedback is so rich that if that person is not
sitting next to a somebody else on your team you're missing all of that learning and so you know maybe an agency or a consultant can help you get off the ground give you best practice this is particularly true if you're in like a well-known space like e-commerce where there are
frameworks you can apply but typically your aspiration should be to have a smart in-house team as quickly as possible you know the way that we work with companies is somewhat tangential to this which is focusing on really acute high leverage problems that we can we
have leverage on because we've seen them before so why are customers churning how do we use monetization to drive growth and we're not trying to replace the team but rather unblock them such that they can execute really quickly and deliver them some insights that would take
longer to get otherwise but in most cases particularly when it's more operational I would say build yourself when are you done like like legs or like if you work for the company you're like yeah I'm working down here like like how do you know when you're like ok I've
completed my mission here I think it's when the mental model is internalized right when they're when they're like the the conversation becomes really seamless like that's that's the right place where they're clearly off off and running with
it yeah and there are some insight about how the product grows who the customer is what they need that is being internalized and active acted upon that we feel really good that they're gonna take it from there how long it's typical in an engagement for you guys
we typically start around 3 months sometimes it go longer than that but sometimes that's all you need to actually get up the curve and understand some of these things and like this is a never-ending problem the exact monetization growth you will always be optimizing but at least for that
kickstart or one of the spark or where the area is to start kind of what are the deep wells to start mining that piece can be relatively quick oh why did you start because all disease versus just soar like joining another company and working growth in another company I
tried to join another company actually and I was working with founders and CEOs and and like the journey at thumbtack from like 30 to 500 was so fun and I was gonna go do that yeah again but I just started to see a consistency of the type of questions I was getting and there
weren't really people on the other side that were able to effectively answer them I think you know as an independent advisor it was kind of hard to provide the help that they needed but when we started pairing it to analytics and user research and the ability to like really
wrestle a problem to the ground that's when we became really powerful so I mean the short answer is we kind of like grew into it organically it's just really fun to get to work with a lot of these founders so we've stuck with it cool so last question which was actually
submitted by Gustav as two people who both advised and work in growth how do you think about growth in the context of improving humanity given that a lot of these products have filter bubbles like can create a dict --iv responses and on the other hand like you're trying to
make the world a better place so how do you balance that yeah so so I'm optimistic here I think there are maybe two reasons for that one of which I think is more valid but I think what one is companies like Facebook and and others maybe got some flak because they
were optimizing for these short term metrics like click-through rate and time on site and those types of things that over time as become more sophisticated and measure longer term metrics you can optimize yeah more you know in keeping with both the interests of the company
and the consumer and so that's that's one angle of it I think like the the one that I give more credence to is that like this stuff these cats out of back right we're like we're not going back to a world where we don't have these addictive products competing for our
attention but we can weaponize these same tools we have learned for good things and so like other products are going to come in you know you see this happening with meditation apps and exercise apps and eat healthy eating apps they're using all the same growth
tactics to turn you into a better person and I'd like to see more more of that and so I think like the way is not to try to like put the cat back in the bag which is to fight back with the channels you know the tactics we've already built yeah I asked this question I thought
about a lot in the last year I think as as a growth team you should have principles you should be like try to figure out these are the principles that I want to follow some of that means not gonna have any dark patterns in our app and then the most important thing is to
ask the question why like if something is really working why is it working like why is it that this is now sort of like changing the behavior of my users and and the more you understand that I think the safer sort of like in the more safer place you are in terms of sort of like
making good decision for the for the company and then I would say much of is that we're talking about is actually tied to engagement driven products so product like you mentioned that fighting for our attention that sort of like where much of this have happened and I
think there is a fundamental problems or like with with the driving the attention and then sort of like having advertising as a monetization engine if you change that and you say actually I am the I'm the customer I'm paying for the product then you get different outputs because
now you're optimizing for me as a customer you're not often I suppose someone else I think there's like there nuances here and I think these are probably the most challenging questions to answer for companies that fighting for our attention especially if you're
free product fighting fighting for attention versus for many other products like I think it's actually massively beneficial to them to do a lot more of this mmm-hmm that's great alright thanks for coming in thank you so much thanks Sam thank you you