and like the you know dead of the Night by that I mean like you know 7 P.M or something on I think it was a pretty sure it was a Friday we just asked for forgiveness and kind of put these questions into the silent flow and ran his Navy test with a small group and you know I'm fully expecting okay this is gonna like hurt our numbers but maybe
it won't be so bad you know and I'm gonna be prepared to Advocate the power of this data that we're getting and I was totally weird I'm thinking with like written started to write like a like the framework for how I wanted to to surface this and we start to get the data for this thing I'm not getting an improved conversion like there's no
personalization nothing passed it just the questions it improved conversion by like five percent like just improve signups and it's one of those like what like okay this like what is going on here welcome to Lenny's podcast where I interview world-class product leaders and growth experts to learn from their hard-won experiences building and
growing today's most successful products today my guest is Laura Schaefer the week we recorded this chat turned out to be Laura's first week in a new gig as head of growth for amplitude taking over for a previous legendary guest Elena Verna prior to amplitude Laura was VP of product and growth at a company called rapid before that she spent over seven
years at twilio as head of growth and PM lead of the growth platform and the experimentation platform at twilio in our conversation we dig into Laura's career growth framework and the importance of carving your own path versus waiting for one to be carved for you we also get into a bunch of tactical and surprising advice around running
experiments making decisions on gut versus data developing your growth strategy and how to sell your product to developers Laura has a wealth of wisdom and I learned a lot from her conversation with that I bring you Laura Schaefer after a short word from our wonderful sponsors this episode is brought to you by public.com I want to tell you about
their new treasury accounts which earn a 4.8 yield on your cash that is higher than a high yield savings account while still being backed by the full faith and credit of the US government treasure yields are at a 15-year high but buying U.S treasuries is super complicated if you go to a bank or navigate an ancient Government website or at least that was
the case now you can move your cash into US Treasury with the flexibility of a bank account you can access your cash whenever you want even before your treasury bills hit maturity there are no hold periods no settlement days just a safe place to park your cash and earn a reliable yield public will automatically reinvest your treasury bills at maturity
so you don't have to do anything to continue growing your yield and you can manage your treasuries alongside stocks ETFs crypto and any alternative assets do all your investing in one place and earn 4.8 percent a higher yield than a high yield savings account only with a treasury count at public.com Lenny this episode is brought to you by EPO
EPO is a Next Generation a b testing platform built by Airbnb alums for modern growth teams companies like netlify contentful and Cameo rely on Apple to power their experiments wherever you work running experiments is increasingly essential but there are no commercial tools that integrate with a modern grow team stack this leads to
wasted time building internal tools we're trying to run your experiments through a clunky marketing tool when I was at Airbnb one of the things that I loved about our experimentation platform was being able to easily slice results by device by country and by user stage EPO does all that and more delivering results quickly avoiding annoying
prolonged analytic cycles and helping you easily get to the root cause of any issue you discover EPO lets you go beyond basic click-through metrics and instead use your North Star metrics like activation retention subscriptions and payments nfo supports tests on the front end the back end email marketing and even machine learning clients check out
EPO at getepo.com get eppo.com and 10x your experiment velocity Laura welcome to the podcast some thanks Lenny it is so great to be here thanks for having me it's great to have you so I asked Elena Elena Elena I'm not even sure to pronounce her name maybe nope Elena okay okay I think I've said it wrong all the time all this time
okay Elena so I asked Elena ferna who's a popular guest on this podcast who I should have on this podcast and you are the first person that immediately came to mind and so I'm really excited that we're doing this and that you agreed to be on well she's the best and I'm really happy that she referred me because I'm just stoked to be here so thanks for
listening to her guidance absolutely and it's kind of a cool time to be chatting you're kind of the the newly minted head of growth at amplitude and so congrats first of all thank you I appreciate that y'all live this is my um day two and a half here I love it some companies there's like a little percentage that shows you how many people have joined before you and I
wonder what that percentage already is we had that at twilio and I got pretty pretty high up there after a while we had like a stack Rank and a spreadsheet and yeah but it's it is funny so wherever wherever that thing exists in amplitude I am like right fresh there at the very bottom so what was the number you got to twilio and he you remember um
yeah no I was I was very proud to crack like the top 50 that was like my claim the same because as people like left right you kind of like move up right right yeah it well yeah right it's kind of it's like on one hand it's like oh very cool I'm like one of the OG's on the other hand it's like oh my gosh like this person's coloring like that bummer it's
a shift but um I'm excited about it for sure so you have this new exciting role and I thought it'd be fun to start to chat about career growth and just how you think about career growth I know you have a framework of how you think about your own career growth it clearly it's worked out so I'm curious to hear about it and see and see how it could be
helpful to folks that are that are listed so yeah can you just tell us about how you think about career growth career growth is often right like it's definitely not a straight line up um but there's definitely some kind of Frameworks and methods that have worked really well for me and I think to dive into it it's first good to talk about
the one that I most typically see people use to try to grow their career and and why that can be a little problematic which is that I see most people are you know try to work really hard kind of the job that they have right within the role that they're that they have at a company do whatever you can to grow there show your manager all these things I've seen
people keep spreadsheets it sort of wins it's a performance reviews you know maybe you try to get better advocating for yourself maybe try to get peers to notice or your manager's peers and that's all good like it's all good stuff but the problem with it is that you're limited to what your manager's ability is to advocate for you to promote you
and you're also limited by the explicit trajectory of your role at that company and where the there's room for that or not in the company and then often that perception can sometimes be a little bit in contrast to what your perception is right and also other things that happen like you know your manager leaves and then you kind of have to restart with
someone else right so the method that I use kind of tries to take that power back a little bit and something that I learned really early on in my career is very lucky to learn by accident was at a company called bandwidth which is like my first kind of real job and you know bandwidth is now a public company and they've done all
kinds of crazy amazing things but I don't anyone was just 50 people and I actually joined in sales and I was you know just hungry to kind of make it succeed and grow and you know bright eyes and everything you know first kind of real job but I realized after a few months of being in sales that I was often like repeating the same thing over
again like using the same thing to sell over and over again and it's like gosh like this is an ideal for the customer so I'm gonna call me and ask me these questions and wait and get these answers and all this stuff and it's ideal for the company because they're like paying Commission on this every time like that's not going to be efficient for
growth and because we were small I was able to catch our GM and I was just like hey um you know I've noticed this pattern where like I'm repeating things kind of over and over again and like the rest of the same thing like I think we should put that online I think we should make that available so they can just like see
it and then buy it because we had an online checkout process and I was kind of expecting him to be like oh well I know it's important but you know for this another like we need to like you know we need to do it this way and obviously we've thought all about it and kind of thinking like oh I'm gonna come in like this new person who's gonna help
me understand what I'm like missing here there's like a little bit of that that I was expecting and he goes wait wait like tell me more about that what do you mean and by the end of the conversation he was like hey why don't you like go do that why don't you go build that like experience why don't you put that stuff and like in a self-serve flow and we
called it e-commerce manager and it was like kind of growth before this girl this is like 2010 and that that moved me into a totally near a new position and the main learning that I had from that was which really kind of took life at Twilight I absolutely worked for me there I'm happy to talk about that too but the core that learning was your
executive team and executive teams at companies are often very sharp but the nature of their day-to-day just does not link them with customers right and that means that over time especially as a company grows they often lose access to some of the best insights and you know in the heartbeat of the people who they're providing value to in contrast
to folks that are closer to the problem and so that means that your superpower is in really pulling those insights in and bringing them to life staying close to the customer there's not a single leader or executive it isn't going to be stoked to hear about valuable customer insights that highlight problems they might not be seeing and there's a lot of
those so especially you know when they align to North Star metrics those those ones are sort of the powerful ones that was the way that I grew my career at Phil and I'm happy to share kind of that Journey too yeah it'd actually be cool to hear maybe another example of that but I think I think an interesting thing that comes up
for me here is sometimes you may have an awesome idea and it may not immediately happen it may not be like yes or let's move on this right immediately and I think it's important to just recognize like they're not going to follow all your ideas but they're always looking for better ideas and they like to your point they may not
have the information that will lead to an idea that you will have because you're like on the ground dealing with real problems day-to-day so I think it's important to recognize you're not gonna always get your way and that's normal yeah totally and right and it's it's kind of about like almost like building up your your individual like brand a
little bit and I think one of the most powerful and accessible ways to see that's learning about your customers there's always those people at companies it's like oh well you know I mean she just knows our customers or he just knows our customers they just know our customers like they just know and it's like well they just know like let's ask
that person let's get their feedback and those people often have a good amount of kind of you know brand recognition and powers in the company and they're often thought of when the company needs to do something new or different or you know if someone is you know hiring maybe they're thinking about that person for like a cross team thing so it's one of
the ways that you can kind of build build that that brand and again it's it's I think it's a sweet spot because it's something that is very valuable to all everyone all the way up to the most senior leaders which we can talk about here in a minute and so it's going to be valuable for you and a valuable tool kind of no matter where you're at in
your career again that's not always an immediate payoff but it often does give you a trajectory outside of just your role and just your manager it gives you something a little bit broader so maybe a simple way of describing to kind of mirror back what you're saying is kind of carve your own path don't necessarily assume your managers will
give you the path that makes most sense for you or even give you the biggest opportunity just like propose hey I think this might be a better opportunity and I'd love to pursue it I'd love to hear the twilio example if that's generally yeah so I actually joined when I joined Twitter there was no growth team at all like not even a breath of it
uh I joined in product marketing and I was leading our product marketing for messaging lines and you know but I followed the same guy that I just mentioned I made it up you know my own kind of personal policy to like hey I'm gonna do my job and I'm like do well and keep notes of things I'm doing well and all kind of stuff because it's good but
I'm also going to get to know our customers and I'm engaging our customers really well I'm gonna pay attention when I'm connecting with them not just about the space I'm in but broadly what are some of the pain points and things are articulating that are relevant to the business and what we're trying to get done and one of the things that came up
was that users were struggling and folks are struggling to get started don't use twilio and that contrasted so deeply to some of the things that our executive team was saying and had high conviction in our company Hyundai conviction which is a tool it was so easy to use in fact it was like top three things about twilio that we were really trying to get
out of the brand we're so easy developers love us they say we're so easy and there's always like there were tweets coming all the time like you know developers saying like oh my gosh like they got started in a couple minutes so there's all these things that kind of made that compound and made that conviction stick but as I was talking to
customers I was hearing a very different story and it made sense as we were penetrating new markets adding more products we write in complexity and we were pulling in folks who were a little bit less motivated and those things contributed to people saying this is this is difficult and so you know at the time you know what this this wasn't a
you know 50 person America just kind of go to the floor and like go to school and be like hey like there's this thing I heard about like I think we should do something about it but there was another tactic that I could take and this is I just started sharing during a voice the customer report I started sharing my insights started writing down and just
sharing them and it became this kind of digest and eventually people like hey can I can you share with me can you share with me could be on your list can you share it with me and this was in like a few months of me joining I was doing this and then that turned into hey you should like host a quarterly voice to the customer session or for all of
the product right and and uh and this was a request that was coming from some of the senior leaders of the company and when our uh Jeff Lawson or Cersei at the time heard about he started attending too so now in the session I started pulling in other people's insights too right because this was these were you know now they had a forum for this I
could kind of do that and um and and you know have people send that to me and I could compile it and all of these things and so then you know this kind of established me as that person who knows about the customer even after a short tenure and then you know when you know came time to do annual planning that that year and I joined in 20 2014 at the
end so this is 2015. I pitched this idea hey we think that it's easy it is not here's data that I have the information that I have and I think that we need to start a growth team here and that needs to be a core focus and is able to bring in a really critical um you know kind of partner to that other folks who could support that because I built up some of that trust so
by the time I was making that pitch I had someone like you know on uh Andre Crow who was like the seventh hire at twilio and got to like number three on that spreadsheet or whatever who was really close their SEO being like yeah like we desperately need this I'm seeing this he led uh you know website he basically created the twilio
brand and he like let all the website stuff and he's like yeah we definitely need this so not only did I have that kind of a little bit of trust right for the executive team but I also had folks who were just trusted on their own advocating and supporting this that I was doing and so it was approved with like it's almost very easily I mean I
put stuff together for it but it was um you know kind of the meeting before the meeting had already been done by those other things so it helped me create the growth engineering growth product team at twilia I love just how proactive your advice is here there's a lot of people that don't do well and then just like oh I never
had the opportunity or I can get kept got looked over all this time and I love that there's all this just like here's things you can be doing to get in front of people to provide value to just create opportunity for yourself any other advice along the lines of just like here's the kind of things you could do for yourself versus waiting for
someone to come and give you opportunity yeah I mean I think that's the most easily actual because to do all of our jobs we need to know customers like we need to know about customer insights product we know but then also customer facing teams especially those who want to crack into product your your insights are extremely valuable you're talking to
customers every day you know more about their problems and their pain than a lot of other people do and so you know that's that is by far and away to me that the most powerful and accessible one by by anyone in any role in any space but you know I'll also say that you know that broader concept of just hey like there's things that you know
and things of value that you know that others can benefit from at your company and and building your brand as someone that is supportive smart creative able to solve problems you know make sure that you're sharing that right and so you know maybe you're really freaking good at communicating with brevity I suck at that by the way so like more
power to anyone that can do that I'm actively working on it so so share that like go to your general slack Channel or whatever and just be like hey like just run up like some tips for how to do it you know some ways that I am good at this and those kinds of things can really go a long way towards people starting to view you as you know an SME
and not just the space that you're in but in sort of broader areas and I can always present and open doors for you and other people are sort of looking up to you and seeing you someone who's who's strong in ways outside of just the role that you're in and SME is a subject matter expert is that right yes thank you for unpacking my acronyms
that's another thing that I am actively working on I got you I'll be on the lookout um maybe one last question along these lines is you have any advice for framing the proposal framing an opportunity to you know your manager higher-ups that you see has worked the best broadly yeah so yeah and one thing I want to say too is like with this stuff I don't think
that necessarily does go like counter to what your manager is doing it's more like supporting them like right like I've done this kind of stuff and then it's helped my manager promote me right so it's it's not necessarily oh we'll do this if your manager is like failing you or they you know are not the board in you or they can't support you it's more
like do this because this is going to be an accelerator for yourself irrespective of your manager but then also it'll be an accelerator for your manager and supporting you because one of the things that you know it comes into play a lot when manager to figure out promotions and doing those things is like they'll sit in a room off on calibrations and
you know with a bunch of people and it makes it a lot easier when those people have had some kind of access or exposure or whatever to you um in a positive light so these things can all on you know with you know your manager and and not against but it's just another way of you kind of taking taking back the ability to build that
momentum instead of relying on all of that going through one single other person what I like about your second example is you just did it you just started doing that kind of report for the company it wasn't like hey I have a proposal here's what I think you should do should we do it's just like yeah just do it yeah like
uh ungate your knowledge I think is the the buzzword that I'm I'm hearing right um and you could do that within your I think that's an Elena ISM like see if we can how many times we can we can bring her up but right it's just you know like you can do that within your own you know company like everybody everybody is skilled at things that they aren't you
know explicit to their role or their space and I think that ungating that opens opportunities and if you're not sure then go to my favorite go-to which is talk customers get insights those are incredibly valuable so related people share those when they find them so so be the person that does that another area I
want to chat about is experimentation and growth and data which uh makes sense you have strong perspectives on being the new head of growth amplitude so maybe we start with experimentation you mentioned that you had there's like a really interesting surprising result in an experiment you ran at twilio that kind of maybe changed your perspective
on experimentation and what you think might work and not work 100 yeah I'm sort of a fortunate have could have two mind-blowing experiments it really shifted the way that I think about growth so one of them one of my favorite ones happened very early on actually so after I created this girl team like one of the things that I saw as as to me an issue was that at our
signup flow we we just asked people for a username and an email like a password and that was it that's actually relatively common at the time it was just kind of a while ago now like everybody kind of start is segmenting users but we didn't and we actually there was a lot of existing conviction around that was okay we retarding
developers developers and you know they just want to do they just want to get their hands on things don't put anything in their way it's going to be disastrous we don't want any Shenanigans here with these these folks let's just let them in the gates but to me this was a really big assumption to make and very costly one it's like okay if that's the case we
don't know we're not gonna know anything about anyone and we didn't know who was signing up we didn't know what they wanted to do and that hurt our ability to understand how people were performing for a quantitative perspective you know we were a little bit lost with prioritization there's a number of implications here but it's obviously a
very contentious kind of space so I you know this is the very first thing that I that I did on the first experiment that I ran I um I did you know some research understand okay what are the most important questions to ask what would I do really really need to know and it was stuff like you know what what language are you know you coding in and what's
your use case what product you want to use and then you know there's you know one around like are you developer at all or are you something else because there was sort of rumors that you're having not just developers sign up which is this whole other interesting kind of story and I you know thinking these questions that would also you know
potentially be things that that our developers signing up would sort of understand why we're asking that it would feel natural and but anyway you know again like adding anything to the sign up was very contentious but I really just wanted to get a little bit of data on it so I wanted to run a test I didn't have a team I didn't have an
engineering team yet and none of that stuff had built out is Just Me Myself and I but like I said I had started to to build a little bit of trust and pulled in good old um Andre who I mentioned earlier who because he was early employee and just he kind of had access to everything like one of those those people and he had also was you know supportive of of this and kind of
had similar hunches and so and like the you know dead of the Night by that I mean like you know 7 P.M or something on I think it was pretty sure it was a Friday we just asked for forgiveness and kind of put these questions into the sign up low and rana's navy test with a small group and you know I'm fully expecting okay this
is going to like hurt our numbers but maybe it won't be so bad you know and I'm gonna be prepared to Advocate the power of this data that we're getting and I was totally weird I'm thinking about like written start to write like a like the framework for how I wanted to to surface this and we start to get the data for this thing I'm not getting an
improved conversion like there's no personalization nothing passed it just the questions it improved conversion by like five percent like just improve signups and it's one of those like what like okay this like what is going on here and I actually dug into it and what I found from just talking to a few customers went through the flow I'm just like learning about
all they felt about it it was actually for folks it was like comforting you know when you think about it when when users are signing up for your product for the very first time like it's new right this is new that means it's scary they're they're expecting it to be difficult they're anticipating that there's going to be friction and
challenges and then they're not going to figure it out I was like looking for the bogeyman right and that's the headspace it's often the headspace that any of us are in when we're doing something new for the first time like oh let's just be very challenging and so by putting in these questions it's like what's your
language it's like oh like I do I I code in JavaScript and and I can select that well that's something I'm comfortable with that would make my my journey easier like yeah Bingo oh and you're you know that's that's my use case okay like I'm in the right place here like you know it was it was actually giving folks something comforting and and challenging
the notion that this was going to be difficult just the questions because it was aligning to some of the things that they were organically thinking about which is what if they don't support my language or like can can I even do this use case I want to do and so it was just a really interesting you know the takeaway for me for this like the
relationship takeaway was the psyche of the user is so so critical right like that's just as important as understanding your product and the broader like Market you're applying to and all those things like just the psyche of of users new people doing things for the first time with your in your user flow like understanding that it's powerful you know in this simple
like kitschy thing I say is that you know ultimately the learning here was bad friction is bad and good friction is good right there's no such thing as being simple as just all friction is bad which is sort of kind of what I had assumed going into this I love that you uh you're new to twilio and he just yellowed an experiment to
production that's uh it's a big move it ended up being like very you know very helpful for everyone like you know shared the insights from it and like all these things conversion but for sure like you know use such you know processes with with caution uh for sure yeah the right way to do this like advocating for for the engineers here is the right way to make
any changes in production is is you know through or with the approval of engineering but it was um it was it was the right move overall and definitely helped business right so yeah I love it that's great I like that move I think we need more of that probably I want to dig into which actually so what is it you change you
added how many questions and then what were the questions there was a question around like what language are you coding in and then as an option to that it was like oh like I'm not I'm actually not coding like I'm not a developer like so that was uh it you know for us it actually gave us two really really interesting data points one was was like
you know how many developers versus people who are not coding or in our flow and then what language are they coding which was massively helpful not just for like growth and onboarding but our documentation team Doc's team you're gonna like what we should you know would that end up being a critical way for us to gauge Trends over time and catch
things before like you know whatever you know reports would come out the end of the year what people are doing we could start to see it and then also product what product are you interested in using that was very critical for knowing kind of the basics of how to organize someone's onboarding right are you doing SMS you're doing voice you know two
eight one whatever and then use case in use cases like you're doing a point where reminders or are you doing like a you know autoresponder or are you doing Anonymous Communications like for a dating app or something right so those were those were the very first questions wow okay so it was like four four drop down questions
and that increased conversion I love these examples where friction and increase conversion like there's so few of them like you hear about this could work and it's rare and so what did you take away like like what's the pattern you took from this like there's the idea it's good friction but is there something that you're like here's what
is a sign of this is going to be good friction this still alleviated a problem alleviated the problem they had where they're coming in and worried that it was going to be difficult or that they weren't going to be able to figure it out they weren't going to be able to get their footing and I'd say that that's not unique to twilio that's that's
something that I think users experience at any front door at any company any sign up beginning the sign up flow it's like here we go right like Buck like buckling up especially when you know it's in a work context and you know there might be extra pressure on you to succeed or you know for you to make an accurate assessment right so you just kind of you
know I think that's psyche of like okay am I like in the right place because it's gonna do what I needed to do like okay can I figure it out am I capable like that's these are extremely common things for people to feel when when they're signing up and so you know certainly you know that I think can concur to any any place I'd encourage
absolutely everybody to be putting those those kind of experiences within their early onboarding not just you know for you selfishly so you can learn and same with them appropriately but also so the user can feel more confident as or get going like hey I'm in the right place this is going to do what I needed to do but I think that the carry over there is
just the the psyche of the user right and just being so aware that it's not so cookie cutter as what is the problem my market experiences and what can my product you to help them there's also this other thing in the room which is so important to people's success their ability to succeed with your products in your self-serve experiences which is
what is the mentality and the psyche of the person at the various stages in your journey and it's you know if you're if you're not incorporating that or addressing that you will absolutely miss things or things will fail and you'll be very confused as to why we had a great um experiment that I'm happy to talk about where like same same concept but totally
different situation which is later you know in in onboarding one of the things that that we tried to do over time to make twilio less complex was to offer steps like you know if season one onboarding like welcome step one like here's what you want to build great we all know that now okay step one go do this thing step two go to this thing three this thing four
this thing if I bam you're live congratulations aha all these things so you know we shipped that you know got that out there and it was like yeah it was like improved conversion it wasn't like that great it's like man like we went from like they're being like absolutely nothing like Choose Your Own Adventure figure it out go figure it out
good luck to this kind of prescriptive thing and it like wasn't converting as well and you know sort of talk to some users and there wasn't anything particularly obvious that was coming out as to like what the issue was it was like oh yeah like I do step one and we did like mocks of the people like okay now I do stuff too but there was one thing that like I
was hearing that was coming out that was like feels like something and that was that the telephone number the Telecom part developers when they were coming into twilio was like things that were familiar to them apis the language they're coding in code samples documentation things that like the bogeyman right the things that would
like psychologically trip them up telecom phone numbers things that like you know these things that just were completely out of the zone of anything that they'd ever worked with before especially you know early earlier on in twilio's Journey but even now right telecom's very different Beast for most developers and guess what was step one get a phone
number because that's step one anytime that anyone's trying to teach one of these twilio like one-on-one they're always gonna sit down nice and like okay like here we're gonna go get a phone number and configure it and that's what anyone every time will do however in a self-serve experience when you don't have that safe person sitting next to me
like don't worry it's gonna be okay I'm gonna take you through this crazy Telecom Journey they're on their own with that psyche telling them like oh my God Telecom well I can't do that that sounds scary we're getting a phone number configurate like whoa I'm out of my debt right and so what did we do to test this out like test out whether that
was the issue actually and it's first we're in the MVP they kicked them out of the the portal entirely and put them into a docs page where we could kind of manufacture an experience where the first thing they saw was code and either in the docs safe plays the language that they're coding in and then like snuck in there I was
like go get a phone number like let's go configure it right not as like step one not as like the leading thing but sort of embedded and the analogy I have for this is like killing the hot dog so like if anyone ever has if you've if anyone's got a dog or an animal you have to feed a pill too it's like you can't
just feed the pill to the animal it's never gonna happen but if you shove it inside of a hot dog which like looks good and that's exciting then you can get them to consume it more easily and so yes we do peanut butter that's our movie that yeah exactly right yeah hot dog peanut butter all all that right you would you you kind of bury it right you
embed the like scary unpleasant thing and so that's what you said with the phone number stuff that telecom stuff and guess what even they were we good in the amount of the you know console and they're going off and and we had no easy return button it converted better because we were addressing the big problem that was there at the time which
is their psyche they were not ready to come in immediately thrown into a phone number experience that was you know letting the bogeyman and out to party and that's not what was going to work we needed to you know put that bogeyman pill in the hot dog and so you know then you know once that validated then we can actually go through the the business of
putting that into the the onboarding float correctly and then that covered even better but yeah so again the psyche of your user is such a critical thing to be thinking about and if something like very logical isn't converting well sometimes it means that you're battling against the psyche of a user and you want to take a step back and you know
think about and learn about where someone is psychologically in your space it feels like you had this experiment that was like a complete redesign of the onboarding flow and that didn't work and then your second attempt was like a different approach that's like a full onboarding flow and I'm curious you have a take on just when you run experiments
and something we dealt with a lot of Airbnb in other places like you just redesign the whole thing or is it better to iteratively work from where you are today and just experiment piece by piece towards some future much better experience here's what I said to this is that from a high level it's always going to be better to be
iterative the reason that it's better is that roughly eighty percent of the times there isn't the time are hypotheses and the things that we believe will be true or and this is like amazing I mean the there's an amazing article out there I'm happy to share with you you can put in the the um show notes yeah absolutely that really takes a scientific approach
to proving that out companies like Netflix and Microsoft there's over and over again 80 plus percent some companies saying 90 of things fail and so when you the the closer you get to something that is you know you kind of go bury your your you know head in the sand or go into an attic and build something for six months and ship it the
more likely it is that you are gonna ship the 80 wrong stuff right whereas the more iterative you are the more likely it is you're going to catch it sooner and you know failure doesn't have to be a wall it can be a compass right it can be the thing that leads you to the right thing and so you know you always want to as best you can get stuff in front of
customers so that you can get that compass and you know get get that Compass activated know where to go and yeah and so that means doing you know ugly things like to tell my teams all the time if it's not embarrassing you've gone too far right gotta be embarrassing the first thing that was embarrassing kicking people out you know onboarding
spend all this money and like whatever to get them into your Center phone and the first thing when you get out of here right I mean that's that's nuts but if it hadn't validated that would have been a very cheap but very valuable learning instead it was a very powerful cheap learning in the other direction where okay now we know we can invest in it we
know that's the right thing to do so always better to be iterative so that you are letting failure work for you instead of having it to be a trap that you fall into I know that stat you just shared is like per experiment you're probably wrong 80 of the time in my experience launching a whole redesign is is negative 100 of the time I've been
I've grown weary to avoid that as much as possible which is like you kind of know that you taught that as you go into growth and product but you're just like wow come on let's make it awesome just redesign this whole thing especially you know your designer is always like now let's start again let's make it amazing but it always ends up being negative and
you're like okay well it's too late now we gotta launch this thing we don't have time to start again well it's funny I mean in the Articles and you'll see like it was um it was written by Somebody from Microsoft who kind of built the implementation platform and did all these cool things like as he went into actually trying to apply a scientific
method of figuring out like how often people are wrong about their hypotheses and and you know what they're planning to do he's like I wonder if like that applies to us here at like Microsoft there's even for him that kind of question of like and it went it wore out right like it's just you know I think it's challenging to you know when
there's a lot of smart people in the world in this space doing things and it's very difficult to think gosh am I really wrong like 80 plus 90 of the time but when you think about it makes total sense right because what has to happen for something to be successful you have to understand the problem perfectly you have to then understand who's having the
problem perfectly the customer at what time they're having the problem then you've got to put the right solution in front of them to solve that problem maybe is that the problem right now all that stuff right but your solution something off right or maybe your solution is right but maybe it's just not presented it kind of communicated in
the right way you could have any one of those things off and it's not going to succeed right it's not going to have you know the the metric impact you're expecting to have so in that context it's almost like incredible we do succeed 20 to 10 of the time given everything that has to line up and so I think it's one of those things where you
know you really want to go into it embracing that okay this isn't about how smart I am or how good my team is or any of that stuff it's just hey the logic of this is a you know it's challenging to get it right and let's Embrace that and let's um lean into let's lean into to that that knowledge and make it a part of our strategy instead of fighting
against it have you found anything that helps you increase those odds or is it just this is the way of the world and you probably can't significantly increase the chances your experiment works out so here's the thing I mean I think I think there's there's very little that we can do to make that space easier right all those things have to be
figured out and so I I definitely think that you uh that that everybody is going to be in a space where their original you know ideas untested ideas are going to be around that hit rate however the way that you go about validating those can be totally different and you can be very fast about valuing those ideas and that's the key right and you know uh a b
testing is one of the most expensive kinds of ways to validate an experiment right it uh you know often requires design and engineering and the PM or growth person or marketing person who's crafting it right all these things or Investments that take a lot of time even for simple things and then you have the time Factor not only is the thing I
have to run to have and have an impact so all of that is extremely expensive and so I think the key is to just think through okay what are the things I can do to quickly validate what what these ideas are that we and you can do that with painted doors right which is where you test rate the concept and the idea before it exists
versus the actual experience you can do mocks like create have you know create you know if you've got a designer create those mocks with that experience put it in front of people see how they engage with it that can be so powerful you could invalidate tons of hypotheses at that State you know you really want to only things the
only things you want to get to that kind of that deep a b testing environment or ones that have been kind of vetted along the way and that way you reduce your fail rate right because you're you're failing Faster by using other methods so you know I think that's I'm more advocate for that side like let's fail fast by using those tools rather
than figuring out a way that you can rise above where everyone else is operating and figure out ways to solve all that complex stuff better because that's going to be challenging but you can always get better at experimenting and validating things faster this episode is brought to you by writer how much hype have you been hearing about generative AI so much but how do
you take it from a Shiny Toy to an actual business tool that helps you do your actual job writer is an Enterprise grade generative AI platform built specifically for the needs of businesses an already widely deployed it world-class Brands like uber Spotify HubSpot and uipath with writer you can break through content bottlenecks across
your organization from marketing web pages to sales emails and product messages to creating high quality on-brand content at scale and unlike other AI applications writer's training happens securely on your data and your style and brand guidelines so you provide specific to your organization the result is that you get consistent
content in your brand voice at scale get AI that your people will love for a limited time listeners to Lenny's podcast can get 20 off if they go to writer.com Lenny that's writer.com Lenny where do you find the best ideas come from for driving meaningful lift like is it gut instinct type and experience uh bucket or is it data telling you like
hey or here's a huge opportunity in your experience I'm a very like data driven person like I self-describe and think of myself that way because you know in large part because of that I feel really you have to be constantly checking yourself and data is a really great way to do that but uh I definitely think that I would be described as someone who's like going
more by their gut when looking at date end results just because of the way that I approach it which is I'm very comfortable and very common in using qualitative responses and things like that and supplement to quantitative data to make a decision and that puts less of a burden on the quantitative to really make an assessment whether something was
working or not you know one of the things I see uh like I I think sometimes goes against whether folks do although I'm seeing kind of things shift a little is that 95 confidence rate right I like I I came from uh like my background like in college I was in a lab running experiments or really publishing to a journal and stuff and we had to have
that 95 confidence I had to because the things that were coming out of the lab and being published were influencing things like how we do education and how we understand how bias works and when it shows up and therefore how we can combat it things were were wrong and seeing a bunch of baloney like that can cause some significantly bad things like false
positive false negatives in that context it's gonna be very um dangerous for like a better word you think of other you know Pharmaceuticals like there's the 95 confidence rate belongs in some companies in some Industries because the risk of of you know the impact of a false success is very high but those of us converting users and trying to upsell
folks like we're very fortunate to not have the that level of burden on us and we can take advantage of that right and so there are definitely times where I will advocate for and I will push for an owl myself used lower lower confidence intervals and 95 especially if that like doubles amount of experiments that you
can run in a year right end of the day these are all methods that we use to try to validate the hypotheses that we have and if if you have like if you're doing a 95 confidence interval you're still accepting a five percent you know some amount of of false success do that a little bit more challenge you do a little bit more and then run way more
experience if you look at the the net of what your team is doing over the course of your what you're doing over the course of a year you will be positive wow that is a big idea idea of releasing the p-value confidence interval for experimentation and data team Z like any everyone would be happy to like be excited about this probably maybe not
some data scientists on teams do you do that like how do you actually is that how you operate on your teams just like we don't need 95 confidence so I'll say this is actually very critically important you must have this game plan set before you run something a failure mode that I've seen so many teams fall into is they'll run the
experiment or whatever it is and then they'll you like make the data fit yeah the hypothesis right yeah or sometimes alone without a hypothesis just like this is gonna do better things for our metrics but not like a core reason as to why or what you know what exactly are we testing here and so this this is another area where you could absolutely fall
into that trap let's hope we're only good 80. I think it's good that that Laura person said it was cool so I I think that that's fine like that will always be a trap right so it needs to be very deliberately thought of in advance as a way of like hey here's we're going to validate this and always always if you're going to you know accept more
risk of a false success or false positive false negative you want to then be really thinking about how you're gonna Harden your your validation of hypothesis for example let's take that when we talked about with uh trillia where we're kicking people out and we're sending them to that you know that the pilton hot dog experiment right and
we're sending people to the experience to kind of hide the phone number you know in that case right let's say that we were going to accept a lower confidence interval I would very much want to see qualitative feedback to confirm that that hypothesis was true I want to be looking at the qualitative data from the ones where people were
thrown into the existing flow and one's put into the dogs that one of them felt more confident and more like this was really easy to get through and like they they felt better territory and things like that and I'd be wanting to hear from the ones who were in the other one things like oh I kind of got stuck on that phone number like ah like kind of
figure this out but it just it feels like it's I'm out of my depth like I would want to be looking for other things to corroborate um the hard data that I'm seeing right and yes it opens the door to you know whenever whenever you open the door to more risk acceptance you are going to have some false successes there but all of these things together can
overall make it more likely that you're shipping more things that are going to positively influence the customer and again it can't say it enough it is a huge risk in and of itself to not ship as much as you possibly could in a year that is a huge risk given that very high fail rate so you know I wouldn't you know to this data scientists and I've
you know chatted with a few of my time what I try to explain is that that article that data that the 80 like that's hard data right about what a detriment it can be if you are um you know if you don't run enough experience you could literally if you just run 10 in a year on there are maybe two around in fact two of course an
entire year if you kind of take that approach so you know data scientists can understand hey if we do this you know if we we move this down we can like double or triple whatever it is or the number experiments we can run and overall net that's going to result in you know few you know more successes that will overall net us to a positive place like
you can still tell a data story to the data scientist about why you're doing this again this is why I you know when you ask that question identify it's a very data driven person but I think you know some of the methods that I use can uh you know sound at the service level as more like oh I'm going by my gut but again very data driven it's just
embracing the reality of some of the hard data that I don't think we all Embrace or are even aware of sometimes about that that sale rate this is awesome this is a big idea have you written about this anywhere for folks that maybe want to try this Approach at their company and if not you should I appreciate that I it's funny I it's
been it's like on my general life to do this is to start writing some of this down I have three children one of whom is five months old two and four and so um sometimes they'll start to write and then like one of them like crawl across the keyboard and you know like this like you know one of them I mean all of them multiple times but eventually yeah I'll
be very happy to to do that if folks would be interested I um I'm always I'm always happy to do whatever I can to help folks help Empower folks with knowledge to do better because this none of this is like Secret Sauce really it's just sort of learn from experience and it's always better to learn from others experience than your own it's faster so
yeah I would definitely I aspire to Lenny is that I think that's the best that I can say but eventually my kids will get older I hear this and and maybe I I can do so hopefully uh cool so maybe if you're watching this YouTube leave a comment if you want Laura to write in depth about this idea and spread it it's your company
okay I want to talk about growth but I have one last question just along the lines of experimentation is there any other just like I don't know big lessons or takeaways of just of running experiments that would be interesting to share I think we kind of got into this one like a little bit but I just really want to exclamation point underline it which is
that notion of kind of making the data wrap to fit like a concept you know I think a lot of teams are you know feel and are under a lot of pressure to show progress right and what did you do this month where do the metrics move and like you know and and it can cause folks to feel like they they have to do that where it's like oh gosh like everyone's
got this experience where you run an experiment you're like looking at the data refresh refresh refresh oh my gosh it actually perform worse you know or it's like not the same and gosh we got everyone really excited about this thing that we all like worked on really hard like oh my God what am I gonna say in the qbr or you know the the
monthly report whatever it is that sort of these things you know the results come to light um and and so this had to I'd say this that it's incredibly important for growth teams to educate out and for folks outside of growth and leading growth especially to to understand that the the best way for a growth team to succeed the the only way really for them
to succeed is to embrace the fact that they're there to validate to to understand what the biggest opportunities are and to go after them and that is not something that can be done on a weekly timeline sometimes even a monthly depending on the space you're in and what's known and unknown and so any growth team that's that's beholden
to short timeline wins and Improvement is always going to be dangerous that's a that's an environment that's conducive to vanity metric usage and Fab you know massaging the data to fit and ones that are more successful are ones that are reporting over longer periods of time because any growth team given enough time to kind of fail enough time to
learn the right thing to do is absolutely going to have shows exist real success right like not the okay we're gonna make this data fit but like real moving the metrics is up and so definitely you know educating out if you find yourself in a position where you you are holding into that you know share that 80 fail rate like just math
statistics data like that is not you know you cannot be successful in an environment but over time you you can be and so uh that's that's one thing I definitely would would draw on that they spent you end up spending a decent amount of my pie chart at uh at twilio and then also at rapid where I was after that and I'm sure I'll spend some time
at amplitude as well just helping folks kind of understand what is the the healthiest ecosystem those powerful ecosystem for a growth team to operate in and time and expectations over time is a big problem we say pie chart it's like the pie chart of your time like a big chunk of your time goes to this that's awesome I like that I use pie charts a lot is to
describe that same idea just to be a little more concrete there what is the time frame you think is the minimum for a growth team to be thinking across I think it's good especially for for newer teams but even teams in general commit commit to something that you can kind of do over the course of a year and you know low medium high right is is always
helpful in that space I mean you know a lot of times well you know I'm more like you know hey we've got a few bets that we have a few core hypotheses and if they take off that's going to be our high look at like wow so they like we think these things could be kind of lightning on a bottle here but they could also be you know a
bunch of paper Mist so but until we run We're Not Gonna know and if those bear out though then yeah that's that's our high and hey we've got a few things that we think are safer you know um maybe it was sort of validated a bit in the previous year what have you and you know these will be the metric system out so it's helpful to give people
though that that construct it deviates from it very hard deviates from this notion of like here's the single number that we're gonna hit right just things that help people kind of understand that space a little bit better and what to expect and because of that it can be a little bit lumpy like you know there were some things that you released I
mean it's cool it for the most number of years which can be easy to talk about this this construct here but like you know there's one thing that we did that generated like tens of millions of dollars in Pipelines really really powerful and took you know sometimes kind of navigate and invalidate a lot of times we did like that onboarding stuff
that's talking about like kind of like catching those things like that was you know could happen on a little bit of a faster clip but still you know took some time to validate and understand but yeah over the course of a year you should generally kind of be able to commit to movement but help people understand the methods there so that they're not coming
at you on a weekly basis being like and what did you do this these past couple days okay I gotta I gotta follow up on a couple of these things uh what is it that what was that big change of twilio that led to tens of millions of dollars this is part of um the the course that I teach at reforge so um oh amazing good like reforge what's
the course called just one activation retention I think it's um live is my part I think it's um right now we can I'll get I'll give you a link so we can yeah in the show notes but um we had a high level version is we you know this was this was kind of deeper into my my journey at twilio this is uh you know fast forward a few years you know to
kind of blown up this this team and some cool things going on but I was really looking for kind of what's like the next big thing for us to do like what what could that be and I noticed you know remember as you know that the question very back of the day when I asked about like the kind of developer versus not developer yeah we saw that that little
non-developer little dude ad was kind of growing right we were actually the number of people in the ecosystem who were identifying themselves is not a developer we're in the space but very interestingly they were as we kind of got got more refined in our understanding of those folks a lot of them wanted to build with twilio like you know there was a
hypothesis of like oh well maybe they're lost maybe this wasn't pricing like maybe they sent a bite mistake and I was like nope they're here to build they wouldn't build you know and then they kind of struggle through the developer onboarding and someone would succeed in someone but anyway um the it was all about identifying like
what did they need to succeed like were they if we were made them successful could it contribute to Dollars you know one of the core learnings I'd heard from sales at the time was hey it's very challenging for us to get the kind of folks when a developer's not involved yet to to go from zero to one to get something off the ground but man if we
can get them to do that if I can get in one dollar and spend I can get them to five and then five and get them like ten thousand then I can get them 100 000. this whole long journey like hey Laura if your team could just get them like off the ground man we can do so much so yeah the journey is all about okay what what were the things that were
missing an experience we were offering and you know ultimately was they couldn't write code from scratch that was really difficult and also we're going to stand up a server that was difficult but we ended up um iteratively experimenting a way to validate those hypotheses and what's the right way to do this and yeah and it was right it's called a quick Deploy on code
exchange um anyone can go there and deploy an app without having to write code and kind of get in a hall moment there with twilio that is awesome so basically it's like a low code twilio app yeah I ended up kind of being like we had a lot of like like pet names like nicknames for I think probably the one that most succinctly describes it is
just it end up being kind of a create your own demo experience which you know made made you talk about the psyche of people you know you know we talked about how developers Telecom until they can be inhibiting we'll talk about like this you know the non-developable like somebody's the buyers or like the people who are you know it's like buying
decisions like for them it was like Not only was it like Telco but it was like the developer stuff was inaccessible but they still wanted to you know jump in and they wanted to have that experience and so this was a way for us to give them momentum give them comments well geez if I can get this running my development team definitely do it right
and so it was a very powerful kind of uh you know moment where we could really address the psyche of those users get them excited about twilio and then give sales kind of the ability to give something powerful to those non-engineering buyers and folks they're talking to so genius looking back seems like an obvious win I I imagine it was
not one of my readers suggested that I start a series of the story of a feature and kind of walk through the discovery ideation development iteration and this feels like a really interesting example that but anyway I got just a couple more questions I know we've been going for an hour now so but I have questions I don't want to let you go just yet and they're
they're on growth so so one question just you worked at twilia which is very product-led growth you're gonna be you're now to amplitude which is more sales driven and I know it's you're trying to go more product-led I know Elena talks a lot about this how every company needs to have product LED motions otherwise they're gonna be
disrupted by someone that comes product LED and I don't know what hire is which which book would they fall into yeah between the aoli like SLG and plg I mean yeah for me it's um they're they're kind of two sides of the same time coin like you know growth grows in sales it's all it's all to me very thematically the same stuff the difference is that with
growth you are selling with your product and with sales you're selling with person like one to one and you know so so companies need to be you know employing both of those forces to optimally converge Their audience uh you know we're in a world where people are expecting both they're expecting to be sold by your product and sold at the
Enterprise level in large companies by by human beings it's going to listen to their specific needs and really break it out for them and you know if you only have one you're gonna You're Gonna Miss stuff so absolutely I think you wanna those two forces together are working well and obviously they're sort of you know different stages you know things
work differently in different spaces but you know I think when it comes to to amplitude I think there's a huge you know opportunity here I think the key is and that the challenge you know for companies have done like the sales thing and are trying to crack into the plg thing is you know it really comes down to how you fundamentally are approaching
that space and again your users and you know they're where they're at and the psyche of where they're at I think a lot of companies will say well okay like Hey we're gonna do this plg stuff let's take that sales like Enterprise whatever offering that we have and let's you know chop it up a bit and like cut access here and like
cut out this feature here and we're gonna like slap this plan out we're gonna put a price on it and we'll maybe have like hours of debates over whether it's like 10 99 or 104 or like 75 and eventually we'll like someone will in that battle like slap it on because anyway the discussion or the focus is a lot around the product what are we gonna
do with this product how are we gonna crack it open and shift it and and then give it to these you know people all these users these these visitors and what it's missing I think is and a lot of times it's easy to miss is that when we're doing plg and we're shifting from sales to plg we need to reset we need to recognize that you know again
this is this is sales sales via the product what do you do when you're what does a good sales rep do when they're engaging they they understand what the problem is of the person and the space they're talking to so we need the same thing here what are the unique problems of people who are coming into our self-serve space and I think you know
when it comes to a company like amplitude you know you're uh you know a lot of the folks that we'll be kind of looking to address via the the plg motion you know there's a number of things you want to achieve there but one of the primary things is to kind of tap into the SMB market and really give them a really you know startups and give them
a space to land and to grow and you know you have to think like what are the challenges and unique problems that they have because we're going to be using our product to kind of settle them we need to meet them where they're at with the problems that they've got and I think one of the things that is that I've I've observed from being in all these
startups and you know advising some startups is I very rarely I think I've ever come across a startup where they have like the right number number of analysts for their needs in fact a lot of them like don't have any and so what that means is that the CEO is you know being an analyst to create their dashboards for the board and you're the
product manager is like being an analyst to figure out like what the heck's going on and creating the reports for their product and that's happening all over the place is that people are in their roles and they have to be an animals too and I think that that's a problem that especially you know younger companies and early stage companies have and so
when they're you know come back to their psyche what are they caring about what are they thinking about when they're signing up for product analytic product or something they're booking for something that's going to help them feel reassured they're going to be able to actually get to the bottom of the right metrics create the reports that show
things the right way I you know what's the best way to show churn there's got to be a best way like so many people are doing it guess what yes there are some really good ways to do it and there are some really successful you know ways to to set up dashboards for the board people have done that too like there's a lot of that knowledge that exists and
all those templates and Frameworks exist benchmarking are these numbers even good right and so one of the hypotheses that I have is that you know if we we take that perspective and we understand you know that that is the problem that there's a number of things that we can do to really change the way that self-serve experience Works to help convert people
and show them how amplitude can make them kind of that powerful but but the thing that I think sticks across all companies not just amplitude making the shift is just that that when you're doing this do not think this is a copy paste but like chop it for parts thing don't start with your product when you're building out your strategy start
with your customers your users your prospects your people who are going to be coming into your self-serflow make sure you're understanding how their problems differ because they do from the people that you're addressing at the sales lead side and then make sure that you're orienting your experience of product around those people it's
interesting that you almost have to kind of start again as a as a product company as a product yeah because you may need to solve completely different problems that eventually lead to the same place but it's interesting what you're saying that you may end up targeting like analysts or PMS or I know amplitude or has always focused on PM's but yeah and
it's right and there's always the nice thing about it is it's in some ways it does feel like you're starting fresh because you do need to kind of start with the customer again and like what's their problem but in a lot of ways you know you can carry over a lot of the same knowledge I mean that point you kind of know what's working well like
amplitude for example does have a ton of knowledge around what some of the best ways artists at other words like there's a lot of things that they have the momentum going sort of like where do you choose that momentum and how do you put that and curate that in front of users and make sure that they're getting the right things there was a ton of momentum
already there it's just a little bit about harnessing it understanding that yeah where are the gaps because there are going to be gaps but you know anchoring in a customer problem as I think the way that you start any new product any new thing that you're releasing should always think about like the customer and the pain point so no
different than when you're doing you know plg for the first time or cracking into it you need to be thinking again starting again with the problem the problems they have the psyche that they have coming to your your space so that you can build something that is going to effectively make them feel like oh you can solve my problem you get me and and
show them how your product's going to do that final question and this is around developers you worked at twilio obviously twilly assault developers I think rapid where you work right before amplitude also sold Developers selling to developers feels like such a hot space right now there's so many startups that are just building developer tools such a huge Market used
to be not used to be like there is not a market in developers they don't they're not going to spend money there's not enough of them and now it is a big popular spot and so I'm curious what have you learned about building a startup and a product that sells to developers I imagine a lot of Founders building sorts of tools would be really
curious the first is that developers are just a very different audience from any others I've seen so many people who have come in stronged on grills really well or product really well with other audiences and like oh like I'm gonna take all those learnings on a pivot into serving developers and as a being a very steep climb because developers are so
different um and let me give you like a couple just fun facts yeah that make them really different and some of these have kind of some interesting stories one is developers like almost 2-1 do not look at your marketing website at all they go straight to your sign up flow so what that means is all that beautiful context you're setting and you
know the product and pricing all that stuff like very you know very often they're skipping all of it context free and going straight to your sign up and so anything that is not that you know anytime you make an assumption like oh well they probably know this coming in to sign up or like well we don't need a clue that's on the marketing website
like none of that's going to apply to this group of people they are like they're there they're they're looking to do a um I describe them like the analogy of for this group is like they're the the Ikea buyers who you know when like a package comes they're not opening up the instruction manual and reading it and then starting to go through they're in
there tearing open the bags and starting to like pull the pieces together and trying to build it right they'll come up for context and and steps and such when when they get stuck if they're motivated so uh that's that's one thing and then you know another one is just the aversion to talking to sales and I think everybody can you know hearing outside
like oh yeah well I hate sales too man I when I sign up and get bombarded by sales that's the worst I totally get that but I have just developers around this whole other level there was a there was a Fang company who signed up for twilio built a POC launched to production all this in operating that space for months
without engaging once with sales so it's trying to reach them right and I ended up being the one that talked to them like first because they like reached out to support because there was something about their delivery that was was off there like missing a feature and they did not want to talk to sales they ended up talking to me and this is when I was
in product marketing and I was like my first exposure but people did not want to talk it's just sales you know and then there's another one where like a giant retail company where like the engineering team signed up with their personal email addresses so they wouldn't get bombarded by by sales it was only like later that we like found
out it's so but but the thing that's most important these are fun facts but but the thing that is I would say is the most important to leave thing to leave with with listeners here is why like what makes them so different like like why like why what's the deal here and it stems from their Charter and their responsibility so you know if you
put ourselves in developer's shoes for a minute a developer if a developer is required to use your product especially if they're like the primary user the primary Builder it's really important to recognize that they are responsible for that if your service goes down that's their responsibility not just for themselves but their team if the pager
wakes up someone because the service they bought from you goes down that's on them if oh turns out that doesn't work with with the systems that they said it was well that's on them like doesn't integrate with the data the way that you know everyone wanted it to that's on them everyone looks to developer when it's not working right and it cannot
work right in so many ways that's their failure that can they can cost them their job it could cost them the trust to their team custom their reputation you know and that means that the stakes are very high for them every time that they're adopting something new so they can't afford to take someone's word for it especially a sales rep who might be
have some other motivations right from their perspective right they they can't afford to trust your content or someone's word they must do it they must prove it themselves and so that's why they for developers to be bought in they need to do something build something a proof of concept at the very least if not kind of like moving further than
that and so that means they're going to be pretty darn deep in their self-serve experience with you before they're ready to commit and so if if you're a company that is providing uh that requires developers to build right you must invest in self-serve experiences in order to effectively convert your audience and you should be
thinking of them something it can your self-serve function and growth folks someone akin to Salesforce because your sales developers are not going to accept sales coming in and trying to convert them at that stage I love that you always come back to the psyche of the user and how you know in this case developers like here's why
they're responsible for this thing sales people are going to convince them this is going to work and it's not and um that's a really interesting tool and that's a really cool takeaway is there anything else that we didn't cover before we get to a very exciting lightning round well any I think we covered it all man got all my questions
and more so with that welcome to the very exciting lightning round I've got six questions for you are you ready I am so ready what are two or three books that you recommend most to other people I'm a big believer in happiness not just like you know because you know I'm being crunchy or you know we should always have it also because it helps us do our
best work and were more creative and all this so one is simple path to wealth by J.L Collins I don't ignore the data that money is something that often gets in the way of our happiness you know so many smart people that just have not figured out the whole like managing their finances thing and this book will cover all of your Basics it's very easy
to read it's got an audiobook that he does he marries himself simple past wealth Dale Collins he's fantastic what's a recent movie oh wait wait there's more oh there's there's one more oh let's do it happiness which is atomic Habits by James clear it is if you ever want to change something yourself or something like not quite working for you
this guy will give you a framework to change it guaranteed I really enjoyed that book that guy's killing it he was on Tim Ferriss he had a great interview folks that don't want to read it they can listen to that there's a lot of cool tips there favorite recent movie or TV show unabashedly The Great British baking show I love that show I love the
show for like all the reasons everyone loves it shows like heartwarming and like makes you know makes you feel good not lives you but also because it is a competitive show like they're trying to be the best Baker and they're out there helping each other they're like a big family like I've most reality competitive TV shows that I see all of
them are like Cutthroat they're sabotaging so I'm just endlessly fascinated also by like the psychology What's Happening Here I want someone to do like a research paper on it like like what gets the bottom of like they're all like helping each other it's wonderful though wonderful to watch interesting I always come back to psychology with you
I know I know I feel like I'm just I'm like really really Sinking Deep in there but it's true though it's like very interesting to me so I and I love that show what's a favorite interview question that you like to ask in interviews I love asking about a ship or release that is not cherry-picked by the person you're talking to you can get it in a
lot of different ways the things everyone has like a big success story everyone does so it really doesn't actually tell you very much to ask someone like what's a great thing you release because everyone can tell that instead take take that away what's the most recent ship is a really easy one because like recency time but you know there's
other things you can do to like kind of take that out like just give specific parameters for a ship that they shared or whatever and and that will allow you to listen more and learn more about their Frameworks versus the you know outcomes because if you're picking a random ship odds are probably wasn't like fantastic so they're going to want
to talk more about how they approach getting there and that's what you want to know about to know if they're going to succeed what their Frameworks hardly approach that is cool I've never heard that one that is a really clever idea what are five SAS products that you use it in your day-to-day work can't say amplitude I know right um still learning what
which ones we have here um but yeah I'll share I'll just share sort of like the ones that I like like a lot that I've used elsewhere so one is hot jar um hot jar qualaru also work just anything that allows you to put some quick little thing in front of customers get that qualitative feedback we talked about it's a critical critical
supplement to quantitative data to understand what's really causing the change or not causing the change okay or c so that's important I will say amplitude is a fantastic tool that I have used and I I would have said that if I weren't hadn't just joined amplitude so um I got to use it I know I got to use it for the first time at uh Rapids use
amplitude and it was uh it was awesome so again like asterisk because I'm like working there now but I do actually like it and you know slack there's boring everyone says like but I just have to have to hand it to them it makes life so much easier and just not not their way and then Builder which also put in asterisk on that one but I really want
to serve six a lot of people don't know about it and it's really helpful I do advise them so I'm like you know in their corner but this is another one I also say would would be a powerful one I think a lot of teams get stuck they're not able to um they're lying too much of their Engineers to make changes again we talk about rapid experimentation getting
these out out and Builder makes it really easy for folks to do that also like a headless you know know CMS you can like drag and drop headless CMS so they do make it easy for non-engineers to make changes so especially if you're trying to figure out how to figure out how to get around that 80 percent foge man that I mentioned
um this Builder would be a good way and then yeah if you want one more I'll give you chat GPT which is really boring and you know everyone's saying that but I think I'll just say I don't have any like crazy things to say about accept that I do think we all need to figure out how we pull that in to our sexy people who don't do that are probably
gonna lose out you know or or like you know smart AI whatever Bots but that would be it for you Lenny but if you ask me in a few months after I've actually been in amplitude for a bit I'm sure I'd give you a different answer uh that's a good time to plug lennybot.com someone made a and I wrote a newsletter or Dan schipper who created
the bot wrote a newsletter post about how you built this thing and so you could go ask me questions using the content of my newsletter as answers and it's very cool lennybot.com or lenny'sbot.com amazing there we go well there you go I've changed my answer yeah yeah that's all you need uh see more questions what is something relatively minor you've changed in your
product development process that has had a lot of impact on your team's ability to execute yeah that'd be embarrassing like they mentioned earlier like be embarrassed by the first iteration if you were not embarrassed you've gone too far that really speeds up ships and helps people celebrate the unpolished as opposed to feel embarrassed about it so
just embracing that awesome and final question I know you just started amplitude but you have a favorite pro tip for how to use amplitude or maybe hidden feature people may not know but you tell me day two and out but I'll say um one thing that was super cool actually that someone put together on my team at rapid literally or before I left
he put together a video of how powerful amplitude could be when Linked UP And integrated with other things like in this case hot jar and segment there was like a amplitude report that that someone had kind of created and there was something that was kind of an anomaly happening there like users were kind of using something in a way we
didn't expect and amplitude one of the reports surface it but of course then we want to know like why is that happening and so we could take find out what the event is and then using uh using segment find out what that name was then look at hot jar and actually go in and get screencasts of people doing that exact event and from that we were able to kind
of form some really concrete hypotheses about what actually was causing it and so you know obviously you know talking to customers is very powerful but in this case just that simple use of kind of connecting and threading those Technologies together could really get a good picture of that without needing to be engaged customers so kind of the
power I'd say of the the tip would be how you can really amplify well you could have amplitude when you use it with amazing Laura we covered a lot of ground career experimentation growth embarrassment psychology thank you so much for being here two final questions where can folks finding online if they want to reach out learn more maybe send
you an appreciation or two and two how can listeners be useful to you yeah find me on LinkedIn I don't post a lot again I will blame my three children eventually I I promise that I will but I'm pretty good about responding to messages so um so definitely link link with me there and then yeah what what listeners can do I think uh yeah I mean
I'm always happy to hear feedback suggest and all that but I'll just say I also know that it's a little bit crazy out there right now especially folks working in Tech so I'm also cognizant of what I might be able to do to help all of you I I know um there's there's a few places I advise and you know rapid time I know of a few
folks that are hiring um growth strong growth people and and product folks so if you um are are interested in learning more about that don't hesitate to hit me up I want to make sure that that uh I help as many people as I can in that respect because it's you know trying times and it's I'm sure you've heard it and read it but if you're laid off this
is not about you it's not your fault this is crazy world we're in you things will get better and I would be very feel very lucky if I could help even one person land so feel free to hit me up about that too awesome and maybe if you share some links we could include links to open roles in the show notes yes I know this there's a few that don't have
JD's open yet they're that hot off the press but I'm happy to uh yeah service a few things there for sure because I know that makes it easier for people to know awesome we will do our best with the show notes then Laura thank you again for being here yeah thanks so much for having me this was awesome and so much fun bye everyone
thank you so much for listening if you found this valuable you can subscribe to the show on Apple podcast Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can find all past episodes or learn more about the show at lennyspodcast.com see you in the next episode