being able to identify the various micro and macro Loops how they're all connected being able to document them in a qualitative model to communicate a shared understanding of how you grow it's really powerful augmenting that then with you know the quantitative side of things that helps guide quarter to quarter focus and ensure you can be
intentional about where you're investing that becomes a big enabler you know you're never gonna have a shortage of ideas in a high performing growth team so knowing where to focus amidst that kind of sea of ideas is a really important role of the strategy welcome to Lenny's podcast I'm Lenny and my goal here is to help you get better at the craft of building and growing
products I interview world-class product leaders and growth experts to learn from their hard-won experiences building and scaling today's most successful companies today my guest is Ben Williams Ben is a vpf product at sneak which is likely one of the biggest and most interesting companies that you've never heard of sneak makes it easy for
developers to catch security issues in their code and there's a lot to learn from how sneak got started it started through product like growth evolved into product life sales was very Community Driven it was also laser focused on developers which has become one of the most lucrative markets to go after in our conversation we cover how the founders of sneak out their first
hundred users what they got wrong when they tried to monetize early on when they hired their first marketing and sales people how they structured and grew their growth in product teams what they figured out about what she'd go into freemium and what shouldn't and so much more as you'll soon hear Ben is British and so the episode is automatic
Really Gonna sound more sophisticated and I can't wait for you to hear it with that I bring you Ben Williams this episode is brought to you by coda coda is an all-in-one doc that combines the best of documents spreadsheets and apps in one place I actually use Coda every single day it's my home base for organizing my newsletter writing it's where I plan my content calendar capture
my research and write the first drafts of each and every post it's also right curate my private knowledge repository for paid newsletter subscribers and it's also how I manage the workflow for this very podcast over the years I've seen code at evolve from being a tool that makes teams more productive to one that also helps bring the best practices
across the tech industry to life with an incredibly Rich collection of templates and guides in the coded dot Gallery including resources from many guests on this podcast including shreyas Gokul and shashir the CEO of Coda some of the best teams out there like Pinterest Spotify square and Uber use Coda to run effectively and if published their templates for anyone to use if you're a
ping-ponging between lots of documents and spreadsheets make your life better and start using Coda you can take advantage of a special limited time offer just for startups head over to CODA dot IO slash Lenny to sign up and get a thousand dollar credit on your first statement that's c-o-d-a dot IO slash Lenny to sign up and get a thousand dollars in credit on
your account this episode is brought to you by athletic greens I've been hearing about eg1 on basically every podcast that I listen to like Tim Ferriss and Lex Friedman and so I finally gave it a shot earlier this year and it has quickly become a core part of my morning routine especially on days that I need to go deep on writing or record a podcast like
this here's three things that I love about eg1 one with a small scoop that dissolves in water you're absorbing 75 vitamins minerals probiotics and adaptogens I kind of like to think of it as a little safety net for my nutrition in case I've missed something in my diet two they treat age one like a software product apparently they're on their 52nd
iteration and they're constantly evolving it based on the latest science research studies and internal testing that they do and three it's just one easy thing that I can do every single day to take care of myself right now it's time to reclaim your health and arm your immune system with convenient daily nutrition it's just one scoop in a cup of water every day and
that's it there's no need for a million different pills and supplements to look out for your health to make it easy athletic greens is going to give you a free one-year supply of immune supporting vitamin D and five free travel packs with your first purchase all you have to do is visit athleticgreens.com lighting again that's athleticgreens.com Lenny to take
ownership over your health and pick up the ultimate daily nutritional insurance Ben welcome to the podcast thank you very much Lenny thanks first of all for inviting me it's a pleasure to finally meet you I've got to say that kind of what you're creating with the newsletter and now the podcast just such awesome resources for the product growth and why
detect communities so it's a real honor to be invited onto the show and you know I hope there's a few useful things that people can take away awesome man I really appreciate that I have no doubt there will be many useful things people will be able to take away I'm really excited to chat about sneak and the things that you're building there I feel
like sneak is this very under talked about company and also super fascinating company especially in terms of how it got started how it scaled and it's also at the center of so many Trends product-led growth community-led growth focusing on developers to grow and also just Security in general is really interesting and so I'm really looking forward to our chat me too before we get
into all of that I'd love to spend just a minute on your background so you're currently BPR product at sneak and I'm curious how you got to that role and just some of the other wonderful things you've done in your career along the way there I'll try to keep it reasonably short my education is in computer science graduated from the University of
Manchester back in the late 90s had already landed as a job as a developer but ended up having a bunch of other interesting offers one of which was to join a small startup building requirements management tooling for product and Engineering folks this was all pre-agile but that whole space at developer tooling engineering tooling it was something I just felt naturally
drawn to and it's where I really specialized through my whole career I actually joined that company as a Solutions engineer so lots of demos and so on I was really close to the product team but also speaking with customers every day which as is quite common I think was was my path into product there several years and a few Acquisitions later I find myself in IBM with the
rational software developer tooling group I was leading parts of the product strategy there and a bunch of initiatives I actually I learned a ton of useful stuff there but also that I had a strong preference for working in smaller fast-growing orgs versus 350 000 people beer moths after an interest in unexpected interlude leading a devops transformation at a fintech and some
Consulting and Advising I then joined Cloud beasts they're a devops startup with an open core model I was leading product design and growth there stayed with them for three years through several raises and a period of really fast growth before finally joining sneak to build out a growth organization and Lead our developer experience initiatives I have a broader remake at
sneak now than just the growth org but yeah that's how it started awesome to give folks a sense of just how successful and big sneakers gotten one is just what a sneak we haven't even talked about that and then to just some stats about the scale of the company and the business at this point the developer security company we make it ridiculously
easy for developers and their teams to improve their security posture while still moving fast so sneak confined and automatically fix vulnerabilities in code open source dependencies containers infrastructure and Cloud configurations and all underpinned by the best security intelligence data in the market with a laser focus on developer experiences
which is why we're we're really different it's also an amazingly fast growing business with some Stellar plg focused investors and board members from the likes of edsim at bold start to tomorrow Joshua slack CPO we were founded in 2015 last valuation after our Series F was at 8.6 billion we're securing the software of millions of developers now well over 2 000 paying
customers now around 1300 people of which around 500 in r d with nearly 70 folks in our product talk and the people here just create this amazing culture and all in all it's just a really exciting place to be okay did you have any sense it would get to this scale when you joined early on yes I think because I wasn't looking for a new role when I ended up joining sneak
when I was first approached by them but everyone I spoke with along the interview process just became more and more impressed with not only the caliber of people here but the vision the mission for the company and all those things you mentioned in terms of plg community-led growth focus on developers the security space these were all kind
of all things that if you create a Venn diagram of all the things that I'm really interested in professionally super interesting super exciting to me then kind of sneak ended up in the middle of it so it's pretty cool awesome so that's a good segue to where I wanted to start is how sneaks started and how Snee got their first 100 users and I know you weren't there necessarily for
that but I'm curious what you can share about how the founders found their first say 100 users how did they get to their International developers and get people excited I think it's a really fun story so if you don't mind I'll just take a moment because I think it's important to set the stage by doing this yeah just look back at the market in general so
plg or bottom-up and security these were never words that were kind of known to have been spoken together in a single sentence right so security has always been a centralized function Security Programs were historically more about Audits and policing and enforcement versus developer enablement or empowerment a sales motion you saw it was always topped down it was seen as
immovable csos other appsec leaders were the buyers security tooling that was out there at the time they all catered to this Dynamic and these were tools that you know slowed developers down they created a lot of frustration they were met with a lot of resistance not the ideal recipe really for consistent adoption and strong engagement and
retention and ultimately app Security Programs based around those kind of solutions just weren't effective as they really needed to be so it was a realization around this timed with adjacent Market shifts happening with devops that sparked the ideas behind sneak so the founders guy Danny and Assaf they saw a real opportunity just to do things differently they believed
that the most effective way to improve application security posture was a developer first approach they knew that the developers were increasingly caring about the security of their code in the same way that they cared about performance and functional quality of their code but they also knew that to empower developers to own that security
they needed just much better tools with way less friction than they ever had before and their approach was I think looking back super smart focus on a community wasn't the full extent of what we'd think of as community-led growth now but it was close they started with a really narrow early Focus as a single Persona single context single use case
and what that meant for sneak was developers building applications using node.js who wanted to ensure that the open source dependencies they were pulling into their apps were secure how open source software it's a huge accelerant in building modern software the average software application today is at least 75 percent of Open Source libraries and components so this was
increasingly becoming a primary attack Vector for malicious actors who could find a single vulnerability in an open source component and then find it and exploit it in every single application that was using that component and at the same time at least back then you know open source software was much less tested for security vulnerabilities and the maintainers of
Open Source libraries were often less security aware so you get that context and then at the same time node.js as a runtime was gaining traction so there was this increasing adoption in the Enterprise more and more dedicated conferences and the like but the community was still small enough that sneak could meaningfully influence and guy and the
others just went all out on being deeply involved in that Community they were you know they were presenting at Dev conferences meetups they were building online content and so on and the question that they repeatedly posed to the community was do you have known vulnerabilities in your apps and seek was there to help them answer that question and fun kind of facts on the
side if you search for sneak in the urban dictionary you'll see it's an acronym for so now you know but all of this kind of I think really only worked because of the parallel product-led approach so while the answer to the question about how does your product monetize users was much less clear-cut in the early days for sneak the answer to the
questions how how does your product acquire and retain users has always been product LED then the initial version of sneaker was a command line tool was a tool for developers it could be run locally or easily integrated into CI CD pipelines for early feedback it allowed devs to assume more responsibility for the security of their apps and that was just
very different from the typical incumbent technologies that were run by security teams late in the dev process long feedback loops issues thrown over the walls inevitably just frustrating Developers and all of this was just built on this fundamental belief that the only viable path and by viable I really mean sustainably effective the only sustainably effective path for
software-centric organizations to meet the challenge of becoming and staying secure was for them to take this developer-led approach to that challenge so really kind of complete disruption of the industry and developer adoption for that reason was always a key priority for us so with that in mind sneak has just been free to use in some capacity from day one
and the early strategy was always about creating something valuable that was readily available something that solved a real problem in a uniquely differentiated way and making it pervasive so with devadoption you know this core concern a freemium go to market strategy was just the obvious approach So eventually getting back to your original question that's all of the
the context and where they went and how they did it but the first hundred or so users really just came from the founders engaging with the node.js community in the interest that drove I think we probably had maybe around 5 000 free users before there were any attempts at monetization awesome there's a bunch I want to unpacked there because what's
interesting the way you describe it maps to the series that I recently wrote about consumer growth strategy and how the first three steps after you have an idea is to come up with your super specific who kind of your target persona come up with a hook that catches them gets them excited and then go find them where they are and Pitch them your hook so the super specific who for sneak was
you said open source developers working on node.js is that right well it was node.js developers and node.js developers we're building their applications using open source node components got it was there any other constraint to that do you know or those were like the cutest three attributes of a user that's really it the community
was was growing for sure it was big enough to have a decent opportunity there but now the technology wise it didn't mean it meant that a product could be brought to Market in a reasonable time yeah that that's great and then the the hook was basically do you have known vulnerabilities in your code base which if I were an engineer like I don't know shoot I don't know
I'll get a point exactly scared and then so now you know right yeah exactly okay cool and then the where so you said that they went to open source communities do you have any more specifics about what those were was it like a specific Forum was it like GitHub somewhere was it Reddit do you know any idea where those communities lived yeah it was less about
a particular place but more about the community of developers themselves who focused on on node.js so you know a bunch of kind of early evangelism was really at conferences it was I think the Velocity Conference in Amsterdam where Diana zeph kind of first unveiled sneak to the world and uh kind of yeah I went from there I see interesting so in-person events conferences and meetups
probably focused on node.js developers yeah okay awesome what happened after that so that was like the first hundred was just roughly the next stage of growth or did it focus on that for a long time what was the next stage roughly yeah I think that Focus was the really important kind of element there if I can kind of latch on that starting with that narrow focus and building
around Community engagement I think it's a well-proven Playbook now particularly in the developer tooling space like New Relic did it notably with Ruby Community for example But ultimately it was important because of this kind of depth verse breadth approach and that depth first approach that sneak took was important to be able to effectively
validate the solution on the path to product Market fit a JavaScript developer just won't care if you support golang or rust but will absolutely care if a key feature like automated package upgrades just isn't available for their ecosystem of course the bigger problem of vulnerable components in open source across all languages and all ecosystems that's a very widespread problem it
affects the industry at large but that just spoke to the potential opportunity that was there to be unlocked but the key for sneak I think was just not to go too wide too early so they focused on nailing that initial use case for that specific community of node.js devs like I say narrow enough to be able to really focus on quickly building a compelling
solution to a real problem but also wide enough to be something viable from a growth perspective and you know even back then the npm the node package manager hosted around 200 000 open source packages they were downloaded something like two and a half billion times a month by over 2 million devs and the typical node app would have hundreds of dependencies mostly indirect and so
hidden less immediately visible but each of those dependencies brought with it some security risks so yeah I think nailing that narrow and deep use case before expanding wider was absolutely critical and generally just sound advice around finding product Market fit and building solid momentum before costing a wider net and you know it's difficult to
maintain that Focus for sure as the Leora of that big attack can be really tempting but ultimately you have to build a service amount it well to capture it do you have a sense of when that Focus expanded to an adjacent group like how many years into the growth story that happened or was there some kind of Milestone there because I know everyone
imagines yeah we'll expand the question is when and when does it make sense and when's it too early do you have any insights into that phase yeah for sure first I think if we're talking about plg the story with sneak and I actually like that definition of um product LED acquisition from Julian Shapiro when he when you spoke with him and beyond that
maniacal focus on finding product Market fit for your product Founders really should be thinking about how their product is going to grow and that's important of course as you think about taking that that next step so let's assume in a simplistic definition that you found product Market fit as demonstrated by strong retention and then the real question is where are the
new users for your product going to come from and Founders really should have I think strong hypothesis around this you your risk essentially adopting and if they if we build it they will come approach so if your acquisition strategy is product LED then understanding and being intentional about your early acquisition growth Loops I think is an
essential Founder's responsibility dedicating time there to design those Loops into the product it's key and when I say product I really mean the whole product experience considering every touch point in and out of the core application that your users and customers might have with you sneak for example believed very early that we could build out powerful content Loops
via fixed pull requests that we raise on GitHub new users they'll sign up for sneak they'll connect their GitHub accounts sneak will scan their code or find vulnerabilities will automatically create sneak branded pull requests to fix those vulnerabilities other devs in the repo will see and interact with those PRS some of them will follow links
to sneak create accounts so some of them will connect their own repos and so the loop continues so company generated company distributed content Lube it's actually really powerful for us because it's both an acquisition Loop and an engagement Loop over the course of time we extended that Loop by adding support for other source Control Systems Beyond
GitHub we layered on a bunch of new loops and I think if Founders can be intentional about this as you're developing early product iterations then you're going to have a big Advantage when the product clicks with the market and that was built into to sneak as we went along so that was kind of ready as we found product Market fit but I think to talk about specifically how sneak
took that next stage it was kind of a function of when we were chatting before this talked a little bit about some of the failed experiences spinning up a a self-serve Revenue Channel and actually before we get there which I definitely want to get into all that I love this story just shared I hadn't heard of this growth tactic of basically the connector
GitHub account you find all the vulnerabilities push a fix people see that sneak did this for them and it just provides all its value and you're saying that was really effective it's an example something that worked very well first of all the that integration in terms of connecting your code with security scanning like that was a first of a Kind integration like
no one had done that before but the key was that we ultimately controlled the content So Not only was the fixed pull request doing something useful in terms of the code but all of the description of the pull request was explaining about the vulnerability educating users and it was all sneak branded and saying if you find this useful click here come and
learn more about the vulnerability sign up for an account if you don't have one it kept existing users coming back and it brings new users a lot of new users in fact you'd be in there all like do you have no invulnerabilities in your code base click here to find out so is that responsible for much of the early growth that Loop I think that was one of
the loops we also have a couple of from fairly early on as well other content Loops that are more kind of programmatic SEO assets that have both been pretty instrumental in terms of new user growth here it'd be cool to hear about those if they're relatively straightforward to explain and then we can get to the thing that didn't work the self-serve
monetization piece you're gonna get to we have a bunch of Loops actually at this point um like you got it off yeah it's I'm a big loot person funnily enough but yeah we have a bunch of Loops um company generated company distributed content Loops have actually worked really well for us we have a sidecar product called sneak advisor sneak advisor it's basically a a service that
developers use to search and find open source packages when they're considering integrating some within their software applications the unique thing about it is it indexes all of the package managers it learns about those packages it augments them the data about them with a bunch of metadata including of course sneak security scans but we also find out how actively maintained the
software is on on the source repo on GitHub or wherever and so we build this kind of package Health score so anyone searching on Google for a package that does XYZ or a specific package by name sneak advisor will be right up there in terms of the search results they'll land on there they'll get a good idea about that package they can look at similar
packages and it's all of course A Sneak website and we have ctas to say you know if you want to secure your app your application on a Perpetual basis then just come and join us so that's a great loop I mean that's all kind of a programmatic assets there are hundreds of thousands of these package pages but they're just automatically being generated continuously got it so it's
programmatically generated better indexing of Open Source libraries that you can integrate with that is so smart it's programmatic because you can inform on the security vulnerabilities and then the maintenance and uh activity interesting yeah that makes sense that's all data you can just gather that's awesome okay so there's that is there
anything else that's worked really well for you guys to help you grow self-serve one of the recent ones that's really interesting is security education we think of snake as a change agent in helping devsecops Transformations and it's fine kind of having this capability but what we really want to get to is this position where developers truly
understand and can be better place to prevent security vulnerabilities being injected into their code and so one of the things that is again something that's pretty different from the industry from an incumbent perspective is that we believe it's really important to democratize security education and so we have been building this bunch of really high quality but bite-sized
lessons about developer security focus on developers about security issues and vulnerabilities and we surface them again they're out there in the public domain there's no paywall to get access to those all the traditional Solutions you don't need to sign up but you need to pay to get access Beyond more than a couple but these were just they're all out there in
the public domain so that that works really well for us from a company generated company distributed loop as well so cool so SEO and then integrating to GitHub in that interesting way imagine there's also a lot of intra company virality when someone uses sneak at a company and they spread it to their colleagues yeah I mean I didn't I didn't talk about those I think those are
pretty well understood we have both referral loops and and invite Loops as well okay awesome coming back to what didn't work and I think you mentioned that there was a monetization attempt that was self-service oriented and that had some challenges can you talk about that at the time you know a few things were in place valuable products check a strong developer user growth check
strong retention check but the first self-serve monetization efforts I only really saw traction with individual developers paying a hundred dollars a month or purchases in in larger companies they just didn't happen as everyone had hoped there was a really critical part of sneak's history at the time a bunch of investors didn't lean in perhaps shying
away from early conviction with the founders on building strong usage without a proven path to monetization at that point aired at bolster who I mentioned previously he was one of the first kind of True Believers and was I think really key in helping with providing Runway during that time but it was clear that there was a lot of work still to do the team dived in they
really figured out what the constraints were and through that process really learned about the importance of catering for the broader governance needs of the Enterprise buyer and that meant a couple of things first there was a need to build out table Stakes features around governance at scale just things that companies of a certain scale and size expected
reporting robust user management and so on and second that it was time to move beyond that depth first approach right so that depth first approach was absolutely critical in getting to that point but it wasn't good enough to take the next step if you think about it there's a point in a company scale where you start to see diversification of tech
stacks and all of those Tech Stacks need securing it's obvious in retrospect that only supporting developers using a narrow slice of those Tech Stacks wasn't going to meet the needs of the security teams who were ultimately the people who were held accountable for the security of their entire application estate so the teams worked hard over the next few
months starting to build support for additional languages and ecosystems and adding those table Stakes features I think back then sneak were simply ahead of this inevitable curve of developer First Security at the time the only buyers were security teams and Dev First Security for the most part wasn't something that csos and absec leaders were driving but if you look at
sneak through that lens of as I mentioned being a change agent being a key piece of the transformational Journey of our customers devsecops Journeys if you you realize how important it was for us to start to build relationships with those Security leaders so it was that time also that it was the right time to bring in the first sales and Engineering hires as well so
you basically found it couldn't work self-serve independent of sales being involved in convincing the folks at the top which makes sense like how do you trust a company with your security like people at the top responsible for security aren't button makes sense today it's less like that there are organizations where the buying sensor is still very much appsick
but there are also many organizations where kind of technical leaders on the buying decision around security Investments what was always true though even back then was the influencing power of developers regardless of where the buying Center was and imagine as the brand has grown it's gotten easier to convince people like oh yeah look at all
these other logos using this it's probably gonna yeah okay so just to understand is in your experience with sneak it never really worked self-serve monetization it worked as a way to get into a company and then developers started using it in small scale but you needed sales and marketing to really grow monetization is that what you found I think back then yes now it's a very
different story we have a lot of self-serve only customers scaling pretty large so got it that's interesting I rarely hear that you start out with sales being important and it becomes less important or I imagine it's still very important but there's like a segment that has emerged that can self-serve fascinating I think it is important to acknowledge the the
products has always played a really key part art in the sales process for sure that touches on something I wanted to ask so at Sam's you've mentioned him a couple times he's got this awesome newsletter he talks about you guys all the time I think he's very proud of the progress of sneak and he talks a lot about that for developers you got to win
hearts and minds of developers to build something that works any lessons or pieces of advice or folks that are targeting developers to win hearts and minds and get Engineers developers excited about what you're building I think there's two things right so first of all fundamentally for someone to get excited about using a product they've got a carer enough right
they've got to have a problem that you're solving so I think there's two things one there is a a shift that is happening and still happening I think there's still a long way to go for developers to really care about security as an integral part of their job in the same way they think about functional quality or performance you know I think that we're still
we're making strong progress there it's changing all the time but there's still a long way to go there but the reality is that I think in most companies developers have to care about security because their companies need to be secure so the key then is how do I make the job of being secure for these developers as painless as it absolutely needs to be and that means really
meeting them where they are integrating with their tools finding ways to take security to them instead of trying to pull them out of their workflows you know flow is just this incredibly important concept for developers and you want to strive to keep them in that flow for as long as possible so the GitHub kind of pull requests are a great
example of that someone can sign up for sneak and they could theoretically be the only user of sneak and connect their repos all of a sudden we're protecting securing those repos a hundred a thousand Developers could be working in GitHub with that code or benefiting from sneak without necessarily needing to uh to to sign up that's that example of kind of taking
the product to users without pulling them out of their workflows and I think that's absolutely critical as an outsider hearing all this it's a product that magically helps you avoid security issues very little work does a lot of the work for you it's hard to imagine it not working looking at it now and I'm curious what it was about the early days that just
felt like maybe that people didn't believe in this working is it just there was doubt that it would be smart enough to find your security vulnerability issues was it the timing wasn't right people weren't ready weren't like concerned about security enough what do you think it was that created challenges early on because looking back it's like of course this is gonna work how could
it not it sounds just like a magical all-win product first of all don't think the challenges were there in terms of the developer adoption so even when the even when those first kind of forays into into self-serve were struggling in terms of breaking into some of the larger customers the developer adoption the the free user base was still growing
at a at a really good Pace that momentum was just constantly building and it's that momentum that is ultimately fueled the sales lit business as we've gone through the years but it was it was just those few things I think that I mentioned earlier in terms of stumbling blocks that needed to be overcome because when those first sales and marketing hires did join us and we
started having conversations and we also tweaked some of the things in the product to to meet some you know had some breadth had some additional languages ecosystems building those table Stakes features then it kind of really unlocked and that was kind of rocket ship time from then got it so sounds like the biggest issue was monetization can we make money doing
this developers love it they're using it like crazy but well people be convinced to scale this in inside an organization and pass a bunch of money for you okay got it so I want to dive a little bit deeper into your growth team and product team and how you think about organizing teams like that in a product-led growth sales org so the first question just how
did the growth team start at sneak what was kind of the early days and then what does it look like today there was some ad hoc efforts happening in various places we had small growth marketing function we had devrel we also add some ownership of key growth surfaces in r d there was a team that owned the new user onboarding flows for example but it
wasn't until I joined that we really formalized the notion of a growth team it was very kind of ad hoc before then when I joined we created what we call the developer growth group now before then there maybe wasn't a strong an understanding about what a growth team needs to look like how they might need to work differently to the core product teams and I'd say overall it was
much later than you typically expect to see and at a bigger scale you normally are going to start growth teams one or two people three or four maybe and scale out from there but we started much bigger than that but at the same time this bottom-up developer first approach it was baked into the company DNA in terms of how teams think and operate so
yeah we were growing fast even before we spun up the growth group I think the significant change that happened there was it was a transition from a simple freemium approach to a holistic and well-coordinated plg strategy it's much more common to start earlier much more common to start at a smaller scale that we did a sink but it it worked for us because of this kind of
Perfect Storm of where we had a product with that bottom-up growth built in from the beginning we have founders with a deep appreciation for how the product could grow and there was strongly exec alignment and sponsorship for scaling the motion so the problem when starting the growth group it was really the most part more oriented to how we can get the
Flywheel spinning faster as opposed to getting it moving in the first place so where did you initially Focus that team which part of the flywheel right now we have dedicated teams focused on acquisition activation and monetization along with a supporting team who own a growth platform including all of our data and experimentation stack but the macro structure it's changed over time
to enable us to focus on the biggest constraints in our growth model so at the beginning we just focused on acquisition and activation intentionally deferring any investment into specific monetization initiatives around the self-serve Revenue Channel until we felt confident that a we'd built the necessary growth muscles to scale and B we'd figured out some of the more
pressing issues that were present earlier in the user Journey so it's important that we felt really confident about our ability to effectively connect developers to sneaks value in such a way that introducing and optimizing a self-serve Revenue channel would make sense I also really wanted to avoid one of the common failure modes I've seen
around cross-functional collaboration and growth so when I joined there was an inherent tension built into the system it was particularly noticeable between r d and the growth marketing team we had amazing people in both teams you know a ton of really great ideas but many of them were just not being executed on and it was leading to a lot of friction a
lot of frustration ultimately caused by misaligned incentives between the different functions so when creating the growth group we resolve this by ensuring that each of the growth teams were truly cross-functional in nature with everyone in each team aligned around common objectives and kpis every team has Engineers an engineering manager a
product manager a designer a growth marketer decision science support and a basic shape of the growth teams that will be familiar to most but I spoke to a bunch of people over the last couple of years and I've actually learned to my surprise that inclusion of growth marketers in the product teams is not all that common and I personally think there's um
just a lot of opportunity being missed there and I expect that to start to become the norm rather than the exception over time okay I want to talk about that but before we get there you said there's a decision science personality what is that about that's cool that's right so we started off from a fundamental bi data analyst perspective but over time we we wanted
to apply a much deeper level of analysis on the data such that we could start to build in predictive models that could help us make better decisions and can ultimately Fuel and power some of the the in-product experiences so yeah we spun up a decision science function and yeah those folks are very smart is that similar to data science or is that a
separate it's a okay it's cooler you call them decision signs people versus data science because that's so much more actionable yeah I think so yeah wow that's cool all right I hadn't heard that before makes me think of Annie Duke and all the stuff about how to make better decisions and I love have you is this something anyone else does or is there some of you came
up with calling I'm not sure but I don't necessarily think what we're doing is revolutionary there but maybe the name I'm not sure yeah the name is cool I haven't heard that before it implies a bias towards action versus just we're gonna pursue a bunch of cool stuff with data interesting okay and then you said that there's a growth marketer embedded on each team so maybe
just broadly what makes up these teams which you touched on briefly and then what have you learned is the value of having a growth marketer embedded within each team it's important to have balanced teams with strong diversity across multiple vectors focusing on functional diversity at the moment which is kind of what you're asking about with having growth
marketers on the team one of the big benefits you get is a broader palette of ideas but also a bigger toolbox when it comes to execution which generally translates in an ability in a growth team for them to test and learn faster with more parallel yet at the same time aligned threads perhaps I can give a a recent example there so yeah having a growth marketer
in an acquisition focused team led us to some lightweight experimentation on the website in creating an SEO optimized page it was something that was really high performing both from the perspective of traffic and conversion but it didn't require any engineering resources to create so the growth marketer in the team and they decided together this was something worth
pursuing but the growth marketer was able to kind of execute that independently while Engineers were working on other things but then based on the success of that the team went on to build out a functional sidecar product that allowed users to basically try sneak without needing to sign up by simply pasting their code in for us to scan and giving them some results there
and there so you know we saw really great results with that visitor traffics or a significant increase sign up rate dropped a little bit as we'd expect it would but overall new users had a big bump and those users had much higher intent which we saw play out with increased activation rates awesome okay and so there's uh essentially four teams
under the growth umbrella there's acquisition activation monetization and then this kind of experimentation platform team is that right yeah that's right and that team is also responsible for making that data available elsewhere in the organization as well product-led sales is a a really important motion for us and so taking the the knowledge we
have the Insight we have around Behavior with users and their teams and their companies within the product and making that available to the GTM teams outside in smart ways allowing them to focus on the things that are most important to focus on that's a really important part of what that team does it's interesting you guys are the epitome of product like sales that's
this new trend of from plg to PLS for sales it's obvious that they're a big part of this whole process and the fact that monetization happens almost all through sales is interesting so that's interesting cool okay and that's not a question just a thought talking about one other thought I had is so you talked about SEO being a really important part
of your growth what is the the person team like to do the seops the right content I imagine they're on the acquisition team there's maybe a Content person that lives within that team we have um actually one of those the smartest um SEO people I've ever met within what's their name let's give them a shout out if you want Anna yeah cool um Joanna well she's she's part of growth
marketing but she works extremely closely with the growth teams and she's got a few people in her organization and we bring them into specific SEO focused initiatives when we're looking to build loops around that yeah incredibly important to have someone like that who understands that had a far deeper level than than I could ever hope to how SEO
works and particularly in terms of keeping on top of some of the things that Google are constantly doing in terms of their algorithm changes and does she actually do the writing for editorially for I guess even the programmatically make Pages or there's someone she outsources but what she does do a great job of is providing the kind of continuously updated guidelines on
how content should be structured to to lead us to good results and then it's just engineers and PMs that end up writing the things wow cool that's exactly right this episode is brought to you by vanta helping you streamline your security compliance to accelerate growth if your business stores any data in the cloud then you've likely been asked are you
going to be asked about your stock 2 compliance sock 2 is a way to prove your company is taking proper security measures to protect customer data and builds trust with customers and partners especially those with serious security requirements also if you want to sell to the Enterprise proving security is essential sock 2 can either open the door for bigger and better deals or it
can put your business on hold if you don't have a sock 2 there's a good chance you won't even get a seat at the table but getting a sock to your Port can be a huge burden especially for startups it's time consuming tedious and expensive enter vanta over 3 000 fast growing companies use vanta to automate up to 90 of the work involved with stock 2. Banta can get you ready for security
audits in weeks instead of months less than a third of the time that it usually takes for a limited time Lenny's podcast listeners get one thousand dollars off vanta just go to vanta.com Lenny that's v-a-n-t-a.com Lenny to learn more and to claim your discount get started today what advice do you have for folks building growth teams and maybe there's
a similar space or just B2B PLS oriented businesses in general what have you learned about what is important to get right this is uh this is a big topic I have a lot of thoughts here based on what I've seen work well but also not so well I think I can broadly bucket things here into maybe three main topics so the first will be people and process the
second would be strategy and the third would be data now they're all related and they all have to be working well to be effective in growth starting with people I'd maybe touched the first element there in terms of balance teams now you've got to have those Balance teams with diversity and ability to create great ideas but also ability to execute
that's the first thing but still on the topic of people I have this kind of mental model that potential at its core is unbounded but that a bunch of things situationally prevent people from fulfilling their potential you know it might be how they're thinking about something it might be organizational it might be in relationships with co-workers or in broader Team Dynamics
it might be in them not even being in the right role even so fundamentally I think it's the role of managers and leaders to help them identify those things and work with them to find ways to thrive and grow and I've seen this have particular significance in a growth org where people are just naturally less good fit now that doesn't mean that
they're not amazing talented humans it just means they aren't going to do their best work in a growth context so when you're starting a growth team you'll often be doing so with internal moves so this can be something that's maybe a little bit easier to miss than with external candidates where you're likely testing for those things explicitly
during the hiring process I'll share an example with Developers so the devs that really thrive in a growth context are the ones that are motivated by moving quickly iterating to create measurable impact they're not attached to their work they Embrace imperfection as part of the process they happily discard their code their ideas even you know they're curious and they're
always looking for ways to be closer to their users now those are the folks that generally make great growth engineers and I've also known incredible Engineers that are most motivated when they're working on really deep technical challenges and love the process as much as the outcome and they've struggled in growth of course it's it's never that simple in
reality there's nobody who's really at either extremity of that Spectrum but it's really important to try to answer the question can this person do their best work in this environment so and that's a really key part of it and yep you need to also make sure they're well equipped from those kind of skills and knowledge perspective as well so they need the right skills and knowledge
to be able to do their best work in the context of your growth process so the ideal state is that every growth team member has common vocabulary they're comfortable with the growth process they can work well with data and an experimentation platform they understand the data they have the right skills and mindset education I think plays a big part here
so we're big fans of reforge but we've also developed a bunch of internal programs to align and up level the teams something we learned I think as an example is the importance of starting simple and going deeper as the teams build experience so for example when it comes to experimentation don't try at the beginning to introduce multivariate testing or concepts like
sequential sampling as an alternative evaluation approach you know teams are still trying to just dip their toes in this in this this water it's kind of a recipe for for a lot of mistakes so um yeah there would be some advice there but people also need to be well aligned I've talked about this actually on another podcast that I did recently but
what I mean by that is their execution needs to be aligned with an evolving growth strategy the growth strategy needs to be aligned with and at the same time influence where the company's going the growth strategy needs hooks into the product strategy and ideally there's some overlap and alignment of kpis so the growth teams and the core product teams are swimming
in the same direction the skills and experience in the team need to be aligned with the strategy if you've got to focus on activation and acquisition then someone who's you know plans and pricing expert probably isn't the right set of skills at that point in time but you also leaders need to plan to ensure those skills are available as focus of
the growth team inevitably shifts over over time at the end of the day I think everyone needs to be able to easily answer the question why they're there why the work they're doing is important I don't know if it's too far off track here but I have a vision and Mission framework that I like to use that leads to I think great simple statements of an imagined better future
State and your role in getting there so I can talk about that a little bit if it's yeah let's do it that sounds too good to pass up cool so the vision is the Nirvana state that you aim to enable for your users and customers in five to ten years it's something that could equally be enabled by your competitors if you don't execute effectively or
efficiently or quickly enough you can always prefix a vision statement with a in the future dot dot now it's something that should be bound to your target market so not too wide and not too narrow and critically it should not mention your company your product or anything solution related at all completely agnostic of those things and then the mission that's the thing
that you're going to relentlessly iterate on to take you incrementally closer to the Nirvana State described in the vision it should answer how you'll realize the vision by describing what your fundamental approach will be in other words what you will do and how you'll do it it should ideally aim to encode any unique differentiating Advantage you have so if I think about
sneak at Large you know when it comes to Advantage we might mention our unequal security intelligence and knowledge of application context and one of the neat things about this framework is if you find utility in doing so you can apply at every level from from the company level all the way down to individual to you it doesn't mean the process is not
difficult and you need to spend a lot of time but it gives you this set of kind of a framework a set of bounds that help you really come out with well-crafted statements that kind of stand the test of time is there an example you could share of the Mission Vision whether it's sneaker or something else yeah yeah I'll give you one for the growth grouper sneak so great division
every developer securely unleashes their creativity and the mission is to connect every developer and their organizations to the value of the sneak platform with frictionless self-serve adoption and expansion interesting how the vision is so big Beyond sneak's current focus in to your point this kind of trickle down right there's a company Vision admission and
it's the growth team's Mission Vision awesome exactly I want to shift a bit and talk about your product work so we've talked a lot about the growth org I'm curious what the broader product org looks like like how many teams do you have how do you structure it is a like product focused user-focused outcome focused and then just how does that team work with the
growth team sure happy to go into a bunch of that stuff I know I've mentioned earlier kind of three areas of building a growth team I don't know if you want me to cover those other two because we talked about oh yeah okay let's do that so let's finish that thread cool so people in process was the first and we'll cover process really quickly I think on the
process side at a minimum here you need well understood documented growth processes practices work in cadences teams in growth need to work differently in some ways from r d teams working on core products assuming otherwise and just giving growth responsibility to an existing Team without working with them to implement appropriate ways of working
I think it's a common trap ideally you get to a point there where the growth process is something that's continually refined and iterated on trying to build in more predictability but what I've found I think most singularly most important in a growth process is facilitating a rapid learning Cadence and providing the means to socialize those learnings surfacing them
in the right place at the right time so they can be leveraged in other context and if you think about experimentation it's not about delivering outcomes it's about generating learnings that the organization can leverage effectively to deliver outcomes might not be now might not be tomorrow or some point in the future but the sad reality is that without good
process learnings easily end up unused and Gathering dust and you have to ask them what was the point so as people in process and you know you know when that's on the right track when you start to see enthusiastic sharing of learnings when you see regular contribution of ideas coming from everyone in the form of well-crafted hypotheses that are based
on data and learnings and when you see a wide variety of folks just assuming end-to-end ownership of managing and running experiments instead of delegating that to the product manager or an engineer in Tech lead so yeah that's that's people in process and strategy is I mean yeah throw in a question real quick before we get to the last piece sure so learnings just to
kind of touch on that I've heard more and more pushback on the idea of learnings being an outcome because a lot of like as a leader you're not going to be like cool we learned so many things but nothing really got done how do you think about the tension between yeah learnings we want to learn we also want to like move metrics grow the business
and learning's our way to inform decisions more than even just learn how do you think about that kind of balance ultimately you're there to create impacts right there's no getting away from that but learnings is the means it's the same as there's a a quote that I love from Farid muscle around focus on the user's path to Value not on monetization because if you focus on the
form of the latter will follow and that's the same thing with learnings and impacts here you know if you try and focus on the impact itself might struggle if you focus on the things you need in terms of learnings to take you step by step that will pave the path to to creating impact I imagine your okrs and goals are still like move this metric some percentage
but you think of that as an output and the input is let's learn a bunch of stuff about what works and doesn't work and use that to inform we're going to double down on exactly yeah okay you've got to focus on when you build a strategic opportunity you're thinking about the outcomes but you don't just go right to the end State you've got to think about what's the
quickest way we can test this hypothesis and from there what do we learn from that and what do we take into the next set of experiments and it kind of you're Paving this path along the way you you know kind of that rough destination how you get there you don't know at the start and that's what that's the the part that the learnings take you on cool
okay and then strategy we're talking about strategy yeah so strategy it's a good one at a very basic level you need to be able to answer questions how do you require users how do you retain users how do you monetize them I know you talked with Elena on that in more depth but from there you need you know you need more detail it's going to guide the growth teams and where they
look for strategic opportunity how they approach that the best way I've found to articulate a growth strategy that fulfills the promise of usefully guiding the team's execution it's the loop-based model reforge as a specific kind of documentation that that I think is great around that be able to identify the various micro and macro Loops how
they're all connected being able to document them in a qualitative model to communicate a shared understanding of how you grow it's really powerful augmenting that then with you know the quantitative side of things that helps guide quarter to quarter focus and ensure you can be intentional about where you're investing that becomes a big enabler you know you're
never gonna have a shortage of ideas in a high performing growth team so knowing where to focus amidst that kind of sea of ideas is a really important role of the strategy and early on you know you probably have one or two core Loops but inevitably you'll need to layer on and connect new ones over time and having a framework for doing that and having the
framework for regularly revisiting that in the context of growth team learnings changes in broader company strategy evolution of the product new features being added and so on Market shifts that becomes a big up level of for teams to be able to create timely impact so the model you create there is what enables you to know it anytime where the biggest
constraints to your growth are and allows you to balance your growth Investments accordingly at sneak for sure you know we've had times where we focused on a broad set of constraints and opportunities and other times where we've had a much narrower Focus for example on driving improvement to activation and engagement even more narrowed like doing that through empty
States for example and when I unpack the strategy piece real quick how do you operationalize this idea of you have this growth model growth Loop you figured out here's the ways we're growing how do you tactically connect that to your strategy is it here's how we're growing here's where we're going to focus this quarter optimize this part of
the loop or is there some other way you write it down basically very simply it's first of all alignment on how you grow the different Loops how they work the roles of different teams and the roles that they play in fueling those loops and getting them to spin faster I think that's the first thing having that common vocabulary and understanding second is understanding the
the way they work in terms of what is constraining them what are the inputs to them what are the opportunities for making those Loops spin faster how can multiple loops work together in a macro context and particularly using the data there to be able to identify and understand and get alignment around where are the biggest constraints in your growth model overall and therefore
where are the things that we need to focus on as a team over the next quarter two quarters and so on and that it's constantly changing you know as I mentioned there's a bunch of stuff that needs to feed back into that model in terms of growth performance or the learnings you're making and so on that means it's changing and you're constantly reevaluating and it's guiding
kind of quarter to quarter planning cool so just to make it even more concrete for folks that are listening so one of your Loops is this integration with GitHub where you automatically fix their code and create a PR you may find at some point somebody clicking that link to becoming a user is too high friction too many people are falling out so one
strategy for one of say the activation team could be we're going to optimize this part of the funnel so when clicking from GitHub to sign up as a user and get them to a point where they've found value right yeah exactly that sort of thing or earlier on for example it might have been without taking that same example of that same Loop and this is sufficiently
far in the past that it's easy for me to talk about but we start with GitHub we want to now expand the scope of that Loop we've seen success with it we see the performance of that Loop growing but we think there's opportunity in saying let's um let's expand that by adding support for bitbucket and now all of a sudden we spin up that same loop with
bitbucket and then gitlab and so on awesome okay we could go a whole whole hour on just that piece so we'll move on and we'll get save that for v2 data was the last thing and our growth team growth team just can't function without dating that even with early stage products before you have the needed volume of traffic to be able to run formal tests in a in reasonable time
frames you still need to have signals that help you learn and inform your decisions both Quant and of course qual through speaking with users my advice there is really just to invest early the infrastructure the tooling and at a given scale the dedicated people as well they're going to be core to building out the growth team and they'll enable data to
ultimately pretty much inform all critical decisions so snake was interesting that we didn't have a problem of not having enough data in fact there was an abundance of data like too much we collected absolutely everything and the problem I identified very early was that we didn't have enough behavioral specific data and we weren't intentional enough about the
data that we were collecting and how we were collecting it which made the data hard to trust and that that's a that becomes a big problem so we invested in tooling and processes for building out event tracking plans and now we test conformance to schema of the instrumented code in our CI processes so we have absolute confidence in the data so that was you know get into a point of
trustworthy behavioral data was absolutely key but the reason I I say invested early here is that to remember that it also takes time to accrue an data that you can learn about and make some key decisions around retention for example also that you have enough data to to run regressions on be able to inform definition of your activation metrics or
engagement States and so on so the earlier you can start collecting the better so yeah people in process a strategy and data and you absolutely need all of those things to build and run an affected growth team when you get all of those things working it's like rocket fuel for Focus creativity but at the same time you know slowing down to put the
maturity in place as an Orc scales that the pace sneak has it it's pretty much impossible you still have to get stuff done while you're kind of building all that stuff out right you have to accept that you're going to make a bunch of mistakes along the way you have to be 100 comfortable with that and you have to treat those mistakes as learning opportunities that that provide levers
for improvement and it's also useful to you know you asked about kpis and what that team are responsible for and that is one way in which a growth team absolutely needs to make impact and it's the way that primarily they're going to be held accountable but it's also I think useful to think about the efficacy of the growth org not just in terms of
the impact they drive via experiments and new product experiences and on core growth kpis but also how they enable and up level the rest of the Orcs so for example our entire product LED sales process it's powered by an evolving model that describes our understanding of users teams accounts the usage and adoption patterns and signals that best inform where and how our GTM teams
focused you know insights from growth teams often have utility far beyond growth but people can't know if something is useful to them if you haven't shared it with them so the learnings made in the growth teams even those from mistakes or failures we socialize them widely and visibly we want every R D team to be leveraging experimentation where appropriate to
learn to create business impacts so one of the things we did here is creating a paved Row for adoption of Behavioral analytics and experimentation stack coach teams on getting started to make it as easy as possible for anyone to start to reap the benefits of the platform built out internal education programs on on data-driven development
on experimentation built internal tools to help with metrics design and so on and then building core platforms Services as well that that are useful for people outside of growth so we built services that power contextual onboarding and and originally that was the intent but now those same Services can be used anywhere in the product to to give contextual experiences I know
that was a bit of a ramble but hopefully there's one or two useful things in there yeah there's two things I wanted to quickly follow up and then we can talk about the product team you said you socialize learnings and experiment results is there anything anything you could share there about tips to do that you have a tool you do use that for is
there someone posts stuff in slack is it an email that goes out I'd actually socialize learning such that say A salesperson can do something with it as so this first of all there's a bunch of slack channels like sync runs on on slack basically so there's a bunch of slack channels even when we're Banning experiments those are kind of wide and in the open and we invite collaboration
on those but from a ceremony perspective we try hard to institutionalize ways to generate and leverage learnings it's something you know I feel pretty strongly about so we have these Team level impacts and learnings reviews Loosely modeled from a Blog years back now I know six years I want to say the Brian Balfour wrote about a similar ceremony from these
HubSpot days now if I had to pick one meeting that as the most important in the growth team it would be this the teams continuously document any learnings from data exploration from experimentation from user research and so on they document that in their weekly impact and learnings document some teams find it better to pair up and advanced into dedicated learning
sessions to Deep dive on specific relevant topics but however it comes together it's all kind of put into that document when it comes to the meeting itself it's usually run by the PM most of it is spent discussing learnings that have been documented their implications how they can be leveraged in follow-up work where they might have relevance to
other teams and so on a relatively smaller part of the meeting is also spent looking at Key metrics some teams have actually split that out entirely into a separate meeting and then no time at all is spent reviewing what the team have actually been doing it's more on kind of the outcomes and the learning so that's at the team level but then we run
that same meeting at the group level on a monthly basis so that's run by the product engineering or marketing director for the growth group and that's where all of the growth teams come together they share some key learnings um can't share everything of course what they're doing is picking specific learnings that have potential relevance and utility across the other teams so
also as a standing agenda item for our user research team to share what we call developer insights that's one of my personal favorite meetings to attend it's always recorded and socialized with the rest of the company after words and yeah I'd say that's really important but there's a bunch of different ways in which we're selling out that information
constantly so cool and this is a meeting that anyone can come to like a sales person comes to to see what interesting stuff the product team has learned recently from experiments how cool would you be able to share a template of that document that you put together that we could include in the show notes yeah for sure sweet and then the meeting you're set it's run by
product basically like some of the product functions and the idea is to share things they've learned in recent experiments say in the past month typically the team level ceremony it's PM LED you know that's from more from a facilitation perspective the learnings are all brought forward by the various folks in the team and each one of them who's contributed to learning will talk
about that learning with the group and facilitate the discussion from there and then at the group level those are read by the directors for the growth group and each team it might be an engineering lead it might be a tech lead it might be a designer it might be the product manager we'll talk about some key learnings from that month makes sense if
there's nothing PMS are good at it's facilitating meetings so that makes sense okay and so that's a good segue to just chatting a bit about the product org I'm curious just like how you structure the product team and then how that works with the grow team looks good yeah so the product org what do you want to know broadly how do you structure it just
like how many teams do you have do you align it across by outcome or is it by a surface area of a product and then is the growth team like a Js into this product org is it like a unit within the product org or is it integrated but I don't think it is so yeah just how do you what does that look like org chart wise yeah I think we've got a fairly common
pattern for how we structure our products and wider r d org so most of the org is functional ownership based there are a lot of really complex domains in the core products so having that localized knowledge is important to be able to kind of own and run the code the team ship the growth teams on the other hand are structured by outcomes we talked about
that already owning areas like acquisition activation engagement monetization these outcomes and team remits change over time as we talked about but the teams here are often working on areas of the product where they don't own the code which I think is a the key difference between how we structure our growth teams and products and with some exceptions you know one of
the growth teams actually owns the onboarding flows and so on so that does require a lot of trust it requires very transparent communication mechanisms built into how we work one of the meetings that that we have regularly is experiment plan reviews they're kind of ad hoc ad hoc meetings they're led by the experiment lead could be the PM
could be anyone else in the team but the important thing is a bunch of people will be invited there especially stakeholders from other teams where we might be experimenting on their surfaces and that won't be the first time they've learned about this we'd like the training actually include them in co-designing the experiment plan so they're kind of fully on board with it
but absolutely kind of inviting them into those experiment and Views really key if we're going to run an experiment on that surface you know we need to make sure that everything in that experiment plan is watertight especially from a scheduling perspective because the last thing we want is a week and a half into an experiment for some change to happen
within within that surface from the product team unaware that an experiment's happening and completely invalidate the experiment cool and then in terms of just org wise do you have a lieutenant that is responsible for just say the product team and then someone responsible for the growth team or the director's report up to you and you have a bunch of reports how do you how's that
oh so so our CPO on the exact team manage he runs the entire product organization he has four I want to say VPS that are in different areas so there's a couple of VPS that are in different areas of the product so first of all the our application security products secondly our Cloud security products third is uh platform and fourth is what we call
developer Journeys which is the area I own which has a few groups within there one of which being the growth group got it okay makes sense okay there's just a couple more questions I want to ask that are like very tactical specific before we get to a very exciting lightning round to close this out one is with a freemium product there's always this question of what to put into free
and what to put into the paid plan is there anything you've learned about how to think about that what should be in for you and what should be behind a paywall was it on your on your show whether they know that she talked about kind of things that promote your growth model being good to Landing free and things that add friction yeah being that's right yeah so I I really like
that guidance um I'll add that for many businesses there might be some cost of service element to consider as well you know of providing a feature to free users is cost prohibitive due to the volume then that's obviously something you're gonna want to reserve for paid I mean ultimately that was the whole reason cited behind Heroku recently removing
their free plan entirely I think your plans from the from free to the top they should have well-defined understanding of a Target customer the use cases they should map out or you should map out the motivations for motion between each you're really clear about what are the drivers for someone to take a step from one plan to it to the next forsake the real drivers to
move from free to a paid plan for example is when you want to secure business critical code and you start having needs around governance and compliance and the other the other interesting to mention is how you approach trials like with most things I think we're still figuring that out at sneak I don't know that there's ever a perfect answer or
even a correct answer here certainly different from product to product we have a self-serve trial to support time boxed evaluation of some of the capabilities that are reserved for our paid plans but we're intentional in revisiting the model periodically you know it's important I think to regularly challenge yourselves to ensure you don't fall into the Trap of Simply assuming
what was best fit in the past is best fit now and in the future now what if the trial duration was limited by some dimension of usage instead of time or what if we didn't have a child at all but put more into the free plan with appropriate limits and why making those changes impact our growth model it's not always easy to answer those questions
but I think there are some ways that you could you can help test there for example you know you might cohort trial users and teams who have low engagement and don't convert during the trial and then when the trial ends drop them back into a new enhanced free plan and monitor engagement there so there's some things you do but it but I think that habit of
continuously challenging yourselves and reevaluating whether the model the specific delineations between the plans and how you support evaluation and emotion between those plans I think it's really important to do that and also when it comes to plg and sales we talked about the self-serve motion obviously it's big and important for sneak but you know that the sales lead
motion critical critically large as well and significantly impactful you need to think about the plan design and those motions across both aspects of plg and the sales lead motion when you have when you have a strong plg Foundation that is inclusive of a product LED sales motion you're going to be in a really powerful position from the perspective of having a significant
volume of Highly qualified leads that are coming from the product we actually track a metric that we call product driven Revenue which basically accounts for all revenue in customers where we saw meaningful value-based activity in the product before there was any sales contact and that really tells actually a super interesting story about the plg
efficiency of your company across all revenue channels self-serve and sales LED and what's what's fascinating there is that the product driven cohort contribute a relatively greater amount to net retention so when you think about packaging you know you really need to think about and understand that macro level contribution of the freemium motion and know what you're trying to
optimize for kind of balancing Revenue today versus potential future revenue is that increasing net revenue retention from product LED leads mostly because they start at a cheaper price you think or is it more that they just end up being better customers it's a great question I'm not sure I have a good answer no problem I have still trying to figure
that out it's a good prompt to have people yeah adding more customers and seats you talked about the importance of figuring out the trial length and what to put in the trial and freemium and things like that is there anything that just has surprised you something you've learned from iterating on that comes to mind I wouldn't say surprised me
per se but it's something that I think has is perhaps obvious in retrospect and that is that companies of different sizes of different complexities of different industries from those that are you know very highly regulated to the other end of the spectrum they're gonna take different lengths of time to need to evaluate properly so being able to cater for those
in some way whether it might be dynamic trial lengths or whether it be trial links that are based on usage or things like that I think that's um it becomes really important that's something to be thinking about for sure awesome that's a great learning I know Elena talks a lot about how trials often screw you because you don't give people
enough time to really evaluate in a company so that makes sense last tactical question that I wanted to touch on is around activation and activation Milestones I'm doing a survey right now with Yuri that I think will come out before this episode airs but anyway in real time I'm curious how you think about setting what the activation Milestone is for a new sneak user so
maybe just share briefly how you think about what is an activation Mouse and why is that important and then how you define that for sneak like what is the Milestone of a user is activated for your product first of all what is activation so for us activation is indicative of the team forming a habit around the usage of sneak and when I say the usage I
actually mean deriving core value which is ultimately fixing vulnerabilities it's not just logging in right it's not even just finding vulnerabilities it's fixing vulnerabilities so building a habit around that and the reason I say team instead of using is and we actually base most of our definitions of activation engagement around teams it's really important because ultimately
security is a team sport that team might be one person which case the user is equivalent to team but often a team is multiple people and we actually expect different people to fulfill different parts of the team activation Journey we also want to enumerate aggregate level activity around fix that sometimes what happens off platform where we can't
explicitly measure it at the user level so in the activation process we have set up moments aha moments and habit moments now habit moment that we Define as a team being activated it's related to fixing vulnerabilities within 30 days of Team creation and the reason that is is chosen it's because there is a significant correlation with Downstream
and in that case with activation three-month retention and retention again based on again not just coming back and logging in but still fixing so teams that fix the vulnerability within their first 30 days are much much more likely to still be fixing three months later that's really interesting yeah I love that how did you come up with that number it was there a decision scientist
that looked at some kind of inflection of like at 30 days and it's probably not exactly 30 in real life but it's like a nice round number that's close enough right yep that's it so there absolutely was a decision scientist in involved thankfully we had to collect a lot of Baseline data first so after we built out the data platform we needed actually
to wait a bit to build a good data set we did a huge amount of Quant analysis a lot of spelunking of the data applying ml models along with a bunch of supporting call research as well but we started really with identifying the Corpus owners and their use cases different roles of different users within the team-based activation Journey defining our retention metric which is
still fixing so whether a team is fixed along with natural usage behavior and expected natural usage frequency and then we found the Habit moment that ultimately most strongly correlated with improved long-term retention and most of kind of the numbers side of things came out via ml Ops platform but after that we then worked back to figure out that's
the Habit moment that's what we see as activation but the Step steps that teams need to get there so what's the aha moment before that what's the setup moment and what are the individual steps that the team needs to go through to to reach that setup moment so yeah that's the overall process and ultimately it's a really strong model that allows us
then to feed in the set of user level behaviors that we know can influence those different steps on that part to activation awesome I'm excited to get this post out and that's a really good anecdote of how a company comes up with it and what they said I also just thought of a another question I may start asking everybody I know I keep saying we're going to wrap this up but
here's a question if it doesn't work we'll get rid of it you mentioned a bunch of tools that you use and I'm curious if you have to think of what are the five most important slash valuable SAS products to your organization other than like the obvious ones like Salesforce what comes to mind when you say organization do you mean sneak at
large or do you mean the growth let's say let's start with growth and see where it goes okay so I'm going to say amplitude first of all uh segment as a means to be able to get our data from the products to amplitude and to everywhere else that cares about it whether that be a kind of Downstream bi system snowflake can look around top of that or system marketing automation
systems like Marketo and stuff like that so I'd say amplitude I will next say full story which is you know absolutely fantastic for kind of session replays of course and kind of it bridges that gap between call and Quant right I would say userinterviews.com which is comparable to usertesting.com both of them amazing services in terms of getting fast curated access to
individuals to participate in user research break here's another one so sprig is a fantastic kind of in-app survey platform which is what we primarily use it for but it also does a bunch of other cool stuff in terms of being able to test ux designs as well in app how many have I said I think four if there's anything else you could add a fifth okay if not
we can move on that's awesome this is a really nice test yeah I'll stick it at full okay is there anything for the wider product team that also comes to mind that you guys find really useful a table in fact a table for for growth and products it's just so flexible it's just you can do anything in it in fact if we think about growth I should have
mentioned that first because that's where we keep most of our experiment plans and the knowledge base and our user research base okay I like this question I'm gonna start asking if this is great okay that was a precursor tour very exciting lightning round I'm just going to ask you a bunch of Rapid Fire questions just answer whatever comes to mind we'll keep it short and quick
question one what are two or three books that you recommend most to other people I have been dreading this question is there oh there are too many so I'm just gonna I'm gonna pick a couple that I've enjoyed reading in recent months so for product and growth Geeks like me or in fact anyone with more than a passing interest in data I'll recommend how to
measure anything by Douglas Hubbard second up you had Marty kaganon and he mentioned the book Sprint by Jake Knapp and John zaretsky and I love that book but personally their make time book it was something that radically changed my relationship with information and I recommend that to all time staff product people out there and for number three I'm gonna say This Is How They tell me
the World Ends by Nicole pearlroff which gives an amazing view into the world of digital Espionage oh I read that book Tim Ferriss recommended that at one point that is a wild ass book very beautiful cool I love these recommendations all right that's it cool great choices okay great favorite podcast other than the podcast you're currently on maybe acquired with Ben
Gilbert and David Rosenthal I just wish I had enough time to listen to them all yeah they're pretty long I was just at an event where they interviewed someone live as like a live podcast that was very cool those guys were Pros what's a favorite recent movie or TV show so movie turning red on Disney plus which we love watching with our kids it
was just fun TV show the most recent curb season had me in tears as usual they haven't had a new one in a while right so that's a little bit out there yeah a little bit cool I'm ready for the new one's coming oh it is season 12 yeah I don't love watching that show my wife loves it cringe painful but anyway watch it anyway and my wife's actually saying
she can't watch it because she cringes too okay or reverse I love the awkwardness it's good for a product leader to have that enjoyment favorite interview question that you like to ask candidates give a couple here if it's not cheating too much first is one I like to ask when hiring for for anyone actually really um it's fast forward three years what's different
about you then a lot of people will default to telling you where they aspire to be in terms of role or title but what I'm really looking for is signals of humility of self-awareness around areas of personal and professional growth so you know people who can be open about where they think they need to work on to to grow themselves as people I love that also
just generally throughout interviews I'm looking for curiosity so day to day good PMS will be asking why as much as my six-year-old son does which is a lot so I'll try and discern that through the course of the conversation it's not really a question but something I'm looking for and then maybe I want to flip it because building on something um that Adam fisherman was saying his
theme of evaluating the people dimension of folks you're potentially going to work with when you're interviewing with a company and this was a question I got asked myself recently by a candidate which I just thought was brilliant and that was tell me about the diversity Equity inclusion and belonging initiatives that you've recently personally been involved with and that
it just felt like a really great way for them to be able to test the alignment of their personal values with those of someone they'd be working with really closely so I love that awesome by the way I love how many callbacks to other episodes you're making you're definitely a power adopter of the podcast and I really appreciate that there we go
last question who else in the industry do you most respect as a thought leader as a leader in general so maybe I'll cheat on this one a bit too and I'm not going to combine it to the when you say industry I think security industry but I'm going to look at the product domain yeah and specifically product operations and in my mind there's not many people
who know more in this area than Christine itwaru or pindo so if you ever get a chance to talk with her I know that would be a fun conversation many gems would be dropped I think wow I will get her on this podcast that is my new goal I had not heard of her and that's awesome thank you for the suggestion then this has been awesome so many nuggets and stories and insights I so
appreciate you being here two last questions where can folks find you online if they want to reach out or learn more or maybe come work at sneak and then how can listeners be useful to you firstly I'm I just want to say a big thanks for having me on Lenny I love talking about all this stuff and really appreciate you being willing to let me bend your ear a little bit as to finding
measure thanks as to finding me I'm I'm a bit of a twit when it comes to Twitter I generally spend a bit more time over at LinkedIn but you can find me on both of those platforms at semanticbin and in terms of how people can be useful to me I am starting to take on some additional clients in advisory capacity so feel free to get in touch if you think I
could help I can't not ask about semantic Ben and what the story is there and before we let you go actually when I was at IBM it was a focus that I had on linked data um yeah became semantic bench all right hello there we go it's stuck awesome all right Ben thank you so much for being here and thanks for listening thanks Danny Take Care
thank you so much for listening if you found this valuable you can subscribe to the show on Apple podcast Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can find all past episodes or learn more about the show at lennyspodcast.com see you in the next episode [Music]