the issue for us at the time was that we took people away from the investment in our core product to go do those other things like we moved people right and so the Trap there is that you leave yourself right for disruption in your core because someone else can out invest you in that core and so if you're the leader in some core product our takeaway here is you should continue to out invest everyone else in that core and
then invest you know the profits that come out of that core into the next Metro invest profits and not people or or Venture Capital which is maybe like net present value of profit or something to that effect but don't take people away from the core to go to this other things because then then you end up distracted welcome to Lenny's podcast where I interview world-class product leaders and growth experts to help you get
better at the craft of building and growing products today my guest is Vijay ayengar Vijay is currently head of product at mixed metal it actually has a very similar career trajectory to myself where he started as an intern at Amazon and he was an engineer for a while at Uber then he became an engine manager at mixpanel but then he shifted it from an end manager to director of product and now header product at mixpanel you don't
often see people moving from an Android leadership role straight to director of product so it was really interesting to hear what he took from his engine experience and brought into his approach to product leadership but we spend the bulk of our time talking about what he's learned from the journey that mixpanel has been on where they started with a simple product then scaled to a number of different products solving many
problems for customers and then made the hard decision to scale back to just a single core focused analytics product we talk about why they made that choice what they learned about when it makes sense to expand to new products and when you probably shouldn't and how they approach that organizationally I also talk about how mixpanel builds product how they think about product philosophy how they prioritize and also what you're
probably doing wrong and how you set up your analytics for your own product with that I bring you the J Iron Guard after a short word from our wonderful sponsors this episode is brought to you by Pando the always-on employee performance platform how much do you love the performance review process um yeah it's time consuming subjective biased and there's rarely any transparency with the rapid shift of
distributed work it's a struggle to create the structure and transparency that you want to help your employees have the highest impact and growth in their careers hando is disrupting the old Paradigm of Performance Management including a continuous employee-centric approach so employees stay engaged see their progression in real time and know exactly when and how they can level up with pandu managers can leverage
competency-based Frameworks to effectively coach and develop their teams and align on consistent growth standards resulting in higher quality feedback and higher performing teams visit pando.com Lenny for more info and get a special discount when you sign up and reference this podcast that's pando.com Lenny this episode is brought to you by notion if you haven't heard of notion where have you been I use notion to coordinate
this very podcast including my content calendar my sponsors and prepping guests for launch of each episode notion is an all-in-one team collaboration tool that combines note-taking document sharing wikis project management and much more into one space that's simple powerful and beautifully designed and not only does it allow you to be more efficient in your work life but you can easily
transition to using it in your personal life which is another feature that truly sets notion apart the other day I started a home project and immediately opened up notion to help me organize it all learn more and get started for free at notion.com Lenny's pod take the first step towards an organized happy team today again at notion.com Lenny's pod welcome to the podcast thank Lenny great to be here huge fan of the the Pod so
glad I can contribute I definitely want to talk about mixpanel's Journey both as a product team and a product but before we get there as an engineer you're in a long time engineer and then you became a product leader is there anything you had to unlearn as an engineer and the way you thought about leadership and product and business one of the things after you've been in engineering for a while is that developed this um tendency to
to immediately respond with no to new ideas and I think the engineering perspective this is because you spend a lot of your time building and maintaining ideas that maybe are half thought out or didn't really go anywhere and you just feel like the full brunt of the the maintenance cost of that and if you build up this the scar tissue and this immune response to say no to new ideas and it's a hard no like no we're
definitely not going to do it and I think I had to unlearn that moving into product because you know you get a lot of ideas coming from a lot more places in the organization and ideas are are fragile in their agency and it's you know hard no can really kill a whole direction that you could potentially go they could be very high reach and high impact so one thing that I found is the best way to get to a no if you ultimately need to
get there is is to try to make it work like start try to make yes work and and document how you've tried to make guesswork and do that earnestly not as a you know as an exercise of just an alternative that you're considering try to do it sincerely and get to know after trying to make guesswork um and so that's that's one thing I've been trying to just apply my engineer problem solving brain to to do that instead of
thinking about how it might not work and something saying no is that something that you recommend Engineers work on like looking back I know as a PM it's like oh I love when Engineers say yes this is awesome I'm gonna help everyone learn to say yes but as an engineer obviously that's a challenge often what do you recommend to folks that are Engineers currently that maybe want to improve on this or or should and how
they think about saying yes they know when they're asked about something new I think some of the best Engineers that I've worked with actually already do this but by default they're able to balance both in their head it's ultimately this balancing active you just want on call you were woken up three times at 3am due to various bad ideas and the next morning you wake up and then stand up it's like hey can we
do this new thing you're you kind of have to have that empathy and do that so yeah I think the exercise is just like just take 10 minutes to consider the idea just sincerely consider how might we make it work and if at the end of those 10 minutes it's like futile and there's no path it's fine to say no and it's a good good instinct to say no I'll actually in a lot of cases but yeah I would recommend that to Engineers I
think it would have been better in my career for sure if I had learned that sooner I want to spend some time on the mixpanel product Journey it's been an interesting uh roller coaster I think the company's been around for how many years since 2010 2009 2009 yeah so you know it started as kind of a very simple product analytics product back in the day and then as you do with ambitious companies you
look for more problems to solve you look for more problems to solve from your customers so as I understand it you guys added a lot more products to the suite of mixed panel products and then I know that there are some challenges with scaling that and maybe the products didn't stick as much as you're hoping and what I understand is recently moved back to just a single core analytics product and so I'd love to just hear
that journey of what that process was like what you learned as a product leader and as a company about kind of scaling expanding trying to solve a lot of problems and then coming back to One Core straightforward problem McDonald's started in 2009 as provide product analytics to epd teams I think early on it saw a lot of success because it built this in-house database called ARB which stands for arbitrary segmentation and
that was necessary because events data which is the fuel for product analytics is a few orders of magnitude larger than than most other types of data that people collect as you need a specialized approach to deal with it and so that I think spurred the first wave of explosive growth because product analytics was a really burning problem at the time people were shipping mobile apps like crazy and they needed a
solution that could scale and that was kind of a durable mode for make final for a while and I think because we had this SDK that was installed on so many apps and we had this really scalable event collection and analytic interface it was just natural to expand into a few adjacencies that would leverage those same Technologies the first one was messaging being able to send targeted messages to users which is you know
something that's fairly natural you might want to do especially if you have an SDK already installed yeah the other aspects that we've added to do was did infrastructure and try to be sort of the single source of truth of data and companies and what ended up happening was that by 2018 we we had this big churn problem we had something like 40 churn Revenue churn our core product and when we dug into it it wasn't that people were churning
because they didn't need product analytics anymore they had the need they were just churning to competition because we were just not up to the market in terms of the features we had in our core and when we dug into why that was it was just that we had a 50 engineering team that was Building Products across three domains right product analytics messaging and they had to check her stuff our engineering team
was just spread too thin to address all those core gaps and functionality and so we made a really hard call at the time we said the hard no uh to those two other categories and decided to focus our entire engineering team on closing the gap on product analytics and innovating there and from a process standpoint how we operationalize this was we threw away all our planning and all the execution and the work that we
plan to do so far and we did something very simple we took all the churn reasons that our customer success and sales teams have been painstakingly collecting for years group them by category which was like roughly product features we needed to build sort of descending by ARR took the top 10 things and made that our roadmap I just gave every engineer you know direct access to customers and give them a bucket to go
work on which I think goes against about a million product best practices out there of just doing that but I think given the context at the time we needed to optimize for Speed and speed comes when you have extreme Clarity on what you want to do and focus and so we really just optimize for Speed in that time and so in that first year we we moved really quickly and we shipped something like 100 features in that year
and closed a lot of gaps again not these are all vanity metrics like measuring number of features doesn't mean anything and what year was this by the way Ron this is 2018 to 2019. um got it yeah so we moved really fast shipping all these features and instantly saw the improvements to to win rate and the retention but you know one of the the cracks that started to emerge was we neglected the holistic design of our
our product at the time right and if you're shipping features that quickly it's you don't have time to stop and think like where does this go and how does this fit into our overall system architecture and what started to happen was that we were hitting diminishing returns with some of these features and like not considering the holistic design and consistency meant the reach of every feature was low right like you had to
rebuild it for every part of the product that we were in So at the time we made we spun up the second stream that was very design-led and I think this is also coincided around the time we adopted figma it's a really broad design a seat at the table of the company and and we just set up this goal to make design one of our our key differentiators so this this design driven initiative was really about how can we think about the system
architecture of our product what are the key building blocks of McDonald where do they need to fit how few of them can we have which is a really important step and then how will users discover them and how do they relate to each other but I think this realization was born out of the the fact that like so many great products win or lose based on their architecture cutting notion for example like that
pages and blocks architecture is is so strong and you can hang so many features off of those core building blocks in a way that has such high impact on on region and discovery of those features so anyway we did that in parallel with the an in continued that grind on on core gaps and so the end result of that phase which is from you know 2018 to maybe late 2021 2022 was our retention went from about 60 to 90 and our NPS
went from 16 to 50. so I think yeah I mean there's there's a lot in there to unpack but refocusing on the core really helped us uh achieve those results got it yeah I have a lot of questions about this so interesting so that phase that you went through where you sorted things by potential ARR was that the phase of expanding to multiple products or that was post we're going to focus on analytics and go all in there
oh that was posts focusing on it okay yeah yeah and you're saying that you had a stream of just build all the features that were lacking that are causing customers to churn and in parallel there was a track of let's build this product such that it all connects and works together well and it's really well thought through long term the first thing I might have made it seemed like it was just the buckets or features we
did take the step of turning them into problems and being clear like exposing Engineers directly to the customers that had those problems and then invent a solution to solve them so I mean it Loosely there were features involved but a lot of them are kind of core problems we needed to solve but first approach is so interesting like it's kind of like yes we will make more money if we focus on these features to your point it ends
up being just a bunch of features and products that kind of maybe don't synergize looking back was that a good idea to approach it that way at least for a while it highly depends on your context in a very competitive context where there are just table Stakes features that customers need and that's been validated by the market you need to optimize for Speed more so than anything else but it is an approach
that outlives its usefulness pretty fast and we we put that approach behind us relatively quickly after that phase um and I would actually say like that design driven phase was the next phase where it was okay we're not bleeding on the table Stakes anymore but we want to make a holistic product that have high range high impact on the features and and is actually usable and so that was I think a follow-on phase that's necessary
obviously depending on your particular circumstances and competitive Dynamics you can sequence them differently but I think it was the right call to just sort of like that on-call thing again where you know when you're in trouble you got to get out of trouble you can't mow your lawn while your house is on fire he could have put out the fire and then deal with everything else yeah so that that's kind of the approach we took
what's an example of a feature or product that you launched within that first track and then what's an example of something that came out of the designer LED approach if anything comes to mind I think out of the first track oh man there's so many that were just core like we didn't have a good cohort product at the time like just being able to create behavioral cohorts of users and create them from any any report that
we built right and I think that I mean it's just table sticks in analytics to be able to do that so that was one of the first things we built in was fairly obvious there's a lot of other things in like more advanced types of funnel analytics and flows visualization that was you know really Interactive I think on the design LED phase the biggest thing I think was visualization consistency and making our charts
interactive in a consistent way across all across our entire product um and so there's two things that enabled one was that every time you added a new visualization or a new enhancement to a visualization or like how something is sorted in one report it just instantly applied everywhere so just the reach was multiplied for everything we added and the other thing is it just made the product more
accessible let us add dark mode so it made emitter visualizations really stunning and really easy to see what the takeaways were and then every new visualization you added inherited all those benefits I'm trying to think about like being at a company that goes through this phase of hey we're just going to build a bunch of stuff that we know we need and it feels like it's hearing it it's like oh yeah and then we're just gonna
make it all look great and connect and work well I imagine that wasn't planned and I imagine that wasn't easy to get people to maybe slow down on just building more products and features or push it in a direction where it's all going to make sense can you talk at all about what that process was like like how hard it was to shift from We're Just Gonna Knock through all this checklist of things to like let's just yeah bigger
let's slow down let's spend a lot of time designing it was definitely more messy internally than I described it one of the key junctures was when we had this really talented design team and we were putting them on these very tactical projects that that was like frankly like that was very engineering but right and design would often come in at the end and be asked like hey can you just make this look nice and put some pixels on it and
it's just such a waste of your design team to uh to have them do that but at the same time the pace was so high that they didn't have time to come up for air and do anything else um and so there's actually this moment where I was an engineering manager at part of this and you know had a meeting with RPM and our head of design at the time and you said hey we can actually do the next three months of projects about any
design which was a kind of controversial thing to say but we're doing this so that you can take three months with a set of designers and go think about the system architecture of the product and we'll be you know we'll wait for that to be done before we do any architectural things that might impact the architecture and I think that gave designers like a bit of breathing room to go do that just like separating them
for a bit from from the Tactical fire because what was happening instead was we would get towards the end of the project bring design in and they would use each project as an opportunity to squeeze in like oh and we can simplify here and that's just a classic way to blow up scope at the end of the project because there wasn't a dedicated space for design-led projects and I think that that was kind of a key friction point
that we ultimately had to decouple for a bit and then regroup and say okay now like what's our strategy and and just take on projects purely for the sake of improving consistency reach depth of our ux also looking back the process he went through adding a bunch of products to solve more customer problems something every founder and product team thinks about when should we add new product lines
when should we expand Beyond The Core I'm curious what you take away as a lesson and what you'd advise other Founders and companies when it comes to when is it time to expand and think about a new product and a third product and a fourth product I don't know if there's a hard and fast rule here I can just maybe say what made sense and didn't make sense in our context the issue for us at the time was that we
took people away from the investment in our core product to go do those other things like we moved people right and so the Trap there is that you leave yourself right for disruption in your core because someone else can out invest you in that core and so if you're the leader in some core product our takeaway here is you should continue to out invest everyone else in that core and then invest you know the profits that
come out of that core into the next Metro like invest profits and not people or or Venture Capital which is maybe like net present value of profit or something to that effect but don't take people away from the core to go to these other things because then then you end up distracted and the other thing aspect of that is that those secondary products we took on were in categories of their own and it's really tempting and you often get
dragged into building you know enter accidentally entering another category and then you'll end up building these bolt-on products that are the end best in their category right and like the adjacent categories for analytics are like cdps or message targeting or feature flagging or something but you know there's not that many people that need the sixth best CDP or the eighth best feature flagging or the 10th best
message targeting tool and it ends up being you know in aggregate will contribute five to ten percent to your Revenue well seriously accelerate your growth rate and then takes Engineers away from the core product and so those are the circumstances that we were in and I think if you're seeing turn to your competitor on your core product and you're not Best in Class on any of the other ones then maybe it's time to
reevaluate and then the last thing I'll say there is that it's also 10x more painful than you think to cut mild successes than than anything else and organizationally painful and there's teams that have whole road maps and you know it's it's a really painful experience so you know you have to think really hard before you you kick those off that is a really really insightful advice makes me think about if you bundle good enough Solutions
there needs to be kind of this anchored tenant that like I will not give this thing up and I'll use like the third best version of something else if you have it right but if you're not that valuable and important you're not going to convince people to use something because they're you're competing against the best in every category exactly yeah that is really interesting I've been doing this kind of series on how
different companies approach Building Product and I have a few questions I'd love to ask around the product development process and mix panel sure the first is just like how do you plan how do you plan I know it evolves but just how do you plan currently like how long are your planning Cycles how far ahead do you plan in detail these okrs maybe maybe I'll start there that's three kind of sub questions we have
these kind of unsolved problems and analytics that we're going after for us that's like people always want more power more Simplicity better Data Trust faster onboarding better collaboration Better Price performance and so we largely organize our teams around those problems and those missions one quick aside there is that you know some of those problems have attention with each other like power and simplicity there's
there's a trade-off there right and we want one team to own both so that they can they're kind of forced to to confront that tension and beat that trade out and so that's kind of how we think about generally our product team is these cross-motional epd teams Each of which that's focused on solving these long lived paired problems protectors are like our core analysis team focuses on that power Simplicity trade-off problem
in terms of planning the way it works is that we plan on a six-month time Horizon um and I can talk about our most recent planning cycle actually because we're just completing it yeah let's do it yeah basically it started out with the this strategy memo that our leadership team wrote that basically just conveys this is where we want to go as a company in the next year and here's how the product
team can contribute most of that I just established these key pillars we shared that with the teams and they took that and also combined that with all the quantitative and qualitative contexts they're constantly consuming about the problem they're working on and and our customers and ideated and developed the series that's for the next six months which I think are some extent similar to okrs where I bet the anatomy of a bet is
that it's probably want to solve our hypothesis on the solution and then some plan to win like some plan to actually get there and a way to measure that you got there and I think one of the unique things that that we did relative to other companies that do planning is I think it usually is sort of this W process of there's the strategy memo and then teams generate bets and there's a review and then they go back and I iterate and then they
finalize and we kind of collapse the middle part of the W where myself and our head of design actually spend time with each of the teams actually ideating on the vets and participating in the solution Discovery process going into the the jam sessions and adding thing bus to Keys ourselves with ideas and and thoughts on things which we did because we aren't a huge product team and we we're not going to do like 50 things in
a half we're going to do maybe 10 to 12 things and so that's enough that we don't we can do something that doesn't scale uh if that enables high band with communication between us of the teams and it ends up being more messy and unstructured I bet in that in that phase because we're just we're in there contributing ideas as well but by the end of it I think the team leaves feeling both more confident in their
bets because there's more thought that's gone into it and then more lines up to bottom line you know why we made certain decisions and so I think that that's one thing that's different and then we conclude that process with the Roadshow where we present to the rest of the company I'm going to get their feedback as well how long is this process generally the teams did uh pre-work for a couple of weeks like two weeks in
December and then we did a two-week sort of Sprint on on solutioning and ideation in January like the first two weeks of January awesome and what's the end result of planning for each team did they deliver a document with like here's our strategy here's the big bets here's a road map is there a template you pass around how do you kind of get to a thing that people share and present and comment on yeah I think there's
basically three artifacts that are kind of linked to each other so the first is uh we use notion um and so we have a database a notion called bets which is where you know each page in the database is a bad and and that has a template yeah so it's like kind of roughly what I described with a few more sections but what problem are we solving what's the evidence of demand what's the region impact of this problem how do we know
we're successful um and what's the key driving hypothesis beyond the solution um and then a rough plan and then that's tied with the presentation that's kind of like a tight summary of that that has like one slide per bet and then is also tied with more of an execution Focus how do we sequence and staff this thing and eliminate dependencies which the engineering team contributes to so I think those are the three artifacts that
are linked together foreign this episode is brought to you by lemon.io you've achieved product Market fit you're able to activate engage and retain your customers but you don't have the engineers that you need to move as fast as you want to because it's hard to find great Engineers quickly especially if you're trying to protect your Burden rate meet lemon.io lemon.io will quickly match you with skilled senior developers
who are all vetted results oriented and ready to help you grow and all that at competitive rates startups choose 11.io because they offer only hand-picked developers with three or more years of experience and strong proven portfolios only one percent of Canada to apply get in so you can be sure that they offer you only high quality talent and if something ever goes wrong lemon.io offers you a swift replacement so that
you're kind of hiring with a warranty learn more just go to lemon.io Lenny and find your perfect developer or Tech Team in 48 hours or less and if you start the process now you can claim a special discount exclusively for Lenny's podcast listeners 15 off the first four weeks of working with your new software developer grow faster with an extra pair of hands visit lemon.io Lenny I know you have some insights on
vertization and some strong opinions on how to prioritize can you talk a bit about that and have advise your product teams to prioritize one really common framework in Partition is rice reach impact confidence effort and I think it's simple and fairly robust which is I think generally good qualities of a framework but one of the traps with rice that we observed is that the c and e the confidence and effort
tends to cause you to prematurely deprioritize potentially high reach high impact bets really Innovative things and we encountered this on one of our teams early last year where we just rised everything all the ideas and a lot of the high reach high impact things ended up at the bottom because confidence and effort were just so murky for them as they should be typically for for high reach high impact ideas and so one
exercise that we push our teams on is just ignore the c and e for a little longer than it's comfortable and and just sit with those high reach high impact ideas with like engineers and designers in the room committed to actually trying to solve them like give it a fair shot and you'll often find like if you spend a week on that set of ideas you can get pretty far in understanding the confidence and the effort you can
probably find a higher confidence or effort way to do them then add the CNE back in and then rice as usual and the goal is to end up with a reasonable mix of innovative bets incremental bets and then ones that are you know technical that or product that you need to address I usually just cut out the C myself I find that it's not that powerful do you do this in a like a Google sheet do you use this in notion how do you actually
recommend teams do this participation or just eyeball it not super opinionated on the exact tools that teams use I think this is like a team local exercise typically but most teams use no shed and just simple tables or databases induction um for this I think the other thing on prioritization that's always tricky is estimation and you know every engineer will value estimates are all lies and if you say
it'll take eight weeks it'll take eight months but one I think the core problem with estimation is you're asked to estimate things before you know what the thing is and and it's just a strange output to be expected to produce and one approach that I found really interesting is from this book called Shape Up by Basecamp which they said you have appetites over estimates where instead of making the estimated output of flattening you make
the time box or an appetite the input and you say like we want to solve x problem and we're willing to invest six weeks it's only that problem obvious question there is like you know how do you pick that time window it just seems arbitrary and so the base camp people suggest just pick six weeks for everything and they're really austere about like if you can't scope Hammer something down to six weeks you're doing
it wrong which I think is uh it can work and has a lot of benefits that you it creates a rhythm in your company but one approach I found that works better is you pick a reasonable sounding appetite and just explore the two to three options around it pick six weeks and then say what would we do differently if we only had four weeks or eight weeks and you'll kind of naturally find the efficient Frontier of you know cost and
and impact and then align on that and the important thing is that you check in after that time period and say is there any new information that's just we should continue did we uncover the biggest risks and are are we just on the long tail of things and and actually be honest with yourself about about that I think that's important regardless of what framework you use I really like that so does that how you actually
operate you create a time box we have four weeks for this project whatever we get done we ship whatever we don't we push up we operated that way in engineering like particularly on the infrastructure side because we had this series of projects that would just take forever and the longer it takes the longer it's going to take and so we've done that exercise quite a bit I'd say more on the the more engineering heavy
projects than than others but we're starting to adopt it more in the product side as well the main exercise we've taken on the product site is more the consider what would you do differently with different time boxes approach just a thought exercise yeah it's a good thought exercise and it just forces everyone to truly score the requirements right like critical nice to have is nice but really if you know in two weeks you're gonna
get pulled off to do something completely different what would be a complete solution that addresses the core problem and it forces you to build like the meat of the problem in first as opposed to just doing the things that are surrounding it that's cool I really like that I've done that myself I'm curious if anyone ever does the shape up process for real or it's just like we will ship anything that is ready within
six weeks and not actually have like specific deadlines or kind of like concrete goals of products they need to ship in specific ways yeah well I think the shape up process if you run it all the way they do their ideas that you can actually predict on a six week time Horizon so you you can just hammer down scope to something that is complete it needs to be complete it can't be Milestone one that's like a half-baked
thing in six weeks which I think that rigor like the rigor they applied to that across the board you need to do it all the way you can't adopt the process happily I think it's the is the challenge otherwise you end up with people shipping like Milestone one and then moving on which is not the Complete product makes sense uh a couple more questions around how you build product you mentioned that you
have a unique approach to keeping product teams close to customers and I'm curious what you've learned there what you found to be helpful and just kind of yeah keeping product teams close to your customers I think this is one thing that is something we invested in pretty early on it makes battle actually around that time in 2018 when we refocus on our core product one of our sales Engineers Aaron built this automation where
you piped all these customer gaps that we got that were reported by our customer success and sales teams typed that into slack in just a feed and what this created was this culture where all engineers and designers could consume that raw feat of direct points of customer with no gigkeeper no process to access it no pre-aggregation right and I think this scale is pretty far like at a product team of our scale and
with our reach of customers we don't get so much feedback that someone couldn't read it in 20 minutes every day and for like four or five years in engineering every day I would read all the gaps that we got and many Engineers would do that and one of the the rituals that it's enabled is we'll find that Engineers will go into that channel and react with a message with an email Emoji which means I'm going to email this
customer and find out more right and they'll email the customer and say hey I'm the chair that built this feature I saw you said the specific thing can you tell me more I'd love to understand they asked them five wise and then they improve the product on their own and I think that culture is just so important and it's just it just empowers all engineers and designers to think like think like a PM a little bit which I
think takes a little bit of the load off on the PM to be the gatekeeper of all that information and then over time we've evolved it quite a bit as our data stack is involved so we now not just take customer requests but we take things that are posted on Twitter and and NPS survey feedback and you know win loss notes for more competitive deals and pipe them both into slack and into notion so that we can both get the
real-time feed and then we can sort and Aggregate and tag things accordingly but the key artifact of this is that it's all open uh there's no gatekeeper behind that process that sounds both amazing and wild do you still allow Engineers just to email customers and ask them questions about this stuff or is that harder to do it as you've grown oh no we still allow that yeah wow that's awesome one nice thing
about the stack actually the data stack is that it's all basically all these feeds information land in our data warehouse which is bigquery and and from there they're pushed out via a reverse detail tool we use called census to slack and notion if that makes it no code one of the benefits of that is that we can enrich all of these feeds with you know who's the account what's their AR who's the CSM and like all that other
contact information so it's usually not like an engineer is blindly reaching onto a customer customer without letting a CSM know if it's like a million dollar job or something the idea is just like trust them with that context and they can tag the right people and make the right call I'm so curious how that gets prioritized and how PMS are looped into all that but we don't have to get too deep in that that's uh that's a really
cool process I haven't seen that before where Engineers or just emailing customers digging into questions and problems the Trap Of course is what you just called out is like you can be reacting to everything all the time and certainly if you ship a redesign right like the first two weeks of that there's going to be a bunch of feedback that's like I hate this go back and I think that's sort of an organizational muscle
you have to build to balance the reaction and that's just a thing we've had to practice doing but I think the trade-offs are worth it awesome one last question along these lines can you just talk about the tools you use like the SAS products you use to run your product team for collaboration communication nodes docs you mentioned Notions as an example I think our stock is actually fairly standard these days
so we have slack Zoom for communication notion for docs and any long-form writing and it's a Wiki and database and big magic Jam or for design and whiteboarding I think what's actually more interesting is our data stack and the the productivity we get out of that like I briefly touched on this where basically all of our data gets EPL out of all the systems we have lands in bigquery gets joined and
modeled and then pushed out via census to all the other tools in our stack and I think that's been a huge productivity in unlock because you can build internal tools with very little code if you can write SQL you can build an internal tool basically and that pushes information to the teams that need it and so that I think just has unlocked a lot of these types of things like automated qualitative signals with no code in a reliable way
and then if someone's like oh can I get ARR on this yeah sure it takes two seconds to to do that so I think the that data stack has been a huge productivity unlocked for us awesome if you guys shared that anywhere online just to show kind of the stack you guys have built we have a couple blog posts that talks about our our stack for so we use this both for kind of our plg infrastructure and like our product list
sales you know like defining a bql and alerting a new user if it's that criteria but then we also use it for internal tools uh yeah we have a few blog posts on that topic sweet uh we'll follow up and include some of that in the show notes yeah definitely final line of questioning you're one of the smartest people in the world on product analytics heading product for bixpanel I'm curious what you think most people
get wrong when they're setting up product analytics for their site their product their company this may be a bit of a hot take because I think so many people there we go so many people still do this but I think the biggest mistake is is setting up analytics using client-side sdks client-side tracking so um like web and mobile sdks like putting a mixed panel.track or segment.track in your web app or your mobile apps and the reason
it's a hot take is that for many people that's product analytics and SDK tracking are synonymous they're like all right make final means SDK I have to put an SDK in my web and mobile app but that's a mistake because it we've just seen time and time again it leads to poor data quality and difficulty to maintain that that data so the problem on web is just due to ad blockers and other unreliable things in JavaScript
world you end up dropping 20 to 30 of your events and so it just doesn't match your internal databases and then on mobile there's two problems the first is that you have to reinvent tracking for both IOS and Android because it's two different languages and two different platforms generally speaking and so you end up with in many duplicate events that symmetically mean the same thing but are just different because of the
two platforms and you might have two teams owning that and the second issue which is I think even worse is that you are kind of beholden to clients updating their mobile app to get to the latest version that has their latest tracking so if you want to add new tracking it'll only apply to people at the latest version and Beyond whereas yet all of your old tracking whether it's broken or you made a mistake is
still out there in the wild and so you're constantly getting events that are that are old and broken and so what we recommend instead and that we've seen a lot more customers adopt recently is just track events from your servers right instead of your clients and that is three benefits one it's instantly cross-platform web and mobile and TV and whatever other platform they're all going to go through your servers so you instantly get 100
reach the second is it's an environment you control so if you want to update tracking you can update it it updates for 100 of the users and the third thing is that and this is I think maybe unintuitive but it's true is that Engineers have been tracking events from servers forever it's called logs right and events are just launched with a user ID in them and so they don't need to deal with learning a new SDK and dealing
with all that they just just have to track logs that have some structure and a user ID in them and they're tracking events and so if it's easier for the developer it'll get done in a higher quality way so I think the really simple advice there is just just start tracking events from your servers instead of from your clients and if you need to supplement it later on with context that's only on the client you can add
that on later but server side should be the default maybe the last question is just what's changed most and how companies work with analytics in the past few years and then just where do you think things are going in the space of analytics yeah so I think one huge trend is the rise of the data warehouse so these are you know snowflake bigquery redshift and you know they're really scalable and they speak the SQL standard
which has led to this explosion of tools that have emerged around them and make it really easy and cheap to load data into the data warehouse and then also easy to push data out of the data warehouse I think tools like Concepts do that and this has a few implications so the first is that the data warehouse becomes the center of gravity for all data in your company whether it's product marketing and sales data they
all land there and I think the that's really valuable today in this product-led growth world and a lot of ink has been filled by bad but like from a data standpoint that means you know all these teams need to be operating after the same version of the truth and that version of the truth is sitting in the warehouse and it just needs to be joined correctly the second thing in terms of where things are going is
that events um like like a Time series of users did this action at this time are the universal data model for analytics and the reason for that is every action every interaction a customer has whether it's with your sales team like a Gong call or with your marketing team like they clicked on an attribute a marketing article or their product which is more well known those are all events they can all be marvelous
events and it's super granular super intuitive as a way to understand what what people are doing and it's really powerful because oftentimes you want to ask questions about sequences of events right like which users spent This Much Time on My pricing page and then looked at three developer dots that's probably a user I want to reach out to or you know so many things can be modeled off of that and so I think data warehouse is becoming the the
loading dock for all this data which can be very easily modeled as events but it's not a very great analytical tool for events because SQL is optimized for rows and tables and joints and not events and sequences of events and segmentations of events and so one of the things that we're spending a lot of time thinking about is how do we get that really rich trusted comprehensive data set from the data
warehouse into a tool that's optimized from the UI down to the data model for events because that unlocks like really fast intuitive exploration of data on data set that people already have and Trust so that's I think one of the big trends we're excited about and what we see is the future interesting anything mixpanel is in a good place to help people do that is how you see this like the something that companies like yours
will help people solve or this is something everyone's going to have to figure out for themselves or there's like a whole new class of startups launching to help them make the mess out of data warehouses no there's always a new class of startups joining through the analytics space it's never never a dull moment yeah I think this is something that we we're looking to solve because I mean Alex don't need as good as the data and
if people are already collecting great data in the warehouse I mean we integrate with the warehouse really well then then we get access to that good data increasingly what we've been seeing is that companies like in the reverse CTL phase like census and high touch are effectively Reinventing the CDP Reinventing data movement tool like segment on top of the data warehouse and so really our strategy there is just
tightening our integration with those tools and we've seen just huge growth in people using their data warehouse as the source for events like not adding SDK tracking anywhere just saying I already have events sitting here I trust all of them they're from all parts of my business why can't I do analytics on my support tickets and my gong calls just as easily as I can do it but my user behavior and so I think that's that's
the thing we're seeing and we're investing in awesome anything else you'd like to share before we get to our very exciting lightning round you know we open talking about the the transition from engineering to products and I think one of the things that's just been really fruitful in my career both on the engineering side and products that is just adopting that product mindset and getting closer to
customers consuming the Raw Feed of customer context taking every opportunity to talk to them and I I'm really excited to see you know things like this podcast and your newsletter and other forums for engineers to also develop that for my mind that and get closer to customers because I think long term that just means better products and services at lower and lower prices which is just Innovation right so
I'm really excited to see more of that in the world here here with that we've reached our very very exciting lightning round I've got six quick questions for you I'm just gonna go through them pretty fast whatever comes to mind your share and we'll see how it all goes sound good let's do it okay uh what are a couple books that you recommend most to other people on the business book standpoint uh there's this book called the goal by
Elijah goldron and it's kind of an old book but I like it because it's sort of written in this like fast-paced Thriller you know model it's like like a fiction book but it's about this idea of a theory of constraints like finding constraints in a system and how you can remove them to improve productivity so I found it just like a fun read and also really insightful non-technical books that I I recommend to folks uh
particularly those that live in SF which is cool gray City of Love by Gary Kamia who's a long-time SF president and it just goes into you know the like the history and the communities and the geography of San Francisco and I've just discovered so many little pockets in the city from from Reading uh reading that book so it's uh something I recommend to people who live in San Francisco what's your favorite other
podcast that you enjoy other than this podcast I'll do a non-tech one uh for this so I'm a huge fan of the show The West Wing and so there's this podcast called The West Wing weekly that goes into each episode of The West Wing and brings in actors from the show as well as folks from from government uh to talk about each episode and it's just a delight to listen to that you if you love The West
Wing wait so they go back to the old show The West Wing and talk about old episodes with politicians yeah uh that's cool yeah wow exactly so the show's over uh the podcast is over so oh okay you have like all seven seasons but I think it started in 2016 or 2015 or something so got it so cool favorite recent movie or TV show that you really enjoyed pretty mainstream TV States so I really enjoyed um we crushed and and Severance
that's two good shows I really enjoyed last year awesome favorite interview question that you like to ask people that you're interviewing a big set of open-ended questions and so one of the questions I ask in the behavioral interview at the start is walk me through the story of you from college to now or high school to now if they're a more Junior candidate and a couple interesting things here like interesting
to see where people spend most of their time talking and where they don't um and also you know how they describe the other people in on that journey and and do they use like a standard framework to describe everyone or do they you know go into each person uniquely it's just tons of follow-up questions from from that final question is who else in the industry do you most respect as a thought leader gotten a lot of inspiration from from
Gibson Biddle and his product strategy um medium thing and in particular the there's a piece on on proxy metrics and like the shape of metrics you should use which I found is like a really the way he frames it is a really elegant way to measure reach and impact at the same time of your metrics and then also a big fan of shashir from Coda specifically is ethics on on eigen questions framing problems and uh the one on marginal
Ultra contribution it's really interesting amazing both uh guests of this podcast and people I love great choices Vijay this was awesome thank you so much for joining me two final questions working folks finding online if they want to reach out learn more about what you're up to and then how can listeners be useful to you I'm on uh on Twitter and Linkedin I think my handles will be in the show notes uh not Super
Active there but I definitely check the Amazon would love to connect on either of those and then how can listeners be useful to me yeah I mean ultimately it makes panel we're building a product for product teams so two things if you haven't used McDonald's in the last four years we've changed a lot as I've described on the Pod um so you know check it out and uh happy to take any feedback to help us improve the product
awesome Vijay thank you so much thank you it's been great [Music] thank you so much for listening if you found this valuable you can subscribe to the show on Apple podcast Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can find all past episodes or learn more about the show at lennyspodcast.com see you in the next episode [Music]