there are probably I call them techno-utopians who would say feed all data to the algorithm give it an objective and it will do the right thing and I was like the reason that falls down is the algorithms don't understand a long-term effects often nor do they understand how people might
respond to it nor do they understand your intent for the product and I think it's really important for product managers to play that role that is our job when you are working on algorithmic heavy products your job is figuring out what the algorithm should be responsible
for what people are responsible for and the framework for making decisions welcome to Lenny's podcast I'm Lenny and my goal here is to help you get better at the craft of building and growing products today my guest is Adriel Frederick Adriel is a VPR product at
Reddit refocuses on incubating and scaling new products within Reddit before that he was director of product.lift where he led the marketplace teams and the pricing teams over the course of five years and before that he was in early pm at Facebook where he spent four years leading the
user acquisition team Adriel is one of these incredible product leaders who's way too under the radar because he doesn't spend all day on Twitter and instead is execute shooting and building great products one of the goals of this podcast is to highlight incredible
product leaders who you may not be aware of and Adriel is a great example in our chat we talk about the origins of growth hacking how to get better as a product leader waste increased diversity at your company what it was like to work on Facebook's growth team early on the
future of AI and a lot more it was such a joy chatting with Adriel and I am really excited to share this episode with you with that I bring you Adriel Frederick this episode is brought to you by linear let's be honest the issue tracker that you're using today isn't very helpful
why is it that it always seems to be working against you instead of working for you why does it feel like such a chore to use well linear is different it's incredibly fast beautifully designed and it comes with powerful workflows that streamline your entire product development process from issue
tracking all the way to managing product roadmaps linear is designed for the way modern software teams work what users love about linear are the powerful keyboard shortcuts efficient GitHub Integrations cycles that actually create progress and built-in project updates
that keep everyone in sync in short it just works linear is the default tool of choice among startups and it Powers a wide range of large established companies such as versel retool and cash out see for yourself by product teams described using linear as magical visit linear.app Lenny to try a linear for
free with your team and get 25 off when you upgrade that's linear dot app slash Lenny hey Ashley head of marketing and flat file how many B2B SAS companies would you estimate need to import CSV files from their customers 40 and how many of them screw that up and what happens when they do well
brainstorm our data about a third of people will consider switching to another company after just one bad experience during onboarding so if your CSV importer doesn't work right which is super common considering customer files are chock full of unexpected data and
formatting they'll leave I am zero percent surprised to hear that I've consistently seen that improving onboarding is one of the highest leverage opportunities for both sign up conversion and increasing long-term retention getting people to your aha moment more quickly and reliably is so
incredibly important totally it's incredible to see how our customers like Square Spotify and Zora are able to grow their businesses on top of flat file it's because Flawless data onboarding acts like a catalyst to get them and their customers where they need to go
faster if you'd like to learn more or get started check out flat file at flatfile.com Lenny Adriel welcome to the podcast it's good to be here Lenny thanks for having me man it's absolutely my pleasure I actually found out about you through a guy named Jules Walter who we both know
he's a PM at YouTube and I actually asked him who should I have on this podcast that is maybe a little bit under the radar that is just amazing and immediately he suggested you and so I'm really excited to be chatting man now this hype praise coming from Jules Jules
my boy I love him he's such a great guy awesome product manager and dedicated to the craft that like just like being in this presence yeah we're gonna get him on this podcast at some point he's busy with some kind of secretive project that we can't talk about but he's rescheduled
I'm gonna talk about jewels all day but he actually has the 10th most popular guest post on my newsletter still how about that wow yep he's awesome enough about Jules so to give listeners a little bit of context on yourself can you just give us like a 55 second
overview of all of the wonderful things that you've done in your career oh we'll do real fast so big highlight about me is I'm originally from Trinidad and Tobago an island in the Caribbean came to the rest for college double e got seduced by Consulting and did that
for a couple years worked in oil and gas electric power heavy Industries love that stuff but also like writing code on the weekends for fun so I thought I should move into Tech and I did worked it into it and helped develop their first iPhone app which was you know I think back in the day worked at a
startup growth team at Facebook for four years working on user acquisition which was really fun and I get kind of like strong performative experience I had quick stint and biotech and then worked on Marketplace at Lyft so Rider pricing real-time driver incentives matching
writers with drivers and then a lot of the operational tools that we use to manage our Marketplace and so that's a bit of my journey in maybe 45 seconds that was great I don't have a timer for these but that sounded right so we're going to talk about a lot of the things
that you've learned along the way at all those places you can also share what you do now awesome yeah so I'm the vice president of product management for Reddit X which sounds like we're out launching balloons into space but that's not exactly what we're doing doing we're
more of a team at Reddit that's thinking about the evolving modes of interaction with Reddit so content temporality the audience that you're talking to if you think about it Reddit is primarily about a synchronous conversations between Anonymous strangers about shared
interests sometimes other people find answers to their questions on Reddit but we're looking into and on the X team evolving that's a look at problems like helping people communicate faster and easier about shared interests perhaps xinjin would there have been conversations with maybe it's about
something other than a shared interest or maybe they have something else in common that brings them together maybe bringing video audio and other media into being a part of the products and playing with permanence and things like that whoa I felt like you were going to
go into a metaverse direction as their metaverse angles to this not really I think we'd look at that as a potential technology but our primary focus is a lot more on I see it modes of interaction and platforms that are a lot more at scale today got it is there
anything coming out in the near future we should be looking forward to I imagine you can't talk about too much of what you're actually working on more concretely I think there's a few things that we've done recently that have been fun we have a an avatar Marketplace that
we've been working on recently where creatas have been able to make art put it up for sale on Reddit and make that available for other folks to buy and use and that's been performing amazingly well the underlying technology behind it is nfts and we thought that technology
was really important to use because it gives a Korea a public way of acknowledging their rights to a piece of content and so they have some form of Ip protection especially in the marketplace where you're doing something like selling digital Arc we thought like that was incredibly important I think the
technology behind nsds has been used for some really nefarious things but I think we're still in the infancy of using these Technologies appropriately there's a lot of like terrible use and a lot of uses that are wastes but I think there's some gems in there and we're hoping to
find some of those sweet I will avoid getting pulled into a web3 rabbit hole here but that is very cool something I wasn't planning to ask about but I'm curious because I was just talking to some other guest about this topic is the idea of these kind of r d ish teams at
larger companies and companies that have been around for a while I know you're relatively new there and this kind of may be a new thing but I'm curious is there anything you've learned about how to set up teams like this and Investments like this these kind of
long-term Horizon bets are indeed teams yeah I think it's really good coming to this from being on the other side of it if you think about where I've been I've been on growth and a Marketplace which is as far as you get from seeing like we're on the new stuff kind of team and
what I've seen happen a lot is organ rejection that like this thing looks so different to the rest of the body and the rest of the organization that you get some form of rejection of the ideas entirely so I think what I've learned is a few things so the first is the rest of the
company needs to see what you're doing as being core and critical to the mission it can't seem like these guys are just playing off in a corner and something that isn't related to what we are doing every day because I think that leads to some of the like resentment because you
can imagine any team internally is fighting for resources and they look at this group as having resources that they can't get they're like oh we got to get rid of that because they're not helping us do what we are here to do so you have to be part of the core Mission otherwise
you're going to have problems culturally with that so I think that's one thing the second is it has to be everyone's success so if you end up doing something on one of these r d teams it should just be the r d team that wins everyone should feel like they win and that this
kind of relates to that first goal I was talking about I think the third is you have to set up the work that these teams are doing such that people don't believe all Innovation is gonna happen on that team it can't be that like okay we're just stuck with the operational stuff
and they're getting to have all this fun other teams are still going to innovate but maybe we're taking on something that other teams don't have capacity for that the organization needs and is part of the core Mission so I think that's been a lot of what I think about when working
on and setting up these teams is to make sure that we are part of the organization and everyone wants to hug us this being yes you are one of us not you kind of need to go off in your little corner and behave amazing that is really helpful so just to kind of recap you want it to feel
like it's core and critical to the company you want it to feel like it's everyone's success it's not just oh Adriel over there is doing great but we're like stuck with these terrible hard problems and then this idea of not all Innovation is going to be just
coming from that we can all innovate but they're just working on this one specific Innovation yeah awesome okay great another question I definitely wanted to ask you so you said you were born in Trinidad and Tobago not something that you hear very often in Tech I'm curious your background and your
journey to what you do now how did that impact the way you lead the way you build product the way you just think about your career broadly yeah it's not something that I really think about consciously but it affects me every day and it's tough not to see it in retrospect I was
the first black product manager at Facebook oh wow and so it's tough for me to not see that having some effect on what was built or how things were built or on me so it's pretty meaningful but I think one of the ways to see how it affects things is actually just to
understand a little bit about Trinidad it's kind of its own little unique animal so that is an island in the Southern Caribbean all the way at the bottom next to Venezuela it's a really diverse place so ethnically it's 35 Indian like for Beast India 35 African 25 mixed and that
last five percent is everything under the sun European and Chinese Arab Etc and then for religions it's about 60 Christian but then that's a lot of different forms of Christianity in that 60 20 Hindu seven percent Islam media diet is a mix of British and American TV
you have a really broad range of incomes but then schools are melting pots so you don't have as much of the kind of class and income segregation with schools that you get in most of the West and so when you have that kind of a Melting Pot of ethnicities religions media consumption
and kind of socioeconomic status in one place you learn a lot of them because in school you're mixing up with everyone one of the jokes we have is trinette probably has the most public holidays of any country because you have to celebrate everyone's holidays from Diwali for Hindus Idol fatir to
Christmas I have friends who were fasting for Ramadan I know a lot of the names of Hindu gods and I always loved shocking my co-workers with my knowledge of this stuff so that gives me this really different perspective that shows up at work so I'll give you an example
something I've noticed in almost everybody I worked with in Tech as we work on mobile devices people make an assumption that one phone number plus one device title one person and growing up in Trinidad I just knew that wasn't true someone who is using a prepaid phone
could have their number change all the time so that one person can have multiple phone numbers just because they were using prepaid you have phones with two SIM cards that was pretty common and our phone is and definitely was a really expensive digital device it's a computer
so it was often shared and people couldn't just have one for themselves so when I was working on user acquisition and designing registration for Facebook that knowledge was incorporated into the design of the product in ways that I think other companies not caught on to
yet and I know for a fact that a lot of that thinking that went into designing how you think about a phone number and advice and it's use among one it's how to say peering with an individual has been helpful for Facebook's growth back then and even after I left I know that's still been providing benefits so
that's a simple example of how just being in that environment and soaking up information could help product design in a way that I think would that happen if I and others like me weren't there you said you were the first black pm at Facebook I didn't realize that how many
PMS were there at that point when you joined oh man I remember we all fit into this conference room called Canada and that was probably like my second week it was probably about maybe 30 of us yeah yeah is there anything that you learned from that experience about just like how
to how to help with diversity at a company like did Facebook do this well have you seen other companies do this better is there something you could share there for folks that are trying to work on this man that's a tricky one so the two parts of that what was that like
for me I think it went quickly from being a little bit of imposter syndrome like that date when I was sitting in that room I was like dude I'm one of 30 people working on Facebook yo what am I doing I don't belong in this group this is crazy and then I recognized after talking to a
lot of the other PMs and the engineers it's like no no they want me for what I know from my perspective because they're really trying to grow this product globally and being this guy from Trinidad working on growth with the perspectives I just mentioned was
appreciated I think I was lucky enough to be on the growth team and having Zetas on that team who really valued diversity I think about some of the teams I was on and they were awesome I joke about them sometime I remember being on a team where I was a black Trinidadian product manager with a
female Israeli engineering manager a female Brazilian Tech lead then the rest of the engineering and design team was from all over the world we had Russians Chinese some folks from Slappy countries and it made designing products fun because a lot of times when
you're building a product and you want to think and get into your head of your customer you have to go out and talk because you don't necessarily get them really well man we didn't need to on that team we would just argue with each other we would think about how our friends would
use it how our cousins would use it and we are covering a broad swath of the world when we were arguing about how to design a product and I think the original leadership of the growth team I think starting with chamat But Then followed up with hobby so I knew that
and kept bringing in that diversity of again ethnicities religions cultures from all over the world so that you could actually build a product that way and it just makes you so efficient because an argument that might take two weeks to resolve because you have to go recruit a panel of users and talk to
them and figure out what's going on we kind of knock out in 15 minutes we're just throwing it back and forth with each other that I can't stress how much that's important for building products that you want people across the world to use you gotta have your teams look like
the world it just makes you so much faster it's not perfect you still have to go out and talk to folks because we still have our own kind of monocultures that form that we need to get out of but it helps a lot it's your second point about diversity and how to Foster it man
from the beginning of my career McKinsey to today at Reddit I've been in rings where everyone's asking the same questions about how those fiction and here's what I've seen work when you recognize that you get business value from it then it all of a sudden becomes something that you look out for
and you take care of that's it and there's definitely a lot more to it but I think when it goes from frankly something people feel they need to do to be PC or for cultural reasons or because they're getting social pressure to do it to something that you really recognize
concretely no I get value from this and you are willing to take the other steps to have a culture a company that utilizes it then it becomes easy because when you bring folks in from diverse backgrounds they retain and that's always the number one step to growth as
you will know from attention you have to retain them you have to retain diverse talent and so you have to have an environment that values it cares about it and uses it and rewards it because it's part of the core system of the company then once you have that
working it becomes a lot easier to recruit because people see you valuing it and bringing it in and wanting it and it's not just like lip service that you're paying that's been what I've seen to be true in all the conversations I've had on the topic that first piece is interesting that it
answers the second piece which is the point you made about how having a large diverse Global Group of employees early on especially for a company that's trying to go Global International is so powerful they just save all this time you don't have to necessarily interview
people that you don't already have yeah there's something that feels like the approach to not doing it that always feels Colonial it almost feels like we're a group of people sitting down in this like Tower in this country in this relatively like sterile environment and
don't worry we know exactly what you need in these other parts of the world it's just it just doesn't work well so it doesn't feel right to me also yeah awesome thanks for sharing all that that was really helpful there's another topic I definitely wanted to spend a little
time on which is this interesting Trend that I noticed when I was looking at your LinkedIn and your background you worked at Facebook lift Reddit and interestingly they're all very like in the news full of controversy type places people like to turn them down and show all the reasons that they're doing bad
things to the world and imagine as a PM that's just like a challenging place to be and the fact that you've been at three different places I imagine you've learned some stuff about how to operate as a product leader at a companies full of chaos and fires and bad parent things
like that so is there anything to share about what you've learned there I think the biggest thing is that as a PM you are a leader you have to provide a buffering or damping effect on the team and that goes two weeks sometimes we're doing stuff that everybody thought was
amazing this is the best thing we've ever seen and you kind of got to bring people back down to earth and go look that was cool but we got a lot more stuff we are really not there on providing the value that we want to provide to people in the Wheel World so slow your role
and recognize that there's a lot more to do and then when it's terrible and the Press is telling you that like you're the worst thing to ever happen in the world you kind of have to also go back and say guys slow down we're not anywhere near as bad as what they think
you see and know what we're doing and they're gonna misunderstand us sometimes and so pull your team up at this point in time and keep charging forward with the mission I think some controversy is necessary so I may be in a different point on that one I don't think you're going to have any
meaningful influence on the world without changing some pattern of behavior and if you're choosing a pattern of behavior there's somebody who's invested in that pattern of behavior and that's going to create some conflict the most fun news stories to read involve conflict so that's always going
to make for a great story and put you in the press for Facebook it was traditional media and other social networks were one side of the fight and then Facebook was the other side of the fight and then it became other tech companies now that always makes for a
great story with Lyft it was taxis and unions and so you have to recognize that you're always going to have some bit of a challenge now the really hard part about dealing with this is understanding what criticism is valid and how much of it is just because
a source of power is being changed so I'll give you an example let's see what left Rich Medallion owners in New York I had no sympathy for them when they were complaining about trying to bandlift because when I was in New York City putting my hand out to get a cab they would drive
right by my black ass and so I'm sorry I'm like I do not feel that much empathy for it but I think there were really legitimate complaints about the structure of driver p that were coming up and that were behind I think some of the complaints and some of the big stories in the press and some
of the big kind of legal action that was taken paying for pickup time when a driver is on their way to pick you up or when they drive somebody far out of town and they have a deadhead to come back into a place where they can work that's real that's a real problem
that I think we got called out for that we weren't paying enough attention to and it got us off our ass to go fix it I don't think we've when I say we but I'm not there I don't think the problem has been fully solved but I think as a PM listening to this you kind of have to
find the truth behind it and try to find a way to work on that and not get too lost in responding to the specific criticism and so to walk this line between kind of going yeah some of this controversy is just part of the game versus like nah this is really valid yeah dude to figure out where that is
you got to do what is so cliche but like you have stay close to your users and so to give you an example of how I did that when a lot of the complaints were happening about driving on Lyft I drove I would just pick up the car and I would get up and I go drive and I'm like let
me go feel this for myself let me go see what these guys are talking about Mark give you a story about Rick I still remember this drive I did with Rick and Berkeley so I'm at home I just get in the car I turn on the app it's time to go driving I get up King 15 minutes away and I'm
thinking do this I go do this right now sky might cancel on me I'm not really getting paid for this but maybe the ride is worth it so I drive one over I'm dodging traffic pedestrians drunk college kids stop signs I make my way over to Rick he's coming up Japanese and
he's about 80 years old jumps in the car and then I push the button to figure out the destination and it says the ETA to the destination is two minutes so I was like hey Rick you get this right what's going on it's like hey I had a little bit too much to drink I'm
worried about breaking my hip so that's why I call her right and so I went from wanting to curse Rick out for making me drive 15 minutes to come pick him up to feeling like all right no no there's real value I'm providing you and driving in just two minutes but
I recognized that wasn't embedded in the structure p Rick would have been happy to pay for my 15 minutes to come pick him up but we weren't one giving drivers compensation for that nor were we finding a way to pass that through into pricing for Rick it's a much more difficult problem than it seems from
that simple example but it clued me into why drivers were complaining so then I went got it I understand what we need to do so when there were all the pr was going on about EB size and prop 22. I was out driving and I was out sitting with the team trying to figure out how we're going to
design a product that helps pay a driver for this still keeps prices reasonable for users doesn't create bad incentives where you end up with writers not getting picked up when they really need a ride because I didn't want Rick to break his hip he still needs a price
that makes him feel like it's okay for him to take that right and finding a way to balance this out is actually more complex than you might think and that's what I stayed focused on whether prop 22 passed or not I was ready for either side with a solution that was going to
work for Riders and drivers that was the job and so I think for PMS it's it was so easy to get sucked into like the press and it's like yo plan the work work the plan go back to your job that's what you're supposed to do solve for customers in the middle of this and then you figure out how to
communicate it well what I love about that strategy is it also helps you see that it's not everybody that is worried about something I think if Airbnb like all hosts are pissed off about this one feature there's going to be a revolt and then you see your point you talk to some
like nobody even knows about it nobody cares everyone's fine and so this there's so many benefits to what you're talking about doing which is yeah talking to customers not just paying attention to the loud voices absolutely you know I also have empathy for
reporters too the story that with the headline some Airbnb hosts are annoyed by the Jews right I mean come on but it's not but it's not a great headline I reckon I said they have a job to do and sometimes they hold people accountable and sometimes they're getting people to read
a story that maybe has a bit of hyperbole in it and so they have to do their job and I have to do mine too yep this episode is brought to you by EPO EPO is a Next Generation a b testing platform built by Airbnb alums for modern growth teams companies like
netlify contentful and Cameo rely on Apple to power their experiments wherever you work running experiments is increasingly essential but there are no commercial tools that integrate with a modern grow team stack this leads to wasted time building internal tools we're trying to run your experiments
through a clunky marketing tool when I was at Airbnb one of the things that I loved about our experimentation platform was being able to easily slice results by device by country and by user stage EPO does all that and more delivering results quickly avoiding annoying
prolonged analytic cycles and helping you easily get to the root cause of any issue you discover EPO lets you go beyond basic click-through metrics and instead use your North Star metrics like activation retention subscriptions and payments nfo supports tests on the front
end the back end email marketing and even machine learning clients check out EPO at getfo.com get eppo.com and 10x your experiment velocity you share this really heartfelt story about Rick what's your most stressful memory of working at Lyft I think the most stressful time was when I had to
unwind up a product I did and actually make a better version of it it was really a pricing algorithm change of something behind the scenes nobody would really see but this was a fairly big initiative that we worked on we had experts in Revenue management to work
like phds and people who wrote The Textbook on the subject helping advise us on this we built this model you launch it and you're expecting like this big change and it goes put it just does a little bit and then we work at it and we work at it and work on it and eventually we get it
to be good and it works really well in three cities we start rolling it out to more cities and it's a pain in the butt to rule it out to more cities because it's super complex and eventually we get it rolled out to maybe a hundreds and then someone says all right cool I want to change
prices and oh we struggle for months to implement price changes and man the sentiment like around this product was just rough for a while and I remember being on our walk after like a particularly bad week of this and trying to figure out what I was gonna do about this thing like do we
stay at the chorus after a while the answer is kind of simple even though it was emotionally difficult and the answer was like yo we gotta rebuild it there was no answer where we couldn't have a product like this we needed some ability to be able to influence prices so that
we could actually run an effective Marketplace the current solution didn't work it wasn't as operationally flexible as we needed it to be because we didn't consider that requirement when we were building it and we got caught up in the kind of algorithmic complexity and sweet
sauce of it and so I recognized that we just needed to own up to it tell everyone we didn't get it right and we needed to come at it in a different approach it was actually more flexible operationally and we did it I think the big learning at least in
that business was you have to think about operational requirements and operational control as a first order requirements and I think when a lot of us were building products at a lot of the other consumer internet companies you didn't have to think about operational control
you gave the algorithms an objective you feed them some data you let it run You observe it the nation's doing nothing crazy and you tweak it but you didn't need to have day-to-day operational and strategic control over the product and we just needed to snap our brains into
being able to put people in the loop with the algorithm for folks that haven't worked at a company with this kind of on the ground Ops Team can you just unpack what that is like operational control what does that actually mean in in Practical terms okay so I'll give you an example so lift is
in back in the days lips is in 300 cities probably roughly across the US and in every single one of those cities you don't have exactly the same pricing it's a little bit different and so sometimes you might need to make a gene seasonally because traffic gets worse or because fuel prices were
different or because it's a new tax or because your competitor did something that you need to respond to and your algorithm cannot see this it has zero visibility into this and so you need a person in the loop to not only give that visibility but also to make a decision
about how you respond because I think also in let's say you're in Chicago and there is a snowstorm and you need to change the way let's say you need to update pricing so that it handles the increases in driver pay that you need to create and get people out during a
snowstorm you don't know exactly how you want to respond every snowstorm is different and a person has to make that judgment call and provide the right information to the product to be able to utilize it now algorithms were handling a lot of that and they could generally respond but to
be a lot more precisely needed a person to help handle that submit that call got it cool thanks for sharing that so you're making this point about when you're at a company that has a big operations component and obviously the core Central product team you're sharing
some learnings about what you've learned to work in that environment so yeah I just wanted to come back to that for sure for sure so the main thing you said is just treat Ops is a first order component when you're designing the software is that the big learning I think it's not just treating Absa Z kind
of first order requirement the bigger picture for me was like when I look across my career is the algorithms need people to help make judgment calls and so I saw it really I got a heavy lesson in it I left but when I look back I recognize it was there in Facebook too
it just wasn't in my domain there is always a judgment call that has to be made between how often are they going to be ads versus how often are we going to show organic stories from your friends and family how often are we going to show content that you might be interested in
that's not quite in that group Topton might be want to show you things that help you find your friends or help other people find their friends and that is a judgment call that varies for different markets in different situations and there may be algorithms behind the scene
that are making that call for every single person in real time but there still have to be people applying some strategic judgment to that and I wasn't in the position of needing to do that at Facebook but once I saw how much I needed to do it at Lyft and I kind of looked back at history I saw
that it was there too but I think there are too many people who don't see this and believe that there's an algorithmic solution to everything I think as a product manager and especially product managers working on systems that are heavy on machine learning or operations research and
optimization to think about where you want a person to make a decision and where you want the machine to be Off to the Races and to think about that as a product design problem because there actually is a human computer interface so you have to think
about that you need information about what's going on let's see it left what's going on with my market how long does it take for somebody to get picked up how expensive am I versus the competition what are my goals in this market and like how am I performing
today with that give somebody information but also give them the tools to execute the right decisions without creating trouble and that's like a design problem that's a first order product design problem like anything else that you have to think about and I'm not perfect to it but I would
guarantee that there are people thinking about those same kinds of problems that other companies that reminds me of just listening to Zuck on Joe Rogan and he made this point that when you look at a post you can like add a little Emoji reaction and you can have a little angry Emoji reaction
and he made the call that we're not gonna use the angry Emoji reaction in our algorithm in any way we're just gonna ignore that because naturally you'd be like okay people are angry that's interesting let's show that because it's interesting to people but he specifically wants to avoid anger and
facilitating anger probably because a lot of the feedback that they've gotten exactly and I think they're probably I call them techno utopians who would say feed all data to the algorithm give it an objective and it will do the right thing and I was like yeah
the reason that falls down is the algorithms don't understand a long-term effects often nor do they understand how people might respond to it nor do they understand your intent for the product and I think it's really important for product managers to play that role like that is
our job when you are working on algorithmic heavy products your job is figuring out what the algorithm should be responsible for what people are responsible for and the framework for making decisions is there an example that comes to mind where you did that or didn't do that well or someone on your
team should have just something to make it a little more concrete even let's assume that you are a person working on price and you say like great I have an objective that is I would like to win market share in a region okay and you less that's an algorithm to see
I need you to optimize prices such that you maximize market share but what would the algorithm do drop your prices to the floor all the way to the floor and then you don't make any money okay great so then you say okay what's the next step on that let's give it a
constraint let's set some Target that we might want to have for how little profit we might be willing to take okay go do it now what if the guy on the other side is doing the exact same thing both of you will hit you constraints and then the game will stop okay great so now it then flips do oh we
have to choose where we want to win and so I think one of the things we did that I'm particularly proud of is building products that help people see and understand that game a little bit more and decide where they want to I think the first two pieces of that are not
shockers but like that conclusion at the end where you get to oh wait I need to create a tool that gives people information to them decide how to play this game is actually what's critical interesting so kind of what I'm hearing is a lot of the work is giving humans
more information versus giving machine learning algorithms more information and there's a lot more leverage potentially there giving humans more more ways to tweak and dial let me respond that a little bit more it's more about giving people the information that they can use
for decisions that they alone are good at and giving machines the power to amplify a person's intent so one of the ways I like to think about it is all software in any form including ml is just a tool like a screwdriver and you could try to put a flat head into a
Phillips and maybe it'll work a little bit but it's better to use a Phillips screwdriver and we're tool designers generally and especially in product development function you figure out how much do I put into the tool and how much I leave it up to the person and I give
the person the ability to choose what they want to do I give them a screwdriver a flat head a Phillips a Torx and you let them decide how they want to use the tool for the application at hand and so going from that analogy to concretely with ML you see look machine learning is going to be amazing at
optimizing for a given objective but it's not going to understand the constraints or strategic choices I need to make the constraints and strategic choices that we need in the external world are always going to have to be decided by a person you make that incredibly easy for
people to do an intuitive for them to do and then you go that algorithm can then amplify their respect by making decisions on hundreds of thousands potentially millions of individual decisions to take that person's intent and amplify it given all the information that they can learn in
that single context so I think about this designing an interface and making an extension of yourself rather than a black box on its own that you just need more information to is that helpful yeah it makes me think about a neural link and what elon's trying to do I don't
know if this is how he thinks about it but the way but why guy described it as elon's worried that AI will take over at some point and so he wants to build a tool that connects straight to our brain that can access the power of AI to kind of have a chance against just a real
computer you know in the future even then you've got to make sure the person is still in control I hear that thought and I go okay you build the interface but then losing control right is the person still uncontrollable did they become a slave to the machine
and you just made a better interface to make them a safe oh I am not yet as worried about these versions of them taking over thus far and maybe I haven't fathomed what they can do they still seem like tools that need our guidance to be useful even the most amazing we've we've been seeing the
image generation and I've seen some of the Cutting Edge like text Generation stuff they can fool you into believing that they're like near human capability but there is a lack of decision making and judgment that I see coming out of them I see them as being again extensions and
useful like text generation algorithms a lot of them can't write a paper for you and that's the thing people are scared of because it's it still requires your judgment to decide now when you decide what the ceiling topics are and something you've read let's say you're
doing a book report you've decided what the topics are you can help you write the paper faster for sure but it can't write the paper for you it can't choose the topics that like your background in history and interest find useful or compelling to tease out this isn't where I was
expecting your conversation to go but I'll add another thought here because it's interesting the way I think about it is there's nothing like magical about our brain and so if that's true why isn't there a world where we could just completely simulate it Sam Harris talks
about this a lot that it feels like once you get close then it could just accelerate so quickly Beyond human potential like it'll start from like 20 as good as a human to like 40 50 60 and then it goes to like 10 million times better it can move so fast Beyond us very quickly so I think that's where a
lot of not that I'm afraid of this but I feel like that's where a lot of fear comes from you could just like Dolly come in absolutely co-pilot just like holy yeah our brains are good with linear thinking not exponential so I've heard that I already met that like yes
this is increasing exponentially and you can't fathom it I'm like yes that is definitely potentially true completely see that possibility and recognize that I have that like cognitive defect in being able to understand it and even if it's a remote possibility we should be
paying attention to it so I'm all for paying attention to it given the message the high cost of the low probability outcome is still a high cost and so it's still worth paying attention to yeah okay good tangent I wanted to chat about your learnings at Facebook we've been
chatting about all these other places and especially about growth just stuff you've learned about growth and growth hacking and I was thinking about this interesting world that Facebook is in slash meta where on the one hand when they started I'm talking about growth hacking like Facebook did a lot of
growth hacks emailed all of Harvard he had all these interesting dating thing happening and got a lot of controversy it was all these interesting tactics to start Facebook but now people use Facebook to growth hack and grow like Zynga famously a few other places so all
that to say I'm curious whatever you learned about growth slash growth hacking premiere time at Facebook and other places I think growth hacking has traditionally assigned like finding those small changes you can make to a product that give you outside's impact
that is absolutely valuable well I've seen people get lost is they assume that if you do that alone it will work you can grow your way into something successful if you just find those few hacks and Patch them together and there's something about that that I find disrespectful to the people using the
product it's like you assume that they have no intelligence and they will catch on to what you're doing eventually you know they all saying fool me once fool me twice you know it kind of applies so if you don't have a product that's providing real fundamental value to people
you can be a one-hit wonder I have a flash in the panel grossack your way into something that might last for a few months but like people will catch on to it and they'll disappear so I think that stuff is helpful especially early on to get your initial traction
but you gotta have some people like and want to continue using and when I think back over the products we did that really move the needle they were all things that just focus on the marginal user and figure out how to make the product easier it's easy to get seduced into thinking that there is a
fast secret way to do it and I'm like no the vast majority of it was just hard work and finding ways to solve the real problems and what are those real problems I mean pretty damn simple but we just grinded on them for a long time and just stayed on it one make it easy
to find the product number two make it easy to get into the product three stupid easy to find your friends and then once you did that you were off the races and like those were the things we were doing over and over again I think another big piece of it is
reminding people that there's something interesting here and building the habit of coming back to the product it was also part of it but like we just grind it on those few things over and over again and some of the really big wins were attacks they were just paying attention to
little I'll give you an example I remember sitting one day thinking about how to help people find their first few friends and we would do this thing where we'd have recommendations if you get one or two friends you'll be off to the races and we could find you more people that
were in that same friend group so I thought about the way the people you may know algorithm worked I get one or two friends they would find your mutual friends and then would help find you more of those kinds of folks and I was like you know what that does is it
spirals you down one friend group but it doesn't get you all your other friends I remember just like looking at somebody using the product and recognizing that we were only taking them down this one path so I was like Matt how do I see all your friends and so we had this idea
that we came up with that would do it I'm not gonna let that one out and um it was like Game Changer it's like absolute Game Changer especially for new users helping them find those first few friends in a few different friend groups which then meant we could get you down
one group and another and just continue building out that rap just by using recommendations because we had a great tool for seating it and that was not easy that was not a hack that was hard work I also remember like one of my favorites is something Tom Allison Tom Allison I
think no is responsible for Facebook app and when he was working on like the engineering manager for one of those teams but there was a change we wanted to do to one of these algorithms and it was a big it wasn't a hack and it was gonna take a few months to pull off
and Tom just hit it in a corner just didn't let everybody know that we're really going to change the way this product worked he had a really smart guy working on it as teaching and like they just hit off in a corner rebuilt the product in the wheat needed to be built to make it easier for us to operate it
and scale it and then put it out there and of course it crushed it and they were incredibly modest about it but it was not a hack and it came from them looking at this deep problem of finding that thing that mattered and then saying we need to make a fundamental change to
make it easier to recommend friends to folks and just grinding on it and so one of the things I recommend for people when they're thinking about growth for their product is figure out what the core actions are and then grind on them think about removing them removing friction and some
of them but yes sleep staying at it and as you grind on it you'll do little hacks you've got to figure out how to put you know write text in the button and get it above the full create the right copy like all the things that we traditionally associate with growth
marketing you've got to do those things but to me that's table Stakes I'm just doing good product communication with your user but then like you got to think about this person who can't yet figure out your product and is trying to seek this action and making it stupid easy
for that I got a million more examples of that one but that's the game it's not just finding some trick to spot my site I love that the way I think about this that I've heard well described is just there's no silver bullets just many lead bullets yes and a few massive cannonballs every now and then
every now and then there's some cannonballs what's an example of a cannonball as you think about that sign up with full numbers which is now like par for the course that was a cannonball getting sms's delivered to people all over the world doesn't sound glamorous
really hard to do without Cannonball good friend recommendations another big one there's more I'm not going to go into all of them what I mean by Cannonball here is that there was Sometimes some really big fundamental changes you needed to make to the product to make these things work got it
so you think about that in terms of investment not necessarily the investment impact plus massive investment cool I have so many questions along these lines okay I'm gonna pick a couple one is Facebook is famous for this kind of activation Milestone of getting 10 friends or seven friends
whatever it was like there's some number of friends you gotta get and the good things will happen were you involved in that do you have any insight into like how that came to be Is that real that decision came before me I saw it I understood the dates and I worked on
this problem what I thought was brilliant about that was not the there's nothing metric it was the designing it to be understood and communicates it what I think is fabulous about it is that you're talking about it now because it's memorable and it got people to take
the right actions to start chasing the gold it's literally nothing magic about the number or the Deeds but basically it was a way of seeing like get people as many friends as possible as fast as possible and if you said that generically to someone they'd be like yeah I kind of get it but yeah
I'll go do that when you create a discrete number and a discreet time and there is a concrete goal to cheese and there's a number and a graph that everybody can look at and see we are going to go make that thing go up the organizational effect of that is galvanizing so what I thought was
brilliant about it is and as I've heard the stories you know this is all secondhand there was a lot of debate about what the numbers should be and what time frame should be and at some point Zuck just said 10 14 days go and it just just got people past the
academic debate of like all right got it as many friends as possible as fast as possible let's go I love that that's exactly how I've always thought about it that it's not the number exactly it's just a rallying cry that everyone can just get around and just go it doesn't
need to be this perfect number that has like incredibly correlated link to retention or anything that it's just like this is good enough it's probably directionally I just try to do this let's just go there are downsides of it some of them are really funny I remember
looking at a graph of like retention versus number of friends and like what actually dropped with 11 or 12 versus 10. because somewhere in code somebody had done something with 10 Friends of the limit to help improve retention and it just shut off at 11 or 12 and it came back up
but I was like you know what that's fine that's completely fine because the if we didn't get that like organizational momentum that graph would have just been lower so I could take the Kink where it drops it's fine you also mentioned this term marginal user and I thought it'd be
helpful just to unpack what you mean by that for me it's a person who is just on the cusp of taking the action you want to take talking to the concrete example when working on registration I would try to find a country where we had a lot of growth but for some reason our
conversion rates were terrible so we had a lot of traffic but conversion rates were terrible and I was like okay that's the marginal user this is the person who's just on the cusp of coming in wants to come in as you can see by the traffic but we can't get them in so why and when you go
to the extreme and you find that person who's the worst right and I most likely it was a person on a feature phone on edge trying to access Facebook in a country that was far from one of our data centers and then you go like all right what's wrong with this person's
experience let's go check it out you're like oh you see everything that's wrong with the product so then it gives you your list okay the language is probably wrong we didn't get that are we detecting the country properly so that we can actually get their phone number
formatted properly probably not oh man it's far from the data center so that connection's slow and they're on edge oh that's terrible and you just see and package up all those it gives you everything that's wrong and that can just start figuring out what to do with them something I caution people against
though is don't use the data alone to figure out who the marginal user is it'll give you a clue where they are and what might be wrong would you be some hints it's not going to give you the answer you have to go watch them to find the answer because I think in a lot of
these like data-driven places somebody will say great just create a funnel figure out all this drop-offs of the steps of the funnel look at it yourself and then figure out what might be wrong and go fix those things but what I think happens is often there's a problem
that's orthogonal to that funnel that you can't see from looking at the data and you have to go look at the person and talk to them I remember one example we had is like was watching someone sign up for the very first time for Facebook in India and they're about to put their
name in it and they're asking like so what what name are you going to put in they're like okay my full legal name all right cool is anybody in the real world call you that no I was like oh dude we're screwed if you send out friend requests it's not going to get accepted
because nobody knows who this person is and in the reverse it they find you they don't know who this is and so I was like yo you're you're gonna look at some problem deeper in the funnel yo what's going on with my accept rate and then you're going to go terrified that little mini funnel and then
recognize that you had a problem that happened ways back and so when you're thinking about that marshmallow and you say you gotta go and look at them talk to them watch them use it try to get into the issues yourself as much as you can and then make the call from there on
what you do but that detail isn't going to tell you this is going to give you the answer I'll just tell you how bad it is well I love that connects back to the same advice you gave in all these other contexts of just talk to people don't rely on just this aggregate data now
don't get me wrong like I built the experimentation platform at Lyft I'm a guy who loves data and loves using it and looking at experiments I think it's just too easy to try to sit on your laptop pull up a funnel or pull up some charts or look at an expect results and
think that's going to give you the clue to what to build it's a compliment it's not the only thing and I watch people fall into that trap of assuming especially when you're working at companies with lots and lots of data you fall into the Trap of thinking that you're swimming in answers because you
have all this data and just need to like tease it up yeah just go out and talk you'll find it faster I really like this advice of when you're trying to optimize things focus on your marginal user and there's two parts to it that you talked about there's the next most likely
person to sign up and then there's like the worst case and going to like them to see all the things that are wrong and have beam your like North Star make this person successful make so many more people successful is that how you think about it yeah I do so you know marginal
use I think is a fun word to think about because you think of what I mean you think about the person who's right on the cusp but I like to go to the worst it shows me everything that's wrong but the marginal user thinking helps you prioritize one thing to do next so like
that person that that example marginal use I was talking about they're not future full with Edge dude there's a lot wrong that's just going to be tough but I might look at that experience and go all right let's see somebody was like perfectly equipped and best phone and a
great internet connection in that country what would still be wrong I was like oh language is still wrong and the latency to their phone is actually still pretty high to our data centers which is why it's taking a long time to sign up I could still fix that so that how you can
see the worst case tell you everything but then decide what is marginal by removing a few of the barriers and you know what difficult for you to attack and then see which ones are closer to being resolved awesome I wanted to ask one more question about experiments at
Facebook back in the day so we talked about there's all these lead bullets or some cannonballs maybe a silver bullet Somewhere In your experience what percentage of experiments end up being impactful and successful okay that's a difficult and different question so I'd
say probably 60 successful 40 like you should turn off but within that 60 I think there's a hidden cost to the experiment which is that like you're futzing around with something small you could have used your time on something bigger and more meaningful but you're fussing around with a bunch of these
small ones some of the small things were incredibly meaningful and you needed to do them so I think this is actually it's almost like the same problem about I don't know which of my marketing is best you have to try a bunch of stuff and then figure out what was terrible you
don't know before you do it before you do the experiment what the impact is but sometimes what I've seen is let me take a bunch of them as a program and let's say you have over the course of three months you're gonna experiment with 10 things you might have been able to push
on two really big ones and what I've seen is there's a laziness and this is not this is broadly this is not just a Facebook this broader there's a laziness that can creep in where you're just finding a lot of little things because they're easier to come up
with they're easier to design and think about it's easier to build it's easier to talk to your boss and see remove the number by 0.02 percent and like if you feel good about doing those few small things and so it creates this incremental thinking we're just trying to do a bunch of small
things that just don't mean need to lay out up to something big I think what's healthy is having a good portfolio because basically you say like look I'm gonna have using our analogy from for the from before I'm gonna have some cannonballs I'm gonna work on a couple cannonballs and I'm gonna have a bunch
of red bullets I'm not maybe it's 80 of your energies on those big cannonballs 20 on the light bullets and what it like having a constraint like that force you to choose the few experiments that are actually probably the really good ones and it's not just a whole bunch of crap
that you're trying out and is that actually how you divide up those bets broadly is that like a rule of thumb you have or is that just numbers you're putting out there those are just numbers I'm putting out there it's always going to be I've got call beasts on where you
are I think depending on the stage your product is at it should be a different set a different bias very early on when you're building a product you kind of know what the big things are you've talked to enough people you have enough let's just go build it you should not be
playing around with experiments it might be a hundred percent cannonballs just go knock the big pieces out don't worry it'll work also the cost experimentation is time so if you are experimenting on every little thing and waiting for the data to come in and then also screwing
up some other part of the product because your experiment's on 50 50. it's just not worth just starting the big things out but as you get more mature the balance needs to switch in the portfolio probably you know probably aren't that many big cannonballs anymore probably
just one and this will be a lot of the refinements that you need to work on and by then you have this scale that the time to experiment isn't as high and the cost of experimenting is lower so it's fine it's good to do with that way okay one last question before we get to our
very exciting lightning round so you've moved from Icy a while back at this point to now vpa of product at one of the most trafficked sites on the internet and I'm curious what skills have you grown or had to grow most as you've gotten more senior in your career
organization design and empathy wow I love that oh dude for a long time and I think this is I and many others I have this idea that the people who were the smartest are the ones who arrives people are the most technically competent of the ones arise people who are the best
individual contributors of the once arise and somewhere along the way I had that idea disabused of me and I recognized like the job's different it's more about building a great team creating the writing sentence for the team on blocking them guiding them and helping them work efficiently
those mattered way more than anything else and I guess one of the ways I slowly recognize it is like as I started going up in my career I recognized that if I wanted to have more impact I couldn't do everything myself there was just more that needed to be done um in today's world you can't do
anything meaningful by yourself you need a lot of people to do stuff with you there's nothing meaningful that gets done by any single person even though people like to make you think that in their hustle porn that they post online so it may be just a step back and think
about what helped me be productive environments when I was productive and how I could do that for others because then that would just actually help me and so there were simple things like clear goals helping people feel safe and understand that like you've got their
back making it easy to do their jobs but my job is to make sure the processes for you doing your work and the people who you have to interact with are just better reason and everything just runs useful and so that was like lesson one it was just like designing a good
organization culture skills people processes Etc all necessary that's one piece the second is like empathy first step of that was just like you have to have that as a PM for your user but I think it's different to having it for up here and another function or somebody
else on one of your teams and the hardest part of it is that they say getting in somebody else's shoes the hardest part is taking my own shoes off basically going yo okay I came into this there's something I wanted I wanted get rid of that now just talk to this person and try to understand what's
going on with them what they care about for like their life goals and motivations what they're scared of what they're excited by how you might be able to help them once I was able to get out my shoes clear my mind try to get into their head then I could be like all
right cool let's find a nice happy middle ground in the middle here about that something that works for both of us together and sometimes for me it was yo what I care about I'm good I'm gonna let you do your thing I've gotten into your shoes I need to leave you alone like
you're good other times you know I'm ready to push but I think once I have the empathy I'm then able to think about what we as an organization broadly want to achieve and try to put the two shoes on at the same time and find something that works for both of us so what's that
common objective and I think that's how I try to approach almost every conversation as especially being a guy who looks different talks different comes from somewhere else first thought they might have this and unconsciously it might be like this guy in one of us
but then once then you get clear to them that we have the same objectives we're about the same thing and I want to know what's going on with you so that I can help you achieve what you want to achieve Do It Go problems go away okay I know I have to let you go and you have
to get back to real work so we've reached our very exciting lightning round the final part of our little chat and basically I'm just gonna ask you five quick questions whatever comes to mind share it and we'll go through a pretty all right sound good okay what are two three books that you recommend
most to people the prize and probably now the new map by Daniel Jurgen they are books about the history of oil and the geopolitics of oil it is a fascinating way to understand the world it's like the best books I've seen to understand geopolitics and how they work
and why they work it does it through the lens of oil which explains way more than you might think and so this comes from the early part of my career working in energy I will link that into show notes I've not heard that one before what's a favorite podcast of yours other than this one oh
of course it's the easy one away uh revisionist history with Malcolm Gladwell just gives you a different look into things and I'm also a huge car nerd like deep into modifying and tweaking and tuning cars so there's this esoteric one called HP Academy that I'm into but
mostly listeners will not speak to that wow very out there and awesome and I think there's a new a season of revisionist History coming out soon yep okay favorite recent movie or TV show last night I discovered Mo on Netflix Moe and it's short for Muhammad it is semi-autobiographical about a
Palestinian refugee living in Houston his journey to seek Asylum and live and work and date in this Multicultural environment he speaks Arabic Spanish and English fluently it's funny as hell but also dramatic it is fabulous amazing okay wow these are all very unique I
love it in a different direction what's a favorite interview question that you like to ask you know these days of work I have to go through the standard interview questions but when I got to play and sometimes when I feel like playing a little bit more I'll see something like teach me something you
don't think I know it's a really good test of what you've heard me say a lot empathy I heard shamat use it once and I kept trying it to see what it was good for and it helps you understand how good somebody is at reading you how much knowledge they have and their ability to
communicate and share knowledge so it was like it actually could test a lot of things at once and a lot of times you learn something it's awesome win-win okay final question who else in the industry would you say that you most respect as a thought leader well look on
the discipline it's shring us the Splinter prop management definitely us I think just in terms of Technology development it's the team behind radiant nuclear what is that while taking a break between jobs I'm studying climate change and energy because of my
background and I just basically became convinced that nuclear is a bigger answer than we're giving a credit for a lot of the barriers are political not technical but the solution they're working on I think is a technical solution to have some of the political problems we have around nuclear which
seems really interesting and I am like really hoping that they pull off what they're trying to do wow I left out there all these recommendations are these are great Adriel I am so appreciative of you making time for this I'm also really appreciative Jules for connecting us this was amazing you are
awesome just two last questions where can folks find you online if they want to reach out learn more and then how can listeners be useful to you awesome before I jump into that thank you for having me on here it's just good to reflect about life and work for a little
bit and hopefully share some insightful stuff with the folks who listen to your podcast so thanks for having me you can find me on LinkedIn Adriel Frederick there might be one other I'm pretty sure I'm the only one and then how can the snow is be useful to me number one keep
listening to this podcast because if everybody keeps listening to the insights that you're teasing out a lot of things will work well and not necessarily me another thing they could do to be useful to me is find somebody that's just different to you and talk to
them for five minutes that's it I think that will come back to me eventually love these and really flattered really appreciate it thank you for being here thanks for having me let me take it easy thank you so much for listening if you found this valuable you can subscribe to
the show on Apple podcast Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can find all past episodes or learn more about the show at lennyspodcast.com see you in the next episode