"you fake it it affect your test you do smoke tests Wizard of Oz tests we used a lot of those in the tapped inbox by the way one of the first early versions was actually we showed the tabbed inbox work..."

"my body as I get older the way it works is you book an appointment you come in you put on some very cool silky pajamas that they give you that you get to keep afterwards you go into an MRI machine for..."

"thousands of fast-growing companies like Gusto com quora and modern treasury trust Fanta to help build scale manage and demonstrate their security and compliance programs and get ready for audits in w..."

"mushrooms people were spending hours it really freaked out Google and the solution the obvious solution was to launch the social network of Google called Google Plus and we all believe in this thing i..."

"social mobile apps like WhatsApp like Snapchat Etc and for me this story kind of was the epitome of what I call today opinion based development we come up with an idea we believe in it we all the indi..."

"after all yeah okay so that's an example of something that didn't work because it was opinion based software I think the phrase used and then there's a different experience with uh with tabs I think w..."

"Google Plus was the tabbed inbox so it started with the it was kind of the reverse of Google Plus it started as a very small idea that no one believed in and we started looking what's behind the city ..."

"at Amazon Airbnb anyone you you will check at least in their best periods they found a way to balance human judgment with evidence they then try to obliterate human judgment or opinion just to superch..."

"want to do so going back to the story briefly before we get into the Frameworks and lessons of the book in the first example of Google Plus basically came top down hey we need to build a social networ..."

"evidence-based approach I I think the founders are very important especially in the startup and scale UPS phase they come up with many of the most important ideas and it's super important that they ha..."

"people that are hearing this might feel like okay I like this idea of pushing back and encouraging the founders to make it more evidence-guided in the case of say Google Plus was was it even possible ..."

"have done better now I'm doing a YouTube service I was not in all the discussions I know probably in Google Plus the reversed serious discussions happening along these lines but it just as a general T..."

"that's why this is the principle I I base the book on you have this concept of being evidence guided people listening may feel like Hey we're having this guy we're on experiments we make decisions usi..."

"much that's usually a sign of uh well it's usually something that you can fix by adopting a more evidence-guided system okay so let's dive into your approach to becoming more evidence guided in the bo..."

"because not two companies are exactly the same and it's very tempting when you write a book not to give any process but that's the part that people actually want the most so it's included as well but ..."

"oh that's a whole different discussion Christina you're the real expert on over there so I doubt I can add more to that but it's through okay I was all part of it but let's start with goals what what'..."

"on each and the metric we usually use to measure value delivered is called the North Star metric I know you wrote an article a very good article about it thank you and in it you listed dozens and doze..."

"they overlap so you might find in the middle some metrics that are super super important because moving them actually moves the needle on everything else can you clarify again the difference between w..."

"objective of the company and these are the two top most key results if you like the top most things so when you go go and work with another team and you say hey why don't you work on my project they m..."

"foreign a place where you can find all your potential customers and get your message in front of them in a cost efficient way if you're a B2B business that place exists and it's called LinkedIn Linked..."

"and what usually happens inside the organization is some sort of Battle of opinions or of some sort of politics sometimes or highest paid personal opinion people you had Ronnie could have you invented..."

"popular is the the concept of product Market fit yeah we created ice he created a bunch of things that we use in product and we don't even know I didn't know he came up with ice Okay cool so the origi..."

"and you can see the number is going from 0 to 10 where 0 is very low confidence we don't know basically anything we're just guessing in the dark and 10 is full confidence you know for sure this thing ..."

"maximum of 0.1 out of 10. according to the tool if you if you follow it then we move into slightly harder tests one is reviewing it with your colleagues your managers your stakeholders the idea they d..."

"you do data could be also what I call Market data that comes from surveys from assessing a lot of of your data by doing a deep competitive analysis and there are other methods where you create a large..."

"confidence like the test you can do an A B experiment early adopter program whatever it is and then launch it some ideas you don't need to test sometimes expert opinion is enough if you're just changi..."

"so there's always like a trade-off between speed of delivery and speed of discovery and and that actually leads to the next layer of how do we combine the two because people tend to think it's an idea..."

"awesome so maybe just to close off that Loop C someone listening is at a bigger company say Netflix versus a series a series beer startup is there something you'd recommend about them approaching this..."

"time so there you need the order of evaluating ideas in a very systematic way in a company like Netflix by the way I don't know if they need this specific method they're very yeah maybe that's a bad e..."

"so some of the people that actually respond to the strongest are actually surprisingly in these companies what I love about your Frameworks and kind of all these things we're talking about is these ar..."

"move closer to being less opinion oriented and more evidence-based which of these Frameworks or models would you recommend first I recommend that they discuss internally what is the biggest problem th..."

"learn and build at the same time as we said and one of the patterns I see is that organizations don't know that they can actually learn at a much lower cost they believe they need to build this elabor..."

"together things that much smarter people invented so uh in assessment you have very easy things things that don't require a lot of work you check if it aligns with the goals this idea that you have in..."

"but until now we didn't build anything now you're ready to start testing building versions of the product and putting them in front of users and measuring the results but initially you don't build any..."

"lot of evidence that's an awesome story so that was in the user research it wasn't like rolled out to people it was a manual individual there wasn't a single line of code that's written this was just ..."

"internally when I joined Microsoft many years ago the first thing I noticed was that Outlook was very buggy and ask people what's going on and we and they told me we are all dog fooding the next versi..."

"so the key point is you don't have to start at the right hand side which is expensive you can start early on and that leads to parking a lot of ideas very quickly you realize they are not as good as y..."

"the party the people who are actually doing the work they live in Edge award they're they're very focused on moving tickets to the dance State on completing burning story points you know pushing stuff..."

"so I suggest changing this and bringing the developers a little bit out of their kind of agile cage if you like and no disrespect to Agile it's a great thing but let's let them do more than just devel..."

"your the gist board as the way you should be Road mapping or is this more of a strategy framework to think about why you should be prioritizing broadly the way I say this is at the beginning of the qu..."

"can we validate this some of the steps will be done by the PM some by the data analysts some user researcher but some will involve the team there'll be some coding there'll be some running of experime..."

"with mock-ups and then a usability test as a prototype and then an A B test yeah basically uh and you can alter this as you go along sometimes you can run multiple steps in parallel it's not always se..."

"you're doing that and people know that the goal is to launch that thing by October forget about learning from forget about evidence guided I recommend using outcome roadmaps saying by October we want ..."

"and the roadmaps can kind of suffocate this process if you decide up front with low confidence that this particular idea must be launched okay so you're proposing people switch the road mapping practi..."

"awesome maybe just on the topic of okrs real quick how do okrs connect all this it sounds like broadly you kind of assume people keep working on here's our metric or key results or objectives and then..."

"zooming out again basically you don't need to take all of these ideas and lump them all together and change the way you work as a business you can start with picking some of these ideas and starting t..."

"prominently there and there's also the books Landings page where you'll find everything you need to know about the book evidenceguided.com well with that we've reached our very exciting lightning roun..."

"gold in all these books Linux really really important books and I think they're not as appreciated as they should a running Lane that's another example what is a favorite recent movie or TV show I'm n..."

"love it favorite interview question you like to ask candidates I like to ask them to design something for a niche audience so a navigation system for elderly people or some sort of laptop for people w..."