"a lot of people were saying like oh this whole idea of Game Theory it's just nonsense and if you really want to make money you got to like look into the other person's eyes and read their soul and fig..."

"you've been a lead on three amazing AI projects so we've got libratus that solved or at least achieved human level performance on No Limit Texas Hold'em poker with two players heads up you got pluribu..."

"hold 'em poker is the most popular variant of Poker in the world so you know you go to a casino you play sit down at the poker table the game that you're playing is no limit Texas Hold'em if you watch..."

"other ones rewards more kind of calculated strategy or or no because you're sort of looking from an from an analytic perspective is is strategy also rewarded in No Limit taxes hold on I think both var..."

"between No Limit and limit what about on the action side when you're actually making that big bet that's what I mean by crazy I was I was trying to refer to the technical the the technical term of cra..."

"that play these games we'll talk about poker we'll talk about diplomacy are you um are you drawn in in part by the beauty of the game itself AI aside or is it to you primarily a fascinating problem se..."

"this gets into the concept of an ash equilibrium yeah so it is a Nash equilibrium Okay so and any finite two-player zero-sum game there is an optimal strategy that if you play it you are guaranteed to..."

"your stack generally speaking now that doesn't include anything about the fact that you can go broke it doesn't include any of those kinds of normal real world limitations you're talking you know in t..."

"some games there's no guarantee that you're going to win by playing a national equilibrium have you ever tried to model in the other aspects of the game which is like the pleasure you draw from playin..."

"fun to play with and fun to watch yeah and I think you know I've I've heard uh talks from like game designers and and they say like you know people that work on AI for actual recreational games that p..."

"they're just releasing the star feel good they do one game at a time yeah and so uh whatever it is whenever the date is I don't know what the data is calm down uh but it would be I don't know like uh ..."

"crazy stuff AI does we have some flexibility to play just like with the game of diplomacy it's a game this is not real geopolitics not real war it's a it's a game so you could you can have a little bi..."

"it regret having not played that action in the past and when it encounters that same situation again it's going to pick actions that have higher regret with higher probability now it'll just keep simu..."

"uh it's a simpler setting sure so I kind of in my brain the word self-play has mapped in you all networks but we're speaking something bigger than just neural networks it could be anything the self-pl..."

"neural nets come in I said like okay if it's in that situation again then it will choose the action that has high regret now the problem is that poker is such a huge game you know I think no limit Tex..."

"complex game chess or poker or go or poker do you know that is a controversial question okay um I'm gonna it's like somebody screaming on Reddit right now it depends on which subreddit you're on is it..."

"they think I have what do they have what do they think I think they have that kind of stuff and um that's that's kind of where bluffing comes into play right because the fact that you can Bluff the fa..."

"how what are the different approaches to the imperfect information game so the key thing to understand about why in perfect information makes things difficult is that you have to worry not just about ..."

"now you take that to Poker what that means is the value of bluffing for example if you're the kind of person that never Bluffs and you have this reputation as somebody that never Bluffs and suddenly y..."

"the balance of how often in the key sort of branching is the bluff or not the bluff is that a is that a good crude simplification of the major decision in poker it's a good simplification I think that..."

"to think about what would I do if I had this different set of cards is there explicit estimation of like a theory of mind that the other person has about you or is that just a emergent thing that happ..."

"start by playing the Nash equilibrium and then maybe if they spot weaknesses in the way you're playing then they can deviate a little bit to take advantage of that they aim to be unbeatable in expecta..."

"deviate from a Nash equilibrium style of play to try to take advantage of those perceived weaknesses and then counter exploit them so you kind of get into the Mind Games there so you think about these..."

"I think you know it we've we're playing for 50 100 blinds and over the course of about 120 000 hands it made close to two million dollars 120 000 hands 120 000 hands against humans yeah and this was t..."

"money at stake where it would basically be divided among them depending on how well they did relative to each other so we wanted to have some incentive for them to play their best did you have a confi..."

"in a game like chess the the search is like okay I'm in this chess position and I can like you know move these different pieces and see where things end up in poker what you're searching over is the a..."

"Nash equilibrium approach because if they're not then you're just going to make more money right like anything that deviates like by definition the national equilibrium is the strategy that does the b..."

"lot from that competition and in particular you know what became clear to me is that the way the humans were approaching the game was very different from how the bot was approaching the game the bot w..."

"outcome and we did some experiments in small scale versions of Poker and what we what we found was that if you do a little bit of search even just a little bit it was the equivalent of making your you..."

"hands Okay so that search how do you fuse that with what the neural net is telling you or what the the the train system is telling you yeah so you kind of like where the train system comes in is is th..."

"all these other hands as well okay but you are you in the search always going to the end of the game in liberatis we did uh so we only use search starting on the turn and then we searched all the way ..."

"the case of liberatis so for liberatis what's the the most number of re-racists have you ever seen uh you probably cap out at like five or something because at that point you're basically all in you k..."

"second best hand like now you get a really tough choice to make and so the humans would sometimes think like five or ten minutes about like what do you do should I call should I fold and um and when I..."

"um so he wasn't scared he was excited he was excited and uh and he all he honestly he wanted to play against the bot he thought he thought he had a decent chance of beating it um I I think he's you kn..."

"um we've focused on the most popular variants so heads up no limit Texas Hold'em and then we followed it up with um with uh six player poker as well where we managed to uh make a bot that beat expert ..."

"moves ahead and you see like what the board state looks like um that's what I mean by search if you take out the search that's done during the game the ELO rating drops to around three thousand so eve..."

"network is doing the searching and I wonder what the human brain is doing in terms of searching because you're doing that like computation the human is Computing they have intuition they've got they h..."

"think it's a really important missing piece the ability to plan and reason more generally across a wide variety of different settings in a way where the general reasoning makes you better at each one ..."

"kinds of of planning that we could do so when the broadest actually beat the poker plays what did I feel like what was that I mean actually on that day what were you feeling like were you were you ner..."

"but still a lot for even any gratitude today it's still tough to to get or even to allow yourself to think in that in terms of scale at CMU at MIT anything like that yeah and you know talking about te..."

"that I you know had to make uh just a fun question what what id did you use what uh for for C plus plus I think I used a visual studio actually yeah okay yeah is that still carried through to today vs..."

"be able to distinguish you know like having a king High flush versus an ace high flush and in some situations that really matters a lot and so they could put the bot into those situations and then the..."

"and so then they would review the hands and try to see like okay could they find patterns in the bot the weaknesses and could they then then they would coordinate and study together and try to figure ..."

"in the long run how did it uh feel at the end like as a human being what it as a person who loves appreciates the beauty of the Game of Poker and the person who appreciates the beauty of AI is there d..."

"it's a different it's different than chess and that aspect like people get that's why you want to look at Betty Marcus if you want to actually understand what people really think in the same sense pok..."

"and so there was this big debate about whether Nash equilibrium and all these techniques that compute it are even useful once you go outside of two player zero some games now I think for many games th..."

"that's true more more broadly in extremely adverse serial games in general but that's sort of in practice versus being able to prove something that's right nobody has a proof that that's the case and ..."

"so what are some interesting things about uh pluribus that was able to achieve human level performance on this or superhuman level performance on the six player version of Poker I personally I think t..."

"matter how would you describe the more General case of limited Dev search so it's basically constraining the scale a temporal or in some other way of the computation you're doing in some clever way so..."

"um but it wasn't the main challenge for poker like I think what neural Nets are really good for if you're in a situation where finding features for a value function is really difficult then neural Net..."

"the skill with which you avoided the question of the greatest of all time was impressive so my feeling is that it's a difficult it's a difficult question because just like in chess where you can't rea..."

"diplomacy yeah so I talked a lot about two player zero some games and what's interesting about diplomacy is that it's very different from these like adversarial uh games like chess go poker even Starc..."

"period is done all the players simultaneously submit their moves and they're all executed at the same time and so you can tell people like hey I'm going to support you this turn um but then you don't ..."

"talk about anything you could say like hey let's have a long-term alliance against this guy you can say like hey can you support me this turn and in return I'll do this other thing for you next turn o..."

"by working with other players so on every turn you can issue a move order so for each of your units you can move them to an adjacent territory or you can keep them where they are or you can support a ..."

"they'll have to retreat somewhere what does support mean support is like it's it's an action that you can issue in the game so you can say this unit you write down this unit is supporting this other u..."

"general is it true that Henry Kissinger loved the game and JFK and all those I've heard like a bunch of different people that or is that just one of those things that the cool kids say they do but the..."

"know it kind of has a nice like wholesome take-home message then that you know war war is ultimately futile and uh and that optimal that feudal optimal could be achieved through great diplomacy yeah s..."

"more vulnerable position because they have to like um they have a lot more neighbors as well got it larger territory more uh yeah right more border to defend okay uh what else is what else is importan..."

"um there's a lot of different scoring systems the one that we used in our research um basically um gives a score relative to how much control you have of the map so the more that you control the highe..."

"sure um and you know it's understandable I mean the game is so incredibly different and so so much more complicated than the kinds of games that people were working on like chess and go uh and poker t..."

"um the the depth and breadth of these conversations is is really complicated and it's all being done in natural language um now you could approach it and we actually consider doing this like you you k..."

"like hey you might be getting attacked by by this other power Okay so what how we're supposed to think about okay so that's the natural language how do you even begin trying to solve this game it seem..."

"to beat humans so how do you integrate the human into the loop of this so what you have to do is incorporate human data and to kind of give you some intuition for why this is the case like imagine you..."

"considering just how much progress there there was in Ai and that that progress has continued in the years since so winning in diplomacy what does that really look like it means talking to six other p..."

"about 40 games with with real humans online uh the bot came in second out of all players that played five or more games and um so not like number one but way way higher than well what was the expertis..."

"that's quite brilliant because I played a lot of sports in my life like as a tennis Judo whatever and it's it's somehow almost easier to go against experts almost always I don't I think they're more p..."

"um the really good diplomacy players are able to to take advantage of the fact that there is that there are some weak players in the game okay so if you have to incorporate human play data how do you ..."

"how's that done just so as a starting point is that with reinforcement learning or is that just optimal determining what the optimal is for intents It's a combination of reinforcement learning and pla..."

"planning is done is actually not using language so we're coming up with this like plan for the action uh that we're gonna play and the other person's gonna play and then we feed that action into the d..."

"you have like an extra function that does the uh estimates the value of that message yeah so we have these kinds of filters that like so it's a filter so there's a there's a good and is that filter in..."

"the the the uh the goal you want yeah and we're filtering for several things we're filtering like is this a sensible message you know so sometimes language models will send will generate messages that..."

"when it is telling you that hey I'm actually going to support you this turn is there some sense I don't know if you step back and think that this process well indirectly help us study human psychology..."

"use to um we're hoping that the the academic Community the research Community is able to use it for for all sorts of interesting research questions so do you from having studied this game is this a su..."

"you're threaten somebody you're supposed to do it politely yeah politely you know keep it in character um we actually had a researcher watching the bot 24 7 for well whenever we play a game we had a b..."

"um so what's required to win like what um what does it mean to mess up or to exploit the sub-optimal behavior of a player like uh is there is there optimally rational behavior and irrational behavior ..."

"and be able to to work with that can you just Linger on that meeting like there's an individual there's an individual personality each player and then you're supposed to remember that but Woody means ..."

"equilibrium would would change things is if it helped you so I what's the dynamic of cooperation that's effective in diplomacy do you always have to to have one friend in the game you always want to m..."

"and the bot will do this like the bot will work with the other players to stop the superpower from winning but if it doesn't really if it's trained from scratch or it doesn't really have a good ground..."

"don't have unlimited data we don't have unlimited neural net capacity um but it gives us some approximation uh is there some data on the internet that's useful besides just diplomacy so on the languag..."

"is um and and then the language can correspond to that intent now I'm not saying that this is you know happening imminently but um I'm saying that this is like a future application potentially of this..."

"yeah I mean I just came back from Ukraine I'm going back there on deep personal levels think a lot about how peace can be achieved and I'm a big believer in conversation or leaders getting together an..."

"but then you have to have human data right you really because it's like the game of diplomacy is fundamentally different than geopolitics you need data you need like I guess that's the question I have..."

"like because it's natural language right you're operating in a space of intense and in a space of natural language that feels very close to the real world and it also feels like you could get data on ..."

"um to elaborate on this a bit like one way to approach making a human-like AI for chess is to collect a bunch of human games like a bunch of human Grand Master games and just do supervised learning on..."

"where he needs to improve his strategy um and so I can Envision this future where data on specific chess and go players becomes extremely valuable because you can use that data to create specific mode..."

"right now it's really hard to learn how to get better in games like chess and poker and go because the way that the AI plays is so foreign and incomprehensible but if you have these AIS that are playi..."

"after somehow find where's the ethics in that and we're back to discussions inside relationships anyway what were you saying oh yeah I was getting like yeah this yeah that's kind of going to the quest..."

"creating AGI systems you've been a part of creating um by the way we should say a part of great teams that do this of creating systems that achieve breakthrough performances on before thought unsolvab..."

"gigantic background model language model and then you do um the training becomes like prompting that model to uh to essentially do a kind of querying a search into the space of the things it's learned..."

"um and then shifting more towards reinforcement learning as time went on and that actually had a lot of benefits I think because it allowed me to look at these problems in a very different way from th..."

"learning do you think one day we'll be able to since you're taking steps from poker to diplomacy one day we'll be able to uh figure out how to live life optimally well what is it like in in poker and ..."

"function is sometimes it's pretty hard to specify the same way that you know trying to handcraft the optimal policy in a game like chess is really difficult it's not so clear-cut what the reward funct..."