
63 segments available
In this segment, Lex Fridman introduces Michael Kearns, a professor at the University of Pennsylvania and co-author of 'The Ethical Algorithm.' They discuss Kearns' expertise in algorithmic fairness, bias, privacy, and ethics, as well as his diverse research interests including machine learning and game theory. This introduction sets the stage for a deep dive into the ethical implications of algorithms.
"the following is a conversation with Michael Kern's he's a professor at the University of Pennsylvania and a co-author of the new book ethical algorithm that is the focus of much of this conversation ..."
Lex shares his personal experience working with Michael Kearns on an algorithmic trading project during his undergraduate studies. He reflects on how this experience ignited his passion for algorithmic game theory and highlights the influence of Kearns' work on his own research in machine learning and human-robot interaction.
"an undergrad early on I worked with Michael on an algorithmic trading project in competition that he led that's when I first fell in love with algorithmic game theory while most of my research life ha..."
Lex recommends the podcast 'Pessimists Archive,' which explores historical resistance to new technologies. He discusses how the show connects past fears of innovation with contemporary societal issues, emphasizing the psychological and technological lessons that can be learned from history. This segment highlights the importance of understanding human psychology in the context of technological advancement.
"again in person after so many years this is the artificial intelligence podcast if you enjoy it subscribe on YouTube give it five stars on Apple podcasts supported on patreon or simply connect with me..."
Michael Kearns discusses the impact of literature on his life, particularly mentioning 'Infinite Jest' by David Foster Wallace. He reflects on his early aspirations as an English major and how his literary background has influenced his approach to algorithmic ethics and fairness. This segment reveals the intersection of literature and technology in Kearns' thinking.
"pessimist archive I highly recommended and now here's my conversation with Michael Kern's you mentioned reading Fear and Loathing in Las Vegas in high school and having more or a bit more of a literar..."
Kearns explains how his work on algorithmic fairness is rooted in philosophical questions about justice and ethics. He shares insights from discussions with his uncle, a moral philosopher, and highlights the complexities of defining fairness in algorithms. This segment emphasizes the need for a balance between technical solutions and philosophical considerations in algorithmic design.
"that literary mind that either you had yeah I'd like to claim there was a deeper connection but but there you know I think both Aaron and I kind of came at these topics first and foremost from a techn..."
In this segment, Kearns discusses the challenges of achieving algorithmic fairness, particularly in the context of biased data. He explains that while algorithmic methods can address some fairness issues, they cannot fully rectify systemic biases present in the data. This highlights the limitations of technical solutions in addressing deep-rooted societal problems.
"algorithmic solutions but I would say the algorithmic solutions take care of some of the low-hanging fruit sort of the problem is a lot of algorithms when they don't consider fairness they are just te..."
Kearns shares his optimistic view that most people are fundamentally good, attributing deviations from this norm to circumstances and the influence of power. He discusses how social norms within professional environments can lead to behaviors that may not align with broader societal values, emphasizing the complexity of human nature and morality.
"it will have to revisit I think things that have been thought about by you know many generations of scholars before us so it's very early days for fairness I think so before we get into the details of..."
Kearns explores how different professional cultures develop their own social norms and rationales for behavior. He reflects on his experiences in both finance and academia, illustrating how these environments can shape perceptions of acceptable behavior and the potential for ethical dilemmas. This segment underscores the importance of context in evaluating actions within various fields.
"that's really interesting so it's the within the social norms constructed by that particular group of people you're all trying to do good but because it's a group you might be you might drift into som..."
Kearns discusses the concept of ethical algorithms, comparing it to traditional algorithmic metrics like running time and accuracy. He emphasizes the need for specific definitions of fairness in algorithms and the challenges of quantifying ethical behavior. This segment delves into the complexities of measuring ethics in algorithmic design.
"other side of the coast Silicon Valley and the problems there as we talk about privacy as we talk about fairness at the high level and I'll ask some sort of basic questions with the hope to get at the..."
In this concluding segment, Kearns addresses the potential for measuring the ethicality of algorithms. He discusses the importance of defining harm and the challenges of balancing different fairness criteria. This segment highlights the ongoing debate in the field of algorithmic ethics and the need for continued exploration of these critical issues.
"and our book is about that part of algorithmic ethics that we know how to kind of put on that same kind of quantitative footing right now so you know just to say something that our book is not about o..."
Kearns delves into the tension between group fairness and individual fairness in algorithms. He explains how traditional definitions of fairness often focus on group metrics, which can overlook the unique circumstances of individuals. This segment raises critical questions about the effectiveness of current fairness definitions and the potential for algorithmic gerrymandering.
"and when you ask questions like will algorithms feel ethical one way in which they won't under the definitions that I'm describing is if you know if you are an individual who is falsely denied alone i..."
This segment addresses the concept of subjective fairness and the importance of incorporating human perspectives into algorithmic fairness definitions. Kearns discusses the need for behavioral research to understand public perceptions of fairness and how these insights can inform algorithm design, emphasizing the gap between technical definitions and societal expectations.
"promises but even that doesn't incorporate I think something that you're hinting at here is what a chime I'll call subjective fairness right right so a lot of the definitions I mean all of the definit..."
Kearns reflects on the intersection of computer science and psychology, suggesting that computer scientists can contribute valuable insights into human behavior through data analysis. He discusses the potential for interdisciplinary research to enhance understanding of fairness and ethics in algorithms, advocating for a more integrated approach to these complex issues.
"I think I think this needs to change actually for many social norms not just for fairness right so there's a lot of discussion these days in the AI community about interpretable AI or understandable A..."
In this segment, Kearns discusses the challenges of achieving fairness in algorithmic design, particularly when considering individual versus group fairness. He introduces the concept of fairness gerrymandering, where algorithms may meet aggregate fairness definitions while still discriminating against specific subgroups. This highlights the need for more nuanced approaches in algorithm development.
"and machine learning more specifically I kind of think of as imperialist research communities in that you know kind of like physicists in an earlier generation computer scientists kind of don't think ..."
Kearns examines the trade-offs between fairness and performance in algorithmic systems. He discusses how incorporating fairness into algorithmic objectives can impact overall system performance, emphasizing the importance of understanding these dynamics in the context of real-world applications. This segment underscores the complexity of balancing ethical considerations with technical efficiency.
"civilization so there's a lot of people now in public discourse doing sort of group thinking thinking like there's particular sets of groups that we don't want to discriminate against and so on and th..."
This segment explores the intricate nature of fairness definitions in algorithms, particularly regarding the intersection of various demographic attributes. Kearns highlights the challenges of ensuring fairness across multiple dimensions and the potential for unintended discrimination when focusing solely on group metrics. He advocates for a more comprehensive understanding of fairness that considers individual nuances.
"still you know very strong determinants of outcome and personality in the light so one I think useful spectrum is to sort of think about that array between the group and this individual and to realize..."
Kearns discusses the necessity of integrating human values into algorithmic fairness definitions. He raises questions about how to weigh different demographic factors in fairness assessments and the implications of these decisions on societal norms. This segment emphasizes the importance of engaging in broader societal debates about fairness and the ethical responsibilities of algorithm designers.
"common scale and you know one of the things we've worked on from a research perspective is you know so we sort of know how to you know we in relative terms we know how to provide fairness guarantees a..."
In this concluding segment, Kearns reflects on the politicization of fairness discussions and the challenges of addressing complex social issues through algorithmic frameworks. He emphasizes the need for researchers to engage with public discourse and consider the broader implications of their work on societal values and ethics, particularly in sensitive areas like criminal justice and affirmative action.
"big optimization of achieving fairness is that kind of what humans I mean most of you know I mean of course you know I don't need to tell you that of course technically one could incorporate such weig..."
Michael Kearns discusses the intricacies of affirmative action debates, highlighting the strong arguments on all sides of the issue. He emphasizes the importance of engaging in broader societal discussions beyond technical algorithmic work, suggesting that researchers must address these debates to be taken seriously.
"just had a debate they have this structure we have a old Oxford style or whatever they're called debates those two versus two and they talked about affirmative action and it was the is incredibly inte..."
Kearns explains the concept of fairness in algorithms, particularly in relation to gender discrimination. He introduces the idea of a Pareto curve, illustrating the trade-offs between algorithmic performance and fairness, and discusses how these trade-offs must be communicated to stakeholders.
"it's interesting to be a researcher trying to do for the most part technical algorithmic work but Aaron and I both quickly learned you cannot do that and then go out and talk about and expect people t..."
In this segment, Kearns elaborates on the Pareto curve, which represents the trade-offs between error and unfairness in algorithmic models. He discusses how researchers can visualize these trade-offs and the implications for decision-making in contexts like criminal justice.
"paper or do you want to really not touch that like you know we want to touch that and we do touch it so I mean just just again to make sure I'm not promising your your viewers more than we know how to..."
Kearns emphasizes the need for stakeholders, such as policymakers, to understand the trade-offs between accuracy and fairness in algorithms. He argues that these decisions should not be made solely by computer scientists but should involve broader societal input.
"disparity between false rejection rates between two groups and you know your algorithm now has a knob that basically says how strongly do I want to enforce fairness and the less unfairly you know we y..."
This segment explores the philosophical debate between equality of opportunity and equality of outcome in the context of algorithmic fairness. Kearns discusses how biases in society can affect algorithmic decisions and the importance of addressing these biases.
"so this curve kind of specifies the you know kind of undaunted models models that are off that curve are you know can be strictly improved in one or both dimensions you can you know either make the er..."
Kearns provides examples of how bias can manifest in machine learning algorithms, particularly in face recognition systems. He discusses the implications of training datasets that lack diversity and the need for algorithms to account for fairness without sacrificing accuracy.
"care about those you know the mistakes you make not being disproportionately on one racial group or another you can you can show this curve I'm hoping that in the near future it'll be possible to expl..."
In this segment, Kearns discusses strategies for addressing discrimination in predictive models, such as adjusting algorithmic objectives to include fairness metrics. He highlights the costs associated with gathering diverse data and the challenges of implementing fair algorithms.
"really important thing to sort of think about so you think when we start to optimize for fairness there's almost always in most system going to be trade-offs can you like what's the trade-off between ..."
Kearns delves into the philosophical challenges surrounding fairness in algorithms, particularly in a politically divisive climate. He emphasizes the need for engineers to consider ethical implications while developing algorithms that impact society.
"or keep incarcerated somebody who would not have recommitted a violent crime so in case of awarding somebody parole or giving somebody Perl or letting them out on parole you don't want them to recommi..."
Kearns discusses the responsibility of engineers in shaping ethical algorithms, stressing that they should not dictate societal norms. He argues for the importance of stakeholder engagement in defining fairness and privacy in algorithmic systems.
"unfairness you want to get rid of is the in the judges mind the bias of having being brought up to society the slight racial bias the racism that exists in the society you want to remove that from the..."
In this segment, Kearns reflects on the unintended consequences of social media platforms on political discourse and society. He discusses how algorithms have evolved based on user behavior and the challenges of predicting their societal impact.
"and and so a lot of the fairness discussion is almost the goal is for it to be a corrective mechanism to account for the injustice earlier in life by some definitions of fairness or some theories of f..."
Kearns suggests potential algorithmic remedies for the polarization caused by social media platforms. He discusses the importance of balancing content exposure to promote diverse viewpoints and mitigate the effects of echo chambers.
"less accurate okay now there's a bunch of ways you could think about addressing this one is to deliberately put into the objective of the algorithm not to not to optimize the air or at the expense of ..."
Michael Kearns discusses how social media platforms like Facebook and Twitter evolve based on user behavior and the unintended consequences of their algorithms. He highlights the role these platforms play in influencing elections and polarizing political discourse, emphasizing the need for awareness of their impact on society.
"it okay things like Facebook and Twitter these are you know these are these are systems right these are social systems and their evolution even their technical evolution because machine learning is in..."
Kearns explores potential algorithmic solutions to combat polarization on social media. He suggests that instead of only showing users content they agree with, platforms could introduce mechanisms to expose them to opposing viewpoints, fostering a more balanced discourse.
"and you know like feed us all basically just the stuff that we already agreed with and I think it you know we've come to that outcome I think largely but I think it's something that we all learned tog..."
In this segment, Kearns addresses the tension between optimizing for user engagement and promoting fairness in algorithmic design. He questions whether prioritizing engagement has led to negative societal outcomes and discusses the challenges of implementing fairer algorithms.
"polarization and intolerance of opposing viewpoints as you know right these these algorithms have models right and they kind of place people in some kind of metric space and and they place content in ..."
Kearns reflects on the potential costs to companies of shifting away from engagement-focused algorithms. He argues that while it may be challenging, there are ways to optimize algorithms for societal benefit without sacrificing user experience.
"from your interests I can it's almost like an exploration button so just get your intuition do you think engagement so like you staying on the platform you because thing engaged do you think fairness ..."
Kearns discusses the complexities of algorithmic optimization, emphasizing that understanding how to optimize for one outcome can inform how to avoid negative consequences. He highlights the difficulty of measuring long-term societal impacts of algorithmic decisions.
"essentially driven by machine learning right it's driven by not just machine learning but you know very very large-scale a be experimentation where you gonna have tweaked some element of the user inte..."
In this segment, Kearns argues for the necessity of human oversight in algorithmic decision-making. He suggests that while algorithms can optimize for user preferences, human judgment is crucial for ensuring that these optimizations align with broader societal values.
"that's right so I'm not I'm not claiming that doing something different will immediately make it apparent that this is a good thing for society and in particular I mean ethical one way of thinking abo..."
Kearns proposes that social media platforms could benefit from allowing users to explore content outside their usual preferences. He discusses the potential for algorithms to introduce users to diverse viewpoints, which could enhance understanding and tolerance.
"think one one concrete thing they could do is they could experiment with letting people who chose to to see more stuff in their newsfeed that is not entirely kind of chosen to optimize for their parti..."
Kearns reflects on the evolving role of computer science in society, suggesting that as algorithms become more integrated into various fields, the discipline of computer science may become less distinct. He emphasizes the importance of interdisciplinary collaboration.
"the platform for the platform's influence on society outside of the platform right and they you know it's easy for me to sit here and say these things yes but conceptually I do not think that these ar..."
Kearns introduces the concept of algorithmic privacy, explaining how privacy norms can be encoded within algorithms. He critiques traditional data anonymization methods and advocates for differential privacy as a more effective approach.
"the focus in your book and algorithmic fairness research today in general is on machine learning like we said is data but and just even the entire AI feel right now is captivated with machine learning..."
In this segment, Kearns discusses the challenges of defining fairness in algorithms. He expresses caution about allowing algorithms to autonomously determine fairness, highlighting the complexities and emotional weight of these decisions.
"algorithm start exploring as well I definitely think you know this is a much narrower statement I definitely think the kind of algorithmic auditing for different types of unfairness right so like in t..."
Kearns argues for a greater philosophical engagement within the computer science community. He believes that understanding the human and ethical implications of algorithms is essential for responsible technological development.
"don't laugh it's like the the cost-benefit analysis to the field of kind of going there right now it just doesn't seem worth it to me that said I should say that I think computer scientists should be ..."
Kearns speculates on the future of computer science education, suggesting that as the field becomes more integrated into various disciplines, traditional computer science departments may evolve. He emphasizes the importance of programming skills across all fields.
"know though I think one of the reasons we decided to write this book as we thought 10 years ago I wouldn't have tried this because I I just didn't think that sort of people's awareness of algorithms a..."
Kearns explains the concept of algorithmic privacy, emphasizing the importance of encoding privacy definitions within algorithms. He critiques traditional data anonymization methods, highlighting their flaws and introducing differential privacy as a more robust alternative that allows for useful data analysis while protecting individual privacy.
"is differential privacy or more broadly algorithmic privacy algorithmic privacy more broadly is just the study or the notion of privacy definitions or norms being encoded inside of algorithms and so y..."
In this segment, Kearns elaborates on the limitations of data anonymization techniques, using the Netflix Prize as a case study. He illustrates how seemingly anonymized data can still lead to privacy breaches when combined with other datasets, emphasizing the need for stronger privacy measures.
"with data what's anonymization of data so by anonymous a ssin i'm you know kind of referring to techniques like i have a database the rows of that database are let's say individual people's medical re..."
Kearns introduces differential privacy as a superior method for protecting individual data. He explains how it works by comparing two scenarios—one with an individual's data included and one without—demonstrating that the risks to privacy remain consistent regardless of data inclusion.
"these notions right is that notions of Adana anonymization removing personally identifying information the kind of fundamental conceptual flaw is that you know these definitions kind of pretend as if ..."
This segment delves into the technical mechanisms behind differential privacy. Kearns discusses how adding noise to data computations can preserve privacy while still allowing for meaningful analysis, showcasing its applicability across various statistical and machine learning methods.
"fingerprint for you is the see of all Netflix users there were just another paper on this in science or nature of about a month ago that you know kind of 18 attributes I mean my favorite example of th..."
Kearns highlights the versatility of differential privacy in statistical analysis and machine learning. He reassures that many common algorithms can be adapted to maintain privacy, allowing researchers to benefit from data while safeguarding individual information.
"a technical definition but like the spirit of it is we we compare to to alternate worlds okay so let's suppose I'm a researcher and I want to do you know I there's a database of medical records and on..."
In this thought-provoking segment, Kearns discusses the balance between individual privacy and the societal benefits of data usage. He explores the potential for individuals to retain control over their data while still allowing for its beneficial use in research and commercial applications.
"happen to you as a result of data analysis it just says that these bad things were going to happen to you already even if your data wasn't included and to give a very concrete example right you know u..."
Kearns introduces the concept of privacy through obfuscation, discussing a project that aims to protect user searches from being tracked by search engines. He raises questions about the effectiveness of such techniques in providing privacy while still allowing for personalized services.
"is done great but how what is the mechanism of differential privacy so that's the kind of beautiful statement of it well what's the mechanism by which privacy's preserve yeah so it's it's basically by..."
Kearns shares his optimistic view on the future of privacy in a data-driven world. He discusses the potential for individuals to have more control over their data and the importance of developing frameworks that balance privacy with the benefits of data analysis.
"by adding noise yeah so I'm a relatively recent member of the differential privacy community my co-author Aaron Roth is you know really one of the founders of the field and has done a great deal of wo..."
Michael Kearns discusses the concept of privacy by obfuscation, emphasizing the need for specific data responses while maintaining privacy. He explores the balance between individual data specificity and privacy guarantees, suggesting that obfuscation could be a viable solution to this challenge.
"999 were not okay so it's the same kind of idea kind of you know privacy by obfuscation so I think that's an interesting idea doesn't give you differential privacy it's also I was actually talking to ..."
Kearns reflects on the future of privacy in an algorithm-driven society, comparing data collection practices in different countries. He expresses optimism about achieving a balance between individual privacy and societal benefits from data usage, highlighting the importance of individual control over personal data.
"others as well so where do you think will land in this algorithm driven society in terms of privacy so sort of China like Chi Fuli describes you know it's collecting a lot of data on its citizens but ..."
In this segment, Kearns discusses the emerging idea of compensating individuals for their data. He raises questions about the economic implications of allowing selective data use and the potential for individuals to have more control over their data, which could reshape the current economic model of data usage.
"data they should be able to limit the uses of that data they should even you know there's there's you know fledgling discussions going on in research circles about allowing people selective use of the..."
Kearns emphasizes the significance of interface design in empowering users to control their data. He argues that enhancing user interfaces can facilitate better data management and privacy, ultimately benefiting both users and companies.
"you know to kind of get more directly answer your question I think I'm optimistic at this point from what I've seen that we will land at some you know better compromise than we're at right now where a..."
Kearns discusses the current economic model of the internet, where users are often seen as products rather than customers. He explores the potential for a new economic model that includes users as active participants in data markets, which could fundamentally change how data is valued and utilized.
"yeah I mean the other thing that has to be said right is that you know it's a cliche but you know we who is the users of many systems platforms and apps you know we are the product we are not the cust..."
In this segment, Kearns introduces game theory as a mathematical framework for analyzing interactions among individuals. He explains its relevance to algorithmic game theory and how it can help model complex systems with many interacting agents.
"consequences speaking of markets a lot of fascinating aspects of this world arise not from individual humans but from the interaction of human beings you've done a lot of work in game theory first can..."
Kearns shares his admiration for the concept of equilibrium in game theory, particularly Nash's work. He discusses how equilibrium can emerge in competitive settings and its implications for predicting collective outcomes in complex systems.
"but cooperation is not an equilibrium in prisoner's dilemma and so my work and the field of algorithmic game theory more generally in these areas kind of looks at settings in which the number of actor..."
Kearns explores the connections between machine learning and game theory, particularly in the context of no-regret learning. He discusses how these concepts can lead to equilibrium in systems where multiple players interact, highlighting the potential for algorithmic solutions.
"personally than you asked the question I think within the field of algorithmic game theory perhaps the single most important kind of technical contribution that's been made is the real the the realiza..."
Kearns examines how navigation apps utilize algorithms to optimize driving routes based on user data. He discusses the implications of achieving equilibrium in traffic patterns and how this can lead to increased driving times, despite the optimization efforts.
"them first which is you know we have many systems platforms and apps these days that work really hard to use our data and the data of everybody else on the platform to selfishly optimize on behalf of ..."
Kearns reflects on the historical application of game theory in nuclear deterrence strategies during the Cold War. He discusses the complexities of modeling such high-stakes scenarios and the evolution of game theory as more players enter the field.
"to talk about what those other solutions might be and what what the algorithms to implement them are which we do discuss in the kind of game theory chapter of the book but but similarly you know on so..."
Kearns shares insights from his experience in algorithmic trading, detailing how algorithms have transformed financial markets. He explains the progression of automation in trading and the optimization challenges faced by institutional investors.
"right because there were sort of only two nuclear powers at the time and you didn't have to worry about deterring new entrants and who was developing the capacity and so we have many we have this it's..."
In this segment, Kearns delves into high-frequency trading (HFT) and its reliance on rapid data analysis to exploit temporary price discrepancies. He contrasts HFT with longer-term investment strategies, emphasizing the unique challenges and risks associated with predicting market movements over extended periods.
"your your execution so you know this is the kind of you know this is an optimization problem this is a control problem right and so machines are a better we know how to design algorithms you know that..."
Kearns reflects on the importance of human judgment in long-term investing, particularly in navigating economic cycles and geopolitical risks. He argues that while algorithms can optimize short-term trades, understanding broader market dynamics requires a human perspective, making certain investment roles relatively safe from automation.
"cliche goes so that is a you know kind of a sweet spot I think for quant trading and investing right now and has been for some time when you really get to kind of more warren buffett style timescales ..."
Michael Kearns shares a personal story about a pivotal moment during his doctoral studies at Harvard. He recounts a time of uncertainty and contemplation in Boston Common, ultimately deciding to persevere in his academic journey. This decision marked a transformation from student to researcher, highlighting the challenges and rewards of pursuing original work.
"sources of data that are on wildly different timescales right so if I'm an hft I'm a high-frequency trader like I don't I don't I really my main source of data is just the data from the exchanges them..."