All Segments
Richard Craib — How to Open Source Finance | Episode 169
33 segments available
Richard Craib is the Founder & CEO of Numerai, a new kind of hedge fund where data scientists around the world collaborate to predict equity returns using artificial intelligence. Richard joins the show to discuss Numerai’s origins, how it embraces the spirit of open source, why it has its own cryptocurrency and MUCH more! Important Links: - Numerai - https://numerai.fund/ - Richard’s Twitter - https://twitter.com/richardcraib?s=21&t=98b6M5KTDwZJ4hwFwVcsaQ Show Notes: 0:00:00 Intro 0:01:28 Main podcast 0:03:13 The genesis of Numerai 0:07:53 How and why Numerai gives away its dataset 0:17:19 Getting users to put skin in the game 0:21:31 How Numerai scores users; becoming comfortable with the process 0:25:38 Size limits & leverage 0:28:48 Reactions to Numerai 0:31:53 Numerai’s cryptocurrency 0:36:19 How Numerai differs from Quantopian 0:38:08 Data, optimization & LLMs 0:47:16 “Monopolize intelligence, monopolize data, monopolize money, and decentralize the monopoly” 0:53:30 Numerai’s relationship with its data scientists 1:00:20 What could go wrong? 1:07:20 “Life is long”
Segments Timeline
The Global Talent Pool in Data Science
Richard Craib discusses the untapped potential of data scientists worldwide, emphasizing that talent is not confined to Wall Street. He introduces the concept of providing obfuscated data to machine learning experts, allowing them to model without knowing the specifics of what they're predicting. This innovative approach aims to democratize access to financial modeling.
"where's all the talent in data science and actually it's online it's not necessarily on Wall Street it's all around the world what if you could give away the data but obfuscated so they have no idea w..."
Numerai's Unique Hedge Fund Model
Craib explains the foundational idea behind Numerai, a hedge fund that leverages global data scientists to predict equity prices. He shares his journey from a quant at a traditional firm to creating an open-source hedge fund, highlighting the importance of collaboration and innovation in finance. The segment also touches on the challenges of market neutrality and the fund's impressive performance.
"portfolio is constrained enough and then they'll get better because hey we just burned a bunch of models foreign [Music] with yet another infinite Loops my guest today I feel so simpatica with them yo..."
The Concept of Open Source in Finance
In this segment, Craib elaborates on the need for an open-source approach in finance, contrasting it with the traditional closed systems of Wall Street. He recounts a pivotal moment in his career that inspired him to create Numerai, focusing on the potential of sharing obfuscated data to enhance performance without compromising confidentiality.
"of money in 2023 because you did really really well last year which was a very difficult year to do well in uh Richard welcome I can't wait to dig into this well thank you that's a that was a great in..."
Obfuscating Data for Better Predictions
Craib discusses the innovative method of obfuscating data to protect sensitive information while still allowing data scientists to build predictive models. He explains how this approach enables machine learning experts to work with abstract data, fostering creativity and collaboration without the constraints of traditional financial data.
"the future and in the best Quant funds uh have managed to do that and numerate as unique idea uh which is what I had when I was had my first job was well where's all the talent in data science and act..."
The Challenge of Data Interpretation
In this segment, Craib addresses the complexities of interpreting financial data and the importance of separating data from its traditional context. He shares insights on how data scientists can leverage abstract data to uncover patterns that may not be immediately apparent, emphasizing the need for innovative thinking in finance.
"to accept that the idea that it can't be confer and much like you've done I set up the uh the system almost exactly it's going to be worldwide we were going to invite data scientists in we were going ..."
Building a Diverse Data Science Community
Craib reflects on the importance of creating a diverse community of data scientists to drive innovation in finance. He discusses the potential for collaboration across different backgrounds and expertise, highlighting how Numerai aims to harness this diversity to improve predictive modeling and financial outcomes.
"of uh person uh and cognitive ability to to make sense of it um so I I definitely love the idea by the way he told me at the end of it uh oh by the way Jim do you have about 500 million dollars lying ..."
The Role of Machine Learning in Finance
In this segment, Craib explains how machine learning plays a crucial role in Numerai's operations. He discusses the various algorithms used in financial modeling and the significance of finding patterns that can generalize to future market conditions, showcasing the innovative spirit of Numerai.
"place it is it is um I had my first boss was a was amazing uh and he was uh he helped me with in so many ways but there was one discussion we had that was uh kind of interesting to me which was I had ..."
Navigating Regulatory Challenges
Craib touches on the regulatory landscape surrounding open-source finance and how Numerai operates as a registered investment advisor. He emphasizes the importance of compliance while fostering an environment that encourages innovation and collaboration among data scientists.
"then you sort of mess up the feature to a large extent that uh it doesn't even look like it's like a p e ratio um and and then you can share it with anybody and anybody with data science background wh..."
The Future of Open Source Finance
In this concluding segment, Craib shares his vision for the future of open-source finance. He discusses the potential for continued growth and innovation in the industry, driven by collaboration among data scientists and the democratization of financial modeling through platforms like Numerai.
"thing where we're going to obfuscate the data uh because uh talking to all the data scientists I knew they're like yeah that's why that doesn't make any difference uh and you know I I used to make peo..."
The Open Source Spirit of Numerai
Richard Craib discusses how Numerai embraces an open-source philosophy by allowing users to retain their intellectual property while using Numerai's data. He explains the challenges users face when they expect a different experience and the importance of avoiding overfitting in machine learning models. Craib emphasizes the responsibility of Numerai to provide quality features that prevent biases in model training.
"model but we do know you used our data and that's cool for the users and sort of the open source Spirit where you can keep the IP uh that yourself you're not handing over the IP to anybody and that's ..."
Staking: Putting Skin in the Game
Craib elaborates on Numerai's innovative staking mechanism, which requires users to put up capital to participate. This system ensures that only serious models are submitted, as users risk losing their stake if their models perform poorly. He explains how this approach has improved the quality of submissions and reduced the likelihood of users exploiting the system.
"not overfitting and and so tell me about the process uh first off actually uh what has been the biggest impediment to open source like regulations how how at the out of your uh Masters at the SEC uh i..."
Creating the Meta Model
In this segment, Craib explains how Numerai combines user models into a single 'Meta model' signal. He discusses the process of weighting models based on the amount staked by users, which reflects their confidence in the model's performance. This method aims to create a robust trading signal that is neutral to various market factors.
"okay excellent so walk us through that end you you've you've got all of these data scientists sending you their signals what what happens within numeri uh and what what do the what does the team there..."
Evaluating Model Performance
Craib outlines how Numerai evaluates the performance of submitted models based on their ability to predict residual returns. He explains the correlation metric used to assess model effectiveness and how successful models can lead to increased stakes in Numerai's cryptocurrency, incentivizing quality contributions from data scientists.
"staking um things got a lot better very quickly you know no one was making a thousand pounds and hoping to get lucky because they would have to stake all of them and they would lose on more than half ..."
Trusting the Process
Richard Craib shares insights into how Numerai manages its portfolio and the confidence they have in their process. He discusses the importance of having a diverse range of positions and how the collective input from data scientists helps mitigate risks. Craib emphasizes the continuous improvement of models and the adaptive nature of Numerai's strategy.
"the model um and and that is why you know if a factor does badly numerize unlikely um to do badly because we're we're running neutral to uh to all the factors that's absolutely brilliant because you'r..."
The Role of Leverage in Quant Funds
In this segment, Craib discusses the use of leverage in Numerai's quantitative fund strategy. He explains how leverage decisions are made based on the volatility of the assets involved and the importance of risk management. Craib contrasts high-volatility strategies with low-volatility approaches, highlighting the careful consideration that goes into leveraging positions.
"brilliant how how do you then transfer so let's say let's make it me and I I come up with through O'Shaughnessy Ventures I've got a team here that is doing exactly this and I think ah you know what I'..."
Institutional Reactions to Numerai
Craib reflects on the initial skepticism from institutional investors regarding Numerai's innovative model. He shares the journey of gaining trust and securing investments from significant players in the finance industry, including Canadian pension funds and Ivy League endowments. This segment highlights the challenges and successes Numerai faced in establishing credibility within the quantitative finance space.
"got it so um what is the output uh and first off so many questions uh like have you ever uh seen the output from The Meta model and just scratched your headed bot oh my God I I it's gonna be really ha..."
The Birth of Numerai's Cryptocurrency
Craib explains the origins of Numerai's cryptocurrency, NMR, detailing how the decision to use crypto for payments emerged from a user request. He discusses the advantages of using blockchain technology for staking and the unique features of burning tokens, which enhance trust and transparency in the platform. This segment highlights the innovative approach Numerai took in integrating cryptocurrency into its business model.
"allocators you have for Quant but it did take many years you could have a Quant that just hated so you could have an allocator that just hated Quant altogether I don't like anything Black Box okay the..."
Skepticism Towards the Crypto Market
Richard Craib shares his mixed feelings about the broader cryptocurrency market, expressing concerns over speculative investments and the risks associated with ICOs. He contrasts Numerai's approach to crypto with the pitfalls seen in the industry, emphasizing the importance of using cryptocurrency for practical applications like staking rather than speculative trading. This segment provides a critical perspective on the evolving landscape of cryptocurrency.
"got started with playing in crypto but the reason to make our own cryptocurrency was to do the staking I mean not many people had even heard about staking in 2017 when we started doing it um and it's ..."
Differentiating from Quantopian
In this segment, Craib discusses how Numerai differs fundamentally from Quantopian, particularly in its use of machine learning and staking mechanisms. He explains that while Quantopian focused on rule-based strategies, Numerai leverages advanced algorithms and a community-driven approach to enhance predictive accuracy. This comparison highlights the innovative strategies that set Numerai apart in the quantitative finance space.
"hope that the thing goes up after they buy it uh and numerai when we created NMR we actually gave away the NMR for free to the users on the platform and that was the right thing to do because we wante..."
Data and Optimization Strategies
Craib delves into Numerai's focus on data quality and optimization as critical components of their success. He explains the importance of having a robust data set and how Numerai's unique approach to data acquisition and model training allows them to remain competitive. This segment emphasizes the ongoing commitment to enhancing data capabilities and optimizing trading strategies.
"traded Traders uh so that made it less attractive as an investment um Switching gears many of our listeners probably would when they're hearing what you're doing the the first group that might come to..."
Exploring New Data Sources
In this segment, Craib discusses the types of data Numerai utilizes and the importance of historical data for machine learning models. He highlights the need for extensive data sets to train algorithms effectively and the potential for expanding into other asset classes. Craib's insights underscore the strategic focus on data as a key driver of Numerai's predictive capabilities.
"let's make sure we grow the data and the people talents um the final word is optimization this very it's a very subtle thing maybe you you know maybe if you take your feature exposure down to zero you..."
Integrating Language Models into Finance
Craib shares his interest in leveraging large language models (LLMs) to enhance Numerai's predictive capabilities. He discusses the potential for turning text-based data into valuable features for stock market predictions, indicating a forward-thinking approach to integrating advanced technologies into their existing framework. This segment highlights the innovative direction Numerai is exploring to stay ahead in the finance industry.
"do you think that there would be any value in kind of the traditional Quant sense you mentioned and I know much more about it now since by association with stability the the need for locked up data as..."
The Master Plan: Monopolizing Intelligence
Craib outlines Numerai's ambitious strategy to monopolize intelligence, data, and money, ultimately decentralizing the monopoly. He explains how an open hedge fund model can attract top talent without the traditional barriers of entry. This segment delves into the significance of creating a fair and transparent system that empowers data scientists while building a robust hedge fund that leverages collective intelligence.
"a really good job turning text-based data into incredible new features for the newer users that are very uncorrelated and different to uh what they have right now have you noticed any um inferences th..."
Navigating Challenges in Open Source Finance
In this segment, Craib reflects on the challenges faced by Numerai in the evolving landscape of finance, particularly regarding regulations and market access. He contrasts the barriers of entry for traditional hedge funds with Numerai's model, which democratizes access to financial markets. Craib emphasizes the importance of maintaining efficient markets and the role of innovative platforms like Numerai in fostering a more inclusive financial ecosystem.
"monopolize intelligence that's sort of talking about the Talent how can we be really quite definitively uh the the hedge fund was by far the most Talent and you might say well the way you do that is y..."
Empowering Data Scientists at Numerai
Craib discusses the relationship between Numerai and its data scientists, highlighting the unique structure that allows contributors to retain intellectual property rights over their models. He explains how this fosters a collaborative environment where data scientists can thrive without the constraints of traditional employment. This segment underscores Numerai's commitment to reducing friction in contributions and empowering talented individuals in the finance sector.
"I I love the way you break it down what could go wrong um I mean yeah it is this there used to be a lot of answers to their questions and then now to her uh you know we I do I do think the U.S it's it..."
The Future of Crowdsourced Intelligence
In this concluding segment, Craib speculates on the broader implications of Numerai's crowdsourced model for other industries beyond finance. He discusses the challenges of applying similar approaches to fields like healthcare, where data dynamics differ significantly. Craib emphasizes the unique nature of financial markets and how Numerai's model is tailored to adapt to the ever-changing landscape of stock trading, setting a precedent for future innovations.
"and I would say that some some startups I'll say Robin Hood I don't like Robin Hood but uh I'll just I'll just talk about Ramen that it's like you can't say you're democratizing access to the market a..."
Navigating Risks in Quant Finance
Richard Craib discusses the inherent risks in quantitative finance, emphasizing the potential for bugs in trading systems and the importance of precision in this high-stakes environment. He shares insights on how even minor errors can lead to significant financial consequences, highlighting the need for experienced engineers who understand the gravity of their work.
"participants and new data sets and new information coming into it so that's why it's especially good for this um and I don't think you should you could benefit that much from doing crowdsourcing on ot..."
The Challenge of Market Volatility
In this segment, Craib addresses the unpredictability of market volatility and its impact on performance. He explains how the dynamic nature of the stock market presents unique challenges for quantitative models, and discusses the difficulties in hedging against unforeseen risks, particularly during events like the COVID-19 pandemic.
"funds would say you met all that Justice guys we found a bug today in our trading system um but it obviously happening it's not like immune from all the problems of software and bugs so you know there..."
Understanding Drawdowns
Richard Craib reveals Numerai's current drawdown situation, comparing it to past performance during significant market downturns. He explains the implications of increased volatility on their strategies and how they manage risk while aiming for higher returns, providing a candid look at the realities of running a hedge fund.
"future data releases and uh and they might have heard performance uh for a little while not a lot but a tiny bit I mean these are three features at two thousand it's not like catastrophic um but it is..."
The Importance of Action in Ideas
Craib reflects on the difference between having a good idea and successfully implementing it. He emphasizes the significance of taking action and the challenges that come with executing innovative concepts in finance, showcasing his journey with Numerai as a case study in effective implementation.
"job you're an engineer but this is a danger a job and and you can't make mistakes so it makes for a very different environment to end up becoming a very slow hedge fund if you start caring at that lev..."
Life Lessons for Investors
In a thought-provoking moment, Craib shares two key ideas he would like to instill in the world: the importance of recognizing that 'life is long' and the reassurance that 'AI is not going to kill us all.' He discusses how these perspectives can influence decision-making and investment strategies, particularly in a rapidly changing technological landscape.
"the fact of running so much uh so much concentration uh you know even if we're not back to neutral we're still not immediate if another covered what coveted 19 would have happened that's actually a re..."
Coping with Market Hysteria
Richard Craib addresses the current climate of fear surrounding AI and its implications for society. He critiques the hysteria that often overshadows rational discourse and emphasizes the need for patience and understanding in the face of technological advancements, advocating for a balanced perspective on AI's role in the future.
"um one of the reason is the fact that we increased volatility on purpose right we we wanted to take on um more risk now that we're comfortable with the with the system so we've sort of in the past we'..."
Connecting with Richard Craib
As the conversation wraps up, Richard Craib shares where listeners can follow his work and insights. He invites the audience to connect with him on Twitter and explore Numerai's platform, encouraging engagement and discussion around the innovative approaches to finance that he champions.
"uh wow you you got me ready for a much bigger number than ten percent it's like us secretly no I know it's 10 sounds pretty chill if you've ever invested in uh venture or crypto or it doesn't bother y..."