
21 segments available
This presentation was recorded at GOTO Aarhus 2012. #gotocon #gotoaar http://gotocon.com Martin Fowler - Author, Speaker, Consultant & General Loud-mouth on Software Development ABSTRACT Martin gives a rapid introduction to NoSQL databases: where they came from, the nature of the data models they use, and the different way you have to think about consistency. From this he outlines what kinds of circumstances you should consider using them, why they will not make relational databases obsolete, and the important consequence of polyglot persistence. TIMECODES 00:00 Intro 00:42 History of NoSQL databases 07:51 Definition of NoSQL 09:36 Data model 10:04 Key-value data model 10:37 Document data model 17:36 Column-family data model 20:20 Graph data model 25:47 NoSQL and consistency 40:28 CAP theorem 44:40 When and why to use a NoSQL database 50:52 Is NoSQL the future of databases? RECOMMENDED BOOKS Martin Fowler & Pramod Sadalage • NoSQL Distilled • https://amzn.to/3ChIpu7 Martin Fowler • Patterns of Enterprise Application Architecture • https://amzn.to/3lp4sIq Martin Fowler • Domain-Specific Languages • https://amzn.to/3nzOIFk Martin Fowler • UML Distilled • https://amzn.to/3kahjyA Martin Fowler • Analysis Patterns • https://amzn.to/3Emabar https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #NoSQL #PolyglotPersistence #Programming #MartinFowler CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos & other perks: https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1
Martin Fowler introduces the NoSQL track at GOTO Aarhus 2012, emphasizing that this talk serves as an introduction to NoSQL databases. He highlights the importance of understanding the history and context of NoSQL to appreciate the practical experiences shared in later talks.
"so welcome to the track on nosql databases although it seems like we've already had a couple of tracks on nosql databases um my name is Martin Fowler um stenosi has hosted the track he asked me to kic..."
Fowler discusses the history of relational databases, their emergence in the mid-80s, and their benefits, such as data persistence and concurrency management. He also addresses the challenges developers face with relational databases, particularly the impedance mismatch problem, which complicates the integration of in-memory structures with database tables.
"going to talk a little bit about the history of nosql databases because as if with many things to understand why something is the way it is it's useful to know how on Earth it got there in the first p..."
Fowler explains how the rise of the internet and high-traffic sites like Google and Amazon necessitated a shift from traditional relational databases to NoSQL solutions. He highlights the limitations of scaling relational databases and how organizations began developing alternative data storage systems, leading to the NoSQL movement.
"complete dominance of certainly the Enterprise data space and plenty of other ones as well I mean we saw with the the science work at the large hon collider they didn't really want to use relational d..."
Fowler clarifies the origin of the term 'NoSQL,' which began as a Twitter hashtag for a meeting about non-relational databases. He discusses the accidental nature of the term's adoption and the challenges in defining NoSQL databases due to their diverse characteristics and history.
"movement now it's important at this point to talk a little bit about where this term nosql comes from a lot of people complain about it quite reasonably because they say well it's a really odd term tr..."
In this segment, Fowler outlines common characteristics of NoSQL databases, emphasizing their non-relational nature and cluster-friendly design. He notes that most NoSQL databases are open-source and have emerged from a 21st-century web culture, distinguishing them from older non-relational systems.
"writing a book about the subject if you it's important if you're going to write a book about about something to Define what it is you're writing about my conclusion is we cannot Define no SQL database..."
Fowler introduces two primary NoSQL data models: key-value stores and document databases. He explains the simplicity of key-value stores and the flexibility of document databases, which allow for complex data structures and querying capabilities without a fixed schema, while also addressing the concept of implicit schemas.
"models a little bit more so the most simple data model to talk about is that of the key Value Store the basic idea is you have a key you go to the database tell me grab me the value of this key the da..."
In this segment, Fowler delves into the similarities and differences between key-value stores and document databases. He points out that the boundaries between these two types of databases are often blurred, as both can handle complex structures. Fowler introduces the concept of aggregate-oriented databases, which allow for storing complex data as a single unit.
"here now it by having the no fixed storage schema does give you some options that you don't get with relational databases and and there is a difference and there are advantages in terms of flexibiliti..."
Fowler introduces column family databases, describing them as another form of aggregate-oriented databases. He explains how these databases organize data into column families, allowing for efficient data retrieval. This segment highlights the advantages of column family databases in managing complex data structures and their suitability for distributed systems.
"on what is important though is that both key value and document databases have this common notion of you're taking some complex structure that you can save as a single unit into the database whether i..."
In this segment, Fowler discusses the challenges associated with aggregate-oriented databases, particularly when it comes to querying data in different ways. He contrasts the ease of querying in relational databases with the complexities of rearranging data in aggregate-oriented systems. Fowler emphasizes the importance of understanding the aggregation structure for effective data management.
"picture so the great thing about this is that now when you're taking your aggregate in memory instead of spreading it across lots of individual records you get to store the whole thing in the database..."
Fowler concludes by contrasting graph databases with aggregate-oriented databases. He explains how graph databases excel at managing relationships and traversing connections between data points. This segment highlights the unique capabilities of graph databases and their specialized query languages, showcasing their effectiveness in handling complex relational data.
"to cover some of these models I've basically taken the document column family and key value and lumped them together under this aggregate oriented category and I think that's a useful abstraction at l..."
Fowler delves into the concept of consistency in NoSQL databases, contrasting it with the ACID properties of relational databases. He explains how graph databases often maintain ACID transactions, while aggregate-oriented databases may not require them as much. This segment emphasizes the importance of atomic updates and the challenges of maintaining consistency across multiple aggregates.
"relational approach so no SQL divides into those two c categories all of these are schemes so the graph databases as well allow you to add any bits of data to any node you have all that flexibility bu..."
In this segment, Fowler illustrates the challenges of conflict resolution in NoSQL systems, particularly when multiple users attempt to update the same data simultaneously. He discusses the importance of version stamps and offline locks to manage updates and prevent inconsistencies. This highlights the practical considerations developers must take into account when working with NoSQL databases.
"kind of what really matters but there's also a bit more to thinking about consistency even than that because even in a relational World acid transactions don't mean we get to be completely consistent ..."
Fowler explains the complexities introduced by data replication in distributed systems. He discusses how sharding and replicating data can lead to new consistency problems, particularly when multiple users interact with the same data across different nodes. This segment underscores the trade-offs between availability, resilience, and consistency in NoSQL architectures.
"probably have you'll probably also know how to solve this and basically use a technique which uh in one of my previous books I referred to as an offline lock um basically what that means the usual way..."
In this segment, Fowler elaborates on the trade-off between consistency and availability in distributed systems, using a hotel booking scenario to illustrate the potential for double bookings. He emphasizes that businesses must make informed decisions about these trade-offs based on their operational needs and customer expectations.
"same another thing however that's common to do with clusters of machines is to replicate data to put the same piece of data in lots of places this can be advantageous in terms of performance because n..."
Fowler introduces the CAP theorem, which states that in a distributed system, you can only achieve two out of three guarantees: consistency, availability, and partition tolerance. He clarifies that this choice is not binary and can vary across different operations, highlighting the importance of understanding these concepts in the context of NoSQL databases.
"sorry we can't accept any orders at the moment which is kind of bad for business this is one of the things that drove Dynamo they wanted to make sure that the shopping cart was always available you co..."
This segment focuses on how consistency is perceived differently in aggregate-oriented NoSQL databases compared to traditional systems. Fowler stresses the need for businesses to understand their specific requirements and how these influence the consistency and availability trade-offs, ultimately guiding their database choices.
"choice so this then ties into something you'll hear endlessly about whenever someone talks about this stuff which is the cap theorem everybody who's heard of the cap theorem how many people understand..."
Fowler identifies two main drivers for adopting NoSQL databases: handling large amounts of data and simplifying development processes. He shares insights from his experiences with organizations like The Guardian, illustrating how NoSQL can reduce complexity when dealing with natural aggregates, making it a preferred choice over relational databases.
"where we make these tradeoffs and if you want more um well I'm going to tell you to buy my book anyway so you know what to do okay so the last little segment I'm going to talk a bit about when and why..."
In this segment, Fowler addresses the integration challenges faced by organizations when using databases. He argues that encapsulating databases within applications can mitigate these issues, allowing for more flexibility in choosing the right database technology, including NoSQL options.
"databases because you've got you get get effectively getting rid of that impedance mismatch problem now of course that raises a question that was of course the promise of object databases they were go..."
Fowler critiques traditional data warehousing projects, which often fail due to their complexity. He advocates for a more focused approach to data analytics, where NoSQL databases can play a crucial role in aggregating data from diverse sources, including logs and spreadsheets.
"for no SQL databases to thrive that this is a good practice anyway even if you got relational databases you do not want to be integrating through integration databases that cause no end of trouble bel..."
Fowler introduces the concept of polyglot persistence, suggesting that the future of databases will involve using multiple database types, including NoSQL and relational databases, tailored to specific application needs. He emphasizes the importance of selecting the right database for the problem at hand.
"given a big advantage to the the Amazon because they're able to mine all this information so with all of this does this mean that no SQL is is the future of databases that relational databases are goi..."
In this concluding segment, Fowler differentiates between strategic and utility projects, advising that NoSQL databases may be suitable for projects critical to competitive advantage. He shares positive experiences with NoSQL implementations, reinforcing their role in the evolving landscape of database technology.
"you got to deal with organizational issues relational dbas are not going to like this in fact for some people that's a big Advantage but let's not go there no SQL databases are immature they don't hav..."