NoSQL databases have become very popular. Big companies rely on them to store hundreds
of petabytes of data and run millions of queries per second. But what is a NoSQL database? How does it work, and why does it scale so
much better than traditional, relational databases? Let's start by quickly explaining the problem
with relational databases like MySQL, MariaDB, SQL Server, and alike. These are built to store relational data as
efficiently as possible. You can have a table for customers, orders,
and products, linking together logically: customers place orders and orders contain
products. This tight organization is great for managing
your data, but it comes at a cost: relational databases have a hard time scaling. They have to maintain these relationships,
and that's an intensive process, requiring a lot of memory and compute power. So for a while, you can keep upgrading your
database server, but at some point, it won't be able to handle the load. In technical terms, we say that relational
databases can scale vertically, but not horizontally, whereas NoSQL databases can scale both vertically
and horizontally. You can compare this to a building: vertically
scaling means adding more floors to an existing building, while horizontal scaling means adding
more buildings. You intuitively understand that vertical scaling
is only possible to a certain extend, while horizontal scaling is much more powerful. Why do NoSQL databases scale so well? Well, first of all, they do away with these
costly relationships. In NoSQL, every item in the database stands
on its own. This simple modification means that they're
essentially key-value stores. Each item in the database only has two fields:
a unique key and a value. For instance: when you want to store product
information, you can use the product's bar code as the key and the product name as the
value. This seems restrictive, but the value can
be something like a JSON document containing more data, like price and description. This simpler design is why NoSQL databases
scale better. If a single database server is not enough
to store all your data or handle all the queries, you can split the workload across two or more
servers. Each server will then be responsible for only
a part of your database. To give an example: Apple runs a NoSQL database
that consists of 75,000 servers. In NoSQL terms, these parts of your database
are called partitions, and it brings up a question. If your database is split across potentially
thousands of partitions, how do you know where an item is stored? That's where the primary key comes in. Remember, NoSQL databases are key-value stores,
and the key determines on what partition an item will be stored. Behind-the-scenes, NoSQL databases use a hash
function to convert each item's primary key into a number that falls into a fixed range. Say between 0 and 100. This hash value and the range is then used
to determine where to store an item. If your database is small enough or doesn't
get many requests, you can put everything on a single server. This one will then be responsible for the
entire range. If that server is becoming overloaded, you
can add a secondary server, which means that the range will be split in half. Server 1 will be responsible for all items
with a hash between 0 and 50, while server 2 will store everything between 50 and 100. Theoretically, you've now doubled your database
capacity: both in terms of storage and in the number of queries you can execute. This range is also called a keyspace. It's a simple system that solves two problems:
where to store new items and where to find existing ones. All you have to do is calculate the hash of
an item's key and keep track of which server is responsible for which part of the keyspace. Now, in this example, the range of 0 to 100
is a bit small. It would only allow you to split up your database
into 100 pieces at most. So, real NoSQL databases have much bigger
key spaces, allowing them to scale almost without restrictions. Besides great scalability, NoSQL is schemaless,
which means that items in the database don't need to have the same structure. Each one can be completely different. In a relational database, you have to define
your table's structure, and then each item must conform to it. Changing this structure isn't straightforward
and could even lead to loss of data. Not having a schema can be a big advantage
if your application and data structure is constantly evolving. At this point, it's clear that NoSQL databases
have certain advantages over relational ones. But that's not to say that relational databases
are obsolete, far from it. NoSQL is more limited in the way you can retrieve
your data, only allowing you to retrieve items by their primary key. Finding orders by ID is no problem, but finding
all orders above a certain amount would be very inefficient. Relational databases, on the other hand, have
no trouble with this. There are workarounds for this issue, but
only if you know how you're going to access your data. And that might not always be the case. Another downside is that NoSQL databases are
eventually consistent. When you write a new item to the database
and try to read it back straight away, it might not be returned. As I've explained, NoSQL splits your database
into partitions. But each partition is mirrored across multiple
servers. That way, a server can go down without much
impact. When you write a new item to the database,
one of these mirrors will store the new item and then copy it to the others in the background. This process might take a little bit of time. So when you read that item, the NoSQL database
might try to read it from a mirror that doesn't have it yet. This is not a big issue in practice because
data is replicated in just a few milliseconds. And if you want consistency, most NoSQL databases
do have that option. So, in summary: both NoSQL and relational
databases will be around for the foreseeable future. Each with their own strengths and weaknesses. So now you know how NoSQL works, let's look
at a few examples. Cloud providers heavily promote NoSQL because
they can scale it more easily. AWS has DynamoDB, Google Cloud has BigTable,
and Azure has CosmosDB. To give you another example of their scalability:
during Amazon Prime Day in 2019, Amazon's NoSQL database peaked at 45 million requests
per second. That's mind-boggling! But you can also run NoSQL databases yourself
with software like Cassandra (which was developed by Facebook), Scylla, CouchDB, MongoDB, and
more. Before ending this video, let's quickly talk
about the name "NoSQL." It's a bit confusing as it can be interpreted
in two ways. First up: "NoSQL" can mean "not only SQL,"
pointing to the fact that some NoSQL databases partially understand the SQL query language,
on top of their own query capabilities. And secondly, it's often called "NoSQL" in
the sense of "non-relational" because it can't easily store relational data. So that was it for this video. Please subscribe if you learned something
from it, and I hope to see you in the next video!
