(upbeat music) - [Tutor] Databricks provides
a unified open platform for all your data. It empowers data
scientists, data engineers, and data analysts with a simple
collaborative environment to run interactive and scheduled
data analysis workloads. Databricks is from the original creators of some of the world's most popular open source projects, Apache Spark, Delta
Lake, MLflow, and Koalas. It builds on these technologies to deliver a true Lakehouse architecture, combining the best of data
lakes and data warehouses for a fast, scalable and
reliable data platform. Built for the Cloud, your data is stored in low
cost Cloud object stores, such as AWS S3 and Azure Data Lake storage with perform and access
enabled through caching, optimized data layout,
and other techniques. (upbeat music) To work with your data, you can launch clusters
with hundreds of machines each with a mixture of CPUs and GPUs needed for your analysis. If you're on a large data team, policies can define
limits on cluster sizes and configuration. There is a Databricks
runtime for data engineers and data scientists as
well as a runtime optimized for machine learning worklaods. (upbeat music) See how easy it is to create a cluster with up to 390 workers. In the data science workspace, you can create collaborative
notebooks using Python, SQL, Scala or R. (upbeat music) Just like you can share
your Google docs with your colleagues and groups of colleagues, you can also share these notebooks. Plus, built-in commenting
tied to your code, helps you exchange ideas and
updates with your collegues. (upbeat music) In addition to using notebooks for exploratory data
analysis as you see here, many Databricks users love
the powerful integration with machine learning
frameworks like MLflow. Here, we're training a
model and testing it. But we can also look up at the
top here and see the MLflow experiment tracking, which records the
previous experiment runs, and you can see important
variables like their accuracy. Now MLflow is just one of the
integrations that Databricks provides with popular
frameworks for machine learning and data science. Databricks also supports
a variety of other open source libraries, which
are popular in the community. Want to know more about
what data your colleagues have shared with you? Take a look at the data tab
where you can see individual tables with schema and sample data. Importantly, you see the history of operations performed on each table, (mumbles) the transaction log. Now why does history matter? Well, it's important for compliance and security audits in many industries, but it also enables you
to explore your data by another dimension, time. Let's see how by opening up
this SQL analytics interface. The SQL analytics interface
gives us the ability to create visualizations and dashboards as well as query our Lakehouse
with performance exceeding or comparable to
traditional data warehouses. We achieve this level of
performance, reliability, schema enforcement, and scale through advances in
Delta Lake and Delta Engine. Delta Lake is an open format storage layer built on top of Parquet, which adds ACID transactions
to your Cloud Data Lake. Let's show you how the transaction log enables Delta Lake Time Travel. Here, we're looking at a
series of loan risk scores based on where a property is located. When we originally created this dataset, in version zero, we didn't
have any data for Iowa. We didn't have any loan
applications there, but as time went on and
we reached version 40, you can see that Iowa is populated with a loan risk score of eight, probably signifying that in
the middle of the country, there's a little bit less in
terms of natural disasters. Now let's show you the SQL
that powers these queries. And here you can see that
we very simply have added a version number into
our SQL query to indicate when we're querying the data from. This uses the Delta
Lake Time Travel feature in order to find the data at a particular point in time. Well I hope you've seen
how simple and powerful Databricks can be for
your entire data team. Whether data analysts, data
engineers, or data scientists, they can collaborate together to do their data plus AI on Databricks. Learn more at databricks.com.
