- Hello everyone. Thank you for coming
on a Thursday afternoon to Mandalay Bay. My name is Sumit Thakur. I'm a senior manager of product management for Amazon SageMaker at AWS. And today I have with me my
colleague Giuseppe Porcelli, who is a principal ML
solutions architect at AWS and Marc Neumann, who is a product owner for the machine learning and
AI platform at BMW Group. And together we are going to talk about how to scale your
machine learning development using SageMaker Studio. Thank you, gentlemen. Katie. So before I begin talking about
how to use SageMaker Studio to scale your machine
learning development, I wanted to do just a quick recap. So SageMaker Studio was launched
at re:Invent back in 2019 and it was launched as the first fully integrated
development environment for machine learning. It brought together everything
that you need for ML into one single unified visual interface, which means you have purpose built tools for every step of the
machine learning workflow accessible within the
same visual interface. So tools like data labeling,
feature engineering, building models, using code editors, and then for tuning, deploying, and managing the end-to-end
machine learning workflow, you can find the tools for every step in the same visual interface. This also means you can
quickly move back and forth between these steps, make changes, observe the
outcomes, and iterate fast to quickly roll out your new
machine learning capabilities in your organization. And today, tens and thousands of customers are using SageMaker Studio, which gives us a highly
privileged opportunity to work with these customers
to understand the trends that impact the machine
learning developer experience. And I'm going to discuss
some of these trends. So one of the first trends
I want to talk about is just the increasing
pace of AI adoption. One of the key indicators of AI becoming pervasive in our economy is just the growth in AI investments. The AI Index Report published by Stanford
University early this year estimates that the global
corporate investments in AI grew by a staggering 13
times over the last decade. And the trend is not stopping; it's going to accelerate further
because of generative AI. What it means for developer experience that developers are asking for tools which are highly performing
and makes them more productive in quickly getting from
data to model to insights. The next trend I want to talk about is the proliferation in data and AI-specific job roles
and specializations. Given that machine learning is so complex, it's highly unlikely that one person possesses
all the skill sets needed to develop and deploy a model. For example, a data
engineer might specialize in pulling together data from
multiple different sources, integrating them together, and then transforming
them into a form and shape which is useful for analytics
and machine learning. While a data scientist may specialize in doing feature engineering
on top of this data and then choosing algorithms to build and experiment with models. And then a machine learning engineer might have the skill
sets to take this model, deploy it into production, and automate the process of deployment and retraining the models at a cadence. Given this rapid proliferation
in AI-related roles, more and more machine learning developers are asking for tools
which are purpose built for their specific task. And the third trend I want to talk about is the continued impediment in taking your machine learning models from prototyping to production. So Gartner survey estimates that only a little over 50% of the models go from prototyping to production. Now the tools, the skill sets, and the processes for building a model are substantially different from that of deploying the model. And developers are asking for tools which are more self-serve, so the model builders can
quickly take their prototypes into production. To make it easier for all of us, I'm going to summarize these key trends in the form of four Ps. The first two is performance
and productivity. As I said, to keep up with
the pace of AI adoption, developers need tools
which are highly performing and make them productive by removing any
undifferentiated heavy lifting out of the way. The next P is preference. Given the proliferation
of data-related roles, developers need tools which are purpose built
for their specific tasks. And finally, productionization. Model builders need more self-serve tools to take their prototypes into production. And in order to address all these four Ps, the four key pillars of
the developer experience, I'm super pleased to announce the launch of next generation
of SageMaker Studio that comes with a blazing-fast
startup experience, an even broader selection
of the code editors, AI-based assistance for all your ML tasks, and new tools to automatically convert your machine learning code into pipelines. Some of these tools were launched
in our leadership keynotes over the last two days, with the latest one,
being the Code Editor, which was launched in Werner
Vogels' keynote this morning. We are gonna do a deep dive
on each of these innovations one by one. First, let's talk about the performance, on how the new blazing-fast
startup of Studio meets the performance standards of a modern day machine
learning developer. We have done a couple of things to improve the startup experience. So first of all, the
initial setup of Studio has been significantly simplified, Which means in one single click, you can now set up a
SageMaker Studio domain, with IM-based identity
and access management, fully managed connectivity
to public internet, and underlying compute and storage for building your machine learning models. Setting up studio is
now literally as simple as flipping a switch to
turn on a smartphone. Now once you have set up the studio, you can launch Studio with a simple click and it loads up the
application in your browser in less than five seconds, giving you instant access to
all the tools in your browser. Now one of the first
things you might want to do is to start writing code, and for which you want
to spin up a code editor. So what you do is you go and
you create a SageMaker space. A SageMaker space is
like a workspace in cloud that stores your compute,
storage, and runtime preferences for your development environment. The space comes with many capabilities. For example, you can stop a space, come back to it later, restart it again, and resume your work
from where you left off. All your work is automatically saved on the underlying EBS storage. You can also scale up and
scale down the compute and the storage of the
space with a simple click. So for example, you could start space on a lightweight CPU instance when you are doing data exploration and then immediately flip
to a beefier GPU instance as you begin your model training. The space also comes
with several capabilities to share and collaborate
artifacts with your team members. So you can attach a shared
file system like Amazon EFS, onto a space to share your code, datasets, and artifacts with your team. You can even create a shared space in which you and your
peers can come together and review and co-edit
the files in real time. Once you have the space set
up, you can launch an IDE such as JupyterLab or
RStudio from the space. The JupyterLab loads up in your browser in couple of seconds, and it comes pre-configured
with SageMaker Distribution. SageMaker Distribution is a data science and machine learning runtime, which comes pre-configured with
several popular data science and machine learning packages so you can quickly get started with writing your machine learning code. And we are gonna give you a
demo of this entire experience. But however, I'll show
you a couple of slides about how the visual
experience looks like. So this is the homepage of
the new SageMaker Studio. You can see on your top left, there's an applications gallery from where you can pick up a
code editor of your choice. Let's say you pick up JupyterLab. It immediately opens the model
in the center of the screen to help you create a JupyterLab space. All you need to do is to select
the compute for your space, storage for your space, and the runtime. And you can see defaults
for each one of them, where the default for runtime
is the SageMaker Distribution. Once you have made these choices
or reviewed these choices, you can quickly launch the space and get started with writing your code. You can run multiple spaces in parallel, each with a different combination of compute, storage, and run times if you're working on
multiple different projects or multiple different
experiments at the same time. You can come back to this page, review all your running spaces, terminate a space, stop
a space, resume a space. You can do all of that from
this one central place. And as I said, we'll
walk you through the demo of this experience very soon. Let's talk about the next P, preference, where developers are asking
for their preferred tools for their specific tasks. As you saw, SageMaker
space comes with support for both JupyterLab and RStudio, but our customers have been telling us they need more choice. They need IDEs with
much more robust support for debugging, refactoring,
and deployment tools. And one of the IDEs they
have been asking us for is VS Code. So I'm super excited to announce the launch of new code
editor in SageMaker Studio built on top of the Visual
Studio Code open source. This is built by AWS and it uses the Visual
Studio Code open source code, which means all your familiar
VS Code shortcuts, terminals, and debugging and refactoring tools work in Code Editor. It also comes pre-installed
with an extensions marketplace powered by Open VSX from
Eclipse Foundations, which gives you access to more than 3000 VS Code compatible extensions. And one of the popular
extensions pre-installed in Code Editor is AWS Toolkit that gives you an easy access
to many of the AWS services like Lambda, S3, Redshift, CodeWhisperer. So you can quickly get started with building and deploying
applications on AWS. And finally, similar to JupyterLab, the VS Code runs on a SageMaker space, so you get all the benefits of the space. You can attach a shared file system, you can resize compute
and storage on the fly. It comes pre-configured
with SageMaker Distribution so you can quickly get
started with writing code. Now with the broader choice of IDEs available within SageMaker Studio, your machine learning workers can come in and pick and choose a tool of their choice for the task at hand. As an example, a data
scientist might come in and pick up JupyterLab
IDE from the app gallery, an experiment tracking
tool from the tool gallery, and then begin building and
experimenting with models while a machine learning
engineer might come in and pick up the the VS Code
IDE from the app gallery and a pipelines tool from the tool gallery to start deploying and
automating model pipelines. Next, I'm going to talk about
the third P, productivity, where developers are asking
for more efficient ways to get from data to insights. The new SageMaker Studio comes built in with generative AI-based
assistance for all your ML tasks. Your editors like JupyterLab and VS Code both come pre-installed with automated code generation
powered by CodeWhisperer, Which means you can now generate Python and data science code by simply adding a comment in your IDE in plain natural language. You no more have to go
about spending hours in looking up code
samples from the internet, debugging those samples, remember API syntax, you can just get started
coding in the IDE. In our internal studies,
developers who use CodeWhisperer were found to be 57% more productive than the ones who didn't. The IDEs also come built in
with an AI-based chat companion. So you can have a natural
language conversation with the chat companion to do
tasks beyond code generation. So you can ask for advice; for example, to explain a code sample, to debug an error, or to
even ask recommendations to refactor your code. The chat companion is very
powerful and highly customizable. You can pick and choose
the foundation model that runs and powers the chat companion. So for example, you could choose to run Claude V2 model developed by Anthropic available on Amazon Bedrock, or you can choose to run StarCoder developed by Hugging Face available on SageMaker
JumpStart model hub. The JumpStart Model Hub comes with now more than
150 open source models for multiple different tasks, and all of those models
can easily be deployed from Jumpstart interface
onto SageMaker endpoints, and from there, they can be used to
power the chat interface. So you can play with those
models right in your IDE. It's a pretty fun experience, so I would definitely encourage
everyone to give it a shot or try out different
models in the IDE itself. Next I'm going to talk about the final P, productionization. On how to build tools
which enable model builders to self-serve, get their
prototypes into production. (Sumit sniffles) Here we have done a couple of innovations. We have significantly changed
the SageMaker Python SDK. We have introduced a new set
of decorators in Python SDK using which you can
annotate your Python code and automatically run that
code as a job on SageMaker. The decorators automatically
determine the runtime needed to run the code as a job
based on a requirements file or a YAML file that
you provide at runtime. What this means for you
is you can take a code that you have written on your laptop and have been debugging for a while and you are satisfied with the code and quickly scale it out to
a beefier instance on cloud without making any code changes, just by adding decorators. And this works not just
for one Python function. If you have multiple Python
functions in your code where each function is
designed for a specific task, you can stitch them together
into a pipeline on SageMaker. All you need to do is to
annotate those functions with a step decorator in Python SDK. If you're a notebook developer and want to run your notebook as a job, it's even simpler In SageMaker studio. You go to SageMaker Studio,
choose the notebook, and you can create a schedule for running this notebook as a job. And SageMaker automatically
triggers the job, determines the dependencies which were used in authoring the notebook, and then uses it to create a container and run this notebook
as a job on SageMaker and pushes all the outputs of the job back into SageMaker studio for you to review and monitor. You can even orchestrate
multiple such notebooks in the form of a pipeline. You can actually create a DAG of many of these running notebooks. So for example, you might have a notebook which takes in data from S3, performs some feature engineering, another notebook which
takes these features and performs a hyper parameter tuning while another notebook which takes model candidates
and do model evaluation and generate evaluation reports. You can stitch them
together into a pipeline using the new SageMaker Python SDK. And finally, we have
dramatically simplified the model deployment process. Now you can deploy a model onto SageMaker in just two simple steps. The new model builder class
in SageMaker Python SDK allows you to pick up a model artifact and call out a function model.build where SageMaker automatically
determines all the ingredients for deploying this model onto an endpoint, which includes determining
what inference server to use. Could be a Triton server,
could be a Tensor server, depending on the type of model artifact. It determines what would
be the appropriate compute for running this model. It also automatically
generates the code scripts, which acts as a glue in passing the incoming inference requests into a form that is suitable
for consumption into the model and packages the response
generated by the model into a rest API output. And then SageMaker takes
all these ingredients and deploys them onto an endpoint which is high throughput, low latency, and a fall-tolerant endpoint. Now to give you a demo of all
of these awesome capabilities, I'm going to invite
Giuseppe back on the stage to help us with a live demo. - Thank you. Thanks, Sumit. - Thanks, man. - So before we get into the demo, let me spend a couple
of words on the use case that we are trying to solve today. So we want to solve a
pretty simple problem, which is detect failures in
machinery with machine learning. Well, it's a hard problem, but it's simple based on the data that we're gonna work with today. We are gonna choose the AI4I
predictive maintenance dataset from the University of California Irvine Machine Learning Repository, which is made of a few features that you can see here on the slide. Like the type of machinery, temperature, the rotational speed,
the process temperature, and well, the variable
that we want to predict is the machine failure variable, which is a binary variable. So as you can imagine, we want to train a binary
classification model here to try to address this use case. And what we're gonna do
is use SageMaker Studio to build, train, deploy
the machine learning model. Plus we will use Amazon
SageMaker pipelines to orchestrate the end-to-end, the entire machine learning workflow. Great. Let me move to the demo. As Sumit mentioned, we have extremely simplified
the onboarding experience to SageMaker Studio and you can literally onboard
to Studio in one click. Sorry. If you click on this button
here, set up for single user, what will happen is that
a SageMaker Studio domain will be configured automatically for you with a single user profile being created with a set of default configurations with an Amazon SageMaker
full access execution role with default networking
configurations and so on. And you can soon get
started with this domain. Otherwise, if you want to set
up Studio for organizations, we have also simplified that process with a wizard-based user interface that allows you to customize how SageMaker Studio has to be configured in your organization. Great. For this example, we already
have a pre-created domain, just this one. Let me get into the domain and then open SageMaker
Studio for this specific user. Great, we are now in
the new SageMaker Studio user interface. As you can see, we have the
application gallery on top left, the left navigation, and the landing page here,
the main screen here, where you can choose whether you want to work with
JupyterLab or Code Editor. And you also have the getting started site of resources accessible. Since we want to train this
binary classification model, we want to solve this problem. As a data scientist, the
first thing that I want to do is use JupyterLab to explore my data, to pre-process my data, to then build a machine learning model. So I'm gonna create a
JupyterLab space here. Let's give it a name. And the space will be ready in a second and we will be able to
customize the configuration as Sumit mentioned. So we can configure the storage that we want to allocate
to this specific space. We can choose to add
lifecycle configurations to customize the way the
environment is configured through shell scripts, which are executed at
bootstrap of the instance of the underlying instance. We cannot touch a custom EFS file system as well as we can choose a
variety of computer resources including CPU and GPU instances as well as fast launch instances which will run in in less than a minute. So let's select one instance for example, and we use as the image,
as the runtime image, the SageMaker Distribution
here, which is set by default. Now let me run this space and I want to spend a couple of words more on the SageMaker Distribution. Back in May, we launched the Jupyter current SageMaker
Distribution project. This is an open source project. What we have done is we have put together in a single distribution, the most common frameworks and libraries for machine learning. We make sure we have built this
as as a current environment. We make sure that the dependencies and the versions are correct, let's say in this current environment. And also we have
delivered the distribution in the form of a Docker container. So you can go to GitHub, even
pull this Docker container, even try using the SageMaker
Distribution locally. The SageMaker Distribution powers both JupyterLab applications and the Code Editor spaces
in in SageMaker Studio. So this is pretty important because we can move from
one environment to the other using the same run time. So keeping the run time consistent without having to make sure
that we customize the run time when we move from one place to the other. Great, as you can see, in less than a minute minute
we have our space ready, but I have already a
space with all the code that we need pre-created. Let me open JupyterLab here. So JupyterLab loads, as I said, this is a repo that have cloned
for this particular example. Let me get into the first notebook. As a data scientist, as I said, I want to make sure that
I can explore this data, pre-process this data,
then fit a trainer model. As I mentioned, the SageMaker Distribution already contains the most
popular frameworks and libraries. So here I want to work
with Pandas, XGBoost and scikit-learn. What we're gonna do is we're gonna use scikit-learn and Pandas to transform the data, to do the data exploration
and pre-processing. Then we are gonna use XGBoost to create the binary classification model. You can definitely customize
the runtime environment, installing additional
libraries as you want to, and in this case, we are installing a visualization
library, which is Seaborn. Then we download here the dataset from the University of
California Irving ML Repository and we start with some
exploratory data analysis. I mean we can run some
of these cells here. What you can see here, we are just taking a look at
the dataset then describing it, then checking the value counts
for the machine failure, so how many positive
versus negative examples we have in this dataset. Then we are plotting this information, standard plotting operations. Then at some point, what
I want to show you is, hey, I want to compute unique
count of values for columns in this data frame, and maybe I don't remember the
API that they need to call. And what we can use is
Amazon CodeWhisperer, which is running on the JupyterLab now, which is providing us the
suggestion for this function. Another thing is I want a function that removes a column from a data frame. Okay, so here we see the
CodeWhisperer suggestion, I can accept these suggestions
and I have my function ready. So it's pretty powerful tooling that can help us developing
within the notebook itself. Great. Let's move on here. What we are doing is we
are removing some columns that we don't need for
this specific problem, we don't want to include. Then we are plotting the variables to spot potential correlations
using Seaborn here per plot. And then we get into the
data preparation code, the data pre-processing code. So this code is defined as
a standard python function as you would expect, which is called pre-process, which takes us input our
data frame with all the data and does a few operations
like doing training, validation, and test splitting. Then it applies some
transformations using scikit-learn, specifically we are scaling
numerical attributes and we are one of encoding
the categorical ones to then fit the the model later on. And then this function
returns, as you can see, the transformed data, so
the pre-processed data plus the model itself, this featurizer model
made with scikit-learn, which is able to transform our data. So I can run this function and yeah, well, we have our new model the
featurizer model here, this is the representation of this model. And we can look at the data
that has been transformed, standard pre-processing operations here. Then we move into the model training. Here again, I have a train function which takes as input to
the pre-process data, so the training and validation data plus a few hyper parameters for at the XGBoost algorithm. And then it fits the XGBoost model. That's what it's doing here. We are training here the
model, the XGBoost model. The return of this function, the return value of this function, is the XGBoost model. Great, we execute the
function we are fitting, we are also computing some metrics like the accuracy, precision, recall, which are pretty convenient
when we are working with a binary classification problem. Great. Now last thing that you
would do as a data scientist maybe is using these two models to run some inference on the test dataset, which is what we're doing
here in this function. Great. Now let's say we are satisfied for what we have done
here in the notebook, and we want to scale this code on different compute infrastructure. Why would I want to do that? There could be multiple reasons. Is because maybe my dataset size increases and I want to use a larger instance or a set of instances to train
my machine learning model or maybe because I want to automate the execution of my code. So I am in a notebook, but I have to make sure
that my code can also run as part of a pipeline, for example. Great. How we can do this. There are at least a
couple of options here. One option is use the integrated notebook
scheduling functionality, which is available in
JupyterLab environment. So what you can do here is, hey, I want to schedule the execution of this full notebook
either now or on a schedule. So either I execute it on
demand or on a schedule. And this runs in the
form of a SageMaker job. So we run on SageMaker infrastructure. Okay, the other option, which
we're gonna show you here, is the remote decorator
that Sumit mentioned. So what you can do is
with one line of code, we are importing here the decorator and decorating the function,
the Python function. What we do is we run this code and then exactly the same function that you're running here
in the notebook is moved, is executed as a SageMaker training job on separate infrastructure. So what the decorator is doing
is it's taking this function, is understanding the configuration of your runtime environment, and then executing this function as a SageMaker training job. If I want to monitor the
execution of the job, I can go back to SageMaker Studio, I can go to Job Screens, look at the training jobs here, and as you can see there is
one job that is executing, which is these pre-processing jobs, is the pre-processing code, is now running as a separate
job on separate infrastructure via SageMaker training. Great, you can look at the information of these jobs like the input artifacts and some other configurations as well as what I want to show you is that the runtime environment here has been picked automatically. So this is the SageMaker
Distribution again that has been used as the Docker container to run our training job. So the runtime environment
consistency here is on both the JupyterLab environment, the Code Editor environment,
we'll see shortly, as well as the jobs. So when you move to the jobs, keeping the environment consistent allows you to reduce the amount of work that you have to do in terms of configuring this run time. Great. Our jobs are running, we don't have to wait for the execution. Now what I want to do is, let's say that I have
executed my two jobs, one for data pre-processing, the other one is for the
actual training of the model, and I have my two models, the featurizer model and
the actual XGBoost model that does the classification. What I want to do now is I
want to work with a ML engineer to deploy this model to
a real-time endpoint. What can I do is I can go
back to SageMaker Studio and go open Code Editor now. So I'm not going to create
a space in this demo, it's the same procedure that we have seen for the JupyterLab. We just opened Code Editor, that is already running in my environment, and we will see the Code Editor
environment loading here. So, great. Let me open, well, I can do it from here. Let me open the specific folder that I have on my user home directory with this repo again cloned and we move to the deployment module. Great. I have defined deployment
as a standard Python script. The deployment procedure at high level consists of the following steps: we load the two models to memory, the featurizer model
and the XGBoost model. Then we create two SageMaker models. So a SageMaker model
represents some extra metadata that you need to deploy the
model to a real-time endpoint. To build the SageMaker model, we will use now the
new model builder class that has been introduced by Sumit. Then what we do is after we build the
the two SageMaker model, we combine the two models, is what is called the
serial inference pipeline in SageMaker, which is just one of the ways to deploy machine learning models to real time endpoints in SageMaker. Why I want to do that? Because I want to make
sure that the inputs that will provide to the first model will be just a raw data that come maybe from the field. Then this model will transform the data and then pass to XGBoost model
for the actual inference. And finally we deploy the model. What I want to focus on specifically is how a model is built. So let's look at the
new model builder class. The model builder class takes as input the actual XGBoost model. Then we tell a few other parameters like the requirements that
are required for inference, and the most important thing
we pass the schema builder. The schema builder defines through an example input and output. Through the schema builder, we are able to automatically compute the marshaling functions for your model. So how data has to be
serialized and de-serialized. At the same time, you can customize this
through extra classes. In this case, I'm using
the Request Translator because I also wanted to farther customize the way the inputs are
provided to my model. You might be asking, "Hey, what happens if I want to customize the way the actual inference happens?" I've done this for the scikit-learn model where by narrating the
inference spec class, you can also customize the way the actual inference works
through the model builder. Why? Because for the scikit-learn model, I didn't want to call the predict method. I wanted to call the transform
method to transform our data. Great. That said, what we can do is
we can open a new terminal, we go to the Deploy folder
and we do python deploy.py. What's happening here is
the model builder class is packaging our models, is understanding automatically the configuration of the container images that we need to use for inference. Is setting the appropriate configurations for the serving stacks, the
underlying serving stacks, and is now deploying the
machine learning models. So there is an end point
indeed that we are creating, which is called SageMaker
BTD endpoint plus a suffix. How can we verify this? We move back to SageMaker to the UI, we go to the endpoint section, and we see that there is
an endpoint running here that is in creating state. So this is the endpoint that
we are building right now. Great. At the same time, if we
want to test some inference, I'm taking an endpoint that
that is already up and running. What we can do, we can
do minus run inference, which is an extra argument that I've provided to this script to execute some inferences
and see the results for a positive and negative example for our predictive maintenance problem. So based on the attributes, I believe it's the rotational speed which is influencing mostly here. We are getting a positive
and negative result here. Great. So we have seen how to
do that exploration, pre-process data, build the model. Then we have done a model deployment step. Now we want to stitch all
together in a SageMaker pipeline. So how can we do this
using the new remote, sorry, the new step decorator that has been introduced by Sumit? Let me move to the workflow
here and open the pipeline. But before going into
the pipeline details, how the pipeline is defined, I just want to show you the steps that we're going to create. So the first step is, as I
said, the pre-processing step, as you can realize, this
is exactly the same code, exactly the same pre-processing code that we were running the notebook. It's the same function that I
have moved in a Python script. Then we have the train function. Train function again,
it's the same function that we were running in in the notebook, moved in a Python script, no specific SDK code to be written here to define the step itself. Then I have an evaluation function which is very similar to the test function that we had in the notebook, but we are also returning
an evaluation report from this function because after we train these two models, before deploying, we
want to store the models in the SageMaker model registry and we want to attach an evaluation report so that I can then access
the model registry, look at the specific version, and check the performance of my model. Finally, we have the register model step. This step consists of, first of all we want to
create the two models, scikit-learn SageMaker model
and 8XG boost SageMaker model. These are the same functions that we used for deployment,
it's the same code. And then we put together
these two models here as a pipeline model and we register the model in
the SageMaker Model Registry. attaching the evaluation
report, which is here. Last step in the pipeline
is about the deployment. This is very simple. What this is doing is taking the model from the model registry and deploying to a SageMaker endpoint. How do we put all these steps together? Now how do we stitch it together? We have to take a look at this function which create the steps. What we do is we are
using the step decorator. A decorator is a Python function. Here we are not using the
decorator with the annotation, let's say syntax, but we are using the
decorator as a function and the reason is that we are
implementing a modular code. We are engineering our code so we are implementing modularity here. We are passing the various functions that we have seen so far: pre-process, train,
test, register, deploy, to the decorator. Plus we are making sure that the outputs of the previous step are passed to the subsequent
steps accordingly. So with standard Python code. Finally what we do, it's a
standard SageMaker pipeline code, we create a pipeline, we pass
the steps to the pipeline, and we define a few parameters like eta, max_depth, and deploy_model, which is a Boolean variable to decide whether we want
to deploy the model or not. And then upset the pipeline. We insert the pipeline
definition in the system and we start the
execution of the pipeline. Right. Let's see how this works. So what the decorator
is allowing us to do now is to convert the Python
functions that we have seen into SageMaker pipeline step automatically without worrying too much
about writing extra SDK code. We are now adding the
pipeline into the system and starting its execution. And if we go back to SageMaker Studio, what we can do is look
at the pipeline screens. This is my pipeline. We have one which is in executing state, which we started right now. Then we can take a look at
the graph of this pipeline, which is indeed a
pre-process train evaluation model registration in the model registry, and finally deployment of our model. If we take the specific
execution, we can see where it is, now, it's in the pre-processing stage. And we can also take a
look at the parameters that were used to execute this pipeline, plus additional information
and additional metadata related to the specific execution. Last thing that I wanted to show you is about the SageMaker Model Registry. So one of the steps of the pipeline would register a new model version, a SageMaker Model Registry
so that we can decide whether we want to promote
this model to production by comparing this version maybe with the previous
versions of the model, we opened the so-called ModelPackageGroup, which lists all the versions of the models that they have produced so far. We take for example this version 10 here and here you can see the valuation report that I had attached to the
model in the model registry. So with the accuracy metrics
for the XGBoost model. As well as you can look, for example, at how this model would be deployed for realtime inference. So all the information about containers, environment variables, how the serving stack has to be configured have been applied automatically through the model builder class. So great. This concludes my demo. And I'm very happy to
invite here on the stage, Marc Neumann from from BMW Group. Will tell us how BMW is a scaling machine learning development using Amazon SageMaker Studio. - Thank you, Giuseppe. Did you know that BMW Group is a multi-brand automotive actually? Of course we are known for
BMW cars and motorcycles, but there are also other brands. There's like uplifting
lives with our mini brands and the Roy Royce luxury automotive base for BMW group as well. And did you know that in North America, BMW Group is actually the
largest automotive exporter? So there's a plant in South Carolina, I'm going to be there next week, and they're producing there the SUVs, the X Series vehicle models, and distributing them worldwide. Of course a large proportion
is being sold in the US, right? But they're distributing this worldwide. So BMW Group, my team is responsible for building on providing the AI and machine
learning platform services, and we are working a lot with
different business segments to help them build the
actual AI solutions. And for example, there's one team, we work real close with them, they're responsible for the
automated quality inspections in our worldwide plans. So as well in Spartanburg. And what they managed over the last years, they ramped up a global solution that achieves this high
quality that we need, these high-quality
inspections in the plants. And what they actually do is
they equip the production lines with sensors like video
cameras, acoustic recordings, other sensor measurements. They do those recordings and they use AI models
basically to evaluate. So what you can see with
the picture right there is a camera pointing
towards the production line, and every vehicle that's
passing, an image is taken. And in this case it's
about presence detection. So this AI model that is getting the image is checking for the path
that is highlighted in green, whether it's there or not. There are other use cases
checking for the right part that's been used, checking
for any anomalies, right? Or checking for distances
between those parts. And why is it relevant, right? Because they doubled the number of models over the last years each year and now they have hundreds of
models in the production lines that need to be trained and developed. And then there are any changes; for example, regarding
lighting in the plants or the production setup changes, they need to retrain those models. It's quite some effort. And for this they need a
capable environment to do so. They would like to focus
on their model development not on setting up the environments. And that's where we come into play. And for use cases like them and the growth that we
have at the company, we needed to redefine our approach. And that's the story I wanted
to share with you today, how we did that. But to understand, that's not the only segment
where we benefit from AI. There are more like just some examples. So for example, in product development, we are using machine
learning and AI as well. We have to design the features and models that our customers admire. For example, what we can use, with machine learning there, is to identify the control levels that have an effect on the performance of our vehicle functions, yeah? Just for explainability. It's not about using
the model in production, it's about understanding
basically the control levels. There are many other cases of course. And in supply chain and logistics, it helps us to predict
our demand more accurate and reduce the planning efforts. It's very important in these times, looking at the last years with the semiconductor
shortage, for example, to manage these scenarios. Vehicle production, we've
seen the quality inspections. There's more like predictive
maintenance, for example, in vehicle production. And then in sales and after sales, it helps us to improve the
interactions that we have with our customers. And even for our workforce, AI helps us to be more efficient, and what's important in a large enterprise is for like employees, it's like to find the right information. And of course you've seen all those generative AI rack use cases here at the conference. This of course is also something that we are trying to leverage. Yeah. To scale AI in those business processes, we believe that there's a strong platform so that the individual use case teams can actually focus on their use case and don't have to do all this setup work. And that's why my team is
providing these platform services. And before we do this deep dive on machine learning AI development, I wanted to give you a very big overview on the platform services that we have, how we look into scaling. So one thing we have is
AI business services. It's basically a cluster on services that are not domain specific but can be used in all the domains. So for example, there's an
API on doing text translation, an API on extracting
information from documents. And what's good with that is they don't need to reinvent the wheel, they can simply use the service, integrate in their
processes, and all it's good. And then there's more the
custom machine learning AI part, like we've seen with
the vehicle production, they need to build AI models. And for this, we provide the
tools for like data scientists and business domain experts as well, and we dive deep on that. And then we have the management
for unstructured data like images and audio data. We need to record it, manage
it, and provide labeling tools and labeling services
to annotate the data. And then for those
digital data assistance, they benefit from a platform where they can develop those assistance and operate them in a safe way. And with ML ops, we provide
the best practice solutions to industrialize the AI
models that we have developed. We've seen in a demo that there will be more smooth integration between AI development and production. This is something that
we are going to leverage in there as well. And then the last part, which is especially becoming
more and more relevant now these days, it's like governance for AI. So we need to have this overview
on our AI use cases, right? We need to know where
they are, what they use. And we have central repositories
for all the AI models to have like a safe
revision-proof storage for it and all the metadata associated with that. And we even have now
risk assessment processes to assess the individual AI use cases. So that's like the
overview on the portfolio, but today I wanted to deep dive on the machine learning development part, like the code centric part for data scientists,
machine learning engineers. And for us, like way back, it started all with specific
purpose environments. So looking back like seven years ago, we've seen like an increase on use cases that work with unstructured
data, with images, with audio data, with texts. And what they mostly did is they took larger pre-existing models and did transfer learning to our data to solve those use cases. And what they needed is
sufficient GPU performance, GPU memory, in decent amounts. And that's why we ramped up this deep learning platform for them and they could use
Jupyter notebooks there, terminal in a containerized environment that we've run on premises
in our data centers and manage this environment for them. And then there came those use cases that wanted to use more sensitive data and then we saw the limits with the quantum platform approach and it ramped up a new platform which provided the necessary
level of security for them. And then there's of course
many data scientists, machine learning engineers
that work locally, do their experiments there and many different other local solutions with other software vendors. And the challenges that we had with this fragmented approach is of course it's like the
multitude of operational and governance overhead for us in the end. And for users as well, if they're working on different use cases or different aspects, they had
to use different platforms. Yeah. We had real challenges
to manage the compute demands. Yeah, because we need to plan ahead, we need to purchase
hardware, ramp up things, we have to like exchange
existing hardware and so on. Really tough. And we had those peaks
where users were unhappy and we had those times where the hardware was not properly used, like on weekends, for example. So really hard to manage
and that's why we decided to overcome this
fragmentation and challenges and come up with a solution, like a one-fits-all solution that would replace those platforms and motivate even the local users to switch on a central solution and bring all their workloads there. We coined the name for the
solution Jupyter Managed because we wanted to make clear that there's a one-fits-all approach now, which should take in all
the combined requirements from the different use cases. So starting with tools, yeah, we needed to provide
all the tools that they need and there will be a growing
demand in that of course. And everyone needs to be
able to access their data. So may it be data in
our group-wide data lake in the cloud data hub or data in on-premise databases
or uploading local data, we need to supply that. Of course the new solution
need to be elastic and scale with the demands, it was one of the major requirements. And have a low operations effort. From a security perspective, we had to bring in our corporate standards to have security incident
event reporting in there and integration with our identity and access management systems, malware scanning for the people that
upload data, for example. And we needed to support,
of course different teams up to different legal units, having people from the BMW Bank on there, people from the the automotive in there, and separate clearly their
environments in a compliant way. And last but not least, we wanted to foster
collaboration in the teams with sharing data and
sharing code inside tooling. And we looked at different alternatives and finally we came to SageMaker Studio and said we want to use that solution. And because SageMaker Studio already fulfilled many
of the requirements, which allowed us and my team to actually work on the integration. So we didn't have to build
the platform from scratch, we could work on the integration. And I wanted just to provide you a glimpse on this provisioning and
onboarding process for teams, how we do that at BMW now. So what we have is a
BMW-specific service catalog for IT services. So a new data science teams goes there and requests a new workspace and they, for example, have
to specify their cost center because we need to charge
the costs internally to them, right? And then an automated process kicks off and this automated process will create a new AWS account for them that is managed by us. And we provision all the
infrastructure into that. Like the the SageMaker
Studio domain, for example, our custom kernels that we need, the lifecycle configurations that we need, and some of the security
features that we add on top. And then once that's set up, all the users can enter a central portal and access their workspace basically. And when they're in Studio, they can directly import
data from our data lake. They can integrate with
the on-premise database and basically create value directly. And they don't have to mess around with all the configuration
and the setup and the security and all of that. We are taking care of that. So as I said, SageMaker Studio allowed us to focus on this integration and I just wanted to
mention like a few features that we really value,
but there's a lot more. For example, choosing
different instance sizes for the different workloads
is very good for us because we have one platform
for all the different cases. They can (audio cuts out) SageMaker training with
distributed training jobs for larger workloads. We can bring in our custom kernels with BMW-specific libraries to interact with our
data lake, for example. And there's the option
to schedule notebooks. We've seen it before in the demo. It's very important for some of our users. So all in all, looking at the solution
now with SageMaker Studio is very good for us because we benefit from
those new features. Features like we've seen today, features that are added constantly. We also value that SageMaker Studios building on open source components like we've seen today with VS Code, with with JupyterLab, for example. That's very good for us.
We really value that. So what's next for BMW
group with SageMaker Studio? So we will continue onboarding
teams to the new solution. Not everyone is onboarded
yet, so that's still ongoing. And what we are working on is integrating our
MLOps solutions for AWS, which we already have for some years, so we already have a
SageMaker-based master solution. Yeah, and we are going to integrate it for a smoother experience. We of course would leverage the features for increased developer and
and data science productivity like code generation. Many users look very ambitious to this faster startup times of notebooks. So this is really valued. And last but not least, of course, we would provide access
to large language models and other foundation models inside our development environment. For example, for people
being able to analyze texts, doing clustering, for example,
with large language models. So with this, we see us well prepared and really appreciate
the new Studio release. And with that, I would
invite you back on stage, Sumit and Giuseppe. - So really thank you for
for the demo and the release. I really appreciate it.
- Awesome. - Thank you, Marc. Thank you for giving us insights into how you're using Studio at BMW. And you have been a great
audience, by the way. Thank you. Thank you for coming
today for this session. (audience applauding)
