[MUSIC PLAYING] YUFENG GUO: From
detecting skin cancer to sorting cucumbers
to detecting escalators in need of repair,
machine learning has granted computer systems
entirely new abilities. But how does it really
work under the hood? Let's walk through
a basic example and use it as an excuse to talk
about the process of getting answers from your data
using machine learning. Welcome to Cloud AI Adventures. My name is Yufeng Guo. On this show, we'll
explore the art, science, and tools of machine learning. Let's pretend that
we've been asked to create a system that answers
the question of whether a drink is wine or beer. This question answering
system that we build is called a model,
and this model is created via a
process called training. In machine learning,
the goal of training is to create an accurate model
that answers our questions correctly most of the time. But in order to
train the model, we need to collect
data to train on. This is where we will begin. Our data will be collected
from glasses of wine and beer. There are many aspects of drinks
that we could collect data on-- everything from the amount of
foam to the shape of the glass. But for our purposes, we'll
just pick two simple ones-- the color as a wavelength of
light and the alcohol content as a percentage. The hope is that we can
split our two types of drinks along these two factors alone. We'll call these our
features from now on-- color and alcohol. The first step to
our process will be to run out to the
local grocery store, buy up a bunch of
different drinks, and get some equipment to do our
measurements-- a spectrometer for measuring the
color and a hydrometer to measure the alcohol content. It appears that our grocery
store has an electronics hardware section as well. Once our equipment and then
booze-- we got it all set up-- it's time for our first
real step of machine learning-- gathering that data. This step is very important
because the quality and quantity of
data that you gather will directly determine how good
your predictive model can be. In this case, the
data we collect will be the color and alcohol
content of each drink. This will yield us a table
of color, alcohol content, and whether it's beer or wine. This will be our training data. So a few hours of
measurements later, we've gathered our training data
and had a few drinks, perhaps. And now it's time for our next
step of machine learning-- data preparation-- where we load our data
into a suitable place and prepare it for use in our
machine learning training. We'll first put all our
data together then randomize the ordering. We wouldn't want the
order of our data to affect how we
learn since that's not part of determining whether
a drink is beer or wine. In other words, we want to
make a determination of what a drink is independent of what
drink came before or after it in the sequence. This is also a good time to do
any pertinent visualizations of your data, helping
you see if there is any relevant relationships
between different variables as well as show you if there
are any data imbalances. For instance, if we collected
way more data points about beer than wine, the model we
train will be heavily biased toward guessing that virtually
everything that it sees is beer since it would be
right most of the time. However, in the real
world, the model may see beer and wine
in equal amount, which would mean that it would
be guessing beer wrong half the time. We also need to split
the data into two parts. The first part used
in training our model will be the majority
of our dataset. The second part will be used
for evaluating our train model's performance. We don't want to use the same
data that the model was trained on for evaluation since
then it would just be able to memorize
the questions, just as you wouldn't want to
use the questions from your math homework on the math exam. Sometimes the data we
collected needs other forms of adjusting and
manipulation-- things like duplication, normalization,
error correction, and others. These would all happen at
the data preparation step. In our case, we don't have any
further data preparation needs, so let's move on forward. The next step in our
workflow is choosing a model. There are many models that
researchers and data scientists have created over the years. Some are very well suited
for image data, others for sequences, such as text or
music, some for numerical data, and others for text-based data. In our case, we have just two
features-- color and alcohol percentage. We can use a small
linear model, which is a fairly simple one
that will get the job done. Now we move on to what
is often considered the bulk of machine learning-- the training. In this step, we'll use our
data to incrementally improve our model's ability to
predict whether a given drink is wine or beer. In some ways, this
is similar to someone first learning to drive. At first, they don't know
how any of the pedals, knobs, and switches work or when they
should be pressed or used. However, after lots of
practice and correcting for their mistakes, a
licensed driver emerges. Moreover, after a
year of driving, they've become quite
adept at driving. The act of driving and
reacting to real-world data has adapted their driving
abilities, honing their skills. We will do this on a much
smaller scale with our drinks. In particular, the formula
for a straight line is y equals mx plus b,
where x is the input, m is the slope of the
line, b is the y-intercept, and y is the value of the
line at that position x. The values we have available
to us to adjust or train are just m and b, where the
m is that slope and b is the y-intercept. There is no other way to
affect the position of the line since the only other variables
are x, our input, and y, our output. In machine learning,
there are many m's since there may
be many features. The collection of
these values is usually formed into a matrix
that is denoted w for the weights matrix. Similarly, for b, we
arranged them together, and that's called the biases. The training process involves
initializing some random values for w and b and
attempting to predict the outputs with those values. As you might imagine, it
does pretty poorly at first, but we can compare our model's
predictions with the output that it should have produced
and adjust the values in w and b such that we will have
more accurate predictions on the next time around. So this process then repeats. Each iteration or cycle of
updating the weights and biases is called one training step. So let's look at what
that means more concretely for our dataset. When we first
start the training, it's like we drew a random
line through the data. Then as each step of
the training progresses, the line moves
step by step closer to the ideal separation
of the wine and beer. Once training is
complete, it's time to see if the model is any good. Using evaluation, this is
where that dataset that we set aside earlier comes into play. Evaluation allows
us to test our model against data that has never
been used for training. This metric allows us to
see how the model might perform against data
that it has not yet seen. This is meant to be
representative of how the model might perform
in the real world. A good rule of thumb I use for
a training-evaluation split is somewhere on the order
of 80%-20% or 70%-30%. Much of this depends on the size
of the original source dataset. If you have a lot
of data, perhaps you don't need as big of a fraction
for the evaluation dataset. Once you've done
evaluation, it's possible that you want to see
if you can further improve your training in any way. We can do this by tuning
some of our parameters. There were a few
that we implicitly assumed when we
did our training, and now is a good time
to go back and test those assumptions,
try other values. One example of a
parameter we can tune is how many times we run
through the training set during training. We can actually show
the data multiple times. So by doing that,
we will potentially lead to higher accuracies. Another parameter
is learning rate. This defines how far
we shift the line during each step based
on the information from the previous training step. These values all play a role
in how accurate our model can become and how long
the training takes. For more complex models,
initial conditions can play a significant
role as well in determining the outcome of training. Differences can
be seen depending on whether a model
starts off training with values initialized at
zeros versus some distribution of the values and what
that distribution is. As you can see, there
are many considerations at this phase of training,
and it's important that you define what makes
a model good enough for you. Otherwise, we might find
ourselves tweaking parameters for a very long time. Now, these parameters
are typically referred to as hyperparameters. The adjustment or tuning
of these hyperparameters still remains a bit more
of an art than a science, and it's an experimental
process that heavily depends on the specifics
of your dataset, model, and training process. Once you're happy with your
training and hyperparameters, guided by the
evaluation step, it's finally time to use your
model to do something useful. Machine learning is using
data to answer questions, so prediction or inference
is that step where we finally get to answer some questions. This is the point of all of this
work where the value of machine learning is realized. We can finally use our model
to predict whether a given drink is wine or beer, given its
color and alcohol percentage. The power of machine
learning is that we were able to determine how
to differentiate between wine and beer using our model rather
than using human judgment and manual rules. You can extrapolate the
ideas presented today to other problem
domains as well, where the same principles apply-- gathering data, preparing
that data, choosing a model, training it and evaluating
it, doing your hyperparameter training, and
finally, prediction. If you're looking
for more ways to play with training and
parameters, check out the TensorFlow Playground. It's a completely browser-based
machine learning sandbox, where you can try
different parameters and run training
against mock datasets. And don't worry, you
can't break the site. Of course, we will encounter
more steps and nuances in future episodes,
but this serves as a good foundational
framework to help us think through the problem,
giving us a common language to think about each step
and go deeper in the future. Next time on AI
Adventures, we'll build our first real machine
learning model, using code-- no more drawing lines
and going over algebra. [MUSIC PLAYING]
