Machine Learning Made Easy

hello and welcome to machine learning made easy my name is Shashank asana I'm the product manager for statistics and machine learning products at the mat works here's the agenda for rest of today's presentation I'll start off with a high-level overview of what machine learning is and why you may want to consider it will then see some key challenges in machine learning next I'll take a deeper dive into machine learning by solving an interesting problem that involves real-world data to tackle this problem I will introduce a typical workflow that often gets used in solving machine learning problems we'll then go through another interesting example that involves image data I'll show you a live demonstration that uses video from a webcam to identify objects we'll wrap the session up with a summary and some guidance on when to consider machine learning since this is a made easy topic there are no prerequisites basic familiarity with MATLAB is helpful but not required machine learning is ubiquitous these techniques are increasingly being used in today's world to make critical business and life decisions today with machine learning we are able to solve problems in automotive finance computer vision and several other fields that were previously considered impossible so what is machine learning let's take a minute to quickly go over the high-level concepts of machine learning even if you are familiar with this topic this should serve as a quick reminder of when it is actually useful to pursue machine learning we can define machine learning as a technique that uses data and produces program to perform a task let me explain that in the context of an example consider a task that involves using sensor data from a mobile phone to detect a person's activity for example if the person is walking sitting standing and so on the standard approach to solving such a task is to either analyze the signal and write down a program with a set of well designed rules or you may know a priori a set of equations or a formula that uses the inputs and predicts the output the machine learning approach is to learn such a program directly from data in this case we provide the algorithm with input and output data and have it learn the program to solve this task this step is often referred to as training the output is a model that can now be used to detect activity from new sensor data notice I don't explicitly mention a specific machine learning algorithm because there are plenty of algorithms to choose from and each have their own strengths and weaknesses we will see that when we go through this particular example in detail if there are two things I'd like for you to take away from the slide that would be one you need data if you don't have data you can't do machine learning this is a strong requirement to consider machine learning only when the task is complex and there is no magic equation or formula to solve it if you have a formula you can go ahead and implement it machine learning will work but is not recommended approach in such a situation so let's go to MATLAB and take a look at what the machine learning approach looks like the data we are working with consists of six inputs three from the accelerometer and three from gyroscope of a mobile phone the response or the output are the activities performed these include walking standing running climbing stairs and laying this is classification learner it's an interactive tool that lets you perform common machine learning tasks such as interactively exploring your data selecting features specifying validation schemes training models and assessing results on my left here I have several different machine learning models that have already been trained on sensor data right next to each model is a percentage number that indicates the accuracy of the selected classifier on a separate validation set I can use the visualization on the right here to explore my data to search for patterns and trends training a new model is easy I simply navigate to the classifier gallery choose a classifier of interest and then hit train once training is complete you can see the new model in the history list along with the models performance accuracy the higher this number the better the model may perform on new data I will choose my best performing model and click export to start using my model in MATLAB and I can see that the exported model is right here in the workspace I'd now like to use this model along with some test data to visualize the models prediction results this is a plot that shows two and a half seconds of streaming accelerometer and gyroscope data the green bar here on the top is showing the actual activity the person was performing corresponding to the sensor signal right here the bottom bar is green if the model is able to successfully predict the actual activity of the person and it's red if it is if it fails to predict the actual activity you see that the model often gets confused between walking and climbing steps let's quickly recap what we just achieved we use data to fit several different models using classification learner we then chose the one that showed good results and we tested it to make sure it was doing what it was supposed to that seemed quite simple so why does machine learning have a reputation for being so hard to answer that question I'd like to use a quote from a famous statistician who once wrote in a textbook all models are wrong but some are useful why because models are approximations not just machine learning models all models are approximations based on several assumptions but that doesn't make them any less useful machine learning models can explain complex patterns in data but to apply machine learning successfully you need to find useful models and that can be a challenging task in reality there's not one but several challenges in every step of the machine learning workflow data comes in all shapes and sizes it it could be simple numeric data such as from financial data feeds or sensor signals or could be streaming images from a camera or text data real-world data sets are usually messy and not always tabular pre-processing data requires domain-specific algorithms and tools for example signal or image processing algorithm are required to extract useful features from signal and image data statistics algorithms are required for feature selection and feature transformation we often require tools from multiple domains and when working with multiple machine learning algorithms searching for the best model can be a daunting and time-consuming task at the heart of it choosing the best machine learning model is a balancing act highly flexible models may be accurate but may also over fit your data and perform poorly on new data at the other extreme simple models may assume too much about your data there's always a trade-off between speed accuracy and complexity of the model finally machine learning workflows are never a convenient linear workflow we always constantly have to go back and forth iterate and try different ideas before we converge onto a solution my goal today introduce you to a common set of tools and strategies to address these challenges here's a simple two-step workflow we'd like to follow in solving a machine learning task the first step is to train your model we start by bringing in a data which would come from a variety of different sources such as databases streaming devices and so on next we pre process a data using feature extraction or other statistical tools this step is critical to transform your data into format that machine learning algorithm can work with after that we are ready to learn from data if the task is to predict labels or categories we choose classification methods if the task is to predict continuous values then we choose regression methods and then we go ahead and build a model this of course is an iterative process that requires going back and forth to the pre-processing step trying different machine learning algorithms tweaking different parameters and so on the second step of this workflow is to actually use the model so on the Left I have new data and on the right I need to make predictions so what do we need to go from data to predictions first we reuse all the pre-processing steps for the new data so there is no additional work involved here we just reuse all the hard work we spend during the training phase next we use a model from the training phase and make predictions for several engineering problems the second step is often deployed or integrated into production environments like for example onto a server executing automated trades using machine learning let's now use this workflow to go through an example in MATLAB the objective of this example is to train a classifier to automatically identify human activities based on sensor measurements the data consists of 6 inputs 3 from accelerometer and three from gyroscope the response of the output are the activities performed a walking standing running climbing stairs and laying the approach we'll take is as follows first will extract basic features from sensor signals will then use classification learner to train and compare several different classifiers and finally test the results on unseen sensor data so this is Matlock we'll start with the current folder window here since the first step of the workflow is to bring in data I'm going to load some raw since the data I have here in a matte file to import this data all I have to do is pretty much drag and drop it into the workspace window the workspace window helps you keep track of all the variables in MATLAB whether their existing variables or new variables that we create as we move along my data consists of seven variables the first six are sensor input variables 3/4 gyroscope and 3/4 accelerometer XY and Z the last variable train activity contains the activity labels for each observations of the sensor measurements notice all the variables have around 7000 observations let's take a look at what an individual sensor input looks like if I plot a single row of the x-axis gyroscope data we see that it has around 128 points now this is what the data represents sensor data are often collected in fixed width frames or windows each row with 128 readings correspond to two and a half seconds of sensor data now this data is not in a form that can be used for machine learning I must first process all these inputs to extract features for each of these 128 points or two and a half second windows and I have to do this for all six sensor inputs but before we start with the feature extraction let's take a look at what the raw sensor data looks like I have a custom plot here that shows three accelerometer raw sensor data and they have different colors corresponding to the activity the person was performing plots in MATLAB are fairly interactive I can zoom in to see if there are any visual patterns or if there are trends that can help us identify the activity of the person at the quick glance we can see that the sensor values look different for each of these colors for example the orange values here which correspond to climbing stairs is quite different from the purple values which corresponds to standing however at the same time it's hard to visually differentiate walking and climbing because both involve a lot of motion problems such as these are good candidates for machine learning as it's not always obvious that we can define rules for each of these activities before we start with the feature extraction step I am going to create a single table from these six sensor inputs tables are excellent tools for holding mix type data which is common in machine learning in this example we have sensor measurements which are numeric and labels which are categorical values I now have a single table variable in my workspace that includes all the six individual sensor variables another advantage to using tables for machine learning is that I can apply a single feature extraction function to all the variables in this table with a single line of code all I have to do is call whorfin which stands for variable function and pass in my feature extraction function W mean here is a function that computes the average value of each row in the sensor variable in this case I am computing the average of a two and a half second frame of sensor values W mean is a function in my current folder browser and as you can see it's a fairly simple function but depending on your application and the features you want to extract this could be as sophisticated as you please in addition to the mean I'd like to extract two other features which is a standard deviation for each frame and PCA which stands for principal component analysis and I'd like to retain only the first principal component since tables can hold mixed type data I'm going to assign the label activities as a new variable called activity within table let's run the section and take a look at what the human activity data table looks like I have a total of 19 columns and the first six correspond to the average value of the frame the next six are the standard deviation of the frame and the final six are the PCA outputs my last column is the activity corresponding to each observation which could be laying sitting climbing stairs standing or walking now that we have a data ready let's move to classification learner to train our models classification learner is part of the statistics and machine learning toolbox and you can launch it by typing classification learner on the MATLAB command line or you can find it under the apps tab under math statistics and optimization we first start by importing our data from the MATLAB workspace under step 1 of the import dialog box we choose a data set from the MATLAB workspace under step 2 notice that the app automatically selects if a variable is a predictor or a response the app decides this based on the data type however you can also choose to change its role or completely remove the variable for now we'll leave this as it is under step 3 we can choose our validation map validation protects against problems like overfitting choose cross-validation when you have a relatively smaller sized data set as it makes efficient use of all the data choose holdout if you have sufficient data I'm going to choose this option for our problem since we have a lot of data points setting my holdout percentage to 20% instructs the app to use 80% of the data for training and 20% for validation of the models performance the last option is not usually recommended since all the data is used for training as well as testing this leads to a biased estimate of the models accuracy classification learner is a fairly interactive environment with a number of different windows and components and a promise will go through every single one of them as and when they become relevant the tool strip on the top shows a left-to-right workflow all the way from importing your data to exporting your model at the center here is pairwise scatter plot grouped by the response variable this plot can be useful for finding patterns for example this pair of predictors we notice that laying appears to be well separated from the rest of the activities sitting also appears to be well separated but there is some overlap the feature selection option on the tool strip allows you to exclude predictors from the model for this example since we don't have too many predictors we're going to leave this as it is when solving classification problems there is no one-size-fits-all different classifiers work best for different types of data and problems classification learner lets you choose from decision trees support vector machines nearest neighbors and ensemble classifiers and for each classifier type there are several presets that are excellent starting points for a range of classification problems if you are not sure which to choose a pop-up tooltip gives you a brief description of the classifier when working with MATLAB help is always just a click away for further assistance simply click on the question mark on the top right and this opens up the documentation for the app you will find all the information you need about the app right here for now let's go to the following section that has guidance on choosing a classifier here's a nice table that gives you guidance on which classifier to choose depending on the trade-off you want to make for example decision trees are fast to fit but have medium predictive accuracy on the other hand nearest neighbors have high prediction accuracy for smaller problems but also have high memory usage let's use this tip from the documentation and start with the decision tree first training a model is easy simply choose a preset from the gallery and hit train and this will produce a train model in the model history along with its prediction accuracy let's also train a medium tree and a complex tree the model that performs best on the validation set is always shown by a green box in addition to the prediction accuracy percentage there are other diagnostic tools that are useful confusion matrix is a great tool that can tell you how a classifier is performing at a quick glance so this is how you read a confusion matrix anything on the diagonal is correctly classified anything off diagonals have been misclassified a perfect classifier would have 100% on the diagonal and 0% everywhere else let's take a closer look at one activity climbing stairs the way to read this is eighty nine point one percent of the time the model successfully predicted the activity however ten point nine percent of the time the model thought that climbing stairs was walking or in other words the model misclassified climbing stairs is walking we can always switch between different models in the model history to compare confusion matrices another diagnostic tool that's available in the app is the ROC curve ROC curves are used to describe the sensitivity of a binary classifier the shape of the curve shows the trade-off between sensitivity and specificity as we move upward and to the right of the curve we increase the chance of true positives but also increase the chance of false positives in my opinion one of the greatest advantages of using apps instead of writing code is the ability to Train multiple models in addition to decision trees I am now going to Train nearest neighbor classifiers because I know they are faster trained and faster predict when training multiple models you don't have to be confined to these presets if you are an advanced user you can always bring up the advanced pop-up to tweak the classifier parameters these changes are also shown in the model history for convenience we now have about eight models or so in the model history and I trained all of them in a matter of seconds without writing a single line of MATLAB code there are two modes in which you can export your analysis you can either export your model directly into MATLAB or you can choose to generate MATLAB code that lets you automate all the steps we took in building this model without having to write any code let me choose this option to show you what the generated code looks like what I have here in MATLAB is a well commented completely auto-generated MATLAB code we can also see the different steps in a workflow captured by the code for example extracting predictors and response training a classifier using the KNN classifier and setting out how load validation and so on you can always customize this generated code to integrate it into your applications let's go back to classification learner and export a model instead and we can see that the model is now available in the MATLAB workspace we are now ready to test the model on new data first we load in some new sensor data from a mat file we then apply the same feature extraction steps from before we then use the export and model to test results on this new sensor data as we saw before the top green bar shows the actual activity the person was performing and the bottom green bar when the model successfully detects the activity and it is red if it makes a mistake if you want to try a new model you can simply simply go back to classification learner and either export the model directly or generate MATLAB code and train new models to predict with new data let's take a quick look at our workflow to summarize what we just did our training data was sensor signals obtained from mobile phone sensor we applied basic pre-processing techniques to extract features like average standard deviation and PCA we then use the classification learner app to arrive at our best model based on the holdout validation accuracy for prediction with new data we applied the same pre-processing steps on new data and then we use the Train model to make predictions and then visualize the results let's go through another example where we will train a machine learning model using image data the objective of this example is to train a classifier to automatically detect cars from webcam video the data consists of several images of four different toy cars the response or the output is a label for each of these four cars the approach we'll take is as follows first we'll extract features from these images using a technique called bag-of-words we then use these features to train and classify several different classifiers using classification learner and finally this is the interesting part we'll see a live demonstration of the model we train identifying cars in real-time from a webcam feed in my office so let's switch to MATLAB I'm going to clear out my workspace and start clean slate okay so let's take a look at our data so here are a bunch of images and folders and each folder name is the label for the set of images in that folder let's take a look at what the images look like outside of MATLAB so here are a bunch of images for sanddune taken from several different angles and varying lighting conditions and I have several such images for all the four cars when working with images loading all the images in a loop and keeping track of all the files and folders and the labels can be painful the computer vision system toolbox has convenient tools like image set that makes this task easy I simply provide image set with a folder that has all my images and instructed to look at all the subfolders as well another important advantage of image set is that it doesn't load all the images to memory so it's easy to work with when you have lots and lots of images to pre-process my data and to extract features I am going to use an approach called bag of features this is a fairly sophisticated technique to do feature extraction from images and works particularly well if you are interested in learning more about what the function does the computer vision system toolbox documentation has detailed explanation on what the algorithm does underneath the hood for now let's treat this function as a feature extraction tool that we'll use to extract features from images like we did in the previous example let's create a table and assign the labels to the table let's jump right into classification learner and start training our model let's take a quick look at our data we have 200 new extracted features from the images and the last variable is the response I will again choose holdout as my validation and I'm now ready to start training models I'm going to train few models quickly starting with nearest neighbors and support vector machines now that I have a few more restrained let's do a quick visual Diagnostics using the confusion matrix the confusion matrix is mostly diagonal so that's a good thing and we'll quickly export one of these models and see how it performs on real Streaming images before I run this function I'd like to show you the code and how easy it is to obtain streaming images and classify them in real time the function takes in two inputs first is the train model and the second is a bag of features object which I will use to extract features from new images starting a webcam is easy all I need to do is call the webcam command inside a continuously running loop first I obtain a snapshot from the webcam and then convert it to grayscale which means the model should be able to identify cards without the color information the next step is to extract the features from this new image and finally use a predict function and the Train model to make predictions on the new image let's go back and run this function the top plot here shows images from my webcam and the green bar here is the result of the model making predictions on this image this is the classifier I'm using and this bar chart shows how confident the classifier is that this car belongs to either lightning mater needle or send you so let's move the camera around to see how the model performs our different angles and orientation so the model mostly gets lightning right but as we move to needle you see that the model is not completely confident which car it is the probability plot on the bottom shows you how confident the model is in predicting which car this is this is sand dune and finally mater let's export another model from classification learner and see how two models perform side by side so I'm going to use linear SVM and choose export with the default name so I now have two classifiers or two models in my workspace one is KNN the other is support vector machine now let me run this piece of code which compares the performance of these two classifiers side-by-side what we again see here is a video comparing the performance of two classifier side-by-side on the top left in red is a performance of our SVM classifier and on the bottom right is the performance of K&N classifier the important thing to note here is not only is it easy to compare and test models in classification learner but it is just as easy to export these models to MATLAB and test them in real time that brings us to the end of this demonstration let's now switch back to our presentation let's again summarize what we just accomplished using our familiar workflow diagram our inputs consisted of several labeled images on disk we use the bag-of-words approach to generate new features we then use the classification learner app to arrive at our best model for prediction step we obtain new data from a webcam encoded the images and new features and used a trained model to make real-time predictions this brings us to the summary and key takeaways to summarize what we've seen so far let me go back to the challenges I'd set up at the beginning of this presentation hopefully you've seen today how MATLAB is able to address these challenges for the first challenge of data diversity we saw how MATLAB can work with different types of data MATLAB can also access and download financial data feeds work with text geospatial data and several other data formats MATLAB also has libraries of vetted industry-standard algorithms and functions MATLAB also offers additional tools for specific engineering workflows in finance signal processing image processing and several others we also saw how we could quickly build and prototype solutions with interactive app driven workflows that let you focus on machine learning not programming and debugging best practices for machine learning such as cross-validation and model assessment tools are integrated into apps and functions as we saw MATLAB also has rich documentation that has several guidelines that help you choose the right tools for the job finally MATLAB is inherently a flexible modeling environment and a complete programming language with no restrictions to the customizations you can make to your analysis this makes MATLAB an excellent platform for machine learning this brings us to the end of our presentation and I'll try to keep this short in brief I'd like to share with you some guidelines on when you should consider machine learning consider machine learning if you're working on a problem where handwritten rules and equations are too complex or impossible to formulate or when the rules of your task are constantly changing and your program or model needs to constantly adapt since your task is a moving target or when the nature of your data changes and the program needs to constantly adapt and hopefully I've been able to convince you that MATLAB is a powerful platform for every step of the machine learning workflow we are always interested in hearing from you and you can find my contact information on the statistics and machine learning toolbox product page if you are interested in learning more please take a look at the product documentation there are plenty of examples and concept pages that can not only help you get started but also guide you as you master the tools to learn more about algorithms application areas examples and webinars related machine learning feel free to visit the machine learning page this brings us to the end of this session thank you all for listening

Channel: MATLAB

Views: 64,363

Keywords: MATLAB, Simulink, MathWorks, MATLAB (Programming Language), MATLAB Machine Learning, Machine Learning MATLAB, Machine Learning (Software Genre), Simulink (Software), The MathWorks (Business Operation), Statistics and Machine Learning Toolbox

Id: lbNrlwV6Lj4

Length: 34min 39sec (2079 seconds)

Published: Fri Sep 18 2015