Learn Live - Introduction to machine learning (Episode 1)

[Music] all right do we have glenn there oh no welcome everyone to our learn live session and right here off the bat we seem to have some internet issues i think okay oh you are there all right welcome everyone i'm gonna let glenn take over here no worries welcome everyone to another learn live session good morning good afternoon good evening wherever you are on the planet that's great to have you joining in um let us know where you're joining in from we're excited to have so many people here what is the first of a nine part series on machine learning in today's session we're looking at an introduction to machine learning js that's going to take you through what some of the other courses are as well a little bit later my name is glenn stevens i'm a senior content developer here on the learn live team i'm here with my good friend and colleague jason boober jason do you want to introduce introduce yourself to the world hi yeah i'm a senior product manager here at microsoft uh specifically i work on the ml and ai content for our docs.microsoft.com site including some of the some of the docs and of course the learn content uh in fact i kind of acted as the i guess you could say executive producer of uh this uh learning material that you're going to see here uh worked with some amazing authors who did the the real hard work and heavy lifting of creating all of the uh all of the labs exercises code and of course the uh the content that we're going to go through today excellent and i can see on the powerpoint slide that we've got we've actually got a link to today's session right the aka.ms.learn and you can actually go there and look at the whole series that we're working with today and this is just the first of the five sessions that you can see on the screen there's um actually ak dot l ms slash learn ml hyphen introduction that's today's session that's what we're doing which is introduction to machine learning that first and the list of five that you can see on the screen and we're doing a range of others as well we're building classical machine models we're looking at introduction to data for machine learning how to work and manipulate with data looking at things like how to train and understand regression models uh refine and test those regression models and a range of other things too things like classification models architectures and hyper parameters confusion matrixes optimizes all this fun stuff it does sound a little bit complex but we'll actually take you along a smooth path and explain it along the way it's it's very exciting yeah we've got two months to go through it we're going to be doing nine sessions over the course of the next uh next two months so we've got uh a good amount of time to get all of this material into your brains and we'll work on doing exactly that so as glenn mentioned we've got our uh aka.ms link for today's session glenn you want to tell them what to expect out of that well you can expect a really a few things we're going to take you through the process of what's a machine learning model how to create one create a very simple one for today's one it's it is a very gentle introduction and we'll show you how to use it how to work with data a little using a framework called pandas which is really quite common and popular and and essentially simulate what's going to be a little website we'll talk about that in a few minutes and if we just move on to the next part we do want to make sure we are live and interactive i'm wondering where everyone's connecting in from hopefully we've got a few people connecting it on chat it is on our chat we are live we can hear by the the first mistake at the start with the young munich but uh jason any other comments no like you said we are we are live and uh we are on opposite sides of the planet by the way glenn and i so glenn is over there in australia i'm here in the us uh in michigan specifically uh so we're on opposite sides of the planet that means you know we we might expect some lag here and there uh so if you're experiencing that if you see that there might that there's a little bit of lag then then understand that even in today's day of uh of giant undersea cables we occasionally get some intercontinental lag yeah but i can see we've also got some people on the chat from costa rica argentina uh quebec chicago nigeria there's a huge range of people outstanding that's great yeah from all over the place i love that everywhere ever everywhere from where we are and everything in between right so we've got some moderators that are helping us out today jason you want to speak to that yeah so these are just some amazingly talented subject matter experts who are going to be working in our chat channels so if you're in chat right now and you have a question over the material that we're going over again we're live we're going to be able to help you real time with some of your questions of course keep it on topic we are not going to be able to answer every machine learning question that ever comes up uh although you know give it a try so our subject matter experts in the chat window just might be able to to help because these people are really knowledgeable and occasionally they'll throw one of your questions up to us and glenn and i'll feel it as well awesome all right so let's get into today's content all right over to you jason yeah so what we're going to be dealing with today is a very very high level introduction to what is machine learning so we're going to be talking about that today and in fact we'll even still stick with a pretty high level overview in our next session um that episode two will still stay fairly high level still kind of a a um a a survey of everything but just drilling down a little deeper you know after that our sessions after that we'll actually start digging deep into some of the vertical aspects certain types of machine learning and certain things that we need to do as part of our machine learning process so today we're going to address what are machine learning models and that's the very first thing we're talking about right here with this slide so a machine learning model in many ways is like a function that you would code if you were writing code now i'm going to speak to coders somewhat because i think that's a fair amount of our audience if you don't have experience coding don't worry it's not required for this uh actual session for this conversation but but because i think that's a fair amount of our audience i'll occasionally direct some comments at coders and so a machine learning model in many ways does act like a function you get inputs and you expect an output the key difference is that with a machine learning model we're going to use data to help decide what that function should do in common example and one of the early practical machine learning models were spam filters now early spam filters they did try to do things with imperative code like identify words that are just always spam emails but those models didn't work nearly as well as some of the very early machine learning models those functions those those imperative functions didn't work nearly as well because what they were able to do with these early machine learning uh spam filters was look at all of the emails that we all get every day they were able to look at all of those emails and see which ones where flagging is spam and so by knowing that we humans flagged all these emails as spam and then all these other ones that we didn't flag as spam we were able to feed that into some machine learning models and we'll start to understand what that's about in a few minutes and get a better function than anyone could hand code at the time and probably start today that's true one of the nice things about spam filters jason is that as you get more data as you get more emails and you trade it more and more it actually gets better and better right absolutely right and so when you go in even though our spam filter does try to spam filter things if we actually go in and say oh that's not spam we're helping to improve those algorithms and same thing of course when we continue to add more more spam to our list of things that we don't want so we've got a scenario that we're going to be working through today as part of our example it's not a spam scenario do you want to tell us about that sure it's well it's just as useful this is a business that sells harnesses for avalanche rescue dogs and essentially what the business has found out is that people are buying the harnesses for these rescue dogs and also buying the boots but the boots are actually the wrong size but the the business knows that if you're buying a harness size of a particular size the boot size should actually kind of match in some way they don't quite know the formula but they want to use the data in order to work out what that value should be or at least the range roughly of what those values should be with a little bit of domain knowledge so what we're going to do is we're going to look at the data for the harder size for the boot size and see if we can come up with a model machine learning model that actually represents that business business rule for them we could do things like we could warn the customers and we'll see that a little bit later towards the the end of today's session yeah see what we can actually do with with our business once we have the these models to to use how can that help our business yeah yeah but we've talked about models a fair bit but why don't we go through probably a more in-depth definition of what's a machine learning model yeah we don't want to get too in-depth yet but but certainly we're going to take another pass and in fact we'll of course keep doing that keep taking additional passes and getting deeper and deeper over over the next nine episodes but but we already sort of answered the question what are machine learning models that are functions that we create from data but but how do we get there what do we do what data are we using and the answer is that that our model is that that core function but we need a process to turn that that that basically a wireframe of a model i'll say an incomplete model or a model architecture we'll we'll start with that and that's something that we will choose as as the ml engineer or the ml implementer the ml practitioner we will choose this starting type of model but it's incomplete we need to feed it data and then once we feed it the data our process our training process can start to figure out how to complete the model for us now models can be very very very very simple or they can be very very very complex i'm going to show you an example of an absurdly simple model here so this absurdly simple model here has an input a single multiple multiplier a single multiplication parameter and then some expected output so if we look at the scenario glenn this matches the scenario you you mentioned earlier right yeah that's right we've got the the hardest size but we want to know what the boot size is i can see a parameter here that we like multiply it by 2.5 that seems very rough but it may not always fit every particular situation right exactly so if we start with this very simple model surely with one data point we can fit it and say all right yep if we just have this model that is just multiplying by by one number yep we can fit this one data point multiplied by 2.5 and the math works out however this is not going to generalize if i start looking at more of my rows of historic data i'm going to realize that this isn't a very good model at all right and so that's what we want is is a model that can generalize so this one is is probably too simple it's probably not going to generalize now a model that's not much more complicated however is actually useful in many real world situations is the one that we see on screen now so now we've just added one additional parameter but this actually has an interesting property glenn does this look familiar the uh numbers and the the math on screen yeah when i saw this the first time i went oh this looks like my high school maths whereas the y equals mx plus b where you're drawing the the trend line between a bunch of points right it's exactly what it is right so this is plotting a trend line and and we could say all right for our harness when we go across kind of let me let me get my my laser pointer here if i'm going across the bottom of a graph right and and mapping harness size one two three four then i can actually see that that i'd have a another axis there for the up and down for the for the boot size and our parameters here basically can create a trendline and we're gonna see this in our first lab our first coding exercise we're gonna actually see that trend line and see how that gets us a very very good model even though it's not super simple or it's rather it is fairly simple math uh it's just a single multiplication and a single addition to get there now finding what the right things to multiply and add by well that's not necessarily so simple it it will be definitely achievable and approachable for us but it definitely requires some some explanation and we don't worry we're getting there we start by just picking something when we when we once we've selected our basic model in this case the basic model we're selecting is is that trend line and it's sometimes called linear regression or ordinary least squares but the idea is we're just trying to get a trend line a single trend line from a single input of data in this case uh our actually the linear regression ordinary least squares model can generalize to many more inputs and we actually do see that in some of our later episodes not even that much later i think episode three maybe four will actually start working with those um so so here we start with just any parameter value and the reason it's just any is because we have a training process with machine learning we're going to start with just some value then we're going to feed a lot of that data into our training process and it's going to start changing the parameters for us and we can see when we start with our random numbers it's not very good i think this equals two right so so that didn't work but our training process and we'll talk about how that works in just a few minutes our training process is able to realize that those parameters are way too small and start eking those numbers up bit by bit by bit until it's unable to further improve until it's gotten as good as it can get yeah it's interesting though i think in our case here we're trying to look for the the answer with something like a trend line to find the most least incorrect value would that be a fair assessment that that's a great way to put it because what we'll see once once we start playing with that data is that that our real world data that we're that we're using for training it isn't going to perfectly match the trend line it's going to be kind of rough and all over the place but we'll see that the trend line is a good general match for our historic data and therefore will be a good rough suggestion for our future boot sizes you know there's a couple things to to to really notice here and that is that even though i've selected a very simple model here it should drive home the point that all of our machine learning is just mathematical transformations even our most complex models are just mathematical transformations there's just more more of these parameters sometimes billions of them in fact as it turns out with some of those things that we see about in the in social media doing crazy stuff so there can be billions of parameters but it's still essentially mathematical transformations of the inputs and you can have more inputs as well and sometimes even turning your inputs into numeric representations like if you have a text input well there's some tricks to turning that into numeric uh representation that happens even before you get to training your model but you get that numeric representation and then the training of your model and the actual machine learning part is just finding those optimal parameter values and it's just that that that application of multiplication division those sorts of things you know mathematical transformations right so we've got we do have a general should be a little bit of fun we talked about all this theory let's put it into a little bit of practice um and in our session we're going to look at how we can compare some simple data how we can select the model the very simple model the one we've been talking about so far we'll look at training and then also using this model and to kind of estimate a value it should be a lot and let me just share my screen now we'll switch over here and i'm actually running in the microsoft learn environment here one of the nice things we have with this particular series is that it's using jupyter notebooks which means we can actually execute all this in the browser you don't need to set up any infrastructure you don't need to run anything else it's all in the browser as long as you've got a microsoft id and you're logged in you can execute on the sandbox we're not seeing your screen actually and and for for me at least your video is a little jittery so those undersea cables are uh are of course wreaking havoc with a live demo because that's how they do they they they do let me see if i can correct this you want to share the screen back again jason yeah how lively right now that's so live there we go we are so live right now all right i may just have to switch it works unfortunately um yeah so we have this preparation of data we'll select the model i'll just switch over very quickly and i've got that sandbox environment up too so if we need if we need me to drive no problem all right i think we might be going that route no i'm just sharing my screen now hopefully you can see that is it all visible on the screen for you now jason well sort of [Laughter] it's not good enough i'll say that yeah we've got we've got a lot of pictures well you might well why don't you you uh host it for us and we'll we'll run through it you got it here we go like glenn said we're so live right now did this ever happen in rehearsals no no it did not no so you can see on jason's screen here we have the training and running your first model um and that's really what we call a cell in jupiter notebooks if you've not used jupiter notebooks before and it really is a list of descriptive descriptions and also a list of executable cells underneath so we've got some descriptions here about what we're going to do we're going to train and run our first model we're going to go through that that process of looking for those parameters that we need for the model training the data and then executing it but often one of the first things we do is we prepare our data for use and if i go down to the second cell i think jason's already there and we can see details like at the top import pandas now pandas is a quite a common library in python right jason it is very very common for uh for people doing uh doing data analysis or data science work yes so we're using that library there as well underneath that we've got some references to some web libraries we're a graphing library and we've also got a csv file that we're not using just yet we'll use a little bit later in the the next couple of exercises but underneath that we have some data that represents the boot size and the harness sizes of really information that we know that's accurate already so we've got the boot sizes and they're correlating harness sizes so we're going to use that data to actually train the information as we go through underneath that we're we're using the pandas library to load a data frame which is really its representation of data so it can work with that nice and efficiently and then we'll just print out that data set so jason why don't you go ahead and run that cell yeah while i do that let me uh go ahead and and augment what you what you mentioned about pandas so so pandas is a library that lets us in our python code sort of trade our data the way we would in excel or in say a relational database that is allows us to have a two-dimensional model like a table or like an excel spreadsheet and that allows us to do joins amongst those two-dimensional models or or do filtering aggregations things like that and in fact um pandas stands for panel data because you can imagine a panel of data like an excel spreadsheet and so it lets us work with with the data like that but in python so that we can then after doing all those excel like transformations we can then pipe it right into our machine learning code cool so you've gone ahead and executed that if we scroll down we can actually see the output of that execution while it's running and then a little bit below that if you scroll down you should be able to see the the data cells and there should be 50 records there starting at index zero so it should go up to 49 and then we can see all the data there and sometimes it's a good idea to just look at the shape of the data that you're working with just to get a sense of what's going on we'll look at look at that a little bit more now below that we're going through the process of selecting a model and the linear regression mechanism that jason was talking about before is the one we're using here and in fact we're reporting a library here the stats model formula dot api and then we're also specifying a formula where the boot size is dependent on the harness size and then once we've got that formula there that we're creating the model and that might look a little bit cryptic but what we're really saying is we want to create an ordinary least squares model that's that ols and that's that linear regression trendline we're looking for and we're passing in the formula as well as the data that we want to use in order to train that and this is all going to come together once we run this cell and and and the next couple and we ate that visualization you're you're all going to start to go aha with what we were talking about with trend lines and and and linear regressions and and this is all going to come together we're we're two button presses away from correctly and that's a really important thing too as well looking at the data and actually looking at it visually if you can it gives you a really good sense of what the data is doing all right so we've got that so let's execute that if we haven't already yeah and then we'll go through the process of training it so we have everything set up but we haven't done that that last step of of training the data in order to create the model so in this this point here we're putting another library called graphing that we that'll actually do the work of visualizing this information for us and there's actually we don't need yet that comes come come to think of it we actually use it down here but whoops that's all right so we're training the model that that model called fit that's the the process for training and that will actually look at the data look at the analysis and try and work out what the the parameter values are in order to display and once you execute that if you go ahead and click on that json so let me just have you what you just said while that well that executes too we we started above in that previous cell we selected the model that we wanted to use which was essentially the same one that we had on our powerpoint right it was the one that basically just had one multiplier and then add another another term so we multiply it by a term and then we add by a term uh and that's essentially it in this case we are now saying all right we picked that model then we come down here and say fit we're saying use our data to select the optimum multiplier multiplying term and and addition term now by the way we did actually when we selected the model up up here we also gave it the data we told it which data we're going to train with we just didn't actually do the training step yet when we say fit we do that training step we say find the optimum parameters and there they are by the way awesome so then we can actually use that those parameters and and actually let's run the next cell the next cell is really just going to take the scatter plot i call you you've executed already so it's got that that trend line going up and also it's got the the various data points but what i can see json is that the the dots aren't on the trend line shouldn't they be perfect and exactly on the train line this is exactly like we talked about right is we we've gotten the the least wrong fit which is exactly what we want to do there's no such thing as a exactly correct fit and in fact if we somehow created a a line that went squiggly all over the place with with our historic data we'd find that actually it didn't do very well with the new data anyway because because the real trend is that it looks like this line but with some some variance that is actually the trend the variance is part of the trend so so this actually is a very good model yeah and it's that least incorrect model right that trend line of of going up that's good exactly yep and so yeah we can see this is all our historic data that we loaded early on and said okay fit a trend line a predictor to this and what we get to do now in the future is when we have a harness size we can come along this spot here and then go okay what's the matching boot size that's what we do that's what we do on the the next step we're actually going to take that harness size and go what's going to be the boot size for this particular value so yeah we've got that writing cell so we're populating just some parameters the harness size here that's the input and then we're going to apply that to the model and tell us to predict the value of the approximate boot size so that the predictor's probably a good term because it's not actually getting it right but saying this is what we think based on the data we've analysed analyzed yeah i i always wish i always wish that the that the jargon in ml was guess because predict almost sounds like there's like there's like a a time element a future element to it and that's only sometimes the case sometimes we're predicting what happens in the future sometimes we're guessing what the boot size is but you know like you said it's it's better than saying and then saying give correct answer yeah i don't want to call that man yeah absolutely i'm not sure what you said that undersea cable got me on the uh that's all right um so we've got that prediction that's coming through we've got the the value that's been displayed here so if we correlate those values to the graph from before we've got the the harness size of 52 and a half and what would we get typically for the trailer what's that show chase well if i go to 52 and a half and that's just about right here that brings me to just about 36 and a half on the boot size and which is which matches the output so that's a pretty simple case where we've actually just trained our first model and and taken you through some of the steps that you would do to do a very simple analysis with the ordinary least squares which is still an effective model and like you said before jason it's actually used quite a lot in the in the real world yep yes this is absolutely a real world useful algorithm now more often than not we'll we'll see it with more inputs than just one um but nonetheless the rest of the algorithm stays the same in fact we're going to see it with more more than one input uh in that uh episode three episode three or episode four when we talk about regression models also get back to our slides then and feel free to send messages through in the chat as well um more than happy to answer questions that come through maybe our moderators are just answering all of them for us jason that is likely they are really impressive a group of moderators today i have to say so they they are definitely um more than capable of answering the same questions that we are although it's fun for us to occasionally answer a question all right so we're talking about the inputs and outputs of our parameters we've touched on this a little bit already but i i want to focus in on a few key parts and and one of those is just the jargon again man jargon and machine learning is definitely one of the one of the difficult issues probably one of the more difficult issues is just getting around the lingo or the jargon that is used because we borrow jargon from computer science from statistics from pure maths and of course it's a couple decade old discipline now and therefore machine learning just has its own crazy invented terms as well so there's a lot of different jargon a lot of is confusing it can me you can have two words that mean the same thing because they came from different places two people published papers using different terms for the same concept and now who knows which one we should use it's a little challenging so so one of those pieces of jargon that can be confusing is our input versus our parameter now when we talk about parameters in machine learning we always always are talking about those numbers that we can train okay so when i say those those numbers that we can train i mean these numbers right here so so in the case of our ordinary least squares model it was it was that that multiplication term and that that addition term which when we put it in into our into our terms of graphing was our our slope and our intercept as we saw saw earlier so when we talk about parameters we're always talking about those numbers that the training process is going to find for us parameter is not a synonym for input like we might expect when we're just doing imperative coding right with a regular function the function's input and the function's parameters could be taken to be essentially the same thing so that that's all making sense right yeah it is and i was thinking about that while you're saying that it's like often because i as a developer you go parameters well that's what you supply to something to to execute it but in our case the what we supply in machine learning is data and then the output is the parameters that we will execute later right yeah exactly so so i guess it's it's probably also not a bad time to to bring up although you know we have a slide to bring it up later but it's a good time to bring it up in the in the course of our conversation here that that really our training process is part of our development process it's it's not part of our part of our run time execution so so when we talk about as as developers of a function we would have at runtime we would supply inputs or parameters to that function and it would execute and and as ml developers we're going to still supply inputs at run time to a model but that model will already be completely trained at runtime we have a completely trained and ready to go done model at development time is when we're going to train the model and that's when those parameters come into play so when we're developing that's where we where we will actually do our training process later we'll deploy a model we do actually have have a question here so we have among various frameworks pandas scikit-learn pycharm spark tensorflow how do you decide the right framework that is an excellent question um so so a lot of those have slightly different niches so tensorflow pi torch are used for for deep learning um whereas pandas is just used for slicing and dicing data usually preparing to go to something like pie torch or or tensorflow or or or something like that and and then uh on the other hand something like scikit-learn is most often used it's off much the way that we're we're using that stats model library today for classic machine learning models yeah i think we've completely kind of lost you there glenn darn all right i'm solo here we go [Laughter] all right i'm gonna i'm gonna touch on on our objective function now while we see if glenn can can can unfreeze um so so we've talked about training right we've talked about that training process that training process needs something to get better at right so we give it we give it a base model and we give it some data but but we're telling it now go get better go make those parameters better what does that mean make those parameters better that's our objective function is the thing that we supply that says here is the the during training we're going to have you keep executing against the data training process until you get better at this function now in our previous example the the objective function the thing we want to get better at is just being closer having that line be closer to the existing data points and remember glenn said the least wrong model so so least strong in that case is defined as as the distances the total distances uh between our data points and the trend line being the smallest possible and and so we actually have an objective function that that measures that mathematically measures the distance and then from there um we we can keep getting it better when we don't like it and we say oh that's that's too much distance or that's that's that's not a very good model let's see if we can get it better that that's um that's our training process actually let me just jump over this slide here real quick and and continue talking about the training process since since i kind of got there the the optimizer is the other key piece of our training process so the optimizer is the thing that that it's it's the workhorse it's the heart and soul of the machine learning process whereas as the model is the thing that we will ultimately deploy the optimizer is the thing that has kind of the the master for loop that goes through during our training process and keeps improving our parameters until they have gotten good at meeting our objective function so the optimizer is the thing that does that so so that's kind of that master for loop in our training process or you could even call to in some ways you could simply call the optimizer the trainer the training process glenn are you back with us well it seems like i am can you hear me okay jason i can yes it's wonderful that's good do you have anything to add to anything i said over the last few minutes no i i think that's all all quite relevant i i like to think of the optimizer a lot as like the superhero of the the whole process it does the work and and generates the the least wrong or the most accurate representation of what that those parameters should be that's right that's not a superhero for the modern era a superhero that does the least wrong thing i don't know what it is all right so back to my data slide here so our data is is what we've talked about a lot already but i want to highlight one more thing on this slide and that is that during our training process our data has both the inputs and the outputs remember our historic data we needed to know what the boot size was so that we could determine when we were least wrong or or or most wrong or very wrong right we could only know if we were wrong if we had the the answer if our if our guessing was wrong if we had what the right answer was and so during the training process we do need both the inputs and outputs of our historic data now later we'll be deploying our mod when we deploy that working model to production somewhere that working model will only get the inputs so we'll only be giving it harness sizes in our our example and we're kind of then at the mercy of trusting our model to do a good job to to predict the um to predict the outputs now we're not maybe entirely at the model's mercy because of course we can continue to monitor things and we can do reporting and we can continue to improve our model and all that good stuff but but to some extent we are once we publish that model we are setting it free that sound right to you sir it does i think probably what we want to do is is actually look at this in a bit more um more depth we've got a demonstration coming up as well so let's actually yes i might actually get you to show your screen jason as well all right the demo that's okay we'll keep the risk level low yeah excellent so yeah i think we're we're we're mostly going to be playing with that data this time kind of doing a a very lightweight version of the uh the data cleansing and and data preparation process that we'll really dive deep into in our in our module three right yeah that's that's right and it's quite common often because we're working with data we want to make sure it's got the right shape that there's no real structural issues to the data and also you want to do things like see the shape of the data um visualize the the data as well as you're working with it so if we just scroll up to the top excellent we're we actually do we have an interesting question as well let me uh let me see if we can we can talk about that real quick do we have a mapping between what ml model solves what problems uh there there are definitely some some basic um mappings out there and some things that i i guess i don't have one right at hand to show you in in the presentation deck but even microsoft has some some published materials on this but there are a lot of models out in the world and a lot of variations of models um so during the the course of of well this course so over the next nine episodes we'll talk about several of the common types of problems like classification like regression and we'll talk about the common models that that can help with those but in terms of the the laundry list of hundreds of thousands of or hundreds and thousands i guess of of uh model architectures that that exist out there um it changes all the time so we'll we'll leave it to the uh to the the good folks at bing to to help us well there's a couple things to mention there too as well jason um one is we have a session on architectures a little bit later we'll cover some of that material not all that like you said there's a huge amount but i'd also recommend people go to microsoft.com learn and there's a range of other learning modules as well on machine learning not just the series that we're doing but there's a series on things like tensorflow for example you can step through and learn some of that framework learn how to do things like identify objects from from images etc lots of great stuff there but yeah explore if you can microsoft lab awesome point yes hey do you want to start it on this lab now then for real yeah let's do it all right so we're loading the data with panda this time we're actually looking at the csv file um which is i believe that second link in the the chat window there and then we're reading that csv file the doggy boot harness data in so even you could actually just copy that url and paste it in a in another tab in the browser and actually see the the output there cool there we are so you can see it's got the the appropriate shape there of what we're looking at now if we go back and run the cell which i did excellent so we can see that we've got the data there we can see just the the definition or the shape of the data so the head method will only display the first five rows and that's actually often all we need we want to look at the structure of the data so we can manipulate it and in this case we've got things like the boot size the harness size the the gender and the age and years and it might be that we have columns that are just completely useless to us as well with the model if we had something like the the owner's phone number that's not going to have any influence with the boot size whatsoever so we want to do do something like delete those columns and we might find or we might have domain knowledge that the the gender and the agent years has no effect on the boot size as well so we might want to choose to remove those columns or edit them in some way a little bit later so if we scroll down one of the other operations that we we often do is we want to filter the data and essentially get the the range of data that we know that works we might have some some data outside that range that we know is it's kind of a little bit too outside the outside the realms of of working correctly um so in this case here actually let me let me uh actually we're a little out of sync i think so i want to actually just uh jump in and uh once again i'll blame that undersea cable um but i yeah i wanted to jump in and talk about why would we maybe remove you know the the the sex gender uh or the age of the dog because it seems like there's maybe like a fair fair chance that that that would actually be related to to the relationship of the harness and the boot size right how they how the harness and boot correlate it seems like maybe they're related but yeah if you go back to statistics jason as well it might be that they have a correlation but they may not be significant either so you have that effect as well so one of the things is that the people who design this might have some domain knowledge know that well actually gender doesn't play a role in that boot size relationship as well so that there is that little bit of mix but we're using data but but often in this case the the impact it might have might be just irrelevant yeah or it could be that that we've done some data analysis too and realized that that that there's some null values and then there's so many null values that we don't think we can clean it up or just kind of kind of hand wave around it uh or or values that seem completely wrong like the age in years we have 150 on a bunch of a bunch of our records so we know we know the dogs aren't that old so we know something is uh something is a miss so we decide you know what instead of trying to account for that instead of instead of maybe being wrong and accounting for it let's remove that from from from our data yeah yeah so someone's put in dog years instead of normal years that's what that's what you're saying right that's actually yeah that's actually something that you could really imagine happening is is people putting in the translated uh the doggy years yeah exactly cool so we go back to that those columns those are deleted and it's also listing the available columns afterwards as well that was for the the cell before oh yeah and then we did the head and tail here uh once again we talked about before and which is really useful as well because when you're often working in real situations you've got millions of data data rows you don't really want to scroll through a million rows as you're working with the data and it's going to chew up your memory as well on your machine so the head and tail are very useful just to look at the shape of that data now below that we also want to filter logically on that data so for example maybe we want to look at the the the data for those dogs that have a heart size less than 55 that's what we're looking at here and we've got a few operations we're showing that the number of rows in total so we can compare that to the filter a little bit later and then we've got uh is small so the is smaller is a like a binary mask right of of the data so we're saying we want to look at the we want to create a billion column yeah which where the heart of size is less than 55. yep and then yes exactly and then that boolean column right there that we see here actually gets used as a filter to give us a subset of the uh the small dog data only and that's down here somewhere there it is awesome and then below that we're showing the number of rows just to compare or just to look at it for our own representation awesome and that looks like a lot of code as well but just below that we've got a more simplified version of that code where we're taking yeah and let's once again just just just drill in and why would we do this filtering and again i think i think the thing that that i just want to keep hammering home is why did we do this filtering probably because there was some domain knowledge when doing machine learning when doing data analysis data preparation you must understand your data data is not just data data has context data has has a history it came from some place and if a harness size doesn't have an available size of below 55. if that perhaps is the smallest size that's available then we have some data problems in all of the other rows or maybe we know that below 55 were into puppy sizes and puppy sizes behave entirely differently so we need a different model but again it's going to whatever the reason that we did this it's going to come down to some kind of our our domain knowledge about this data and about about doggy harnesses yeah very good point awesome so we've got the structure just a little bit below that where we we're simplifying the the the data into just a single line of code we're getting the smaller ports we're passing in the expression the the boot size is less than 40 and we're copying that as well and then showing the last five rows of that data yep so just another filter operation yeah yeah but this is really the the daily fundamentals of working with the data as well things like deleting the columns filtering it and making sure we've got a clean data to go into the model because if you put rubbish into a model you'll actually have rubbish parameters too and you want to avoid that as much as you can yeah and remember yeah rubbish isn't just defined i know i i'm going to keep hammering at home rubbish isn't just defined by by by not you know somehow finding math that works or somehow running into an error or an exception rubbish is going to be defined by by not getting satisfying predictions and that can be tricky to to to know did i get as good a prediction as i wanted or or as i could have gotten or was i was was my model handicapped was my model held back by um some some poor job of data prep and the only way to know that is to do a good and thorough job of data prep yeah and we'll actually look at verifying a little bit later in the series too awesome so we've got the the chart now of that smaller pause data set and you can see that visualize visually on the screen and we've just got that limit so we can see that the the harness size is uh appropriately filtered or the boot size is appropriately filtered awesome and and you can almost imagine that trend line going through here again of course now all we've done is to to actually plot the data points but but you can visualize it you can you can sort of sit here since we're lucky enough to still only have one input and one output for now we can have a nice two-dimensional graph and we can we can see right where that line roughly would go right exactly um and in fact you you can imagine if you got a smaller data set the the actual trend line those two parameters would be would be slightly different with less data as well exactly right so below that we're also going to do something else that's quite common and like jason i know you're in the u.s i'm in australia so you use imperial empirical i use metric we sometimes we can't communicate with numbers and things like temperatures all the time but we've got the same situation going on here with without doggies and some of the the hardest sizes we have empirical we might actually have some in metric in some other form and part of that data processing is probably making something a little bit more consistent so here we've got a formula for working out the imperial size based on the metric size that that's there for example and we might use that because we might have formulas that work with one or the other i know that when i'm working with with things like temperatures and distances for example i've i've got formulas that work for metric and some the way for imperial and i want to switch those around so this is a good example of that as well yeah and the the point i was going to make is that that while if we look at this all we're actually doing with this is is dividing by a number which is not likely to to actually dramatically affect the performance of our model because our model multiplies by a number and it finds that number anyway it would just find a different model or a different parameter that was 2.54 higher so it's not going to help the model by doing this what's going to help is us in our further data analysis so if what i'm used to seeing is the imperial number if that is what what i'm used to seeing and charting then i want it that way so that i can do my data analysis in a context that me as the as the doggy harness expert i see the context that i'm used to you know the other possibility is just that we're normalizing this if we had a more complicated data set maybe there was a another field that said imperial or you know it was just a uh lookup field that said okay is it imperial or is it metric and then and then the the actual value field we have to adjust based on that so there's all kinds of weirdnesses like that but i thought i thought i would call out and for those of those of you who may have been noticing hey that's just division which is multiplication and we already have a multiplication parameter you're right but this does help the data analysis part which is super important especially when we're otherwise using a relatively simple model like we are we need to get that data rock solid awesome cool and that's what all we're looking at in the second exercise so looking at things like the the data working with the pandas library to get it into the data frame and start working and manipulating columns doing things like deleting it filtering on columns removing rows of data as well and then just graphing it to make sure you visualize that data and it's so helpful when you're you're working with data to visualize it both the shape in table form and also visually with charts i think it's probably time to move back to our slide deck as well jason absolutely so what we've got is is a trained model we actually we actually worked on that in our very first exercise we created a model and we can use that model once we've trained it we can deploy it to the real world and do some work with it and treat it as a function we can give it more doggy harness sizes and expect doggy boot sizes to come back out and because we trained it with our real historical data we can we can make some assumptions that it should be relatively accurate however things change maybe our doggy harnesses supplier changed and the the harnesses are slightly different sizes or maybe an upstream supplier changed and so the leather stretches more there's there's just an infinite number of reasons that that that our our data starts to become less representational over time and plus we're also collecting new data over time as so we're collecting new data that means we have a better chance to fit a better model and so while yes we are going to to start with a blank model when we very very very first start going down the path of needing to to create a machine learning model for a business problem we'll start with blank we'll train it with the data we have and then we can use that assuming that we find it to be good enough and then we can use that but then we're going to have an iterative process as we get better data as we realize that we have the opportunity to create better models maybe just because we're doing ongoing data analysis even while our good enough model was in production we realize we have the opportunity to to improve it and so we will have that development process remember that training is a development process it's developer time and just like developing new code you can be doing that work while your current code is in production you can still be working on the next version and so we can be training different candidate models trying to find one that beats the one that we've got in production with new data or with just better ideas or better data cleansing and then publish that new model as as we see here so i can imagine in our situation jason that as the the rescue harness shot business continues on maybe if they don't get returns from the customers they will assume that the harness size and boot size are related and then use that new data and push that back for more training a little bit later too absolutely yes yes absolutely and then we just keep getting better over time right we just keep getting better um we've trained of course we've talked about this before but let's go ahead and hit it again while we've got a table that's that's the or a slide that wants to tell us because it's good information once we've trained we don't need the historic data anymore right the we we train that model and then that model which was essentially the the the framework of some mathematical equations plus the parameters that help complete what that equation should be so we've got our candidate model or our model architecture rather plus the parameters that give us a complete model that is all we need to deploy the old data that we trained with and any other you know ancillary preparation code that that that helped to clean the the the training data before we trained the model we don't need any of that we think we just need that model and we can deploy that to production and in some cases we can even employ even to deploy it to to other platforms uh so so we have systems right glenn that will allow us to train a model in python but in some cases deploy it to um in environments for java or environmentsfor.net things like that yeah exactly and i i often like to use python as well to build my models and then i'll use use something like ml.net for example to execute those models in my desktop applications so you've got plenty of options as well and there's lots of good existing standards you mentioned a standard in one of our earlier conversations jason onyx probably yeah the onyx i'm actually built into windows yeah built into windows and um it's a thing that that ml.net uses under the hood when it uses a pi torch model for instance yeah cool so you absolutely you've got that ability to move every everywhere as well so go develop your model where you like it and transfer it to where you need to go as well there's even models for mobile apps as well transfer it to places that are supported it's not quite universal universal portability but there's at least some portability with that we actually are up on another demo that means more code more hands-on more visuals all right so let's switch over to the the last exercise we're just going to use this machine learning model in a more realistic light scenario we're going to kind of emulate what the w harness business would do if it had some real data and had a new sale coming in and what that user experience might be we're using a console app for this but you can imagine that this would be very small you might have on a website as well so we want to do a few things we want to train the process and we also want to save it to disk and we want to load it from this saving the disk unloading from this doesn't sound too complicated but it's actually really important when you remember that the training process might go through millions of rows you don't have to train the data every time you you actually want to execute the model so once we train the data we have the parameters we want to save those parameters to disk so when we're executing a website we want to then load the parameters from disk because then we've just got the the values that we need those parameters then we can use that to predict in our case what the boot size should be based on the harness size and and then once we have those results that we can inform the user of what to expect yeah you are having problems with cable again there um absolutely but yeah i i i t to your point uh glenn the the notebook can almost be viewed as two sections because we do have a starting section where we once again prep data train the model uh you know do do that stuff that we've done before but we have a second section to this notebook that you can almost imagine is more like our our our deploy it our run time our things are happening in the real world now section and and those two sections are separated by saving that file because when we save that file and when we then later and reload that file even though in our notebook those two cells are right next to each other they could be months apart they could be on different computers right all those things it's just a file at that point that model is just a file and so we've got it trained it's done we don't need the data we don't need the the old library that trained it and all that other stuff yeah that's exactly right once you you've actually trained the data you just need the parameters and you can work with the the the model there all right so our first step is actually very similar to what we've done in in previous steps where we've actually just read the the data from the the bhutanas csv file and we're displaying the head of the structure just to get a shape a bit below that that's where we go ahead and train the model this is very similar to what we had before but it's actually just done in a single line of code this time we're using the ordering least squares that linear regression model saying that the boot size is dependent on the hardest size and then actually it's calling the fit method at the end of that in order to do the training and then saying that the data is trained and i can see you've executed that as well jason so all the stuff that we've gone through before but the next step there where we're saving and loading a model this is where we're actually saving it to disk so our azure sandbox that we're running at the moment we're storing that in the avalanche dog boot model pkl file and then we're calling the dot which is really the save metal save method where we're saving that model and passing in the file name and then just printing out the the data is saved and at that point we have the model and we can actually use that in an execution environment as well yeah so this is the part where i was saying that we we we saved the file that file could then be sent over to a remote server somewhere to do some remote things to to to run the the model remotely without having any connection to our development environment to our training environment whatsoever right exactly so once we've got that file we can work with it as we need in our case here we're just going to load the the model in there as well and display the parameters the reason we're displaying the parameters is is just to show that the model we have access to that model information in our case here we've got a couple of values we've got the intercept and the harness size so the intercept uh if you remember back to mass that's that's where it would intersect the x-axis at at point zero so it would be 5.719 etc at that point and then every increment of harness size uh would actually go up by 0.585 so that's the slope of the line glenn's video was kind of frozen but he was he was he was showing us a graph with his arm right there yeah that's that's true this graph oh my goodness these overseas cables we've got to replace the street i think so all right let's let's let's talk right we talked before about how with our simple model here we're training the the intercept like you said that's where it's going to intercept kind of the kind of the uh the the origin point but we also have the slope so you know like this but that's not what those say that says intercept and harness size and the reason for that is because that is the slope of the harness size dimension so again if we imagine those parameters being being laid out in a in a chart in an xy uh chart then we can imagine that we're talking about the slope that matches harness size and if we had many more parameters we'd have a third dimension a fourth dimension things get really hard to actually imagine in your head once you get beyond three dimensions but the point is there's slopes in all those dimensions but that plane or super plane hyperplane still is going to intercept the doggy boot axis at some point and that still gets to be called the intercept excellent that'll be a lot easier to picture in your head what i just said once we get into um the the multiple parameters for our regressions which once again more more teasers for uh episode four all right but but now it's time to actually put it into a little bit of practice and then actually show what it might be like or the the experience might be like when we're working with the website for our little doggie boot store and harness store so just below that we've got a python method here load model and predict if we just scroll down there jason and this is actually a single method where we'll actually use that model that's been saved to disk so this method will pass in the hardest size and then we'll we'll look at the details so we'll load the model from that file name that we've already got to find and then we'll populate the the parameters so the parameters here will really be the harness size we'll specify the inputs the harness size will come from the one that's passed to the method and then we'll we'll call it the predict method in order to get what we expect that boot size to be and then return that boot size so this method will is it's called load model and predict but it's basically saying get the the boot size that you suggest based on the harness size and then yeah and so yeah so so exactly what you're saying it's really good to to to remember that at this point we're done with training data it's done we're finished with training data we're finished with deciding what our intercept our slope is all that stuff is done this is essentially a unit test and so what we're passing in here is one number that's one number not not a record not not data not not one with an answer this is one single number a harness size and we're getting out one answer right so it's just now we've just got this function predict that's doing work for us and not using the historic data anymore etc yeah i i like the fact as well if we had to train the model every time there's so much data to work with but when we're executing it's really one multiplication and an addition it's actually super quick so that's one of the nice things about running the models after you've gone through the training notice though that we're asking for doggy boot sizes of 32.08575 boy i don't know if we sell dog boots in that size no but i i imagine it would probably round up a little bit as of that and you you'd want to expect that yeah and the rounding might be to a whole number for doggies like if it was humans would be 2.5 for example increments for shoe size and we actually hear that right yeah we do we do we do some rounding of the data here in the method below so this is kind of like a user experience that someone might have um where we're basically calling check size of boots we pass in the harness size of the boot size and we go through that process of using that method to check what the boot size would be based on the harness size so that's that load model and predict and then we we then round the the boot size that we get back and then if the selected boot size matches the the estimated boot size then we're in a good position and we go and inform the customer let's just say yes we think those boots will fit if it's less than the boots size or if it's larger than the boot size then we give them a warning about the fact that we think that the selection is is not the best fit for you and and would suggest you you get a particular size a wee app with that size as well and the last line of that method is that the check size of boots passing in the heart size of 55 and the booth size of 39. so if we go ahead and run that cell i think you may have already done that as well then we could see that with the the heart size of 55 then it's probably too big for the the dog at that particular size and we recommend a doggie bit size of 38 so that's some really useful value of using data that we already have to to suggest to customers um that they may be buying an inappropriate choice and that saves a bunch of time for the customers they don't have to do a return especially if we're posting it to them and likewise for us we don't have to support the customer as much and it saves us time and money in that process as well so we're not on both sides and a good effective demonstration of a very simple machine learning model yeah and and quite frankly even though this is is is a relatively simple model under the hood you know the one multiplier and then the one edition um you can actually kind of easily imagine someone taking this class right now this course right now and then going back to their their their mom and pop shop um or maybe even bigger than that and opening up a jupiter notebook and just kind of having it at the side all the time to to to help their customers now not everybody has to be an enterprise and has to have has to have a team of of hundreds of data scientists doing doing things with with crazy complicated algorithms you can adapt this to your store that you have your corner store uh and and start helping your customers today just by having a jupiter notebook you don't need to publish it to a website this is absolutely something that could be useful in the real world just as it is awesome so that was our look at the demo we looked at a very basic model we'll look at some other models in later episodes of the series too uh we trained the data we saved the data to disk we loaded loaded the data back and then used it to execute something for the customer really great stuff yeah you know uh we have a question from peter uh he wants to know why did we decide to go with linear regression at the beginning is there a way to decide what method to follow this is kind of similar to to one of the questions we we um handled earlier and in in i think the the probably the real answer is that we're going to address that a lot more in uh in episode two and in episode four um but i will give a quick answer and linear regression is great for continuous values um and there are other options for continuous values so i don't want to say like that's that's the only one or if it's a continuous value then you're you're immediately going uh to to use linear regression but but it's a popular choice and it's one you're definitely going to want to try and at least see how it works out in most cases what i mean by continuous values is um really what we saw with the doggie boot we saw that at some point it predicted 37.0847 um so that's a numeric value where we can get get a value at any point in this in this numeric range and we did actually end up rounding it so so we're playing a little fast and loose maybe with the rule of of it being continuous uh but nonetheless for for numeric values linear regression is a very good choice um numeric values where where a larger number means a larger number in other words even though it's not necessarily having all of the decimals 37 is smaller than 38 which is smaller than 39 so they're they're real numbers they're not like tokens or placeholders that's probably more involved an answer than i really intended to give to that question the real answer like i said is we'll dive deeper in episodes two and in episodes four on that yeah it's a really good question i i think for today's episode it's an introduction series so we we we selected one that was very simple and easy to demonstrate so you can understand the core concepts um it's one of those things the more models you're aware of the more you can make the appropriate decision so just like architecture in general understand as many models as you can okay i have this problem i'm going to apply this particular model also we've also got another question there how do we safeguard user privacy while allowing machine learning development that's a great question do you want to speak to that jason i actually am not seeing the question so would you read that to me again oh here we go how do we safeguard user privacy while also allowing ml development well that is an extremely complex question um to be honest with you if you have um private data um or even things that that that that can kind of impute private data or or imply private data um then then you want to be very careful with that you want to be aware of that um so so and and then at that point when i say you want to be aware of it okay but then what um sometimes the answer is simply going to be then don't do it um that that's actually one very good answer if you need to rely on on on private data or on um on data that is you know of a of a protected nature then sometimes the answer is don't do it sometimes the answer is you maybe shouldn't auto uh auto approve credit card with no human interaction uh if um or auto decline credit cards with no human interaction if your model is based on all this private data and all of this data that's probably somewhat of of protected natures another way is to simply have a human in the loop so let the machine learning models do a recommendation uh but then have a human in the loop to make sure it makes sense and use some of the technologies that we have for for for machine learning models to explain themselves in other words justify themselves they can they can tell you oh i relied on this column or i relied on that column more than the other ones and so that can help but it's it's a it's a big long topic of its own yeah i think there's there's lots of existing resources for ethical uses of machine learning as well even at microsoft here we have our own framework as well for those choices that we use um but yeah that's probably another topic and we do have learned modules on on responsible ai and on the use of some of the tools that are part of the azure ml uh offering cool all right jason i can see you've got the knowledge check uh slide on the screen which is awesome hopefully everyone on the call has gone to that link https dot ms learn live tv where you can actually answer the the questions that i'm about to ask you jason are you ready for more questions and uh hopefully everybody thinks so i'm gonna try to be i'm gonna try to be less wordy we're actually getting low on time and plus man i've just been really wordy answering these questions hammer but it's been interesting good work that's fine all right question one question one what makes machine learning algorithms different from traditional algorithms is it a machine learning algorithms are always more complicated to build than traditional algorithms is it b machine learning algorithms must be trained every time they're used well c machine learning algorithms are shaped by data directly as part of development and traditional algorithms are based almost entirely on theory or of the opinions of the person writing the code well so with a the the idea that machine learning algorithms are always more complicated it should actually be the opposite um for for like for like now machine learning algorithms can be complicated don't get me wrong but for like for like if i wanted to classify spam well the code that imperative code that i would have to write that would be good as good at classifying spam as the ml filters are would be much much more complicated in fact so much more complicated that i don't know how to write code that's as good as the the basic ml models and and neither does anyone else that's why we're using the ml models instead of the imperative code no i hit the button sorry you did we gave it away you know what it's not so bad jason because everyone who answered this question in the chat got it right we've got some smart people they're called yeah eight eight out of eight from the chat or even so yes machine learning algorithms are shaped by the data as part of our development process excellent learning models do you get tweaked during that development or training process all right cool let's move on to question two then question two when do we wanna perform training is it a when we want to use the model b only when we want to improve the model or c every time we load a model from file i think we've covered this a few times but let's drill through the answers all right so is it whenever we want to use a model will we do training the answer is no because again training can be fairly expensive we want to use the model which kind of acts as a simple heuristic for for this real world situation we're modeling so so it's definitely not whenever we're using it um let me skip b and go is it every time we load a model from a file well no because again we're loading the model from the file in order to use it so we're not going to have to retrain it every time we trained it then we saved it to file so that process of elimination tells us that we're going to train the model whenever we want to improve it now the most obvious time when we want to improve our model is when it's brand new and we just opened up our our brand new uh our brand new development environment or jupiter notebook and we said all right we're starting from scratch obviously that's that that's that's that's a model that's gonna need to be improved but we might need to improve it later as we as we go on as well all right so looking at the results most people got this right actually so yeah mostly not not quite everyone that's good that means nobody's cheating that's that's nice to know we know if we've got a couple wrong answers that that it's all real that's all right but this is part of the training process for them as well but they'll learn that it's actually only when we want to improve the model right and and and that's right by the way it's expected we need better questions if we have everyone getting all of them correct that's not we should not be expecting everyone to get all of them correct we want these questions to be a bit challenging we do all right moving on to the third and final question for today what's the relationship between a model an objective and training data is it a the training data is used to make changes to the model these changes help the model get better at achieving the objective is it me the training data is used to make changes to the objective these changes help the objective be more like the model or is it c the model is used to make changes to the training data and these changes help the training data get better at achieving the objective so let's take this one in reverse order so so is it that the model is used to make changes to the training data and here the answer is definitely no our our training data is actually meant to help us tweak the model not not vice versa uh so not that one training data is used to make changes to the objective well no the objective helps us know how good the model is so we're just we're just using that to see how good the model is we're not actually going to be changing the objective as we go so that leaves us with a the training data is used to make changes to the model uh and and this is definitely the best answer in fact it's really our optimizer that uses data and and makes changes to the model and the model's parameters but this is definitely the best of our answers and let's see how do we do in chat we're getting pretty good yeah most people got that i think there was an issue with the chat where the third option wasn't available so but almost everyone selected the first option that the training data is used to make changes yeah buy the optimizer like you said awesome wonderful i think everyone did amazingly well at that that was great yeah absolutely well done folks well done right and that's good we're up we're in a very good place all right and just just uh to summarize jason you want to take us through what we've covered today right so we talked about what do we even do with with machine learning why are we doing it of course the kind of short answer was we're creating something that's very similar to a function but but can do things that we either don't know how to do with with imperative functions or uh wouldn't be able to do very efficiently so it's just another way of getting getting essentially a function something that can process our inputs and give us the desired outputs it's just a special way of doing it we talked about that special way which is the training process now we talk about how to handle our data so that it can work us through the training process and and actually give us uh the desired outcome and that's awesome today but today isn't all of it there's a lot more to go through and it's really quite exciting um like next week we'll be looking at building classical machine learning models after that we've got really a great range of sessions as well looking at the data from machine learning training regression models there's other concepts as well like architecture and rock and a few others as well and in fact there's a question on the chat window i really like um someone's asked i have no experience in data science but would like to i would like to grow into it can you suggest the next steps of getting certified after i finish this awesome series of learn live thank you what well obviously going through this is one of the fundamentals of this but also with all the certifications for microsoft for example you can actually go to the certification page and it will show you the suggested learn training that you should go to for a certification as well hopefully one of our moderators can paste one in that that's probably appropriate as well um jason you want to speak to that yeah actually let me go ahead and flip to the next slide because what i want to do is when you go to to our our um page for this particular module um you will you will find yourself as in a learning path um in fact i think i think you can even just do aka dot ms slash learn ml with no with nothing else i think that'll actually take you to the learning path and this series is walking through option two which which is our our is is the most friendly for this format i'll put it that way however option one includes five additional modules to it that that that interweave and and uh kind of uh reinforce the same concepts but use different libraries use different techniques but they work on the same concepts just kind of building you up and giving you a different uh perspective so that's one great way is just to do the additional modules that are in this overall package but also search the rest of learn we've got some pie torch content out there and on episode two we've got as one of our moderators an amazing cloud advocate um jen looper who has a actually i think one of her one of her team members is actually with us today as well um yeah if you can uh post the um the the deep learning uh curriculum that is out in github if we if we're able to get that i promise we'll have it for next week because jen looper will be here and she was kind of the lead of developing that um but yeah we'll get the link to that so that's some some amazing curriculum that uh works with some deep learning libraries to uh to do uh things like computer vision and nlp and they don't really require uh the fundamentals that that we're doing here to to get into it you can start you can start playing while learning the fundamentals in parallel with us is one way to do it uh so there's a lot of different ways to handle it because some of the libraries handle a lot of the difficulty for you yeah that's great uh and also one of the acronyms you mentioned nlp which is quite popular which is natural language processing thank you for for calling me on that yes i i do i do have a few acronyms that that i that i are so embedded in there that i forget to define awesome you won't let me get away with it i'll try not to so the next slide we've got is actually our q and a slide we've got some time i think for a few more questions uh there's one in there jason uh what is auto ml and how is it different from what you showed us today so auto ml is is a really interesting um feature uh auto ml is or or automated machine learning as as the the features properly called and to get the long name uh as it's properly called um in in our azure ml offering will find it'll actually look for different candidate models for you and it will do models like linear regression like like we did today um it will use things like random forest which we'll use in a few uh in a few episodes it'll also look at some deep learning models some of the some of the deep learning models and it will determine which one it thinks it is best um what's interesting is it's like an outer auto ml does does make things more complicated too so so those of you who don't want things to get more complicated maybe just cover your ears right now it's almost like an outer loop remember our our main training loop helps us find the parameters for the model that we chose if you can imagine an outer loop that chooses the model for you and then the inner loop finds the optimum parameters automl is that that that outer loop uh and sadiq wants to know instead of python library and he thought to use ml.net instead uh so so ml.net isn't uh entirely a replacement server for for some of the the python libraries um it it can be for some tasks so if you're a net developer and you do want to do linear regression for for example then yes you can do all of your work right in ml.net ml.net can also execute some models that were trained in python so you could do the training process in python export them to a model format that we discussed a little bit earlier called onyx and then that that onyx model can be used by ml.net not all the time there are some limitations but they're well documented and then you can anywhere you can can run.net you'd be able to execute that library also of course embed it in your in your.net code if that's what your you know your application and existing code bases then then that helps a lot excellent and i think that's our last question for today as well jason so i think thank you everyone for coming today to today's session um and we hope to see you on the next episodes that we have over the next nine weeks bye for now next week same time bye [Music] you

Channel: Microsoft Developer

Views: 3,831

Keywords: Microsoft, Developer

Id: Bhc3Kn1IkI4

Length: 91min 58sec (5518 seconds)

Published: Tue Sep 14 2021