On .NET Live - Adding Machine Learning to your .NET Apps with ML .NET

[Music] [Music] do [Music] [Music] do [Applause] [Music] do [Music] and we're live hi everyone welcome to this week's on.net my name is cecil phillip and today we're going to be learning about machine learning with emerald.net and you know i think this is a perfect time to show because just a couple of weeks ago we were talking about how the folks from.net interactive were on and there were just so many ml.net questions and so i'm really glad that bree and lewis are here to talk to us about it so why don't we kick it off with some quick introductions bri why don't we start with you why don't we let folks know who you are and what do you do all right hi everyone i'm bree i work on the dotnet team and i am the pm for ml.net uh you might have seen louise and i talking about ml.net and various other conferences or community stand-ups um but yeah so any i'm very happy to be here to answer all your ml.net questions nice and my friend luis hey how's it going folks i'm luis i am a content developer working across a variety of products such as azure machine learning dot network spark and of course ml.net as we mentioned we uh do a lot of these so you you know you might have seen my face uh someplace else but like brie i'm just happy to be here and talk to you folks how you can add ml.net to your apps awesome and also too i want to give a shout out to our folks that are here watching us live so we're streaming right now on learn tv youtube and also twitch and i can see in the chat we have some folks here from turkey um someone's here from austria hey gregor what's going on definitely really appreciate y'all spending your time with us today um before we dive into the topic why don't we take a look at some links i have some some things i want to share with everyone um before we kind of kind of start talking about ml so i'm going to share my screen and uh the first link i want to share oh i'm not sharing my screen come on you could do it there we go all right so first thing i want to share this week on the community stand up on you know on the dot net youtube channel they had a discussion about minimal apis which i thought was really interesting so you know they had david fowler stephen and damian speaking about like some of the enhancements they're trying to make to the developer experience for apis with asp.net core um so again this is already you know was live and it was all recorded so definitely recommend you go ahead and check this out and i have a feeling we might see a little bit of this later today maybe maybe not we'll see what happens right the next link i want to share with folks so again just in case you don't know we are available on dotnetlivetv and if you go to dot.net live what you'll be able to do is you know you can scroll through here you can see all the different shows and community stand-ups and things of that nature that that we have streaming on.net live tv as you can see right now like this is us so you know hopefully we're live and hopefully you're seeing us right now but what you could also do you see here on the right side you can actually click on one of these links and check out some of the other shows and past recordings and stuff so for instance if you wanted to see the ml.net community stand up you can actually click on this category here for instance click on machine learning and then here you can look at you know some of the past live streams and things of that nature that they've done so if you've loved what you're going to see a little bit later today definitely make sure you come over here and check out some of these past recordings for those live sessions that we have and the last one that i want to share um so i'm in south florida so i'm a little biased to things that happen in the southeast um no offense to anybody else but we are having a south southern data science conference um that's going to be happening on july 17th i think it's going to be over the course of a few days and you can register for it right now so again if you're in the southeast or not you know um it's going to be completely online and definitely something worth checking out um what i'll do is i'll take the links for all these and i'll show them inside of the chat so if you're interested in taking a look at you know either that recording or you know this this data science conference you know you'll have those links that you could go ahead and um and then click through now with that being said my screen's going to go away because like i always like to say this is not the celso show today it's going to be like debris and louise show so why don't we start talking really quickly about like ml.net like give me the elevator pitch like why is it why should i care about it what is it for you know i i'm a dotnet developer i've been a dot-net developer for a long time i don't know anything about machine learning right like so so tell me how does this thing help me out sure um well i guess our goal with ml.net is to as we say democratize machine learning for net developers so you are a perfect target customer um ml.net is actually a machine learning framework it's open source cross-platform and it's for net developers so essentially it allows net developers to stay in their ecosystem you can use c-sharp f-sharp to train custom models consume pre-trained models yeah so you can add custom machine learning models to your applications and it makes it really easy because right now there's kind of this problem in the industry where um you say data scientist or a developer that you train in python and then you have this model that's been trained in python and trying to integrate that back into a.net application or get that model into production is not very easy um if you use net then you know it's very easy to go to productions.net and ml.netiz.net so it makes bringing machine learning to production very easy yeah i think for me it's always a case of i mean i understand what ml is from from a bird's eye view right like i understand like some of the interesting things that um it's enabled but where i struggle is always trying to figure out well how do i add it to my app and then what kind of what kind of value can it add for me so that so i think that's one of the things that i'm really looking forward to for us talking about is again for you know your typical line of business developer or you know again maybe you're building mobile apps or you know any of these types of things you know what are some of the really cool use cases that we could you know use with ml.net yeah so i mean there's a ton of different use cases um for training classic or classical machine learning models um there's you know classification regression anomaly detection forecasting and that might may not mean anything to you if you've never done any machine learning stuff but we try to simplify that in the tooling that we have so for instance when i say regression regression means trying to predict a value um and so we we kind of frame it as value prediction that could be predicting that the price of a used car uh predicting the price of a house based on the fee you know how many rooms and bathrooms it has as a practical example there is a company called evolution software who's using ml.net to predict the dr they're using iot sensors in these huge commercial dryers of 50 000 pounds of hazelnuts to try and predict the ideal moisture level of hazelnuts and so that's a value and they're using ml.net to predict that regression value um internally it's used at microsoft for a ton of different things if you've used a power bi anomaly detector or key influencers um powered by ml.net and you know it's machine learning so louise i don't know if you have any other cool examples to add yeah not so much an example it's kind of go taking it back just a little bit you mentioned as somebody who's starting out it may be hard to you know what you know bells or which buttons to push right and then certainly tooling does helps you a lot with that specifically the automated machine learning which um ml.net has but so do of many other frameworks and and even for practitioners right automl does a lot in terms of uh helping uh machine learning uh sort of you know data scientists and machine learning developers uh helping them find out what those what are the best things to choose based on their data right so so it's not even for folks who are just starting out though it certainly helps those folks um but also for folks who are already uh you know experienced and seasoned practitioners um it helps them sort of speed up that experimentation process which which the process of building these machine learning models is very experimental right for sure for sure and so bri i know you had some stuff you wanted to show us why don't i go ahead and put your desktop on camera and then we could kind of go through some of some ml.net stuff totally so i kind of already gave you an overview a little bit of history ml.net started as a research project in microsoft research for text mining and search it developed into this internal framework that's been used for over 10 years and a ton of different popular machine learning features and products and then became cross-platform and open source at build of 2018. that was the first private preview of ml.net and since then we've just been growing and growing um so a point something to point out here is that ml.net is extensible so like i said you can train custom machine learning models with ml.net and then consume those models with ml.net or you can go out and you can train with tensorflow or pytorch um convert that to onyx and then consume onyx and tensorflow models in ml.net and a really um uh popular use case here is you are a developer you work with a data scientist um who works in python so we don't want to pull people away from python we're not trying to pull python lovers or you know data scientists away from python we just want to enable uh net or we want to make.net great for machine learning so data scientists could train let's say in pi torch they could convert that to an interchangeable format called onyx and then you could actually consume that onyx model in ml.net which makes it super easy and um i'll come down here just to show um we talked about some examples so these are some of the more practical business examples these are linked to our samples repo so it actually has the training code and how to consume it there but you can see you know anywhere from product recommendation to object detection to detecting sales uh spikes um you can see a ton of the different examples that you can use machine learning and ml.net for and i want to talk about the perf a little bit a lot of this comes up a lot is you know how does the performance of ml.net compare to um the other you know python frameworks uh machine learning frameworks and so here there's this experimental evaluation done between ml.net scikit-learn and h2o which are other popular frameworks normally used for classical machine learning and they did a bunch of experiments for different scenarios so in this case this was sentiment analysis which is a classification scenario and they used this huge amazon review data set um for the full data set which i can't remember how exactly how large it was but for the full data set scikit-learn and h2o couldn't even complete the training process they ran into runtime errors and ml.net was was able to run to completion um with a smaller subset of the data it was about 900 megabytes of amazon review data set uh ml.net outperformed both frameworks in terms of both accuracy and runtime so that's the amount of time it took to train and test so yeah so you can see you can read the whole paper it's on the site here i'm sure we can provide a link in the chat but there's a whole paper that you can read to see the other evaluations that were done for the other different scenarios there yeah it's always interesting to me when folks ask the question like why do we have this thing in dot net you know what i mean it's kind of like when you look at just kind of look at what programming languages offer in general right like no one asks oh why can i do web development in python why can't i do this thing in go or rust right i think it's just you know we have to look at it from the perspective of you know we need as a microsoft particularly we want our developers to be able to have these experiences regardless of whatever type of application they want to build right so if you want to build again web or mobile you want to build a container thing you want to build a cloud thing you want to add machine learning and or you know even just work on research projects and maybe build something a little bit more inter interactive with graphs and charts and things of that nature like we want to be able to do that and still leverage our skills as dotnet developers and so for me even though i'm not like a data science scientist and i don't work with this stuff a lot for me this sounds like the perfect place that i would start because now i don't have to go learn another language i could be like i already know c sharp i'm already filling with.net i already have visual studio and you know my environment set up the way i wanted to be set up now i just have to learn how this additional piece works so i can start adding into the application that i already have yeah and that's that's one thing is if you don't know python obviously it's gonna be a lot easier to learn machine learning in net but the other thing is even if you did know python and wanted to use python there's complexities and you can take some performance hits trying to integrate python back into net so that's why we make it easy in both cases if you want to stand.net for everything you can with ml.net or if you wanted to train in python and then bring that back into.net through ml.net you can do that as well yep yep for sure um and i think that's all i have here i can talk about some of the use cases but i can also get straight into the into the code and how to get started yeah let's do that let's look at some code sweet so i'm going to open up visual studio and what i've done is and i'll come back to this here i've created a net5 console application and i am actually in these few lines of codes consuming an ml.net model i'm gonna walk through what's happening here um so first i'm prompting the user uh to write a review of a taxi driver so in this case i have a model that is going to tell me if the review left by a by the the passenger is positive or negative sentiment so prompting the user for to write a review and then what it's doing is it's creating this model input that has this comment property so whatever is read in from the console is going to be set as the comment that is left as the review then what we're doing is we're calling this predict method on the sample data so it has this comment here and it's going to return this result the result is going to contain the prediction of positive or negative so here if that prediction is positive we're going to return a positive string if it's negative we're going to return something negative here so i'm going to go ahead and run this and then i'm going to get into how i how you actually train a model like this and again this is just a console application [Music] so if i'm going to write a review of my of a taxi driver um i let's say throw too fast and so we would expect it to say negative and let's see give it a second to so what it's doing is it's actually using the model to predict based on this text that i sent it is it positive or negative so in this case it says we saw you had a bad experience in that case you know it detected hey this is a negative comment um now let's try it you know with a positive comment and see if we can um get a positive result here if our model is performing correctly and this is just taking my free form text and saying hey based on this text it's positive or negative let's say um car smelled good and hopefully we would hope that it would be a positive sentiment yeah i think that should be a positive sentiment yep so in this case it says thanks for the great review so it predicted hey this is positive so um yeah that is consuming the model just in a in a uh net core sorry.5 console application this is and sentiment analysis is very much like a hello world um machine learning type problem um which i've shown here so now i want to show you how you can create one of these models on your own and do that really quickly without having machine learning knowledge really that sounds perfect because you know i'm i'm a big fan of being a lazy developer and i mean having things just ready to go so there's components that i could kind of just plug in and start using like that'll be fantastic definitely so i'm going to use what is ml.net's tooling in visual studio called ml.net model builder um the way you do that is you have your project i'm going to right click on my project and i'm going to add machine learning as easy as that oh that's cool okay yeah so we'll give it a second it's going to open up an add new item dialog if you've used model builder before this will actually be a little bit different some different than the experience we had before before i would just open up the ui and go from there and there was no saved state but in this case you'll see and i'm going to name it let's say taxi fare model and then just for folks that may have not seen this before so you you went you right clicked and you did add machine learning is that something that's just built into visual studio or do you have to like install a thing or create something to get that while it's adding here i'll actually show you i think i have that link open so it does ship with visual studio but it is a preview feature so you have to do a few extra steps in order to enable that um and i'll show you that here this is the getting started tutorial on the website um from dot dot net slash ml so in the visual studio installer and your dot net any of the dot net workloads will have this you want to make sure that ml.net model builder is checked so make sure you modify and make sure that it's checked and then once it is and you're in visual studio you want to go to your tools options environment and preview features and then there'll be this enable model builder thing uh button here you want to make sure that's checked and as long as that is checked you will be able to right click on any net project and add machine learning [Music] got it okay cool and that is what i've done so i right clicked added machine learning did not mean to do that um and it created this um mb config file here um i'll i'll dig into this envy config file at the end of training but what it's doing is it's uh it's just a json file that's saving the state of each step so i'll show you that um once it's done training but this is the ui that opened up as well that is associated with this mb config if you've used winforms before or wpf before it's kind of like the designer and then it has it'll have some code behind so if you double click this it will open up the designer ui so my new model i want to train a model that will predict the price of taxi fare based on say distance traveled number of passengers payment type and a few other features so in that case it's this regression or value prediction i want to predict a value so i'm going to click the value prediction here [Music] and for some of the scenarios you do have the option to train locally with gpu and or train in azure ml um those we have for our deep learning scenarios where you might have a lot more data that you need to train and might need more resources for that in this case i'm going to just train locally with my cpu so my next step i'm going to choose my my training data what i want to use to train the model or fit the model to i have the option to choose from a file or sql server i just have it ready in a file so i'm going to choose it from the file here nice i really like that scenario based like ui you know what i mean so for me that's that's definitely something that i could wrap my head around it's like you know what's my business case for this thing you know what i mean i could pick from the available options and then kind of go forward from there right that was very intentional um was to make it more scenario focused and not you know caught up on the data science machine learning terms there sure so some questions that folks are asking um can we use this outside of visual studio is this ui available outside of visual studio so meaning vs um for mac and like what happens to vs code folks as well yeah so right now um the ui is only in visual studio um but we do have a cross-platform solution which is the cli it's just a net global tool it's called the ml.net cli it does the same exact thing that this does there's a few scenarios missing in the cli we're working to get scenario parity there um but that will work on you know mac linux windows and then we actually had a community member create a model builder extension for vs code which is very interesting um we're in the works on the team trying to see if we can actually open source um this so that's something that we're actually working on so that members of the community if they wanted to extend it for whatever purpose they want they can right now unfortunately it's not so the ui the short answer ui is only in visual studio but the functionality is available across platform with our other tooling i think that could be an interesting use case if folks decided to extend it and then now they could add additional scenarios to that initial that first window that you showed us it'd be interesting to see what what kind of things people decide to create there yeah and we had um a community member who he there's some functionality missing and we've since added it with uh choosing the column type or column purpose of the different um columns in the data set and he ended up creating his own version of model builder in a web app um where that did the same exact thing but with the added functionality that he needed uh which is very interesting yeah so that was pretty cool very cool all right so um you can see that i have a little um sample of my data set there's about 50 000 rows in my data sets just showing the first 10. it has some sort of vendor id a rate code number of passengers um the trip time of the you know the taxi trip time the trip distance whether it's paid with card or cash and then the actual fair amount at the end so we're going to use this data set to train a model so obviously right now we have a bunch of labeled data we know what the fair amount is we're going to use that to get this model that we can then use to predict on new data for data for fair amounts that we don't know yet and again this is called a regression task and so i want to predict the fair amount so that's what i'm going to do i selected my data set i selected fair amount as the column that i want to predict there are some advanced data options here which you can like i said choose um if you want to ignore columns if you don't want to use them for training if you want to change the data type and like changing whether it has a header or not but we do a pretty good job of predicting what your data set looks like but you can override that if you need to so then i'm going to come here and i'm going to train so i you can use a time to train it does suggest it based on the data set size sometimes you may need to train longer i'm going to train for 60 seconds because i'm streaming live and show it sharing my screen and everything this is all using my cpu i'm also going to talk about what is happening when i hit train so luis alluded to this earlier but automated machine learning or automl is this feature that ml.net offers and other machine learning frameworks do as well and what it does is it automates that model experimentation process so in the given amount of time that i give it for training it will iterate through different algorithms algorithm options based on the scenario and the data set that i give it and then at the end of that time in the it will choose the best model based on some performance evaluation metric um in the case of regression it looks at a metric called r squared so we'll let that keep going and i think it does show so one question too i want to pop in with an s really quickly so you mentioned just now that you're doing the training locally is there an option for remote training like maybe i could shell it out to the cloud or to a vm or something and have those results aggregated and pulled back for me yeah so um before i answer that question it looks like i didn't give it enough time to find a model and again that's probably because i'm streaming and everything so let's give it a little more time to train um let's just double the time and see if that helps um so and to answer your question about training remotely and in this case it might have helped right now the only the two scenarios we support for training remotely are um image classification and object detection which are kind of it's part of the subset of machine learning called deep learning um and it will take a lot more resources because you know it's image data so um you can and that's actually well right from the ui will connect you to azure ml and it will train using azure ml and then bring back down your your model as an ml.net model got it got it and then speaking of deep learning um so jose in the chat he's jose is uh coming into us from youtube jose is asking how can ml.net how to call emerald.net and how to call tensorflow or pythorg i've got some i'm guessing he's asking how do i do deep learning slash pytorch slash tensorflow integration with ml.net sure louise you want to answer that one uh sure yeah um yeah so you can extend ml.net if you have a tensorflow model there is a transform uh in ml.net uh which you can i believe it's called uh score tensorflow um and basically you can take your your uh protobuf file or your pv file in in tensorflow and you can bring it in and you can score and make predictions with that model um with pytorch you can do that if you you can score pi torch models if you uh convert them to onyx first so the transform there is called apply onyx and that will essentially allow you to make predictions using your pi torch models so that's how you would do that inside of ml.net and hopefully that that answers your questions jose sure and then we have one more question coming in uh that i'm going to try and get up really quickly this person is asking should we know python as well so i'm guessing it's around not just specifically for deep learning but just in general how much do i need to know python to get into machine learning and stuff or you know can i just kind of get away with being like blissfully ignorant and just focusing on that i'll let louise take that again yeah i think that well to get started with model builder uh i mean honestly you don't even need to know c sharp because the model is trained for you uh or net in general right because everything is sort of done for you you just click this wizard like experience of course later on when you want to deploy it and do other things maybe you want to tweak a few things of course you need to know.net but in terms of python you don't need to know much at all um to work with ml.net actually you need to know not at all to work with uh ml.net um but it doesn't hurt for example if there are for example scenarios that maybe ml.net currently does not natively support right such as the example that we discussed with uh with jose here with python's right perhaps you have pi torch and you working in python there in those scenarios of course it helps because you're going to need to perform some conversions uh in before getting that model into ml.net so um short story very little um but you know obviously it doesn't hurt to to know python sure and then our friend seth wars is is in the chat right now and seth saying one good workflow is to train in pi torch or tensorflow and use the models for inference in ml.net exactly yep nice okay all right i'll let you guys get back to it bring it back to our demo so in the time that i gave it it was able to explore one model and that one model used the fast forest regression algorithm and uh gave me an r squared of 0.81 which is not so bad it's pretty good actually um if i go to the evaluate screen um i can i can test out my model right in the ui so this just pre-filled up by the second row of data so when i hit predict it's using the model just to try it on the ui before getting into the code so it's saying the fair amount is predicted to be about eight dollars and thirty cents if i go back to my data set um the second row of data so it'll have all the same data gate give eight dollars so the actual amount was eight dollars and our model printed eight dollars and thirty cents i would say that's close enough there and so here once i trained the model these three files were added to my mb config um this zip file is the actual model itself so ml.net models are serialized zip files um this this training.cs file is the training pipeline that automl chose for my model so if we come down here this is going to look like a lot it was all auto generated by the tooling this is the data transformations that were chosen based on my data set and my scenario so you can see you know it appended quite a few data transformations and then it chose the fast regression algorithm so it added that what we call a trainer or algorithm to the pipeline and then it actually went and tuned some of the hyper parameters so you can imagine if you're trying to do this yourself you have to one know the data transforms needed for all your data in order to get it into the right format for the algorithm for training you have to then choose an algorithm and then you have to try to tune the algorithm options or hyper parameters in order to get your best model and in this case automl and model builder took care of all that for me and then generated the code and what's nice about this is if i wanted to retrain my model i could just take this file and retrain on the same pipeline which is a really common scenario if you're you know constantly getting new data or data is changing um you can even integrate this with your current cic pipeline and say you get every time you get a new data you can retrain with this pipeline yeah i know since you mentioned ci cd we did have a question in youtube right now and this one's coming from andre andre's asking how can we deploy ml.net and it's trained models so it's great is there a particular artifact that we have or is it just is it just c-sharp code and we're just funneling it back and forth or is there like a thing that has like a trade model so we actually that's actually part of the demo so if you wanted to um hold off for a second and when once we hand it over to louise we'll actually show you let's do that let's do that okay um i'll just do a quick look into this consumption file this is what you need to consume the model um you have this model input class which will be the data schema um we've seen this as you know the columns that we have in the data set and then this is going to be what's returned from the model um we will return a score which is our prediction or the predicted taxi fare um and then down here this is that predict function that um you might remember from the sentiment analysis example where you're putting in your model input that you've set you're creating this new ml or i guess the method is creating this new ml context which is the starting point of all ml.net operations and then you are loading in the model that was trained you're creating this prediction engine a prediction engine is a convenience api for making single predictions and then you're creating this prediction engine based on the data schema for model input and output and based on your trained model and then you can actually you can use that prediction engine's predict function on the input that you set so okay i have this model i want to use it i want to deploy it i'm i'm not the deployment expert i have someone else on my team that does that who happens to be louise so i've added the regression model i have get here so um i have this branch called ad regression model and then i'm going to commit this and we'll save everything there and once that is done i will be able to push it and then we'll switch over to louise um and while i'm doing this if there's um another question we can answer um yeah yeah we do have some a few questions um for sure so the first one that i want to jump in with um again this is one is also coming in from youtube and this person's asking does the designer only work in certain project types so for instance wpf or is it just available everywhere any net application that you did so i this was a console app um i've done it for wpf win forms any sort of web app um you'll actually see a web example as soon as we go over to louise um so just any net application will work um the only i guess caveat is it only produce generates c sharp code right now um but definitely uh producing f sharp code is something that we've has been on our um for a while as well got it and then another question also from youtube uh is there a minimum version of visual studio that you need to use the model builder yeah for the newer versions 2019 for sure um we do we i believe some of our older versions work on 2017 because we you know developed it i think a year ago um a year or two ago um so i would definitely recommend using 2019 for it um because any new changes that we make will be for 2019 and then um the newer version that's coming out later this year got it and then one more question before we let you get back to the demo um so this also coming in from youtube and the person is asking how to retrain a model with data streaming louise yeah so um so so this is a scenario that is currently supported so i presume uh niranjan that what you're referring to there is uh you want to immediately as soon as you get data or perhaps you batch it up and maybe after an hour or so when you've collected enough data you want to go ahead and return your model um so there's two ways to do that one would be to obviously just kick off this process again right so run the code that brie kind of showed off here for your training pipeline run that again with the new data set that you're running um the only thing to keep in mind there though is that it's not incremental retraining meaning that you lose the knowledge that your model had beforehand and you're just completely basically building a brand new model and you know that that is like i said it's just simple as re-running this script here right um there is also the incremental training scenario where as soon as you get in data you remember or it recalls the past data and just continues to add on based on the new data that it gets right so both scenarios are certainly possible um through model builder um the first scenario is the one that's available right where you're training with new data and then if you're using the api there are ways that you can um you know perform that sort of incremental retraining um but the other thing to keep in mind is that it's only on a subset of algorithms that support it so so things to consider there yeah and that kind of makes me think about like versioning right like so as you you know as you and you're doing like these incremental trainings and i'm guessing every training output is like a different version of your model right so outside of how we as developers generally think about like you know version control for our code is there any specific or particular techniques or patterns that you know that are used in the model training world for versioning our models or is it just kind of all the same thing yeah so there's uh something called ml ops which is kind of uh this broad term it's almost like devops for machine learning but it kind of covers um the ci cd portion and automatic retraining as well as experiment tracking model and data versioning um so right now it's kind of being defined in general like by the industry um it's still it's the best practices i think are still being evolved and um it's the space that we're currently working with in as well for ml.net and trying to figure out i think um azure ml does a pretty good job and i think louise can speak more to that because he works on azure ml they have their own ml op stuff and data and model versioning um got it and then let's do one more question and then we'll we'll get back to the demo and andre from youtube is asking which model format files can be used with ml.net um so the output of an ml.model when you train is a zip file but as for outside frameworks you can directly use tensorflow models in ml.net and you can use onyx models as well um so if you trained in pi torch you could convert that to onyx and then consume that with ml.net cool all right all right so i think your code's checked in now so what do we do now let's switch over to louise so he can pull those changes and uh get to deploying the model all right louise you have front stage all right awesome all right let me try the ui here i'm not quite as proficient with the ui uh getting stuff so let's go ahead and pull and you'll see that i'm using uh this branch that brie has created so i should be getting latest changes and there we go so before we didn't have this taxi fare model i'll zoom in a little bit for folks here um there's that taxi fare model mbconfig and again you'll notice that in it is also that consumption file the training file as well as the model.zip which is the actual model asset that we're going to be deploying now at this point what we want to do is we want to pick up where we left off so to do that we go ahead and click on the mb config file for taxi fare model uh you're gonna see that it's gonna bring us right back into that ui um so what i wanna do now is i wanna click on this evaluate tab here sorry the consume tab here and you're going to see that there's a code snippet if i want to go ahead and consume my application in some sort of console app and consume my model there uh i also have a option here to export a full console app similar to the one that bree showed off where she was doing some sentiment analysis um and and you can consume and deploy your model to a console app but you know the use case there may vary something that is a little bit more common is to take these models and deploy them as web api so you expose some sort of uh endpoint uh that you take in requests and the requests would contain the input that your model is expecting it will use the model to then make a prediction and then return the prediction back to you right so that's a fairly common scenario and you're going to see here in the sample projects the web api is actually an asp.net core web api that you can go ahead and create so if i go ahead and click on add to solution here you're going to see that it's going to ask me hey what's the model of the your api that you want to what's the name of your web api i'm just going to call it text model web api and then i'm going to go ahead and add it to my solution now what that's going to do is it's going to scaffold a brand new uh web api project for me and the web api project is going to contain uh three uh well two files really is this program.cs and this mbconfig file which is going to be that model again right so there's that so it automatically embeds the model that i trained uh elsewhere so you're gonna see that this is an asp.net core web api so you may be asking yourself well where's my startup file where are my controllers or my www root directory all right and none of that is there but it's still a full on asp.net core web api and just get out of this for a second here the difference is we tried to simplify it just a little bit because all you need is to have a single endpoint and to have a single endpoint you don't really need to have sort of all the bells and whistles that sort of asp.net core gives you out of the box right so for somebody that just wants to deploy this single asset and handle requests through a single endpoint um this is a way that you could simplify it now the technique that's been that is being used here is known as route to code and that technique sort of leverages endpoint routing um and when you use that technique you're able to sort of shrink down the size of your web api uh into a single file right right so you're going to see here this is going to look very familiar if you're asp.net developer right we start off with web host and we create our builder um these sections here the configure services and the configure are things that you would traditionally find inside of your startup.cs file but in this case again because we don't need a lot we just brought them all into here and we start off by configuring our services the only one service that we're going to need here is this ad prediction engine pool now uh one of the things that you have to note is that the prediction engine and that convenience api it is not threat safe so when you're deploying it to these um sort of multi-threaded and and these applications that are going to be handling a lot of concurrent requests like a lot of asp.net applications do you need something that's a little bit more robust and that's where the prediction engine pool comes in it basically creates an object pool of prediction engine objects that you can then just pick from as requests come in make use of them make a prediction and then you know sort of return them back into the pool to be used at reuse at a later time all right so that's what this service is doing for you um you're gonna see that i'm telling it hey i want this model input to be the input that i'm going to expect in my prediction engine and this is going to be the output and there's two ways actually to to use this prediction engine pool service one is you're able to bring in a model that's in a file now in this case we're using that taxifairmodel.zip file but the thing is that your font you may not always have your model embedded as part of your application perhaps it's in some other sort of blob storage or some other public endpoint right in that case what you can do is you can use the from uri and that's going to pull your model even though you may not be embedded as part of your application it's going to go ahead and pull it down from that location uh where that model is all right um another really neat feature of this uh service here is that it does automatically loading so going back to that sort of retraining scenario um where perhaps you got new data you had to retrain it um and then you you need to sort of redeploy your model or embed a new model inside of your application what you're able to do here is this service has this sort of auto reload um service that's checking in the background i believe by default it checks every five minutes to see hey has the file or has you know the asset that i'm getting my file from remotely has the state of that changed and if so pull in the latest version and you don't have to essentially you know take down or restart your application to do that it does that automatically for you so now that you have your your prediction engine pull service and you sort of configure it to be later used using dependency injection you then go to your configure method where you again tell it hey use routing because we're going to be using endpoint routing and we define a single endpoint which is this uh predict endpoint that's going to be expecting a post request now this predict endpoint will handle some of your requests and do stuff with it and to do that it uses this predict handler in the predict handler it looks very similar to to the handlers that your asp.net applications contain right it takes your http context then inside of this predict handler we go ahead and we request that prediction engine pool service we then just do some parsing of the body we serialize the body we use that prediction engine pool to make a prediction and then we write the output back out as json so that it returns the response to uh you know to what our client or whoever's making our requests here to our application so let's go ahead and test that out locally to see kind of what this looks like so we go here and then we start up our text affair model web api this is going to start up our server and then we can make requests against this so i'm going to be using insomnia here to make requests and again i'm going to be making a request to this localhost 5000 slash predict endpoint and while you're waiting for your thing to spin up really quickly we did have some questions pop in um this one's coming in from twitch and this person's asking can we also add it to an mvc project solution and i'm guessing when he says mvc they mean asp.net framework mvc versus espn net core mvc yep absolutely you you can um again as long as you're embedding this uh you know the zip asset right the ml.net model asset um you can leverage this at prediction engine pool register it using dependency injection right uh for the for use with dependency injection and uh just tell it hey this is the model that i'm going to be using and now you can use it throughout your application you can use it in your action methods right of your mvc and and so on and so forth um and you can use it in blazer as well so yes sure and then uh andre also from youtube is is asking can we train our model offline just using ml.net yeah that's what we did here i'm guessing that's yes yep yeah ml.net is offline first um and you'll also have the option again for those scenarios uh deep learning scenarios to go to azure yeah and then gregor is in here asking a question any plans of adding the same functionality for the time series prediction engine i'm guessing y'all know what with that i have no idea yeah that's a really good question um yeah go ahead no i i so louise and i were actually looking at this recently because we were doing um a demo with forecasting and i believe that you don't need to have this um there was a whole issue on github about it um that's they were commenting i i think you don't need to have it be thread safe do you remember what the reasoning reasoning was louise yeah no no no so so to be clear the for time prediction so for time series predictions it is not built in so you are right gregor that is not built in um but we're exploring as to whether you might need it because time series time series prediction is a little bit different in the sense that there is no model per se right you just give it some information and it tries to um understand the patterns at least that's certainly the way it works it turns to try to understand the patterns just use analytics like that but it doesn't actually build a model per se so so it's a little bit different um but it is something that we're you know we're looking at and um yeah if you know if that's something you'd like to see we'd love to get someone to feedback in the repo yeah nice and then just one more question and then i'll let you keep going um so this one's also coming in from youtube and the question is with api integration it seems that each model only have one endpoint is there any way that we can have multiple endpoints in the same api deployment but have many models yep absolutely so you can register as many prediction engine pools here as you want and there's even uh i don't know i have the application right so i don't know if i'll be able to show that but there's a way here that you can add another parameter called i believe it's model name and you're able to reference it by name so when you're pulling it in here you you tell it get required service and you tell it to get this prediction engine pool you can say i want this particular model to come in to this part of my application so yes you can register as many as you want all right so let's see i think it started off already uh oh i can request https when it's http so when we go ahead and make a request this is again the input right that's that vendor id your rate code your passenger account so on and so forth and you see here that it just returns to me the score or the predicted uh fare for that right so we know that we tested it locally we know it's working um great so what do we do next well this is a net application right so the natural step to deploy then after that or at least one of the simpler methods to deploy is to go ahead and right click and publish right so folks who may be used to deploying applications uh with with within visual studio um you know you can certainly take advantage of that uh right click publish functionality right so if you go here to azure right we go through the steps uh in this case we're gonna be deploying it to azure app service and i'm going to go ahead and create a few things here so we're going to want to select our app service instance here oh wait one second yep it looks like we just got rated with 261 people from lana looks so thank you everyone that are joining us right now that's 261 people that's a lot of people so um what we're doing right now for all of you that are joining us now um this is on.net live and we are talking about adding machine learning to your existing.net applications so if you're you know c-sharp or asp.net or you know you build any types of apps using visual studio using.net this is a really cool way for you to start adding machine learning functionality but not knowing too much about machine learning so i definitely really appreciate all of these people that are joining us now um hopefully you guys will stick around and and consume some of this great content that brie and louise are you know showing aesthetic yeah it's pretty awesome and it actually was very timely because that usually uh takes a while to spin up there so thanks folks and again thanks for joining us and and you know getting to see uh some of this content here all right so then we could just go ahead and tell it that we want to publish it um again this is for folks who are used to doing this uh it shouldn't feel any you know it shouldn't feel like something different right it should feel very familiar and that's the whole point right that at the end of the day the assemble.net models and even if you are bringing in onyx or tensorflow models the way that you deploy them and the way that you get them into production should feel very um very familiar to you so now i'm gonna go ahead and publish it and again what i want to kind of show here is that eventually it's going to go ahead and work here um once i deploy to my public endpoint now uh this might take a a while so while that's happening um i just want to talk about something else and you kind of alluded to uh sure so you'll notice here that we we're using this technique again route to code one of the challenges here is or the trade-offs is it does make it simpler but you lose some of the features like model binding that you typically get inside of the asp.net frame or some of the things that you that you're used to using right so there's certainly trade-offs there but what if i told you that you didn't have to make a trade-off and that it could be even simpler than this um but at the same time you get to basically use all the functionality and and really the full weight of the of the asp.net framework and uh you actually can do that with minimal apis so let me just switch over here to vs code real quick to kind of show you what the code looks like um this is using i believe preview 5 of net 6 um so it's using the the latest bits and i believe this is c sharp 10 that it's using uh and you're going to see here it's going to look very similar in in the sort of create builder functionality except instead of using web host we're using this web application um sort of you know method here or class um then we go ahead and say hey builder i want to register my prediction engine pool in this case we're using one of those uh you know remote models that we have uh you know stored on github um and then we just build our application then we go ahead and we say hey map this predict endpoint and what we're injecting here instead of so very similar to what we did before we had that sort of predict handler except in here we're actually just using a lambda so or sort of an anonymous function right and as you can see here right we're using from services just like you would with your traditional sort of mvc applications to bring in our prediction engine pool and the model input we're sort of bringing that in and parsing from our body all right uh and then all we do is we just say prediction engine pool predict and then to run our application we tell it app.run and that's it like that's that's the extent of what you need to do to deploy a ml.net model as a web api using some of this minimal api uh sort of new app model convention that that should be coming and that was actually uh there's more in-depth uh sort of materials um if you go and check out the asp.net standup that happened yesterday that cecil mentioned at the beginning of the stream so if you'd like to learn more about minimal apis check that out um and then this is just sort of an example of well this is what it looks like when you actually uh want to leverage that app model to go ahead and simplify and deploy uh ml.net models as web apis yeah i think between the new minimal api features that are being built in for espn.net core and then the new c-sharp language features i kind of feel like my net applications keep shrinking right they get smaller and smaller you know i i don't need name spaces if i don't have to you know i don't need that enclosing program or you know public static void main if i don't need it and i can kind of just write code which i think is amazing so now you have pretty much a fully functioning application like a non-trivial functioning application in less than 20 lines of code which is great yep totally and and again in these scenarios because they're so simple right like you for the most part you need one end point um you know that that's that's really all you need um so i think this deployed now i think it said it was successful uh so now if we make a request to that endpoint with again our same sort of body here you're going to see that we get the same response so it was that easy to go from hey we not only got the code to build the model and got that sort of scaffolded and auto generated for us but we also had the actual application that we're going to deploy to the cloud in azure uh scaffold it for us as well and again with the right-click publish sort of uh you know workflow we can easily get that into into azure and out to our customers so i'll pause there because uh there's i know we're actually running a bit low on time here so maybe we want to take some questions from folks uh yeah that's fine i mean we can go over to if we needed to so that's that's totally fine um yeah i mean there's some folks in the chat that are commenting about you running code directly from the main branch and having c 10 and stuff like that on your on your machine um some folks are asking about this uh conference i think you guys are doing a conference this week or tomorrow actually i think actually maybe it's today i think it says it's going to start in a couple hours the virtual.net conference uh i hope it doesn't start in a couple hours it starts uh tomorrow um uh nine about nine hours actually it's the virtual ml.net community conference completely run by the community all sessions are by community members louise and i help co-host a bit but um it's pretty much yeah i have it on my screen here there's the countdown for it there's an awesome schedule of talks and i'll go to the schedule here we'll post the link in the chat as well um this was started last year by two community members john wood and alexander slatte um they just got together on twitter and were like hey we wish there was an all ml.net conference and then they made it happen so uh this is the second the first one was last march or may i believe then there was a kind of a spin-off hackathon in november and then now this is the second annual ml.net our virtual ml.net community conference so i believe there's 13-ish sessions everything from you know the kind of the intro to ml.net to using ml.net and xbox um let's perform anomaly detection using ml.net on a drone flying data from azure iot um so you can see there's quite a few awesome ones here let's play rock paper scissors with ml.net you might have seen i think scott handelman has had a talk with the rock paper scissors and now it'll be integrated with ml.net so a lot of really really cool sessions uh both tomorrow and the next day so we'll post a link in the chat for that yeah i'm wondering if there's a bug in the timer the reason i'm saying this is because there's a few folks in a chat that are saying that they were seeing the timer says it starts in five hours i'm looking on my screen like i like when i refresh my page it says it starts in six hours i'm looking at your page and it says nine hours so i feel like everyone on that thing nine hours is tonight i should have done the math in my head i'm like wait nine hours that does not make it seven a.m tomorrow um yeah there's probably don't don't look at that uh hopefully so the the schedule page should be adjusted to your time zone like i'm in pacific time and this starts you know um in the they they've been taming it on eastern time zone so i'll be up very early for this i'm sure lots of you will be there up very early or late to watch as well but yeah i would go to if you actually go to the schedule page and i can put that link uh in the chat here um you should be able to see it based on your time zone got it got it and then for folks that you know may be sleeping when some of these sessions are going on i'm guessing it'll be recorded and probably available on youtube or some streaming platform that folks could watch exactly yeah virtualml.net has its own youtube channel um and it has all the sessions from last year as well so we will uh record all those and put it on youtube again awesome awesome cool and then would you you all you you both are going to be speaking there right i'm guessing you guys are both this year we're not it's all community levels here yes we'll be helping our co-hosts we'll be helping moderate co-host but um yeah it's all community which we're very excited about got it okay okay all right so then i went ahead and pasted that link in the chat so folks are interested in checking out the conference um i'm sure you could still go ahead and register for it right now if you wanted to and then um hopefully you'll learn some more about ml.net and see what some of the other community members are have been working on some any other questions for us uh none that i want to put on screen um uh here's one from jose jose is asking can i trade my model without visual studio and just using my software made in ml.net so i'm guessing i'm guessing he's asking about like command line execution which we did mention a little bit earlier but you know he may or may not have been here for it so you wanna you wanna talk about that again really quickly about how you can train outside of that visual studio um ui yeah so the ml.net apis like you can build the pipelines yourself so the auto-generated pipeline that was from tooling you can totally build that up yourself and go for a code first approach we also have a layer on top of the ml.net api that's the automl.net api which is abstracting away that portion of it so it's essentially everything that the tooling does but in code so you would say you know create regression experiment put in your data set and the amount of time to train and then it would give you back that model and then on top of that is what we have our tooling on and i think louise had an example of the cli if we switch over to his screen um you may be able to see uh what yeah you can see what the cli looks like yeah you can and we won't go through the training example because i can i know that we're running a bit low on time but let me switch over here yep um okay yep so you can see here i have installed the cli and you can say mlmet dash h and this is a global.net tool right so you install with the.net tool install mlnet and as you can see here it's it's basically what bree was mentioning where you have classification regression uh recombination and image classification and then you see this other train command here uh and a trade command what it allows you to do is we already have this mb config file here so what we can do is we can tell mlnet uh net train and i'm just going to do the h so you can see the different options that you get but essentially it should just pull up where is the path to that training config file and then it's going to retrain based on uh basically whatever has been defined inside of vmb config so you could imagine even using the cli inside of your you know ci cd or some sort of automated process to retrain these models we have one more question that just popped in so this was from jeremy sinclair in youtube and jeremy's pretty much asking how many pictures would i need in order to have a high level of accuracy if i want to detect beans in an image exactly exactly yeah it really does it really depends uh you know there's two camps of like more date more data always helps but again like let's say that you have more data but it's only like one type of bean right like that's not very helpful so you also need to balance it between lots of data but that data also needs to be diverse not only different types of beans but different angles of the same bean you know there's a lot of different um ways that you can do it we actually louise and i did um an american sign language image classification model a while back which it doesn't really work well for that scenario because there's a lot of movement and expression in the face but it did actually pick up some of the symbols or some of the you know the hand motions there which is really really cool um but we needed so for each letter we had to have you know i don't remember how many images we had but it was like all around all different um placement on the screen different angles different lighting um so the more variation you have in your images the better got it um we have some more questions that are coming in from twitch this person's saying i'm a student learning c sharp and i have to make an authentication on asp.net core and nbc do you guys have some tutorials uh louis your content developer do you check the docs yeah no so uh you know on the fly i can't really pull that out but if you go to dots.microsoft.com um there's tons of content there and i think there's also even one of these live shows right i think it's called the 425 show that may be dedicated to uh to authentication that's right yeah exactly so make sure to check if you know you're more of a visual learner check out the show uh but there's also written content in docs.microsoft.com yeah for sure definitely want to shout out christos um so again you can see the 425 show on youtube um they have their own youtube channel and they also have their own um twitch channel as well so you guys search for 425. is it 425 or 429 i don't remember which one it was i forget yeah it's 4 20 something search for that 420 something show for six years and you can find it um some questions coming in from youtube so is there a timeline for the current preview release yeah so ml.net is currently ga so that's not in preview anymore um we have our own we don't follow the dot-net or visual studio release cadences um just to have a little more flexibility right now we try to release every month um i think we've been a little slow on that recently but um the newest model builder release you can currently use um it's on a private nuget feed so we actually have a link and maybe if louise can find it and post it in the chat you can sign up for this one or if you want to wait um for build uh we'll have that new this release out so all the features that i showed if you go through those steps of enabling model builder that will be the version that you get um awesome yeah great great cool so um yeah so kind of like what lewis was saying a little bit earlier we are running low on time so um for youtube is there anything else that we want to show folks or talk to them about or share before we before we wrap it up um nothing on my end i mean i you know we're always here for any questions if you need help getting started if you know it is open source so if you want to contribute or have issues you can you know do that on the net machine learning github repo and we really hope that you come see the virtualml.net community conference it's going to be really awesome that's it's going to be streaming on twitch and youtube and it's completely free you can just watch and um ask questions it's going to be the same you know format here so nothing else to add louise no uh thanks for having us we really enjoy talking to folks and helping.net developers uh you know make their apps more intelligent with ml.net and again like we said we're always here to we're happy to talk if you have questions and uh yeah and thanks for having us folks okay for sure again so thank you two for joining us and then thank everyone that joined us um here in the live chat i know earlier we started out a few countries and i know a bunch more folks joined us i saw some folks join from canada we had some more forks joined from turkey and israel and like tons of amazing places so as always i really appreciate y'all spending this this time with us and hopefully you'll come back again next week as we'll do another on.net live um so see you all have a good day and uh we'll talk to you soon thanks i have to find the music where's the music i could sing it for you is this the music are you gonna think great go from no just kidding that's the encore part of the show not your show if they heard me sing okay i found the music now we're gonna go bye everybody [Music] you

Channel: dotnet

Views: 8,484

Keywords:

Id: THVD4nzi8vk

Length: 67min 47sec (4067 seconds)

Published: Thu May 06 2021