The Future of Machine Learning and JavaScript

awesome thank you very much for being here today it's making it to the last talk of the conference or the closing keynote which I will now put that in my trip report at work and tell my boss so thank you talk today talk today's on the future of machine learning and the JavaScript my name is Asim Hussain you could find me on twitter as draw eight not joash a draw eight my blog about angular and javascript and very very soon I'll have more content on machine learning on my website code Croft TV and also soon to be awesome dev yes I bought it and I am a cloud developer advocate at Microsoft I'm actually the Emil lead for cloud developer advocacy at Microsoft thank you very much recent promotion thank you thank you thank you and we are the curse we we work on the agile platform we are the open source advocates on Asia which is our cloud platform to go to my link there you can find all of us you can find all of our contact details on it's an old one smiler I don't have you on this table sorry I apologize you can go and find us we have people who specialize in all different technologies on the other our contact details there I also work with an Israeli charity called pal internship and what they do is they work with Palestinian recent graduates and they help them get internships in high tech Israeli companies I mentored thank you I mentored about 10 of them last year and I'm very proud to say that this year 17 but all the tech cohort will have a mentor will be one of the cloud advocates at Microsoft so if you are or you work at an Israeli high tech firm and you have openings or availability for an intern go to that website come speak to me I can connect you with Anna who is the program director thank you I also am an instructor on a platform called udemy let me know udemy okay they recently added in subtitling to the platform and they correctly transcribed my name too awesome so my name is awesome but like this slide this next site probably sums up me over the last year so this is not this is not my slide actually this I've stolen this from my friend Ellen her slide Ellen and I go back a long way and we both in to me were both into machine learning were both into JavaScript and so earlier on last year we decided to start our own meetup group in London called AI JavaScript London and then we've held quite a few meetups over the course of the year and what happens is during the course these meetups and just afterwards people will come up to us and give us links to hey did you know that this cool JavaScript application uses oh this one and so we thought okay fantastic let's get all those links together put it online somewhere and that's what we did we made a website called a ijs rocks it would have been a ijs dev you know if we'd wait a little bit but say I chose rocks and if you go there it's got a whole collection of JavaScript powered a I powered JavaScript applications which all represent and show a way of doing AI in JavaScript what's really cool about it is this JavaScript so you can always click on all of them find out how it works but then you can also click on the source you can click on the source code figure out how they were built and hopefully learn a little bit about how to do machine learning and AI and what today's talk is going to be but as I'm going to talk to you about machine learning and AI in Java by talking my way through three different applications that are on the website okay and explain how those applications were built and through that whole process today I'm going to teach a little bit more about machine learning until the end you're going to know a hell of a lot about machine learning I guarantee it okay the first one is the moji fire this is my application I built this one it's not trademarked it's just the moji fire so made sense and what the moji fire does is it does something for something amazing so what it does is you can give it a picture with some faces it then detects the faces in the picture it then detects the the emotion in the face and then any guesses it modifies it correct excellent so it finds the correct emoji to match the emotion and places on the face okay that's exactly what the mojo fire does actually released it as a slack box you can actually go to the Mojo fire calm hit add to slack and it'll be added to any of your slack workspaces it works like this so this is my son by the way isn't he cute you add /mo g phi add it in correct emotion as it to the face it works with multiple faces so you can get that one and add it in here we go I've managed to anse an age-old question which has plagued people for hundreds of years and multiple different moving parts in this application but I think one of most interesting parts of what it does is how does it calculate emotion right that's crazy right how does it calculate emotion on a face but it's actually really simple it's really simple it's only got two steps okay the first step is detecting facial features so what are the different now over all these years we now are aware of quite a few known facial features which every single face has got and if we can detect exactly where they are that's a step one this different algorithms to make that happen lots of different computer vision algorithms it doesn't necessarily need to use machine learning to do it and that you can find a whole bunch of information now there's loads of different jobs with libraries which can give you the facial features given an image okay that's step one you just need the facial features okay step two really simple you're just using your network okay now we all know how to use neural networks in this yeah okay okay I'll explain your networks really briefly just to the rest of you so neural network is based upon real biological concepts ok real biological concepts so this is a neuron you have at least one of these in your head you probably have several million less than you did this time yesterday you have some dendrites going in a body and some axons are going out if enough electricity flows into the dendrites the body triggers some electricity to go out of the axons that's it that's it if you stick enough for these together you get a brain I think there's a little bit more to consciousness than just neurons but teach their own code this up using some language probably JavaScript you probably all know kind of how to start off with this process you just kind of graph theory a little bit so you have some edges going in a node and an edge going out you would then perhaps feed into the edges some features okay maybe that feature is number of coffee shops in an area and the house price increase per years an actual study done in London is directly proportional to coffee shops and that those might be some features you feed into your new neuron and then what you do with the edges you just have a random number you randomize it we give a random number to each edge you multiply them together and you add them up and then you pass that into an activation function and whatever that activation function returns you pumped out that's like the electricity coming in and the electricity going out that's all it is just explain it a little bit more detail watch the screen boom this is it right you're just multiplying these two numbers together adding them up passing through an activation function and passing out that's it that's right when I teach machine learning in a workshop some people off halfway through like it's just it's just multiplication that's all it's just multiplication right it's just lots and lots and some addition that's mainly a large part of what what forms it right you have loads of different activation functions no choice of an activation function it just like you have to understand the data and understand what's going on but it's those different ones this is a very very simple one anything below zero is zero anything above zero is one and that's the one used there you join enough of them together this is what we call a simple feed for or densely connected feed for neural network so we have one input layer one output layer two hidden layers and it's just the same stuff just what I showed you before just those neurons connected together you just create a bunch of nodes a bunch of edges you randomized a bunch of numbers to be the weights math but random it is not random yeah math dot random and there you goes in your network and then what you do you get your data set your training data set that might be a bunch of images a bunch of images that you already know this is a happy images of sad image you already know that you get the facial features whatever they are you pumped them in as the input notes that's what there's maybe that maybe the X Y coordinates of a pupil or the relative distances into cute people something along those lines you multiply that's what we're doing he remember multiplication whose multiply it all together and you look at what it outputs but because you have I have a spare slide yeah but because so this neural network gave us three but we know because we know this is a happy face maybe maybe ten is happy and zero is sad we know that this is eight we labeled their human being said that's actually an eight so we know that this neural networks wrong it's wrong by five five it's wrong by five so what we want to do is we need to that's what we call the loss function because that's what we call we know how far how wrong is this neural network right now it's probably very very wrong because it was a random math dot random so it's probably very very wrong and then what we do is something called back propagation which given how wrong it is we tune these weights these weights that originally random we tune them we adjust them ever so slightly and we keep on doing that you pass through the data again and again the game figures that how long is hopefully the wrongness becomes less and you tune it closer and closer and close and all you're doing is adjusting tuning those numbers so that eventually then you're on that what gives you the data that you wanted I think you can give it a new image I never just never ever seen before and you pumps all this stuff through and it gives you the value of happiness or sadness or something along those lines what you can do then is you can just basically save this data and either kind of wrap it in some sort of API put it on a server and then make a call pass an image to it make a call and come back right you can do that the usually can be quite large maybe don't run this locally on your computer it depends it depends right you can do that but these kind of problems are actually really really common believe it or not right under seige your feature recognition emotional recognition is something that you solved many many times before and then what you do and what companies like my company does is we basically make it available urinate it's your face API I'm I've actually put a request in to have this image changed cuz I think she looks very confused I don't think we should have an image of somebody looking confused on a marketing material but I use for the mojo fire is the face API it's does everything that we've just described has been done this we made available just for your an API you pass an image to the API and it gives you an array one for each face it detects so gives you the rectangle because you remember means put an emoji own image and it gives you emotions from 0 to 1 for anger contempt disgust fear happiness mutual sadness and surprise hands up if you have a bid I can see you a couple of us yes you cannot be 100% happy I've tried you can only ever get to 99% happiness ok just warning that's it that's how the keep up how I built the Mojo fire key takeaway neural networks incredibly powerful we look at the step jump that's been happening in machine learning over the last five six years it's been doing a lot by stuff is going on with your networks new algorithms compute that is working in the space especially deep neural networks they're really powerful but conceptually they're quite simple to understand now in this five ten minutes I probably explain to you the basics the very very basics of a neural network ok did you get much more complicated than that you're not gonna all be able to go away now and get a job as a machine learning engineer ok it does get more complicated but conceptually they're quite easy to understand right up one complete drink this okay tensorflow mobile net and I'm fine I was giving a workshop six months ago in London and I was actually wearing this t-shirt which says puppies yeah and I'm fine so I see has a long story behind it I'm not gonna explain it but I was basically wearing this t-shirt and then one of the students in the workshop later on made this demo tensorflow mobile met and I'm fine and then sent it out I thought was a pretty cool demo and I'm going to talk about how how it works so it's just you can run it in code pen or I don't think I provided the link maybe stop there I send the slides I'll let it run oh you've got AI Jess rocks and find it so it's a pretty simple demo it basically makes it does a search on puppies on unsplash gets an image and then all you can't retell can you then tries to guess what's in the image so then it's gonna do um I'm fine which is come on okay dog terrier okay nine percent all right so it's not the best okay it's not the best but there's an important takeaway from this is the only API request is being made here is to unsplash the detection of what's in the image is all happening on the client side in the browser okay it's using something a technology called tensorflow Zen be here heard of tensorflow anybody use tensorflow Oh a few there are actually some machine learning engineers here excellent so tensorflow is tall does open source by google in 2015 i think like kind of scaled that mathematical operations but it's used a large parts used it's used mainly for machine unless you scale out calculations across GPUs and across hundreds of servers yeah but it's built in see it's pretty heavy way it's kind of something you have to install last year the tens flow dev summit they released ten so J yes that's the important one for this conference is that dot J S at the end of it okay I've been poor thing about ten slow j s first when I first heard about this I thought tensorflow J yes was bindings to ten supply I thought you had to install tensor flow to use tensorflow J s that's not true j s is a complete rewrite of tensor flow in JavaScript completely right what does that mean it means this it means if all you need to do in order to use tensor flow in JavaScript is well if you proper do this and if you still like me and bit old-school using script tags you like that okay but that's all you need to do is import one JavaScript file and you can do machine learning it can do a bunch of different things you can do what I showed you just before where I showed you the neuron Network and the back propagation and the tuning and all of that stuff it can do that with an API but can also do something else it can load pre-trained models higher gets interesting pre-trained models that's what we like all right we just like to import stuff not write stuff that's what we do so the load pre-trained model that there's an ecosystem forming of like three chain models that you can just plug plug into your applications and if you go to TF j s models I think research that on Google you'll you'll find a bunch of them and that's it you know you can train a model and then once the models trained you can save as a JSON file and then you can use it here and you can even if you're using other libraries to do machine learning you can export them in a format and then convert into format which x / GS likes and users write one of the models is called mobile net and mobile net is some thing that has been trained in order to detect one of a thousand it contains like a thousand different things in the world just a thousand that's why it was that Bob party person or the states not very good right but it's very small enough mobile to be able to reasonably be downloaded hmm in the browser okay still free large but it's it's it's a lot less larger lot less larger than the other ones but you saw so that's what he used he used mobile net to build this application okay he's no on that you you wouldn't really use mobile net to do real image analysis is its excuse for something else like chance for learning and a bunch of other things we don't have time to go into today okay but what if you want to do some really more in-depth meaningful image analysis the model will be too large download your browser you have to do something else such as the azure computer vision API there she is again fantastic so no Sarah Dresner fantastic so my colleague Sarah built another demo using the computer vision API last year her thoughts was can you accessibility can you automatically create alt text for images in your applications just by calling the computer sending the image that computer vision API one of the things that the API does it tells you a textual description of what's in the image use that as the alt tag and that's what you did again it's a code pen you can have a look at it you can upload whatever image that you want so she did this one and the API gives you a whole bunch of information but it gave you this text at the tops is a black and white photo of Thomas Edison the text says hey girl do just share electrons because I'm feeling a covalent bond I don't gotta know is that funny I don't know laughs I don't know that sounds like sounds like it's funny the thing that I thought was amazing about this is a black my foe of not just a guy but Thomas Edison right anok and they also detected the text from the image right that's pretty insane right it's doing a lot of really intense cool stuff but even this API gets wrong sometimes as very helpful people on Twitter you know Twitter there's always some very helpful people on Twitter you give some really good advice after you post something on there so after they put and I hope some hopefully this is one of one of you in the audience but after she posted this on Twitter there a couple of people came out and they know a lot know that there are some issues so this one it says the images a star-filled sky the textes four and some some Chinese Chinese sign okay so it didn't work there the next looks why I think could actually be true the next one I mean you don't know right I mean those could be dead stuffed animals for you for all you know okay and I really like the next one I think I think the next one we've got I think 50 percent correct I don't do too well at school so 50 percent for me is passed see so you know sometimes just to summarize there and I think if this won't take away you should remember from this talk this is the take away okay tensorflow Jase doesn't have any dependencies if you want to start doing machine learning in javascript in the browser you can just drop that in little caveat is the node.js version and does have dependencies but anyway the browser-based version doesn't have any dependencies if you want to get started when I when I when I teach machine learning I always start off with nub on that just learning a model using it's very very very easy way to kind of step into it but it doesn't really teach you a lot it's not it's not super useful on its own so if you wanna do something a much more complicated user computer vision API we've got one called a vision API did I tell you I work in the azure team did you guess let me guess okay so last demo my favorite demo image image okay let's watch it this is all running in the browser okay so you can draw the outline of a cat and it will fill in the rest of the cat mind-blowing right mind-blowing and this uses an algorithm called a technique called picks - picks which is it's using in your network and it's using something called a generative adversarial you said that you said that you said that you get a sticker thank you a generative adversarial neural network and these are fascinating I think it's only about two three years is the technique has become known I think they're fascinating so they work and this is why I taught you neural networks so you can get to this point they have two neural networks that are competing with each other adversarial one is called the generator and one is called the discriminator okay the generators job is given some input some input let's say the outline some images which have are just the outlines of cats it's job is to generate pictures of cats but remember you would have just initialized them with random numbers so it's not going to do a very good job of generating chemita this just got me noise okay okay you take all of your outline of cat images you generate you all your crappy cat images and you get them all together and you send them all to the discriminator which then also has real images of cats okay so the input to the discriminator is going to be given a mixture of your generated cats and real cats and the discriminators job it to be able to detect the difference its job is to say this is a real damage this generated cat image and let's say it got it wrong because it's gonna go wrong it's randomized it's just a random number that's not it doesn't know what it's doing it's probably gonna get wrong and when it gets it wrong we have that lost function and then we do that back propagation and we reach unit and we make it better and better and better and it's gonna get better and better discriminating if it got it right though then well then the generators not doing good job right and a generator gets tuned until generator makes better and better so as the discriminator gets better so generator gets better discriminates it gets better generates gets better and we're competing with each other until you reach a point where the discriminator just can't tell anymore between the Rilke images and the fake damages okay and then what you do this stuff you don't run this in the browser this is incredibly compute intensive you get rid of the discriminate we don't care about that you take the generator you export it into some form of the tensor flow Jess likes which is Bob the tense Affairs format and then you load that into your browser with density F dot load model and essentially you can run this in the browser and that's exactly what he was doing oh I forgot to introduce the guy he made it ziad he was still a student at university what was I doing University not much so that's what you can do so he's managed to get this model small enough I think the smallest model he's got it's still 15 megabytes still pretty large but still pretty impressive he's got it working in the browser all right why who cares it's just a cat images who gives yeah who gives a crap about cat images right let's take a look at this all right the image on the left is the input the image on the right is the output okay input output this is computer-generated computer-generated cajones but this isn't the pics to pix algorithm this is the bits of it algorithm yeah this doesn't work in the browser yet yet how about this one one input multiple outputs the input doesn't have to be outlines the input can be whatever you want it to be whatever you want it to be okay so this is another import all the segmented image so that this is the input this is the output the import like because it's segmented you know kind of like a depth knowledge you know a little bit more about what's going on in the image again this is the input this is the output okay she's not dancing dancing image is generated as I said the input doesn't he doesn't even have to be an image the input can be anything anything at all the input can be text the input can be text I think you can read that the top that's the import that you entered in a description of the image that you wanted the flower has long thing thin yellow petals and a lot of yellow anthers in the center the first image is 600 iterations the second image is 1,200 iterations he can't tell with the resolution here but that second images are very good very very good very high definition then we've got another one here the bird is white black and brown in color or the brown beak and this one was actually a bit weird but the rest of them pretty good representations of images I think of eight years ago we were unable to tell the difference in image between a cat and a dog I think that's all we were eight years ago this is what we are now how long do you think it will take before someone just types in an e-commerce website blue theme a contact form and about page go who cares if it's react or Savelle or I'm sorry or or anger who cares right it's just would make me a website how far how long will wait you think we are from that [Music] [Laughter] just in summary this one so Ganz learn to generate new images and that's what's cool if it's a generative they generate things that's what's so cool about them previous like other types and your network algorithms but to tell you if something is one thing or the other Ganz generate things they take a lot of compute to train a lot of computer train but the generated model generator you can optimize it so it can be used inside the browser with tensorflow jeaious if you want to learn a little bit more I want to get introduced to how to build if you throw a little bit into machine learning ok yes with APs but you can go to aks mmog fire and I have instructions I'll teach you how to build a slack bot and you can do all whole / modify and detect emotion and all sort of stuff if you want to figure that out thank you very much for your time please follow me on Twitter

Channel: Coding Tech

Views: 26,714

Rating: 4.7820325 out of 5

Keywords: javascript, machine learning, artificial intelligence, future of ai, future of machine learning

Id: HC38FUMNfc8

Length: 33min 46sec (2026 seconds)

Published: Tue Jul 02 2019

Reddit Comments

[u/VladyslavAfrin]

Only a couple of years ago, an ordinary user could face the results of machine learning (ML) only in entertaining software like Siri or Alexa. Today, its strategies have improved to the point that numerous open and private associations use them. For instance, to guarantee organize security or even analyze maladies in the beginning times. The accomplishment of your business may likewise rely upon advancements in the field of ML? How about we talk about the principle vectors of advancement in this IT specialty.