Building Microservices with Go: 4. More RESTful services

all right welcome thank you all for coming back to watch I am Nick Jackson and I am going to be teaching you about building micro services with go and I'm excited I really enjoy the show it's been a long day today I can tell you that but it's a it's a good day well let's have a quick recap kind of where where were we last time well some of the things that we were looking at last time is we've kind of been looking at like how do we build micro-services but with the go standard library and we want to kind of look at the ghost standard library before we move on to looking at some frameworks because I want you to appreciate actually just the power of the go standard library but also it's it's kind of the bare minimum you sort of you'll also appreciate why some of the frameworks will will give you what they what they can and I think it's about balance but the standard library offers an awful awful lot so I will I will put a link up here of the the previous video but let's let's take a look so where were we at so what we were starting to to look at is we were starting to refactor out our code so we we'd started to to move away from using that pattern of using you know like the very very basic stuff that tan line go server sort of surf funk you know it's a handle fungus is a useful thing to have but in reality we need to be doing things like creating our own serve marks and we need to be able to create our own handler structs which we can then do dependency injection inside but that's very kind of you to say so Dave it's been a it's been a busy old day for me today I literally just jumped off a stream well about an hour ago I'm off the clock now but never off the clock so what what have I been creating so I've been looking at this this sort of handler and I've been kind of starting to sort of replace everything out so we were kind of moving away from us to say from from the model of having everything in one block and and starting to kind of move things out and we're starting to look at routing so we want to look at the the restful principles rest is is is really a useful I'm sorry I'm just going to open that window a really really useful pattern for for micro-services it it's probably the the one which will give you the sort of the most people understand and most people can can kind of integrate with so this is kind of the pattern that we are going to follow we're gonna look at your PC we're going to look at some of the many other things but like right now this is this is where we're at so let me just get the latest per bit of code I'm just gonna pull the latest that we were working on last week there okay and I'm gonna create a new branch so what I'm also doing with the the code is every episode every show I'm creating a very specific branch which I'm uploading to github I'll make sure that that link is in the description below so you can follow along it's it's really easy you're not having to kind of deal with a massive repository you just choose the branch which corresponds to the the show is to show number four so let's do that get checkout - be episode four and we're good so what I want to start - to kind of to do - to look at is keep going with our products so we we implemented the the gap method last time so the method that allows us to retrieve our our data for us service but we want to now start introducing methods which allow us to mutate data so let's let's look at that so what Howard does that work in terms of get a sorry in terms of restful approach so in restful approach what you're going to be doing is using an HTTP verb so an HTTP verb for creating something unfortunately is slightly ambiguous it's going to be either array a post or or a put let's let's take a look at the docs and see what it says there but it says that a post operation according to the restful specification creates a new resource at the specified URI the body of the request message provides the details of the new resource note that post can be used to trigger on operations that don't automatically create resource which is great you know you can use a post to trigger sending an email or something like that we'll we'll look at like the controller based approach as well later and then put so put as the HTTP verb it says that it creates or replaces the resource of this specified URI the body of the request message spaces is a resource to be created or updated so you've got the option there to be able to use put or post to be able to create resources I'm gonna be controversial maybe and I'm gonna say don't implement restful services like that use post for creating new resources and use put to be able to to update resources I think it's going to be much much clearer for the consumers of your of your service and that's what we're going to do here so the first thing that we want to implement is we want to continue with our product API and we want to be able to implement a post to be able to add a new product alright so how are we going to do this so what we need to do we're going to follow this exact same Pat a handler if you remember is picked up and sorry is set here on the serve MUX and the serve MUX is assigned to our server so what we want to do is we want to modify our handler to be able to handle these multiple different request types so I'm going to add a new message let me say if our dot method is HTTP dot method post and in there what I want to do is I want to call my my sort of function so I'm just gonna I haven't created a yeah I'll create in a second so I'm going to do add product and I'm passing it the same response writer and request that I receive and I'm just gonna return so then I'm going to define my my function for the F so a function for that method for that operation it's on the product and we're going to call it add product the signature is exactly the same as the HTTP handler so we're adding a response writer and we're adding an HTTP request oops and that's all we have to do so again let's just let's just do some logging we will also look later on a nicer ways of logging than just using the standard logger there's we can tidy up a lot of things here but it's useful for now handle post product ok so let's just sort of see that in in in operation so I'm just going to run my my service so go run main go and we got that running there and let me just open up a new terminal and I can curl that service so I'm running ladder at localhost 1990 localhost 1990 so if I don't explicitly specify any data or if I don't specify which verb I want to use then curl is just gonna do a get so there we can see that that that get is is working there and we're gonna just pipe that into JQ so we can get it a slightly more prettified response so look at the the kind of the path that's flowing through the code here we're hit serve HTTP we have this statement and the statement is if the request method is methods get so if it is a gap request then call this function a method call get product and return and that returns us that our data so then if we do a post well what we can do here is we can just specify some data and we're gonna gonna specify some some jason I'm just gonna put any old garbage in here cuz it doesn't matter I'm not doing anything with it right now and you can see that if I go over here we handled the post so I'm not I'm not doing anything with that yet that's that's our next step but the key thing around that what I did there is I just did a curl now with curl as a tool if you don't specify an HTTP but you do specify data then curl will automatically default to a post and if I just put that into verbose mode you can see that a little bit more detail you see there the the actual HTTP requests which was sent to my server so that's that's nice and clear so we need to to manipulate our data so the first thing though that we we need to do is we need to take the data from the post and we need to convert it into product object so how do we how do we do that so we when we looked at that previously what we did was we use this JSON encoder so we were using JSON encoder which is a slightly more efficient way of encoding the JSON straight out to the the response writer then then sort of using Marshall which has to bring it in to an in-memory stream and we can do the reverse to be able to create a product structure from the Jason that we're going to pass in so let's quickly just dig into the docks and see that in in action right so we're gonna look at ghost standard library and the package that I want to look at is encoding Jason and Co I'll zoom that up in just a second there okay so encoding Jason so our decoder well what we want to look at this time is an encode err it's exactly the opposite so an encoder sorry all the way around get my brain and order a decoder exactly the opposite of our encode err so we are going to pass as a parameter and IO reader of course if you kind of remember back when we were looking at these things the HTTP request body is actually an i/o reader so we can use that and then we are going to decode the body from the request into a and interface so we're gonna we're gonna put that into a strut so let's um let's see that in action if I get back to my code okay so what what I like I like to do is I actually like to add these these encoding methods on to my my my data structures I think it's a just a nice clean way of encapsulating stuff like this and it kind of keeps all of that logic out of my my products so if I want to use a different encode or I want to take a different method or if I want to do some special pausing before I encode the data strut I can kind of keep that nice and clean and separate so I'm going to add myself a from Jason function so I'm gonna add a method to my product strut and I'm gonna call it from Jason so the source of this is going to be an i/o reader and it's going to return an error if it if it hasn't any problems so what we can do is we can use that encoder right so we're gonna create it so we create we call E and it's gonna be json dot new decoder so new decoder takes a reader we're passing it the reader there and then we can do the encoding so if we just do a dot sorry decode you'll see the decode will copy the the structure of the the request into this this structure here very into your destination structure the destination structure because we have a reference we can actually just pass ourselves here so what's going to happen is this product strut that I'm going to create when I call from Jason with a reader it's going to set all of the properties on Strutt based on whatever the contents of the the reader or the HTTP request is decoder returns an error so we can just quite simply just return that and we'll add some comments on there later but we've got quite a nice clean set up there so then I have my my product so what I want to do is I want to do that conversion so the first thing I need to do is create my new product object so I'm just gonna create prod and it's going to be a new data dart product and then I can call that so error tools fraud dot from Jason and we need to pass at that reader the reader that we're going to use is the response body from the HTTP request so how does this work when you're kind of running things so the the reason that you're getting a reader from the HTTP request is that go hasn't actually read everything from the client at the point of receiving the HTTP request it'll buffered some stuff but it hasn't read everything you might be getting a huge amount of data so in the instance here I've got like something pretty simple it's gonna be like a couple of bytes of Jason but it could be a 10 gigabyte file so you want to kind of read that progressively which is why you kind of want to use something like a buffered reader as opposed to just getting everything in a data slice and go has to buffer all of that into memory and allocate all of the memory for the entire of the requests and all sorts of badness so I'm I'm going to decode that from Jason and I'm going to check my error because I'm a good citizen and I'm gonna say if the error is not nil and I'm just gonna return HTTP dot error my response writer which is the first parameter of error from back and I'm just gonna say unable to unmarshal Jason this is a good IRA message for me it's not necessarily a good error message for the consumer my API but it's good enough for right now and what we're gonna do is we're going to use HTTP status bad request because if I can't marshal that JSON into my product object I probably got a bad request so I'm gonna use the correct HTTP status code so what's my my next step well my next step is that I can then sort of stall that piece of information but let's let's just see how this is working so let's just log it out for the time being so I'm gonna do PL dot and I'm gonna use printf so I can say let's just say prod and then if I use percentage hash V what that's gonna do printf is going to give me a nice representation of the strut I'm gonna see the fields and things like that if I just did percentage V so just the value of whatever that type is I'm only gonna kind of see the values I'm not going to see the fields so we can we can just make that a little nicer so we'll run that all right let's rerun our application and let's redo that curl so I'm just gonna set a couple of the the parameters here so our product has oh if I can find the right button these these properties on it we don't need to set them all but we'll set a few just to check it so an ID and we're going to set the name so I'm just constructing Jason here I'm going to call it well we've got a what we call it a frappe perché how do you spell Frappuccino I've got no idea this is going to - we're going to specify description so that's gonna be a nice cup of tea so the this is just constructing the Jason that we're going to to post the server so I'm going to post that and that's gone okay let's look here so you can see now that what I've just logged out from my my handler is that I've just logged out that data structure so I can see that the the the the sort of the decoder has done its job it is converted the Jason that was passed in into a data product strut which is which is dope we can now save this to our database so we're gonna we're gonna save this to our our fake fake data store here which is literally just an array which is great will promise you I promise you we will look at look at more advanced data stores in a little while but it's not important right now so I'm just gonna create a add an add product function that's going to return a well mate I'm gonna return anything because it's pretty much impossible to return an error and we're gonna pass in a product so that is going to be product like so so the ID I'm gonna auto generate the IDS for for these things because I'm running in a database so I would I would handle that let's just fake that for now I will create something a little bit more more effective let's not fake it let's do it properly so we are going to do func get next ID and that's going to return me an integer so I'm just gonna do four let's get the last product in the list LP which is gonna be product list and the ID the one we want to get is going to be blend product lists minus one and then we're gonna return LP ID plus plus this is exactly how databases work people I promise you it totally is alright let's just add our product anyway so we we need to set our product ID and that's because we are you know we're creating something we shouldn't have the the identifier but it shouldn't be created by the user that's our responsibility so we're going to do product ID and I'm gonna set that to get next ID and then I'm just gonna add it to our collection so we're gonna just do product list product list equals append prada product list and then P is it is it going to be happy about that plus one whatever doesn't matter okay so let's um save that file and go back to our product handler and we can now just add our product so I'm just going to do data dot add product and I'm just gonna pass in the the product that I received there so let's let's run that can I use T type products or shouldn't be type products it should be type product where where where is it complaining products go sixty three bear with me product oh yeah of course well nevermind it's late okay we're running there so again we're going to we're gonna add our T I'm setting an ID but it's going to ignore that so if I just get my my collection so curl local host 1990 let's pipe that into JQ to make it nice and pretty you can see there that I've I've now posted so I'm following that restful principle I posted my Jason it's now been updated in my collection and I've got the ID set and the name and all of those those wonderful things and that's great but the key the key thing is next how do we update it so it's great we can retrieve things we it's great we can we can get things but we need to look at how we do updates so again looking at the can of the restful principles what what I'm going to do is I'm going to use a put so a put traditionally is you will post the entire objects or you will put the entire object and it will do a complete replacement if you want to kind of replace just one or two fields within an object then that's a kind of a path resource but let's see see how we do put because we need to start introducing a slightly different path Convention so the convention that we need to start thinking about is that well how do you know what the the ID is so generally kind of for a for an update one of the the sort of the common conventions is you would potentially do something like this so we would we would send a put and how a sort of resource URI is gonna be like we're gonna put it to an ID with a with an object so we need to be able to get the ID out of the the URI which um isn't as easy as it could or should be but let's let's have a go anyway alright so we're gonna say if adopt method so if we're going to deal with an HD a to big dot method put then expect the ID in the URI so we need to we need to decode this so the the full path of the URI is actually in the the HTTP request so HTTP request has this thing called URL and URL we can actually get the path and what that will return us is it'll just give us the the slash whatever path part of the URL that was called for the server so we let's get our path here and we need to take out of it the just the ID so how do we how do we kind of go about doing that well unfortunately with the go standard library you've got to manage that yourself if you're gonna use a framework like go go real erosion they have handy methods for that and I think this is kind of one of the key things that when we look at those probably next week actually you're gonna really see the benefits of kind of leveraging something but let's see how we can do it by hand so the the first thing that we need to do is we we need to kind of well fish the ID out of there so how can we do that well we can use a simple regex which is it's probably the one of the quickest ways of doing it I suppose it doesn't matter too much and it's a nice way to look and learn a little bit more goal so let's look at regex what are we got in here so regex syntax so we have all of these things we've got find we've got fine string and we can do find sub matches what we're gonna need to use something like a sub match let's so let's let's create the regex and we can get going so our reg X is going to be something like this we're gonna have our route path at the moment that's that's just slash and then we're gonna have an ID so the ID is going to be a digit of more than one number so I can say not to whoops sorry not to nine and I'm going to use plus like that so what I've created is a really simple regex I am going I've the bracket signify a capture group and inside the capture group I'm going to capture digits between not nine one or more off so we can then we can do that it's alright so we're going to use reg X must compile reg X dot must compile and this is what's gonna kind of get us get us going so reg X must compile let's look at that and the docs so we can just take a quick look but that's going to return you the regex object and it's a also going to allow you to kind of do things it'll panic if an expression can't be passed blah blah blah we're all good okay so then I can call my I can call my my my my find all find all matches so the the reason that I'm kind of using find all matches is that I actually sorry find all strings sub-match why I'm using this method is if I've got more than one ID in this URI in this instance I want it to bounce I shouldn't that sort of shouldn't exist it's a bad a bad URI so I can do a little bit of validation on there so I'm going to do that so we're gonna do find all string sub matches we're gonna pass it off string our string is going to be our dock URL your i dot path and it then has a kind of the sort of limit we're just going to pass minus one there which is all good now what this this returns or what are we complaining about found : we were oh of course I don't need to escape tie it in anyway we'll figure that in a second what find all sub match returns it's going to return you the groups and as a as a string array and we can see here the example so we we should only have one so we should only have the the one group so we can say that if land of G is is not equal to 1 then we can just do our HTTP dot sorry error RW invalid URI bad request not found could be any we'll do that for timing and then we're going to return and then we have the the CAPTCHA group so we get the the ID which is going to be an ID string which is going to be the the second element of our our capture group so it's going to be G 0 1 ok so again we need to defensively code that because if we don't have two elements there then again we've got a bad bad sort of URI so I've got two defensively code that it's a little bit painful right but I do have my ID string now but actually ID in my product model is an integer so guess what we need to convert that into an integer so we can just do ID now and we can do oh gosh string con STR con dot a2 I oops - I and we're going to pass it the ID string and that's going to give us an ID so if we get this far we are good so we can just let's just feed off our print line we can just check this got ID okay so let's just compile that and test it I don't URI is oh gosh well I'm I'm shadowing everything on my reg of course and that's not theirs reg I'm pretty sure that's all about URL and it's not string conf it's string conf I never get this perfect never STR calm its is still complaining about lap you telling me you you've not imported string Congo darn you okay alright and that's because a2 I returned an error as well as a thing did I tell you that I think that tooling is a little bit broken and go at the moment I did didn't I it's a little bit broken again if we can't decode that we've got a bad a bad you're right so that's running so let's let's do a put and I'm going to do a put with an ID there - X put whoops and let's look what we got invalid numeric literal at line one column late eight okay and the reason for that is actually is because I'm trying to write I'm pretty sure pause error why am I writing an error out there I'm not but the thing is that actually we're not even coming in to you because we're not even we're not even logging this out look and there's a reason for that so let me let me just save that I'll show you and the reason for that is a way that the serve MUX handles things but let me curl that with a put we are lying I'm like so why is that not working live debugging let's have a look at our URI debugging go microservices there are some more elegant ways than doing this and I will I will show you how they how they work in in a little bit but for the moment where we're getting this bad request so we are because we should be logging our arrows really shouldn't we so let's let this be a lesson to you to always provide yourself with correct information so invalid ID sorry invalid URI more than one ID and here we're gonna say same thing more than one capture group this is stuff that means to us valid URI on unable to convert to number let's just write so we can we can run that now for my doctrine line to the rescue invalid URI more than one capture group let's let's print out G oh no sorry well yes of course it is it's gonna have more than one capture probe it love to is anybody is anybody here on the chat watching what I'm doing because I don't know what I'm doing it and you need to help me you missed that one chat we're um we're making stupid mistakes and I apologize but it is later okay right run that working all good got ID one perfect so now we've we finally sort of managed to get through all of that look at the amount of code that I've had to write just a pause an ID out of a URI that's to say you know the the NGO standard library is a beautiful beautiful thing but as like many things in golang you kind of the the intention behind this was to be a primitive that community and developers would build upon and this is a sort of a fine example of that but we can now call our our put because we're good we validated our our you awry there so we're going to do p dot and we're going to follow the same thing we're gonna do we call it update product I'm gonna pass it the ID which is an int and I'm going to pass it the response writer ATP response writer and then the request like we've done same things before HTTP dari West and then we're just going to return so let's implement that function ever so quickly so Fung P products update products IDE and RW itching P dot response writer are and you know what we could have done here is we're actually at a really good point where we're starting to to duplicate quite a bit of code look at this code here this is going to be exactly the same here so I can I can refactor this code out and put it into its own little function that it's kind of a a night a nice time to do that I'm not going to do that right now I will do that offline and I'll talk about that next time but we've got that we can now update our product so update product we're going to call it the pass of the ID and we're going to pass it the the product itself and let's just very very quickly and add our update product method func update Prada and P product needs to return an error an ID is an integer so what I need to do is I need to find the product first so let me implement that in my my database here func find Prada this is super technical I was going to return you a product or an error and I'm gonna do for that comma that P range product list if I was using a real database I'd still be implementing a bunch of stuff in the same way I want to keep a nice abstraction away from stuff so I can say that if PID ID equals the the ID that I'm looking for ID and then whoops gosh I really can't type and I want to do is I'm going to return that product I'm going to return P and I'm going to turn a real Anil error if I don't then what I want to do is return an error so I'm going to return well nothing so nil and I need an error so rather than defining a kind of an error here and doing it in line what I'm going to do is I'm going to create a structured error because that's going to be easier to receive at the other time so I can say error product not found and it is going to be equal to oh sorry Walmart dot Hara F and we're just gonna say product not found so I'm just creating a structured error there-there's I'll show you why that lat that's quite nice okay so find product when Ali that we're updating our product so the first thing we need to do is does the product exists so we're gonna create FP and we're gonna say a fine product actually sorry I want to get back the position of that as well I'll explain why more dirty hackery of course I was gonna return that oops so fine product ID this is returning me pass and error so if error does not equal nil then we return turn the error otherwise what we're gonna do is we're just going to update our product list so our product list first thing we do sorry is update our our product just to make sure that the ID is correct it's the one the right one that we're supposed to be updating and there's not a mismatch there then we're just going to update our product list so we can our find product returns us in position so I can literally just replace that in there all right now I've got my fine product back over to my update products function remember a data update product is going to return me an error and what I can do here is cuz I use that structured error so I can say if error equals and then I can say data dot what did I call it error product not found then what I'm going to do is I'm going to do HTTP dot sorry I'm just going to HTTP dot error RW product not found as my message and then I can again following that restful principle status not found ok now if it's just a normal error so if error oh I'm sorry and then return if error is just not equal to nil and it would it be nice that it there's a nicer way to do this but I'm doing it quickly so an idea an update is needed Mia maybe if you've got the ID on the product it sort of it depends where where things are I suppose program I found return okay and this is gonna be state as internal status error because there's something else otherwise we're all good so we can just return all right so we should be able to build that and protocol 59 let's have a look 59 59 okay yeah no not enough arguments to return 66 products list and we got this what do we do we call that criminal product list product pluralization and finally 72 we not enough arguments to return and that's because we need to just return this here all right well yeah honestly 52 where are we 52 yeah we're not actually using up I don't care we can go get rid of left and then over here I swear that half my life seems to be just connect correcting my my own syntax errors what are we what are we growing yeah of course that would be a syntax error 108 oh all right cool so we we're up we're up and running there so we're gonna post we're gonna do that update so I'm just gonna run back here I'm gonna post my my sorry put my my tea so I'm going to update gosh well if I could type today I'm going to update product one I'm going to change it to a nice cup of tea so that's going to be one and it's going to be X put the name is going to be T nice cup of tea we push that that all seems to have worked and then we can get our products now and you can see there that we've we've now updated our our objective be a cup of tea so what have we quickly recapping what have we what have we looked at well we've we've seen how we can use the go standard library to implement a restful approach we've seen it that we're going to kind of use those HTTP verbs to correspond to kind of the the crud operations and we've seen how we can kind of break things down we've also seen I think what is quite cumbersome way of having to kind of extract parameters out of the URI to be able to do this and and we're starting to get a handler which is it's okay but but I think we can do better by using some some frameworks so what we're gonna do next time we're gonna do Sunday 4:00 p.m. GMT I'm gonna start looking at refactoring this code so I'm gonna continue with the restful approach but this time I'm going to use the guerilla framework guerilla is is something that I'm very fond of there were two frameworks that I used the very first day that I started working in go some some long time ago maybe six or seven 23 gosh no seven years ago and I used guerilla I use guerilla and I also used a framework called testify testify is a a testing framework in in golang which is written by Matt Riya I recommend you you troll Matt on Twitter he loves to be trolled but this package he wrote and I didn't realize this until I met him and I met him and and we were talking and you know it was like hang on a minute you are the same Mariah who wrote testify he's like yes and I dude that was the first term the first go framework I ever used I was like so so pleased the cool thing is about guerilla which was introduced to me by Ryan who I was working with Ed Martin Spence at the time and testify I still use these today and that they're still happily fulfilling my needs so we're gonna look at these I want to thank you so much as always if you've got any comment please please please anything you want me to show you to see put it in comments if you do like these videos then I nothing would grope me greater pleasure for you to subscribe and like the videos and also if you feel share them because I'm really enjoying teaching this series now I genuinely hope you find it useful so for me I'm gonna go and eat a tub of ice cream which is going to be terrible for my cholesterol and great for my happiness I hope you all have a wonderful day and I will look forward to seeing you next time catch you all later

Channel: Nic Jackson

Views: 25,555

Keywords: go, microservices, JSON, RESTful

Id: UZbHLVsjpF0

Length: 53min 58sec (3238 seconds)

Published: Thu Jan 23 2020