Building Microservices with the Go Kit Toolkit

hey guys my name is tensor today we're gonna be building a simple micro service with go kit go kid is a bundle of different libraries that you can use for building micro services these libraries provide components for logging metrics tracing rate limiting and circuit braking and system observability and resiliency patterns there are three major components in a go kit based application you have the transport layer the endpoint layer and then the service layer transports are essentially just the ways that your micro services can communicate with one another and the outside world in our example we're going to be building a simple HTTP server which allows us to expose our micro service functions these micro service functions they get exposed are what are called endpoints each of the end points represents a single RPC method and so what we do is we take our methods and we convert them into these endpoints and then that allows us to expose them to the transport layer and then the service layer is essentially just the business logic of the micro service itself now the example that we're gonna be using today is just going to be a very simple user module we want to be able to insert a user into a database from a Web API and then we want to be able to fetch that user back using a unique user ID that we generate inside of the business logic alright so let's get started by defining the interface of our service and this is really what you're going to do with most gokit and micro services so you create an interface with the methods that you want to expose to your transports and you use that interface to implement your business logic here we have a file called service that go inside of a folder called account and this is where our logic for this micro service will live and as you can see in the interface we have two methods we have a create user method which takes in context an email string and a password string and it returns a tuple of a string and an error and then we have another method called get user which also takes in the context and an ID string and returns a tuple of a string in an error now that we have our interface for our service let's go ahead and create a struct which will represent a user inside of our business logic our user struct has an ID string an email string and a password string and because we're going to be serving this through an HTTP API we need to be able to convert it to and from JSON and that's why we have these JSON keys here so ID will be the key ID in our JSON object and it can be empty so we put this omit empty key as well then email will be email and password will be password now because we also want to be able to put our users into a database I'm going to create another interface inside of our user deco file and this will have functions that are very similar to the ones that we have in our service interface so we have a create user method this one takes in the context and instead of taking in an email and a password string it just takes in a user struct and also rather than returning a string and an error just returns an error the get user method on the other hand will be exactly the same as the other get user method in that it will take in the context and an ID string and then return a tuple off string and error so these two methods will help us interface with our database whereas the other two methods will be the ones that we expose from our microservice alright so now let's go ahead and build the basic business logic for our service and to do this we want to implement our service interface on a structure and we'll create a structure called service with a lowercase s which will have a repository inside of it and a logger will use the repository so that we can interface with our database and then we'll use the logger so that we can actually see what's going on inside of our micro service now of course we're going to need a function that will allow us to create this service and to create it we just need a repository and a logger this will output a service interface and of course we're going to implement our service interface on the service struct but remember our service interface has the two methods on it so this struck does not satisfy that interface just yet alright so we can go ahead and add these two methods into our file and then we can go ahead and implement them on the service struct so for create user we'll take the logger inside of our service and we'll modify it so that we know that we're executing this method we can call log with passing the logger and then tell it that the method is create user then we want to go ahead and take our email and password string and use them to create a user structure and of course to do this we also need to generate an ID so we can use a library that will allow us to generate a unique user ID and I'm going to use version 4 so unique user ID dot new version 4 and then we'll get the string from that so unique user ID string and we can then pass it into a user struct and by the way this is the library that I'm using for the unique user ID so it's github.com go FRS UUID now that we have the user we can go ahead and call service dot repository dot create user has in our context and the user and if we get back an error we want to log that error with our logger and then return it from this method this method returns a string and an error so we need to return an empty string in this case and then return the error itself then if we do not get back an error from calling create user on our repository we want to then login our logger that we did create a user and we can log in the actual ID of that user and then we can return success as the string and then now as the error so this method essentially is just a wrapper method around a method that calls to our database and of course the method doesn't really care about the logic of the database all that really knows is that this method here takes in a context and a user it doesn't really need to know about anything else aside from that that's important to note because what that means is that we could create multiple backends and repositories for instance say we wanted to do like MongoDB and Postgres or maybe something like Redis we could use these methods over and over again for multiple different repositories in that sense now the get user method is going to be a little bit more simple than the other one and it's going to be very similar so again we're gonna set up the logger and we want to set it up with the method of get user then we can call repository get user passing the ID string and the context and this will give us back an email string and an error if there is one and if we have the error will log it into our logger and then we'll return an empty string and the error in the tuple otherwise we'll log that we got the user and then return the email string and know for the error alright so now we're finished with the logic we can go ahead and create the repo logic that is the logic that interfaces with the database now this code is going to follow a very similar pattern to our service code so we'll create a repository struct and this time rather than having a repo inside of it and a logger inside of it it'll have a DB inside of it and a logger inside of it and we'll use this repo struct to implement the repository interface that we created before and also we can normalize our errors by creating a custom error up here which will just say unable to handle repo requests now for this application I'm going to use Postgres SQL you could use any database type you want and of course you would have to change the logic accordingly let's go ahead and generate our new repository by creating a function called new repo so this takes in an SQL DB and the logger and then it outputs a repository interface which we're going to implement on this struct so we return the repo with the DB and the logger inside of it then of course we need to add the other two methods create user and get user so here we have create user and get user and again create user is going to take in a user and the context the get user takes in the context and the ID string with create user we're going to be putting the user into our SQL database so we need a string of SQL that will allow us to do this we'll have a table called users and that table will have a field called ID email and password and we can just use insert into users ID email password the values will be number sign one two and three before we actually try to execute the SQL though we need to verify that the user has an email on a password inside of it so we can check to see if user dot email or user duck password contain empty strings and if they do then we can return our custom error otherwise we'll go ahead and we'll call repo DB exact context and this will allow us to execute our SQL string so we pass in the context the SQL string and then we pass in the user ID these are email and the user password if we get back an error from this we want to return it otherwise we'll return nil from this method call now with get user we just want to create an email string variable and then we want to query our database and we can just use DB query row for this and we can just pass in some SQL that will select an email from the users table where the ID is equal to the ID that we pass in and of course we can then scan that result into our email variable if we get back an error will return a tuple of an empty string and then our custom error otherwise will return our email string and nil and now we more or less have a complete micro-service so this micro service will allow us to put data into a database and fetch data from the database however we're missing our transport layer so we need to tell the service that we want to serve it over HTTP and one of the best things about gokit is the fact that we could implement multiple different transports for instance if we wanted to talk to other micro services using G RPC we could implement that along with HTTP or we could implement it instead of HTTP we could also implement thrift and there are a bunch of other transport protocols that we could use as well so to implement the transports we need to take our methods and turn them into what are called endpoints and to do this we need to create struct that represent the response and the requests from each of our methods will create four different structs - for the create user method and then - for the get user method I mentioned before that each endpoint in gokit acts like a RPC these are the requests and responses for that RPC so for create user we pass in an email and a password and of course we need to serialize this from JSON and then as a response we'll get back a string that will tell us whether or not it succeeded for the get user request we want to pass in the ID and then we'll get back an email string one thing to keep in mind about these structs is that every single one of them needs to be public that's why I'm using capital letters for both destructive names and the fields so make sure you do that otherwise it will not work properly also because we're using JSON we've got our JSON keys on each of our values here if we're using another transport type we could implement that as well now that we have our requests and response trucks we can go ahead and create an endpoints truck and the endpoints trucks contains the methods that we want to expose to the public we want to expose both create user and get user and we'll convert them to this end point dot endpoint type if you look at the type here it's just a function it takes in context and a interface and then it returns an interface and an error tuple we can go ahead and of course create a function which will allow us to create this endpoint struct and for this we just need to pass in our service we're just going to return the endpoint struct and what we'll do is we'll pass the service into two different functions which will take the methods and convert them into these endpoints so both of these functions will return an endpoint and taking our service and of course since our endpoint is a function we need them to return a function and we can use these anonymous functions as a sort of wrapper for our existing methods we take the request interface that's coming into this anonymous function cast it as a create user request struct then we can call service dot create user on our context and the request email and password and then this will either get us an error or the success string and we can go ahead and pass that success string into our create user response struct and then also pass back the error in the tuple the same thing goes for the make get user endpoint function it takes in the service and then returns an endpoint and we take the request interface cast it as a get user request struct and then we go ahead and call service that get user on the request ID and the context and then this gives us back the email and an error if we have one and then we can just return the get user response struct with the email inside of it and the error on the tuple alright so we've got our endpoints now and our request and response trucks let's go ahead and create the actual server so we've got a function here called new HTTP server which takes in the context and our endpoints and then it outputs an HTTP handler and then we can take that HTTP hen and just serve it using the go net HTTP library since we're going to be using JSON it's appropriate for us to create a piece of middleware to verify that all of our requests and responses will be of JSON type so we can go ahead and create a function here called common middleware that takes in an HTTP handler and then returns an HTTP handler so we just return an HTTP handle func and the function of course will take in the HTTP response writer and then the HTTP request and then we can just change the header to content type application JSON and then call the next HTTP handler now for our server I'm gonna go ahead and use the gorilla moxa library just because I like how it interfaces with gokit and we're also going to be using the go kit transport HTTP library to allow us to access the HTTP transport we want to generate a new gorilla mocs router and then we can set in our middleware function by calling our dot use then we can go ahead and set up our methods and endpoints here we've got a post method on the user path that takes in our endpoint of create user and we use HTTP transport new server to convert that endpoint into the appropriate type to go into this method now of course we also need two more functions if you see here it says that we need a decode request function and an encode response function and these are essentially just functions that allow us to encode the JSON and decode the JSON for our get user method we want to have a get method on the path user followed by the unique user ID that we want to query and again we also need to create two functions one to decode the request and then the other one to encode the response let's start with our encode response function so this will just take in the content the HTTP response writer and the response interface and then it will output an error and we can just say return JSON new encoder to create a new encoder and then we can encode the response into JSON and we can use this function for all of our response trucks now we need to be able to decode our create user request and remember the create user requests will have the email and password inside of it so this function takes in the context and then the HTTP request and then it passes back an interface and an error and we just create our create user request then create a JSON decoder decode the request into this variable and then pass it back now we can come back to our server and slot in these functions for this method but for the get user method because we're getting the ID from the path and not from a piece of JSON we want to create a unique decode function so we're called the decode function decode email request and then we can create it in our request response file so decode email request takes in the context and then the HTTP request and again it returns the interface and an error in a tuple and we can then of course create our get user request and then we can use the gorilla mocks VARs helper function to get back a map that will contain all of the variables inside of our path so we can get the variable by just passing in the key ID and then we can just put it into our get user request struct and then just pass that back from our function and that's it for our service so let's go ahead and wire all this together with our main dot go file alright so in our main echo file we want to start by defining the URI that we're going to use to interface with the Postgres SQL database so we'll call this DB source and we'll pass in this URI this just takes in the username and the password for Postgres followed by the actual address which is just localhost five four three two and then the name of the database in this case it'll just be go kid example and we also want to make sure to turn off SSL mode so we can pass in this flag as well also know that I'm using a library here called github.com lib PQ we're not gonna actually call any of the functions in here but this is what allows us to use the SQL library to interface with Postgres SQL and if you wanted to use like SQLite or MySQL for instance then you'd have to get the ORM for those specific types of databases down in the main function we want to set up our port and by default the port will be 8080 but will allow the user to change it inside of the command line then we need to set up our logger and we can set it up so that it will have a bunch of items inside of it so it has a timestamp it has the color and it's going to say that it's on the service account which is our account micro service and if we had multiple micro services then we could change the service name depending on which service it was inside of then we can go ahead and login info message that will just say that we started the service immediately after we can call defer and call a log here that just says that the service ended so this will execute when the go code exits we then need to make sure that we open our database properly so we can just call SQL down open on Postgres with our DB your RI inside of it and if we get back an error then we want to exit from the application and of course we're wrong this with our longer as well now down here we can parse the CLI flag if we get one and then we can create our context so we just call context top background and this creates a non nil empty context for us which we can then pass around through our micro service to create the micro service we first need to create a repository so we can just call count that new repo passing RTB and/or lager and then we can create the service itself by passing in the repo and the logger into the Carleton News Service we then want to set up a go routine that we'll look for when the user tries to terminate the application we can create an errors channel first and then our go routine we'll check to see if there's a sig termination in the operating system and if that happens if like we hit control Z for instance then it will automatically kick back any errors that come about and kick them through the channel we then need to create our endpoints for our service so we take our service pass that in to make endpoints and then that gives us back our endpoint structure and then we can create another go routine which will execute our server so we just say listening on port and then we put in the port and then we can go ahead and call new HTTP server on our context with our endpoints inside of it and then we can just call HTTP listen and serve and pass in the port and the handler and if we get back an area will get piped into our errors channel and then finally we want to make sure that we properly drain that errors channel into our logger so we can just say level error logger.log exit and then just pass in the errors and that's it for our applications so we're completely done with the code we can drop into our console and now run this and then we can open up something like postman to be able to interface with our API as you can see when we run our application we get back from our logger that we're in the service account and we have our time stamp here and then the caller is the actual line of code that called this and then we get back a message that says service started and then it says it's listening on port 8080 and so now I'm inside a postman and I can go ahead and call localhost 8080 user and I can call post on it and send in a piece of JSON with an email and a password and this will create a user in our database so let's go ahead and hit send and you can see that we get back a JSON that tells us that we were successful did we get back okay success and if we go back to our command line you can see here it says service account has a timestamp the caller so this is in logic go and then the method create user create user and here is the user ID that was created for this user now we can use that unique user ID to call a get method on the same route with that user ID appended to it and this should return the email address that we just added to our database and as you can see here we do in fact to get back that piece of JSON so it gives us the email of test at test com all right guys well I hope you enjoyed this tutorial I may do another one to show you how to add more services to an application like this so let me know if you're interested in seeing something like that if you like this video then feel free to subscribe and like if you have any questions or comments feel free to leave them in the box below and if you dislike this video then by all means download it as much as you like if you want to support the channel and feel free to go check out my patreon and I'll see you guys next time

Channel: Tensor Programming

Views: 33,306

Keywords: golang, go programming, learn go, golang tutorial, programming tutorial, learn to program, learn design patterns, learn to code, learn golang, generator functions, Observable, listeners, channels, go channels, parallel programming in golang, gokit, microservice, go microservice, go kit toolkit, go kit, building microservices

Id: sjd2ePF3CuQ

Length: 25min 11sec (1511 seconds)

Published: Thu Oct 10 2019