Go Programming for beginners | Working with files and JSON

when writing code most of the time we'll be dealing with data we're either storing data in a database or some storage or we'll be sending data or serving data via http json is a common data structure used by microservices and distributed systems for sending data across the wire in a previous video we've learned about the fundamentals of the go programming language and in this video we're going to turn it up a notch by looking at more realistic data and how to deal with data structures specifically json so without further ado let's go [Music] so in part one of our series we've taken a look at how to install the go programming language and also how to run the go language in a container so if you take a look at my github repo i have a golang folder and in the golang folder i have an introduction folder with a readme and this is my introduction to the go programming language if you're new to go check out the link down below to my introduction to go so you can follow along in this video we're going to be taking a look at part 2 which is specifically to deal with json and in this folder we also have a readme which is the next part to the introduction to go for dealing with json so the first thing we're going to need is a development environment so what i'm going to do is i'm going to change directory to the golang introduction part 2.json folder by saying cd golang introduction part 2.json i'm going to copy that paste it to the terminal and change directory to that folder and in this folder i have a docker file which you can see here as well as the readme that we're looking at and if we take a look at the docker file it's a very simple file saying from golang 1.15 running a small alpine container and it also specifies a working directory a folder called slash work and this is where we'll be doing all our work inside of the container we will mount all our source code from the file system into this folder so that we can work inside of the container to build and run this container it's very simple i just say docker build and i specify the dev target and i specify dots because we're in the folder where the docker file lives and then i tag it as go so i copy that paste it to the terminal and that'll build up our container image for our dev environment and then to run it i say docker run minus it for an interactive terminal and i would say minus v and i mount in the working directory and i mount it into a folder called slash work that's our working directory inside of the container and then i also specify the image i want to run which is go that's the one we tagged earlier and sh for a shell terminal so if i run that that'll go ahead and run that go container we've just built it'll also mount in this entire part 2 folder into the container and we're inside the container in that work folder and because we've mounted a volume i can say ls and we can see we have our docker file and our readme i can also say go version and we can see i have access to the go programming language version 1.15.6 so now i don't have to have golang installed on my machine i can simply build and access it inside of a container image this is a completely optional step if you don't want to install go on your machine as per our introduction to the go programming language we always start out with a folder which represents our go repository and then we create a module to hold our application and we start with a hello world base so let's create our application now the first thing we always start with is creating our go repository so what i'm going to do is just create a folder that represents our repository called videos so i say mkdir videos and that'll create a folder on the left here for videos that'll hold our go code and we're going to create an application that deals with videos so then i go ahead and change directory to that videos folder and the next thing we need to do is define a go module so i say go mod init videos and when i run that that'll go ahead and create a new go mod file which you can see on the left hand side here in the videos folder and if i click that file we can see it defines our module called videos and it's running go 1.15 so now we have an empty go repository which is our videos folder and we have an empty standard go module file defined over here so next up let's create our first go file which is going to be our main entry point to our application so what i'm gonna do in the videos folder is i'm gonna create a new file called main.go and then i always like to start with a hello world application so what i'm gonna do is just copy the snippet of code and i'm gonna paste it into main.go so in our fundamentals video we learned how to define a package so every go file is part of a package we define package main we import a standard library which is the fmt package and we have our main function which is going to represent the entry point of our application go will run this whenever we execute our code and the only thing we're going to do in here say fmt dot println hello world so we're just printing a hello world message to the screen to test this code i can just say go build and we can see it produces a binary on the left side here and to run that binary i just say dot slash videos and we see hello world prints to the screen so our code is good to go so in our go fundamentals video we've learnt about how to define a go repository how to create a go module how to define a package for that module and multiple go files that form part of that package we've also taken a look at the fundamentals such as variables types structs and loops and all the basics of go so let's take all of this that we've learned and create a videos app that deals with video data the app will have a function to get a list of videos as well as a function to update the videos and this will demonstrate the fundamentals of dealing with data such as json now i like to keep my code modular so what i'm going to do is create a go file that deals specifically with video so to do that all i'm going to do is in the videos folder right click say new file and i'm going to create a videos.go file and then we're going to add the fundamentals so i'm going to say what package this go file is part of which is package main and then i'm going to import any other packages that we might need so for now i'm just going to leave that empty and i'm going to define the struct of what a video should look like now in part one we covered the fundamentals of go which includes structs so this is basically a definition of what we want our video object to look like so video has an id of type string it has a title a description an image url which is the thumbnail as well as a url to the actual youtube video and then i'm going to create a very simple function in part one we also learned about functions so we say if unc the name of the function which is get videos and we're gonna pass no input you can see that's empty and the output is gonna be our struct and it's gonna be a slice of videos so basically a collection that we're going to get and we're gonna return now in part one we learned how to populate a struct which is very simple so to do that all we're gonna do is remove these comments and we define a struct like so i give the name of the struct which is just video one i then save i then put the instantiation character to tell go that this is a new variable and i say equals and i pass in the format of the struct so this is the struct type which is type video and then i just populate all the variables and this looks very similar to json so i have id colon and the name basically key value pairs and also notice that after every value there is a comma and even at the end so there's a comma at the end as well so what i'm going to do is add two videos i have video one and video two with different data and then i can define a slice by saying return a slice of videos and i can open close curly brackets and i can pass in video one comma video 2. this will create a new slice of videos and return them so now we have a very basic function called get videos that defines video 1 video 2 and returns them as a slice of videos now to use this function what i'm going to do is head over to main.go i'm going to remove this line of printing hello world and i'm going to define a new variable called videos i'm going to instantiate it with the instantiation character and i'm going to assign it to the result of this function so i'm going to call the get videos function and that will return a slice of video struct into our videos variable over here and then to see what that looks like we're just going to say fmt print ln videos so to test that code all i'm going to say is go build that produces the videos binary and i'm going to run the videos binary saying dot slash videos and that will go ahead and print out videos data to the screen so far in this video we've been hard coding our video data in our code base which is great for learning about the fundamentals of go such as variables structs slices and functions but it's not so good when dealing with data because data can change if we want to add or remove videos we don't want to have to go to our code and recompile it usually data lives in a database or a data store a cache or in the cloud data usually takes form of a file such as a json file xml file or a csv but let's go ahead and make our application more useful by removing the data from the code and storing it in a file now an important part of learning go is to learn how to navigate the go documentation by yourself so we're going to be doing that as we go along now when dealing with reading and writing files there's a go package called ioutol which is a standard package part of the go programming language and we can import this package by saying import ioutol i'm going to copy this and i'm going to go to our import statement and i'm going to add that to our import statement like so and the next thing we want to do is remove our video data from the code and place it in a json file so what i'm going to do is in our videos folder i'm going to right click and i'm going to create a new file called videos.json and then i'm going to paste the video data into the videos.json file and you can see this is an array in json with two objects representing a video each now if we take a closer look at our variable where we defined our video struct and we take a look at the json file notice that the fields are the same the only difference is that the struct field starts with an upper case where the json fields are all lower case so the go programming language is able to figure out how to map these fields to the struct directly we'll take a look at mapping fields a little bit later so the next thing we're going to have to do is learn how to read that file so firstly let's clean up this function by removing those variables and let's take another look at the ioutil package and if we scroll down we can look at the index and it shows all the functions that are part of this package and in our fundamentals video we learned about how to read functions and we can see there's a read file function its name is read file and it takes in one input which is a file name or path as type string and it returns a slice of bytes and any error that may occur from reading that file so now we know that this read file function returns a slice of bytes and an error let's go ahead and define that so we go to our get videos function and we define a new slice of bytes called file bytes and an error and because these are new variables we have to put the instantiation character then we call ioutil.read file and we pass in the path of the json file that is the same as the file name inputs of the function over here so now we have the bytes of the file that we've read as well as any error that might have occurred as a result of reading that file as we're building more advanced applications it's important to notice that we should always handle errors for now we'll make sure that the application crashes if it encounters any errors when reading that file because we don't want the application to continue running if there's a problem reading the file we'll learn about more in-depth error handling in a future video so to handle that error what i'm going to do is i'm going to put an if statement to say if that error is not null meaning that there is a problem and there is an error inside of this variable called the panic function the panic function and go typically means something went unexpectedly wrong and these are usually areas that we are not expected to handle gracefully and then what i'm going to do is i'm going to create a new variable called file content i'm going to instantiate that variable and i'm going to convert the bytes to a string that we can read so i'm going to say string open close brackets and i'm going to pass in the file byte slice into this function this is a standard go function for converting bytes to a string and i'm simply converting this to a string so that we can read it i'm going to say fmt.println file content and i'm just going to print that contents to the screen so because we don't have any videos just yet what i'm going to do is just return nil because we have to return something as part of this function and because i'm using this fmt package to print out the contents as a test i'm just going to go up to our import statement and i'm going to add fmt to there as well telling go that we want to use that package i'm then going to say go build and i'm going to run the application by saying dot slash videos and we can see that it's print out our json content over here so now we know it's reading the file and printing it to the screen so now we've learned how to read a file and go now when we read files you can see it returns a slice of bytes and to make it readable by humans what i've done is convert the slice of bytes to a string that might be useful for us but it's not really useful for the application the application expects a struct of videos so what we need to do is convert the json file to a struct of videos and we want to populate a video collection as a slice of videos now for that go has a very useful package for dealing with json that introduces us to two new concepts marshall and unmarshall now working with json and go is pretty straightforward we have a standard package called json and the package json implements encoding and decoding of json the mapping between json and go values is described in the documentation for the marshall and unmarshall functions the two functions that are very important one for converting json to a go type and one for converting a go type back to a json structure so this is very useful for converting a slice of bytes containing json to a slice of videos and vice versa so let's go ahead and import this package by copying this part here and pasting it in our import section so i'm going to go over to our import block and paste it over there and if we go back to the documentation we can see that there are two important functions one called marshall and one called unmarshall unmarshall is the function of converting a slice of bytes to a go type so we can convert a slice of bytes to a videos type marshall is the opposite taking a go type which is our video and converting it back to a slice of bytes now since we are reading a file we have a slice of bytes already so let's go ahead and convert the slice of bytes to a slice of videos now we do this using the unmarshall function which takes in a slice of bytes as well as an interface which is going to be our empty slice of videos now you can see at the top of the function we already have a definition for an empty slice of videos and that slice is called video so what i'm going to do is remove this code over here and i'm going to call the json.unmarshall function and i'm going to pass in our file bytes as well as a pointer to our video slice in our introduction to go we spoke about pointers so this basically will tell go to update the same variable in memory without creating a copy of it and if we take a look at the documentation of the unmarshall we can see it also returns an error so we have to handle it so what i'm going to do is i'm going to say err equals and notice i don't have the instantiation character here because the area is already defined up here so i'm just going to say equals which will override any error that occurred previously and i'll go ahead and handle that error in case another error occurs so now we're converting our slice of bytes into a video slice so instead of returning null i'm going to go ahead and return our slice of videos and to test that i'm going to say go build and to run it i'm going to say dot slash videos and we can see we're now printing out the same thing but this time we're reading it from a file so now that we have data in the real world we'll want to do something with that data so let's go ahead and loop through the data that we've returned and update some terms and conditions in our video description now in our go introduction video we've learned how to write loops so instead of printing out the videos to the screen like this let's go ahead and write a loop which says for every video in the range of videos and what we want to do is update every video's description box and append some terms and conditions to each video so what i'm going to do here is i'm just going to say video dot description equals the video dot description and i'm just going to append some text to that video description i'm going to add a new line character and i'm going to add my terms and conditions for every video and then once i've updated all the descriptions i'm just going to say fmt.println and i'm going to print all my videos to the screen to see the updated content now when i say go build and run that application you can see that my description did not update now this is an important thing to understand with loops when looping objects in a slice go will return a copy of that object so what we've done here is we've updated the description of the copy not the original object inside of our slice so to update each video inside of our slice we have to use the index number so what i'm going to do is enable the index number so i'm going to call it i and i'm going to replace this line with something a little bit different where i say videos and i pass in the index of the video item i want to update and then i say dot description and then i equal that to the same thing and i append my terms and conditions to that video object so this will go ahead and update every video object inside of the slice and not a copy of it and since we're not using the video object at the top here i'm going to go ahead and silence it with the underscore character and i'm going to say go build and when i run it this time we can see now we have our terms and conditions added to the description box so now we've learned how to read a file and we've read that json file into a slice of bytes converted that slice of bytes into a slice of videos iterated it with a loop made some updates and now we want to write it back to a file so for this let's create a save function that takes a slice of videos and performs the marshall function to turn it back into a slice of bytes and write it to a file so for that let's go back to our videos.go and we can see we have our get function now let's create another function called save videos so i'm going to create this function and this function needs to accept an input of videos so what we need to do is define an input here we can define a variable called inputs and it's a slice of videos and note that this function here does not return any output so now what we need to do is we need to take this video slice that we have and we need to convert it into a slice of bytes using the json marshall function if we go back to the documentation we can see there's a function called marshall that takes in an interface which is a go type in our case a slice of videos and it'll return a slice of bytes and an error so we can call that function like so we can define our video bytes which is our slice of bytes returned by the marshall function and we also define an error which may be returned by the marshall function as well and then we say json.marshall and we pass in our video slice and then as we do and go we make sure that we handle that error coming back so we make sure we check whether the error is not null and if there's any problem we hit a panic and prevent any further execution so now we've taken our slice of videos converted it into a slice of bytes and now we can write it to a file and to write it to a file we can take a look at the i o util package again and there is a write file function which takes in a file name or a path and a slice of bytes as well as permission for that file and returns a potential error so what i'm going to do is call that function now we can get an error notice that i did not pass the instantiation character here because the error is already instantiated at the top so i say error equals ioutol.right file i pass in the path and in this demo i'm just going to pass a different path so i'm going to write a new file in this case and not update the existing file i pass in the video bytes and i pass in the permission for that file and then as always i also have to handle that error that comes back so if the error is not null we hit a panic and that is our save function ready to go so now what i can do is head over to my main.go where we retrieve the videos we do the update we print out to the terminal and now what i can do is finally after we've done our update here i can go ahead and save the videos by calling our save function so to test that out i'm going to say go build and i'm going to run my new code by saying dot slash videos and we can see it's printed out to the terminal and if we take a look at the left here we can see our videos updated.json file which is what our save function has saved if i click that we can just format the document and we can see we have our updated videos json file in here now we've learned how to read and write files and how to convert json data into structures that we can use inside of go now in the real world we might not be writing files but we can use the same fundamentals to write these data structures to things like s3 elasticsearch and other types of data storage the key thing here is using the i o util and the json package and knowing how to read and write data using slice of bytes and converting them with the marshall and unmarshall functions is key to working with data structures such as json in go but sometimes there is a case when the fields in json data does not match the fields in our go type or in our go struct for this we might need to use mappings mappings are important in the case where the fields in the json as you can see on the left here does not match the fields in out video struct that you can see on the right here in our case they match but sometimes they may not match because the json might be coming from another system so to do this the go programming language struct type allows us to do mappings for each field so if i go to myvideos.go and i go to the struct i can add mappings to each field and i do this using the backtick and then specifying the field in json i want to map to as an example i can replace all of these fields with this you can see here i pass in the backtick json and then i refer to the field that is matched in the json file so this tells the go programming language what field to pick up in the json file and what to map it to in our field in the struct and it's always recommended to explicitly tell go which fields to map to so now we've learnt about the fundamentals of go and how to work with data structures these are essential building blocks of learning how to work with data structures in memory as well as with data stores and learning how to store data in various systems this can involve connecting to databases we can write our own web service and serve this data up via http we can write applications that connect to other apis or microservices and send data across the wire now stay tuned as we'll take a further deep dive to learn more about these concepts and go and will push our learning to the next level if you liked the video be sure to like and subscribe and hit the bell and if you'd like to join the community be sure to check the join link in the description box below and if you want to support the channel even further be sure to click the join button below and become a member and as always thanks for watching and until next time [Music] peace

Channel: That DevOps Guy

Views: 3,005

Rating: 4.9537573 out of 5

Keywords: devops, code, azure, k8s, cloud, training, course, cloudnative, az, github, development, containers, messge, broker, queues, web, services, coding, programming, golang, go, microservices, javascript, nodejs, python, devs, developers

Id: _ok29xwZ11k

Length: 24min 18sec (1458 seconds)

Published: Mon Jan 11 2021