justforfunc #42: Intro to Go Modules and SemVer

hi I'm Francis Campo in this it's just a funk so welcome to episode number 42 of Joseph funk today we're going to be covering a topic that many have been asking about which is called modules I will not be covering go DEP instead we're gonna be talking directly about go modules you do not need to know about that will mention a little bit as in how to migrate from DEP to go modules and whether that's something you should be doing now but I'm going to start with something as basic as explaining what a module is and why do we need them that's not some fun let's get started okay so the first question is of course what is a goal module and the way I like to answer this is by comparing it to two different concepts that we already know so a module is a way of packaging software that's pretty much where it is right and once we know that we can compare it to other ways of packaging software that we know we have as the name implies package a package is a way of packaging software and also we have repositories what are the difference between those well a package can now be subdivided into working pieces right like you could say that you can divide a package into two different files but now you still need those two files probably right to represent the whole package it's basically an atomic thing while repository contains many packages right and those packages can be move around independently etc as long as you update of import path the corresponding import path cool so what is a girl module well a goal module is not atomic a go module can contain many packages inside and that's actually what normally happens so it's kind of like a bypass ring it is also version we use semantic versioning so it's kind of like a repository right a repository has one single version for the whole repository and then you have all the packages inside but at the same time there's a little bit of a difference with a bit posterior one is it is actually not tied to end a gate concept right there are different things even though they serve a similar purpose and the second most important part which you know it's important but it's I'd say a detail them most people do not care about is the fact that every Pastore can contain multiple modules so a module is kind of like a repository but a little bit more general and we also talked about versioning so I guess that's a good question to ask which is what is versioning why do we use it how do we use it and version it's simply saying hey you know software changes over time deal with that and it's a way of saying hey this version worked for me right Brighton saying this version was the version cut on I don't know October 2nd of 2017 it is actually easier to say you know I'm using v2 right and we specifically use a way of managing those versions which is what I call semantic versioning semantic versioning or semper is pretty straightforward convention on how to name versions you always start with V right so that's important and not everybody does when they go community you're supposed to start with a V so the version 1.0 it's actually V 1.0.0 that is the basic that first version of the version one nice v1 doctor doctor now what are those two other ones right like we have v1 v2 v3 those are the major versions and the idea is that you have a major version for every single non backwards compatible version so say go so and go we have the v1 dot zero of v1 dot one to the three etc no duh 11 now next we'll have dot twelve probably but eventually maybe we'll have go 2.0 right and the whole idea is that once you name that go 2.0 what you're implying is not that there's some really cool new stuff but rather than if you try to use code waiting for v1 and try to compile it with v2 it might not work right so you've broken that backward compatibility and that's what the major version means the second number so 1.1 1.2 1.3 to 1 or 11 for go we have what we call features right the idea is there is a minor version and we increase it when we have not broken any backwards compatibility but we have we have added something new that it's again backwards compatible that's a whole point right so for instance say with added type aliases type aliases doesn't mean that the previous code doesn't work with the new compiler it totally works right the other way around if you write some type alias and try to compile it with go under 0 you will no work but if you get something that was written for going to 0 and compile it with go under 11 it will work that's why the minor version and minor versions basically are safe to move forward and I say basically and practically and in theory because indeed sometimes things happen you know what were I was thinking but something sometimes things happen and it means that the Box backwards compatibility has actually not been protected right so these things happen they tend to be a bug because if it was not about and we broke that backwards compatibility intentionally we should have changed a major version not the minor and the third number is one called the patch and a patch is simply that it's something it's patching the previous version we're not adding new features we just fixing things that were broken so whenever you have a small issue that has been fixed you will see that that number increases you also see this in the go community in general when there's issues that are so important that we want to reach or fit them into the previous version in general you will see these four if security updates right say there's an issue that has been detected in some part less than the library we want to change it we will apply that change to say go one dot 11.1 we'll go to go 1.11 dot dot means something has changed in the since the previous version but it's nothing new it's not a new feature and it definitely doesn't break anything right it just fixes some things and that's what Apaches there's a different naming convention that I actually almost prefer which is rather than saying major minor and patch which implies some kind of size it says breaking feature patch breaking because when you use that number when you change the numbers because you're breaking the backwards compatibility with the previous version the second number instead of minor is feature because maybe there's a lot of code that it's new but you didn't break anything previously right so you can still use that one and then patch which is not breaking things not features just fixing something that needed to be fixed with a patch okay so enough theory and let's start playing a little bit with modules and see how they work I'm going to be using docker for this because that way I will show you how everything is from scratch there's nothing in there isn't completely new docker container every single time I run something okay so you can also do that it's actually available ready go 1.11 so you can run docker run golang wonder 11 and that should work let's do that okay so let's get started by running go 1.11 in a docker container so in order to do that I'm gonna do a docker run RM so it gets delayed at the end I T so it's interactive TTY and then golan 1.11 and okay we got it running and I've I to go version you'll see that these conversion 1.11 dot to the semantic versioning it means that this version 1 of go which is the first one to be stable and never has been broken the 11 means that it's the 11th time we release a new version with new features and 2 is the third version so the second time we fix something in the gold 1.11 version so wonder go 1 dot 11.0 what's the first one apparently there was an issue that is important and we fixed it and now we have one all them and then then we got wonder 11.1 and we did that again and now we get go under 11 don't you that's how these things work so ok so what I'm going to be doing next is going out of Co path go and go path will tell you that that slash go dad go path in in that we are and the route those are not the same place which means that we're going to be running go outside of go pack [Music] now this seems like it's a huge change but it's actually not that big other than for the fact that now you have a freedom to write things outside of gopath you also have the freedom to write things inside of gopath and I still do that just because everything that I do is around gopath I decided that even when I don't look things like rust I actually done blow it was by doing go get - the github.com / brass last rest right so that works it is probably a weird thing that I do but it is the thing that you can do and now you can also write that code completely outside of Coppa so let's try that I'm going to create a new file I'm gonna call it my cut which is not the animal it's actually the cat the command so we're gonna go to my card and now we are going to write some code so let me install them because I'm sure it's not there since it's a completely new container cool and now this director is still empty so when we're gonna do a VM main go package main funk man and what I'm going to do is I'm going to do the simplest cat that simply gets the input from standard input and print it to standard output so that's gonna be a okapi OSTP out OST in let's see and that's going to return an error and if error is done you know lock fatal error import io OS long-running go without go important it's always fun okay so let's see go fun - double you mean the go and go run go build that's going to generate my cut which is that binary there now find my cat found my cat o files Nani stole docker okay so if I do my cat then I do hello it prints hello and if I do LS / my cut it just just works and prints itself right cool so this is a go problem and it's running outside of modules but if I wanted to add extra things say I'm going to now depend on set of log I'm gonna use Lagos and that is github.com slash rips and slash loggers go run mender go yeah it could not fund it anybody to go get if you try to find it and install it but it's saying hey I can do this because you're at sort of go back I cannot store code when you're outside of go path that's not a thing that going also knows how to do well go models to the rescue we're going to create a go module here right so we're going to go from having a directory that has some go code into having a directory that has a go module you simply do go mod in it and when you do that it will complain it will complain because you're outside of a Coppa and it doesn't know what module you're initializing it does know that this is the model you're nationalizing but what is the important the import path that information used to be in the directory of where you were storing that code in south dakota that information is now gone so you need to give it again so go mod in it let's say github comes last time for my cat and of course you can choose any name you want to here and going with the traditional one just because it's easier to remember to be honest okay so now we have go mod let's see what's what's going on there okay so there is go mod but these doesn't do anything yet if you do go build look at that so now it was actually able to understand that you son of a go module because of that go mod file anyone in downloaded all of the dependencies even found things that were wrong like this log is not used anymore so we need to remove it but the important thing here is all of that output so we tried to find github calm / - rips and sliced loggers and we saw that we had the v 1.0 so we didn't have it on our machine so we downloaded from github so downloading github account / corruption then we figure out that we also needed these girls pew and these testify and this stretcher and there's all these things all of those are actually needed not by us directly but but buy lockers itself so let's see what we have under goemon go so now we have required github coms last few reps and slash loggers be 1.2.0 so that is the information that we really care about it's the fact that our module depends on that module and this is where you're going to have all that information if I'm not mistaken yeah there you go it also created got some and got some has way more information I'm gonna put it like a bit smaller so it's easier to read but you can see here that on top of the version that we're using we also have hash this hash here it's a cryptographic hash and the whole point of this is that we know whether something has changed over time right so say that have v1 dot tier right and we know this v1 dot 0 because when we visit that website we have under tags we have V dot one dot zero and that's the last one that's why that's the one we're using right cool simple but what if someone that has committed access to this repository was able to do to change something and then do gate push that's F right like now they're pushing a new version into the same tag how do you know that these change well you know that through goes up these information will change and it will actually complain and say hey you're saying that this is the same version it's not something is wrong right so you're gonna be fixing this and actually let's let's build this run it so I'm gonna do fine I cut cool and then let's see what happens if we say actually we don't want to use V 1.0 we want to migrate down to V 1.1 for instance how do we do that well it's actually pretty straightforward you go here and say we're gonna be using 101 thank you very much and now when you do go Gil it's gonna say ooh actually we need that version so we cannot migrate to that one and also we're gonna update all of the dependencies that we had because of that right because of because of loggers so now if I do go cat mod it still says the same that I left before but go that sum now it's gonna have look at that it has loggers v1 and Lapras Picchu and these might seem confusing right because why do you have v2 we're not using it anymore well the whole point is that we actually going to remember when we said v2 v1 dot - what version we actually meant to the point that if later on if we say oh you know actually downgrading to v1 was not a good idea v1 that one was not a good idea we should migrate back to 1.2 now we're gonna be able to say whether this was the previous one that you that we're using before so now when basically when you're doing when you're reverting you know you're reverting to something that you can trust so gold Ahmad is the way where you indicate what you depend on God some is how we track what do you actually depend on specifically to the point that if anyone changes anything including just a little typo on a comment or something and they just do push - f you will be able to notice right so we have those two pieces and now the question is of course well do I need to put both of them oh my github and the answer is yes I have heard people saying you should not do it because the problem is that if you have to users that are editing got some you're gonna have merge conflicts yeah that's great right because the whole idea is that if you have two different developers that are using a different version of the same library that's a problem that could cause errors that could cause having on one side so more writing some code and testing it versus version one and someone else actually doing over version chewing they don't even notice because actually they have the same tag so it is important to commit your goat song it can't cause problems but those problems are actually features that's not a bug when you have an issue with go some that's actually good is should to have because it means that you're actually taking care of that concept of having hermetic builds which means that everything goes inside of the build okay so now we know how to create a new module go mod in it and then how to add dependencies you can simply do go get in that would work what if we wanted to upgrade back to the previous version so something that we could try is to go get this - you actually go get - you of that vloggers and hopefully these will realize that there's a latest version and we'll go back to that previous version so now go mod you see if you want that you it also added two more in direct indirect dependencies do not worry too much about those basically what we're saying is that those dependencies are actually they're not directly but indirectly and depending on the platform that you're using might be different so that that's why all of these things will appear actually it's a great thing to do to run go mod tidy and that will even find more things it will find the the libraries that you depending on for your tests because when you do go get on go build the test files are ignored what we're doing now is actually paying attention to those two because once you run your tests you want to make sure that you still have that hermetic built with the same qualities so with test works you know that is because all of your code works when it fails it bigger something fells it's not because there's a new version that you cannot expect that was to the machine right so let's see what they're having go some now got that that's now goes some that's pretty big okay so now that we have the basics the next questions are actually all about hey so what do I do with these now right like not everyone is using go modules how do I make it so this would work for everyone well if they're using go 111 that will just work if they're using something previous to that you will probably also work but for some other versions these will not work unless you vendor the dependencies right we can definitely do that so the dependencies on here where are they stored actually they're stored under your go path package mod and you're gonna be able to see all of the things in there all of the code that we've downloaded is there for easy access if you wanted you could even have something like this but remote internally at your company or whatever but in a different server and if you're interesting that you should definitely check out Athena which is a way to manage modules it's pretty cool project so you should have fun take it out I have a link to one talk that earns less when you're gave a gopher Palooza recently about it so if you wanna check it out to be here there I still never know okay so what else can be doing this so go mod vendor now when they do go more vendor look at that that was super fast but now we have a vendor file and that vendor contains all of the source code you depend on right so now we're back to the previous like if you're doing go 1.6 it will definitely not work with modules but thanks to these you can start using modules and make sure that people that are trying to use your code can use it no matter what version they're using so one more thing you might want to consider is using go mod why and go mod why it allows you to answer a very useful question which is hey how do why do I use this right like that's why why do I even have this in my in my godson so let's see go that some have a lot of dependencies how are using this one go I do not know personally but if I do go mod why of that it will tell me that it doesn't know either and that is normal because actually we're not using github.com / pima's art / go do flip but you using something else a module that is inside of there so let's add - m to say actually find something that is in there that answers your question and now we found it we're actually using golden flip / stiffen it which is used by the testify assert which is low used by the testing package inside of lockers which is obviously used by lockers which is used by my cat so we got it now that's why we are depending on that library cool so I showed you how to create something from scratch but what if you really have some code and you want to migrate to use go modules so in order to do that I'm actually gonna use a piece of code that I already wrote long time ago it's called embed MD so I'm gonna do git clone HTTP GET has come and is useful to embed source code inside of markdown I use it all the time for my workshops okay so if we go there you will see that these does not use go mod go some or Deb or anything like that there's nothing here I'm not taking care of the dependencies so what next go mod in it and now I was able to actually figure out that this github.com / Campo / embed Andy how did it figured out I'm assuming is because of the git remote get your origin I think that this is where it's getting information from but maybe not so it's it's an interesting thing to check out if you feel like checking that out but anyway now we have CAD go the mud and we have that information now this is a module it is now complete because we had we don't have any dependencies how do we find those go get da da da just install everything right and look at that the first one is go develop we is indefinite we want to be able to find it here huh we found it so oh yeah because we're already using it from four right go mod it just says we're using go develop here we're using it directly though apparently cabal goes some cool so go there flip perfect what if we do with go mod tidy okay god I'm not cool so these works and now we have something that works easy right now this is a module and we have our god mod go about some and if you want to have the dependence bender is the dependencies vendor or the vendor dependencies you can do go vote go Mont vendor and fine vendor will tell you that we all we all have this stiffly dug up okay so now that we've seen how to create from scratch and also from code Darwin existed bad it didn't have any dependency management yet now we're gonna do one that has dependency management already there which is we're gonna be using our one of my repository to go to Lee workshop and we're going to migrate from DEP to modules so keep clone did have that come / kappa / go tubing workshop which is by the way a break workshop and you should check it up and if you work for cumference and you're interested on me running it let me know okay so go to my workshop and now so we have go packaged on lago package tamil and we have them right so we already have all of that information what happens if I do go math in it well we're copying the requirements from the gold package Starbuck so now we have a golden mod if I do go mod tidy actually let's see what we have in there for now we have some information we have the dependencies by doing go more tidy we find not only the ones we really depend on but also the ones we depend through tests so go the mod so we have a little bit more but also go that some it's gonna be also all of that information that we have and now that we have this there is something we can do which is remove go package tamil go package the lock and even vendor and now we can do go get double dot and achieve this works cool and go test double dot and see that all of these just still works which is great right like we migrated from from not having modules to modules super fast and if we still want to give support to other versions that do not know but modules we can still do go mod vendor and see that we have on the vendor we have all the dependencies that we have so they're all there which is great but now I wanted to do one more thing that it's gonna show thing that is little bit of a pain we're gonna remove go the mod go that song so now we don't have neither DEP nor modules nor the vendor directory tree let me remove them in there so we have no depth no modules no vendor nothing and I'm gonna do go mod in it and this created the the repository which is cool the go mod I created the mod file which is perfect and now I'm gonna do go get done any doesn't work what is going on well these actually kind of interesting right because it's saying hey so you saying you'd have come through Robson with a lot capital S and but actually the one I found was github that comes Arab sand with a lowercase s lovers and if you look at the code you will see that there's a go mod in that go mod here says github.com syrup says syrups and lockers and that is exactly the way you should be importing it if you try to import in a different way it will not work which explains why all of a sudden it complains about uppercase versus lowercase which is a new problem but it is important to take it into account how do we fix it well let's see Goemon is empty so where are we using syrups n' well actually the easiest way to do this is to say we're gonna find everything that is don't go and then grab for sir epson cool we got it so this is our file and that sir Epson lowercase we change that we do go get dot dot dot and now it works and if we do after we're done building it we do go mod tidy it added a little extra thing and CatDog Ahmad looks perfect it's saying hey you depend on lockers which is exactly what we're doing and if we do go that some will have all of the information about the exact version that we're using and we can do again go bender and put those available inside of a vendor directory right there cool so this was a quick intro to modules I hope you got the main things it's a way of packaging packaging your code with all the dependencies of any easy it's easier to share you're gonna be able to manage dependencies you can migrate from nothing or from DEP and everything you need to do is go mod in it and there's some little quirks that you might take in you might need to take into account if you want to know more about your modules I definitely value to ask me as many questions that you want you can find me on twitter as france ask you can also send suggestions to form joseph uncom and also for sure you can leave comments down here what is the next thing you want to know about go modules have you used it are you happy with them do you have some issues tell me more I'm very very curious about this go 1.12 which will be released sometime in favor of enemies will actually contain the final ish version of Co modules so now is the moment to give all of your feedback as always please like subscribe all of those things tell all of your friends how awesome Joseph funk is any of you related on that then I guess it just thanks for watching and see you all in two weeks [Music]

Channel: justforfunc: Programming in Go

Views: 41,913

Keywords: golang, justforfunc, go modules, go programming language

Id: aeF3l-zmPsY

Length: 31min 47sec (1907 seconds)

Published: Thu Nov 15 2018

Reddit Comments

[u/aboukirev]

Thank you, Francesc.

I've been postponing migration to modules. I have no excuse to delay anymore :)

[u/BubuX]

"...we're going to be writing Go outside of GOPATH" (at ~8m45s)

Heresy! (╯°□°）╯︵ ┻━┻

[u/10baller]

I enjoy your video series immensely. Thank you for staying active in the Go community and providing such high quality free content.

Now that I've buttered you up can I make a request? I would love if you did a video on "goroutines and channels" maybe showing some good use cases, good patterns, bad patterns, pitfalls, etc. I still hesitate to ever use them in production code because I feel like I'm forgetting some golden rule that everyone knows but me.

Cheers!

[u/k4kshi]

Can someone tell me if this tutorial is reliable? I want to learn about go modules and need to know if it's the ways it's done nowadays. Thanks

[u/jaekim]

Will this just work if I have dependencies that are in a private github enterprise repo instead of public github?

Do these dependencies all need to have had go mod init run on them first?

What happens if a dependency doesn't have a go.mod file?

[u/Bielefem]

Nice video as always,

I hope go mod sees bigger adoption,

because people break still SemVer and fixing building conflicts is boring.

But I would like a second episode about fixing building conflicts to get more people adopt go mod.

[u/DoctorHoneyBadger]

I did not know this channel existed, but I'm so happy I've found it now. Video was great! Immediately subscribed.