Software Architecture in Golang: Security - Dependencies

hello my name is mario welcome to another software architecture in golf video in today's episode i'm going to be discussing with you security specifically when dealing with dependencies so security is a huge and complex topic i'm stating the obvious right there and we as developers we should really care about it now in this video like i said i'm going to be talking only about dependencies if you want to get more uh details about for example cloud security and those kind of things there are there is this nice be nice a book called practical cloud security released in 2019 and written by chris dodson uh feel free to check it out i will be leaving the link in the description so specifically about dependencies we are going to be adding a few things to the different products that i have in my uh github account my github profile this involves includes the one obviously the tool microservice that we have been using for examples and a few other tools that i read in the past um that i have written in the past and you can you can refer to those as well as always the links will be in the description so we're going to be covering a dependencies regarding the standard library dependencies obviously third-party packages i'm not covering dependencies in the context of the services that we use for that add a value to our own service examples will be data stores or maybe things that we use for storing passwords or maybe uh the tools that we use for content containerization or building and deployment those kind of things so that is again yes it's a dependency but for this video i'm focusing on those two in particular yeah there are going to be a few examples that touch a few of them but that's because of the service that i'm going to be showing you so i'm saying service too much time too much yeah service too much but don't worry about it i will show you the concrete examples okay now dependencies are going to be using github actions and github you don't have to use github actions if you don't have to use github obviously you cannot use github options but you can use any as any continuous integration service that you you want to use in the past when i cover maintainability i was mentioning circle ci jenkins ci um i'm missing one gitlab ci obviously and there are even more more than that but you can use any any ci that you want to use the important things that i want to discuss and want to show you are the different tools that are mostly available in any circle in any continuous integration service so we have the pentabot which is a tool for upgrading the version of your packages automatically and we have this linter called gold sec we have this service called sneak we have this uh service integration thing that is added to github code ql and finally we have a tool that i wrote that is called versions for determining we're giving you um like what versions are you using in different go projects that happen to be using go modules all right so let's jump into the code and i will show you i will walk you through each one of the tools that i just mentioned dependable is the first tool i want to show you is a service that is integrated with github but you can also use with gitlab for example you need to configure it in a different way and it is more or less equivalent to what github offers you now what happens with this one in particular i'm using the videos not the videos sorry the versions project that i have here and i just added this general file this is not not a workflow um in the context of github actions therefore is not testing or running any projects uh or building our binaries or whatnot is just literally running another step in the github actions pipeline so what is this happening is that every time i push a code it's going to be checking you your circle ci configuration which i had here in my project so i i also have an integration with girl with circle ci in the versions project but i'm adding this github dependency that is basically just checking the configuration is valid and every time uh daily it will be checking my go mod if you notice is doing i'm using the gum mod is checking in the path in the root path therefore it's going to be looking for this go mod file and then if there is a dependency that i have right here listed that has a new version available it will create a pull request that then i can merge automatically the whole point of this is that you as a maintainer you don't have to worry about checking the dependencies every now and then of the packages that you use so for example if you go to my pull request in the tutor microservice and you will notice that i have a few of them open for the open telemetry package because i just push these two versions i don't have anything just yet but it's most likely tomorrow or in a few hours or whatever the case may be we i will be seeing more pull requests now the important thing about this like i said it's just a pull request it runs your integration that the configuration that you have and if it builds it builds if it doesn't it will fail like you like you see right here the open telemetry in this case it broke some of the api so it's not building correctly so it won't merge until you let it uh you you until you do it manually that's what i'm trying to say now i will be leaving the link to the configuration of all of this in the description so you can check it out so let's jump into the second tool that i want to show you the next tool will be ghost is a security linter that is included in the golang ci linter so every time if you happen to be using this metal enter it will be running that configuration as well this is really cool because there are cases where you also can make some mistakes when writing go and although the services that i'm going to be showing afterwards can also detect these errors this is a nice thing to have you want to run it locally first so it's really simple to enable you use if you again if you use colon ci you just make sure that it's enabled for example in my do do micro service configuration that i have right here i don't have it disabled therefore everything is enabled and with this gosek will also be enabled as well so again if i go look at the github workflow that i have for testing i'm running well rather not testing but in this case will be linting i'm calling the linter right here with the configuration that is coming from the golden ci file uh the xml file and that's pretty much it i mean pretty straightforward nothing crazy to do and i highly recommend to enable it as well let's jump into the next tool so we can continue with this video so the next one is called sneak a sneak is an online service that you need to pay if you are using private repositories because i'm using an open source project which is a public which is also public in this case i don't have to pay anything for using their service what i did is i linked sneak to my github account and i started adding projects so you go to add project select github and you add the projects that you want to add in this case i wanted to select the go products that i built most recently which is neat videos versions and the tool api microservice example i use most of the times so i'm going to refresh you this a little bit so we can see all the importation and details that just happen so we have the cool thing about this sneak in particular is that it's not only considering goal related dependencies like the standard library and third-party packages but also a few of the details that you may have related to the due to your containerization like docker files and in some cases it could also detect when you have um some uh like uh credentials and whatnot so let's look at neet for example so you have need i have a few issues there is no code analysis issue which will be in cases when you have like i said credentials and those kind of things secure strings and things that you should be do using in the first place or rather pushing it or adding it to your repository repository repository or hard coding those there somehow now we have the docker file and obviously my go mode manager that we are using for managing dependencies in go so i want to show you the docker file first so you can see that it's really cool because it allows you to determine depending on the image that you're using it determines the issues that are coming in that image and you can actually create and fix it via a pull request using the user interface sneak also allows you to automatically create a pull request as soon as those are detected so that's another nice cool thing to have in your automatic integration when you push any code you don't have to have gitlab ci or ndci integration or github actions or whatever because this is integrated directly to your github repository so every time you push something there is a communication between a sneak and github and they handle all those details to determine if you need to change something okay so what i'm going to be doing right now i want to open a new merge request to fix the issue with the docker file and if you notice you just i mean just click a bunch of things and what it's going to be doing is we're going to be trying to open the pull request and there it is so there is a new pull request that indicates all the issues that i found just now running the tool and it will say it will again run my continuous integration which in the case of neat is using circle ci and obviously it's also using the integration with the sneak now the changes that is going to be added is most like you said you just adding the or upgrading the version from 1.14. 1.14.0 to 0.13 using alpine 3.11 really cool right now the other cool thing about the sneak is that like i said it detects you know the dependencies and also the code analysis and i have that one here it's a low in this case because it's just one of those things because i'm using this as an example it says i'm hardcoding a credential but rather i'm just not doing that it's just an example in this case and now if i see the file you will notice that it's in default test file so we can ignore that one for now it's not a biggie but it's really cool because you can detect those errors right here now one important thing that i didn't mention before before but i want to mention it right here which is not part of a snake really but it's more part of the security integration that github has is this way to detect vulnerabilities in our dependencies now with a sneak if i go back to the page you will notice that that in particular that one was not detected it was only detected the one what happened right here okay so if i go to my projects i click the microservice you will notice that that one was not detected anywhere but the github security did detect that one so first of all how do you enable security you go to your settings you click security and analysis and you enable all of these buttons and all of these options with that what happens is that it will give you security updates like this one in in particular that is say hey container d has an issue so we need to check it out and update it now container d i believe is coming from the docker test ori package or a docker test package so it should be okay to update that that one as well so if i go to my project locally and i do a go get let's update it to the late latest because i don't think um that should be a problem so we do a go get a play let's remove all of these we're going to be pulling the latest version and we do go mod tidy just to be safe we do go mode the sum we just make sure that we're building everything correctly we do a goal build build everything and then when this is finished we can push this to remote and see what happens so it looks like it's finished so say oh great and what is the name of this container the we put the theme and go more daddy one more time just to be safe get push now as soon as i push this it says oh github foundable and this is another cool thing about github is that when you're pushing your code to remote it will remind you that there is opponent of bulldog availability now this is still referring to this one but as soon as we deploy the code you will notice that you know it went away because we are not using that version anymore so it's going to be building it's going to take it's going to be taking a while so while this is building let's jump into the next tool i want to show you the next tool i want to show you is called code ql this is another workflow that we can add to our github action workflows that we have so i just did that a while ago it's like i said it's called code ql is using a action right here that is coming from github is an official action and this is actually a company that github acquire so that's why it integrates nicely the cool thing about code ql oopsie the cool thing about code2 is it where did my browser go there you go the cool thing about code ql is also part of the security configuration tab so you go to code scanning alerts let me show you one project that i don't have doesn't have that enable if i go to net for example i go to settings security and analysis i don't have this set up so if i click setup what is going to happen is going to send me to this page with uh for adding the code ql analysis workflow which is what i added right here so this is a literally you know the copy that is coming directly from this workflow and you can literally just follow the user interface and enable it by yourself so code ql allows you to find errors in the code base according to configuration that is part of the service itself so it's a cool way to to allow another service to find building liabilities automatically on your services and projects it's a really cool nice integration the only problem with this is that if you want to use it for private repositories you need to use github enterprise and there is some monetary consideration that you have to think about it in in that case so let's jump into the last tool and then we will revisit everything one more time so the last thing i want to show you is called version this is something that i built for determining the versions that uh the version the packages versions that my projects use in order to know if they need to be synchronized or updated or if they even using the same golf version now this is something that i'm still working on it is not completed but at least it can be useful for the following case if you know or maybe if you are not aware go maintains only up to the two most recent major releases so for example right now we are using 1.17 that means that 1.17 and 16 are only supported 1.15 is sort of like no longer supported and everything below that will not get any security updates anymore so from all the tools that i show you i couldn't find that that allows you to detect that information and it becomes it became it it becomes a little bit tricky because depending on the package or project that you're using maybe you're not using go mod or maybe if you're using go mod you again you're using an old version or maybe if you're using a sort of like a dependency management tool like go uh like what were they uh go dev no it doesn't matter the tools before go mod uh there is no way to determine what go version you're using specifically so it gets really tricky to determine what is the version of go that you're using for your project so this tool of course versions it needs 1.15 is the minimum requirement so it's you know getting a little bit tricky but the way it works it is that you pass in a path of the go mod file for each one of the projects for example right here i'm using the videos project and the to do microservice it will hold on let me make it to example md it prints out a markdown file in this example so what i'm going to be doing is i'm going to just open the market now mark down file to show you i'm going to just copy all of these and i'm going to paste it in this online visualization called dillinger and what dillinger is going to be doing can i make this there you go i didn't want to do that and let me make this a little bit better and you will notice that in here i have the versions for each one of the projects in this case there are two of them 1.17 is the one that i'm using for the two microservice 1.43 is the one that i'm using for this example that i called it alt go so it would be a nice way to determine i used to need to continue working on that versions tool but maybe it could be useful for you for your project and your use case to determine if you are running a really old version that doesn't that is not getting any new security updates anymore so let's jump into the conclusions and i will give you my final thoughts so this is the video for securing dependencies in go i didn't cover other security ways to secure your projects when you're using go i will be covering those in future episodes so don't worry about it remember the key of this video is to give you an idea of things that you can add right now to your project that are automated that you don't have to do too much to keep them working again it depends on the service that you're using maybe you need to pay a license maybe you need to maintain some extra services behind the scenes to keep that service running in case of dependable for example you want you can run it it's an open source project anyways and you can run it locally on your own premises but it's still something that you need to maintain now the linters and those kind of things has the minimum things that you really need to add to protect your services and when you're dealing with external dependencies as well as internal ones related to the go standard library some of the services and again all of these will be linked in the description so you can check it out it allows you to also search for vulnerabilities specific to packages in go and other programming languages although this is a go with specific video you can also see other languages as well so i hope you found this useful as always any comment please let me know i will talk to you next time and take care see you

Channel: Mario Carrion

Views: 637

Keywords: golang, golang software architecture, golang caching memcached, golang caching example, golang snyk, golang gosec, dependabot, golang dependabot, golang codeql, codeql, golang versioning, golang security, golang secure dependencies, golang software architecture security, golang secure apis, golang security dependencies, golang modules, golang gomod, golang maintainability

Id: 5E9QOuop5lo

Length: 19min 0sec (1140 seconds)

Published: Fri Oct 08 2021