Hello everyone and welcome to the backend
master class In this lecture, we will learn how to use
Github Action to automatically build and push docker image
to AWS ECR. First, we have to create a repository to store
our docker images. So let’s open the AWS console, and select
the region. I’m in Europe, so I’m gona choose Ireland ECR, or Amazon’s Elastic Container Registry is a fully-managed docker container registry that makes it easy to store, manage, and deploy docker container images. You can look for it in the search box at the
top of the page. If it’s the first time you use ECR, you can click on this Get Started button to
create a new repository. Or we can expand the left-hand side menu,
and choose Repositories. Right now, there’re no repositories yet. So let’s click this button to create a new
one. You can set the visibility of the repository
to either private or public. If you choose private, then no one can push
or pull image in this repository, unless they’re given access permissions. Next, we have to fill in the name of the repository
in this box. I’m gonna use simplebank. Note that we must follow this naming convention. You can also config the tag immutability. If enabled then it will prevent image tags
from being overwritten by subsequent image pushes using the same tag. I will leave it as disabled by default. Another setting is scan on push. If enabled, Amazon ECR will do a security
scan on the image to help identify software vulnerabilities. I will leave it as disabled for now. The last setting is KMS encryption, which allows us to use AWS Key Management
Service To encrypt images instead of using the default
encryption setting: AES 256. For now, let’s just use the default one. So I’m gonna click Create repository. And voilà, the simplebank repository is successfully
created. Here’s the URI of this repository, which we can use to push and pull images later. If you’re using AWS CLI tool, you can take
a look at this page to see the commands to push docker images
to this repository. Basically, you will have to authenticate your
docker client, Build your docker image, Tag your image, And finally push it to the repository. However, normally we don’t push the image
directly from our development machine. Instead, we will use Github Actions to automatically
build, tag and push the image for us whenever new codes are merged to the master
branch. So let’s open our simple bank project in
visual studio code. First, I’m gonna rename our existing ci.yml
file to test.yml And change its name to “Run unit tests”, because that’s the main responsiblity of
this workflow. Let’s copy the content of this workflow. Then, I’m gonna create a new workflow named
deploy.yml We will use this workflow to build docker
image and later deploy it to production. Now let’s paste in the content of the test
workflow. And change its name to “Deploy to production”. This workflow will only be triggered when
there’s a push on master branch So I’m gonna remove the pull request event
from the list. Then, in the jobs section, We will declare the first one to build and
push docker image to amazon ECR. It will run on ubuntu-latest as normal. Next, we have to write the steps to perform
this job. For this, I’m gonna use some existing Github
Actions. On github.com, let’s open marketplace. Select Actions type. And search for aws ecr. There are many results, But this Amazon ECR Login Action is the official
one, written by AWS. So let’s open it. Here in this page, we can see how to use it. I’m gonna copy this template, And paste it to our deploy workflow file. Now, as you can see, The first step is to login to Amazon ECR. And the amazon-ecr-login action is used for
this purpose. But in order for it to work, We have to provide some credentials to access
our AWS account. In this section of the documentation, We can see how to do it with another Github
action: configure-aws-credentials. So let’s copy this template, And paste it here, as the first step of our
workflow. Now we need to provide 2 secrets: AWS access key ID and AWS secret access key. Note that we won’t write them directly in
this file as plaintext, But they should be encrypted and stored in
the Github secrets, Then will be loaded later as environment variables
when the workflow is run. We will learn how to do that in a moment. Now, besides the secret, we must also provide
the aws-region we want to access. As I’m using aws Ireland, or eu-west-1, I’m gonna copy its name from the URL of
this page. And paste it to our workflow here. Alright, now let’s create some credentials to allow github to access our AWS account. In the console, I’m gonna search for IAM, Here it is. You can read more about it in this documentation
page. Basically, IAM stands for Identity and Access
Management. It is a web service that helps you securely
control access to AWS resources. You can use it to control who is authenticated
and authorized to use your AWS resources. There are several kinds of identities you
can set up, You can see them in the Identities section
of the left-hand side menu. The first one is User, which represents 1
single person or application. The second one is User Group, which let you specify the same set of permissions
for multiple users at once. As you can see in this picture, You can create groups for different team,
with different permissions, such as: Admins, Developers and Testers. Then another identity type is IAM roles. It is pretty similar to IAM user, But instead of being uniquely associated with
1 single person or application, A role can be assumable by anyone who needs
it. You can learn all about it by reading the
documentation. For the purpose of this lecture, we’re just
gonna use IAM user. So let’s go back to the AWS console. Select Users section on the left menu. And click Add user. First, we have to choose a username for it. Let’s say github-ci Then in the access type section, We should choose Programmatic access, since this is not a human user, but an application
instead. This will enable an access key ID and secret
access key For the AWS API, CLI, SDK and other development
tools. Alright, now let’s click Next: Permissions. Here, we will be able to set some permissions
for the user we’re gonna create. There are several options: Either add user to group, Copy permissions from existing user, Or attach existing policies directly. I’m gonna create a new group and add user
to it. For the group name, let’s call it deployment. Then in the filter policies box, Let’s search for Elastic Container Registry. In the result list, we can see the Full Access
policies, which allows all adminitrative access to Amazon
ECR resources. The PowerUser access provides all access except
deletion. Or the ReadOnly access that only allows reading
data from the ECR repositories. In our case, we want to push (or write data)
to the repo, So we can choose either Full Access or Power
User. I’m gonna use Full Access. And click Create Group. Now you can see the deployment group is created, And the github-ci user will be added to this
group. Let’s click Next: Tags. IAM tags are key-value pairs you can add to
your user, Which can help you better organize, track
or control access for this user. It is optional, so I’m not gonna add any
tags for now. Let’s click Next: Review. In this page, we can review all user details
before creating it. One thing we haven’t talked about is the
permission boundary. If you go back to the previous step, where we setup user’s group and permissions. You can see the Set permissions boundary section
at the bottom. Basically, it is a way for us to control the
maximum permissions this user can have. Note that this setting doesn’t grant permissions
by itself, but it just limit the maximum permissions
can be granted to the user. It is also optional, so I’m just gonna create user without a
permissions boundary for now. Alright, everything looks good, Let’s click Create user! And voilà, the github-ci user has been successfully
created. It is added to the deployment group, And the access key for programmatic access
is also available. We can now copy and add it to Github secrets. So let’s open our Simple bank Github repository. Open the Settings tab, And select Secrets section on the left-hand
side menu. Github Secrets are environment variables that
are encrypted, And can be used to provide some sensitive
input data to Github Actions. There are 2 types of secret. First, the environment secrets, which will only be available to one specific
environment. It is used when you want to have different
secret values for different environments, such as testing, staging, or production.. And the second type is repository secrets, whose values will be available for the whole
repository. And this will be the one we’re gonna use
today. So let’s click New repository secret. For the secret name, let’s copy it from
our workflow: The first secret is AWS_ACCESS_KEY_ID. And we can get its value from the AWS console. Let’s copy, and paste it in this box. Then click Add secret. OK, now let’s do the same for the second
secret: Its name should be AWS_SECRET_ACCESS_KEY. And in the AWS console, let’s click Show
to see its value. Copy it, and go back to github to paste in
the value. Then click Add secret. Alright, now the AWS access key and secret
are ready. Let’s go back to the Github workflow. In the second step, the key will be used to
login to Amazon ECR And its output will be used in the third step. Here you can see that, the ECR_REGISTRY variable will take the output
registry from the previous step. We also define another variable for the ECR
REPOSITORY. In our case, it should be simplebank, as we’ve
created before. So let’s paste in the name here. The last variable we define here is the IMAGE
TAG. Basically, we will use the github SHA of the
commit to tag the image. It’s reasonable, because each push to master
will have different commit hash, So naturally, we would want to tag the image with a new version associated with that commit. Now in the run section, we will run 2
commands: The first one is docker build, which will
build and tag the image locally. And the second one is docker push to push
the output image to ECR. All the variables we declared above are used
in these 2 commands. And I think that will be it! Let’s push this new workflow to github to
see how it goes. In the terminal, let’s run git status to
see the change, Run git add . to stage everything, Git status again to see what’s gonna be
committed. Then run git commit with a message saying Deploy workflow: build and push docker image
to ECR. Finally run git push origin master to push
it to github. Oops, the push is rejected. That’s because the master branch is protected, So we cannot push changes to it directly, and a pull request must be created if we want
to merge to master. You can add protection rules in the Settings
tab, Branches section. Here you can see my master branch protection
rule. Basically, I just require the status check
to pass before merging. Which means that the unit test workflow must
be successful. You can also require the number of pull-request
reviews approvals if you want. OK, now because of the protection rule, We have to create a new branch. Let’s call it feature CI build image. Then push this branch to github. Now in the simplebank repo, Let’s open the Pull requests tab, And click New pull request. The base branch is master, And the compare branch should be ft/ci-build-image We can see the changes here. It's looking good. So let’s click Create pull request. You can change name and write some description
if you like. Then click Create pull request one more time. OK, the PR is successfully created. And the unit test workflow is being run. While waiting for it to finish, Let’s take a look at the changes we’ve
made. So the deploy workflow will be triggered on
a push to master branch, The first job is build docker image, It will configure AWS credentials using the
repository secrets, Then login the Amazon ECR, And finally build, tag and push the image. OK, now let’s go back to Conversation tab. The test is still running. And you can see here, the merge pull request
button is disabled. Because it requires the test to pass before
merging. Alright, now the unit tests are completed, All checks have passed. So we can click Merge pull request. Confirm merge. And delete the feature branch. OK, now if we go the Code, On the master branch, We can see the 2 running CI workflows. Let’s open the deploy workflow. Oops, it failed. The steps to configure AWS credentials and
login to Amazon ECR are OK. But there’s an error in the build image
step. The Dockerfile is not found. Oh, that’s because I forgot 1 step to checkout
the code. So let’s open the test workflow and copy
this step. Then paste it to the deploy workflow as the
first step. We’re not running this workflow on a golang
image, so let’s rename it to just: check out code. OK, now I think this will fix the issue. Let’s commit the change. Deploy workflow: add check out code step. And push it to github on the ft/ci-build-image
branch. We can open this URL on the browser to create
a new pull request. OK, the PR is created. Let’s wait for the unit tests to finish. Oh, looks like the current branch is out-of-date
with the master branch. That’s because we’ve merged the previous
PR to master But haven’t updated it locally yet. So in the terminal, let’s checkout the master
branch Run git pull to fetch and merge new changes
to our local master. Then checkout the feature CI build image branch. Now we can run git merge master to merge the master branch into current branch. Everything is up to date. So let’s push it to Github again. Now we have to wait for the unit test. Let’s look at the files changed tab. We only add 1 step to check out code, So it’s looking good. Alright, now all checks have passed. Let’s merge the pull request to master. Here instead of a normal Merge, We can also use Squash and merge To combine all commits into 1 single commit
on master branch. It will help keep the commit history on master
clean and tidy. OK, now let’s check the list of commits
on master branch. On the last commit, we can see the CI workflows
are running. Let’s open the deploy workflow. It is building the image. All the previous steps were successful. We can expand this step to see the details. OK, everything is completed now. All steps are successful. The image is built and pushed to Amazon ECR. So let’s open AWS console to check it! Here, in the simplebank repository, We can see 1 new image with the latest commit
hash tag was just pushed And its size is 26.17 MB. Awesome! So that’s how we setup Github Actions to
build and push image to ECR. Before we finish, I’m gonna show you how
to check your billing. It is very important to control how much you
spend on your AWS services. Just click on your account name on the top
right corner, And select My billing Dashboard. Here you can see your spend summary. Scroll down a bit, We will see the top free tier services by
usage. Right now, we’re only using Amazon ECR With 500 MB free storage per month. And up to now, we’re just using about 1%
of it. But why there’s also Amazon Simple Storage
Service here? Well that’s because ECR uses S3 to store
the docker images. And we’re given 2000 free requests of Put,
Copy, Post or List. So if we go beyond these numbers, we will
be charged some money. If you open the ECR documentation page, And read about its pricing, You can see that, the 500 MB free storage per month constraint
is only applied to private repository. Beside this, ECR also offers 50 GB always-free
storage if you use it for your public repository. So pretty cool, isn’t it? And with this, I’m gonna wrap up this lecture about building and pushing docker images to Amazon ECR using Github Actions. I hope you find it useful. Thanks a lot for watching, happy learning, and see you in the next lecture!
