Hi there everybody and welcome back to another video on the channel Tech with Marco. I'm Marco and I'm the techie guy here and in this video I'm going to show you how to set up crowdsec with your traafik instance in order to secure your web applications. CrowdSec is an open source and collaborative security stack and it helps you to protect your web applications from attacks out there in the web. So in the last video I showed you guys how to set up a traefik instance and now we step up the game a bit further and climb the security step a bit more and that's why we need crowdsec. So if you have never heard of traefik before I definitely recommend you to watch my last tutorial video about it. I'll make sure to link that in the info card up there. And if you've seen it or if you know what traefik is, you should continue right now. So let's start and jump directly into the configuration stuff. I am using DigitalOcean to spin up a new server here. And I'm using the CLI tool of DigitalOcean to just create a new server. So now you can see I just created a new server in the cloud there. And if you guys want to know how that works, maybe you should just let me know in the comments. And if you guys want to use the DigitalOcean cloud, I'll make sure to leave a link in the description below. That might help me to spin up new servers in the cloud if you sign up for that. So I just started up a new server and I added a user data file here in my helpers folder which is called cloudinit and in this cloudinit file I am creating a new user and I linked my SSH key there and let docker install. This is totally automatically done and if you want to know more about that maybe I'm doing a video in the future about that. So now let's check what the public IP of the newly created instance is. And I prepared a statement for that to list the compute instance in DigitalOcean with the tags YouTube. And I think I have two of them here. Yeah, that's correct. But we can see now here I created the crowdsec-traefik instance and that one is here the first and I've got a new IPv4 here and now to make our services available there we have to link that IP to a domain and for that I'm using my DNS setup Terraform automation And yeah, I'll make sure to link that video in the description and also up here. So if you want to know how that works, just check that out. Let's just add these records here and change IP of domain. I'm pushing that into my GitHub repository. Now the pipeline is running and it's changing the configurations. I am using demo to log in. And ta-da! It works. So we got docker installed, no docker containers running. And now in order to integrate crowdsec into an existing traefik environment we first of all need to add traefik of course. So let's head over to our visual code and I have there a markdown file which is describing the tutorial in words, not in a video. yeah so i wrote there we have the traefik setup at first and i made the assumption that we have a working traefik instance out there based on the tutorial i made before yeah we can do that in our docker compose here so for that i am just copying everything which is going until the traefik configuration is finished I'm creating the Docker Compose file and adding the traefik configuration here. Then we need to make sure to create the environment file and crowdsec and the traefik folder which are holding some configurations. So creating directory traefik config and I'm already creating the crowdsec config directory. And as next, we need an environment file. And I'm already copying that here. And I'm creating the environment file with all the configuration needed. After this basic configuration, we now can edit the traefik configuration files. And we have a dynamic conf file. We also have a traefik file. And in that traefik file, we have to change a little thing before it starts working. As you can see here in the entry points configuration, we have the entry point web, which is the HTTP port 80. And we have the web secure entry point, which is the HTTP HTTPS entry point on port 443 and there is a middleware configured and This is some specific crowdsec feature and for traefik to work at the beginning we have to delete this little configuration at first because otherwise traefik is trying to route every request which is coming to HTTPS here and through our crowdsec middleware, and this does not exist yet. Make sure to add comment signs in front of the lines there. After that, we can copy this configuration to the traefik YAML file. I'm changing it to the traefik config directory, and I am creating the configuration file here, and I'm writing that file to the disk. And we can also copy the dynamic configuration file here. So now we made sure that the traefik instance is working and I'm spinning that one up. Oh, that was not the wrong directory, but I'm going back and spinning that one up here. So like the path mappings are working correctly. And now we can see that we have a traefik container running. And we can see we have a working traefik instance here. So now we can come back and add the crowdsec container to the setup. Now I have to copy the second container configuration here. And I am editing the Docker Compose file. And I make sure to add some new lines here and adding the crowdsec container. So now the question might come up what does actually the crowdsec container do in this setup and the answer to that is that I already mentioned in the beginning that crowdsec is a collaborative security stack or a piece of software and it lives because people are contributing to the knowledge base of crowdsec and are reporting that like people or malicious IPs are trying to attack some instances. And with the crowdsec traefik setup here, we are routing every request which is coming to a web service via traefik is firstly routed through crowdsec. And the crowdsec container is asking like the common database if this is a request which is known for being an attack out there in the web. And if yes, then crowdsec is saying, ah, no, no, you're not coming in here. I'm blocking the request right now. So like attackers are having a hard time to try and access like any web service in my setup. In order to do that, we have to integrate some stuff here now. And I just want to point out that we have the environment variable collections. And there we can add some keywords, for example, the crowd security Linux collection and some others. These are some well-known collection which are describing how an attack looks like. Yeah, so we're making our Crowdset container aware of these well-known attacks. And so he can decide if he wants to deny the incoming request or wants to answer the request or forward it to the actual service which is answering the request. So now I added the crowdsec container, but there's a bit more we need to add. So we just created the container and this makes us just aware of the well-known attacks, but we're not doing anything. So we are not like responding or taking these information into account in our setup. and therefore we need a so-called bouncer and there are lots of other bouncers out there for example one for Cloudflare but for our setup we're using the bouncer traefik here and as an extra for demonstration purposes I am also adding a simple web service which is called whoami which just basically gives you information about the request you are doing this is the one here whoami i'm copying that here also editing the docker compose file again and add that one at the end here writing and quitting the file again and there's now one thing we need to do with the crowdsec container so i am starting up the two newly added containers again and now docker is pulling the WhoAmIContainer and the crowdsecContainer. And in order to integrate the bouncer, which acts on the information we are collecting, we need an API key. And we are getting that API key from the just started crowdsecContainer. And we can do that with the command. docker-compose exec, like for execute the crowdsec container and we're using the crowdsec CLI and we're adding a bouncer here and that bouncer is called bouncer-traefik and we are getting an API key in response for that and we need to make sure to add this API key to the bouncer container so that crowdsec is able to act with the container. Make sure to note that down in your environment file or anywhere else where you would like to store it because it will not be shown again. So I am copying the output here and we have the environment file of docker and there is a variable crowdsec bouncer api key so we need to add that one here. and need to write that to the disk. And now we can have a look at the Bouncer traefik container configuration. So it has two environment variables. One is the crowdsec Bouncer API key, which I just created with the command before. and stored it into the environment file. And we need to make the bouncer aware of the crowdsec container. So this one, crowdsec-agent-host. In my case, this is crowdsec-8080. This refers to the host name of the crowdsec container here, and the 8080 is the standard port of crowdsec. So I am editing the compose file again and I am attaching the bouncer traefik at the end. Putting now up the traefik bouncer and now we can have a closer look into the configurations because now we need to make sure that every request is routed through the crowdsec instance and therefore we are jumping into the traefik.yaml file. And there you can see, which are just commented out here, the HTTP middlewares crowdsec bouncer at file. So I'm just adding that one in here back again. And we need to have a look into the other dynamic configuration file. As you can see here the crowdsec Bouncer add file. That means this middleware is configured in our dynamic configuration file. So that means like the add file and if we want to have a look in here we can see under the HTTP that we have some middlewares configured. I have the traefik authorization configured, which is a basic authentication, or not authorization, authentication. I'm collapsing that one here. We can scroll a bit down and we see the crowdsec Bouncer. And in the crowdsec Bouncer, we see a forward auth. and the address of the forward auth is the crowdsec-bouncer-traefik-88 and there is a specific API path here which is just according to their documentation for example if we now visualize an incoming request like hard word for me as a German there's a request incoming into our server with the name whoami.domain.com and in this case this one is routed to our server and traefik is taking that request and now it knows because of its configuration that one is coming through the web secure entry I need to send this request into the middleware first and I am saying here okay please route that request to crowdsec-bouncer at file first and now what's happening here that crowdsec-bouncer is configured here with the crowdsec-bouncer-traefik that is just the internal known address of the container And now the bouncer can check that request, okay, is that, has that request a specific payload? Is that coming from a specific IP? Has it specific headers I need to check? And therefore this bouncer is checking our crowdsec container. And the crowdsec container holds the information about the collections, which I just configured in the beginning. And then the crowdsec container can answer that and say, oh, that's a malicious request. don't answer that and block it. And there's also another nice thing about that because traefik is writing some access logs. And in these access logs, like there are also these information written down and crowdsec is able to parse these access logs. And for that to work, I am writing the file path here for the access logs in the traefik configuration to var log traefik access log. And I am mounting that file here. You can see the var log traefik I just configured here. This is the internal docker path. And I am mounting that into the directory var log crowdsec traefik. And in my crowdsec container here in line 41, I am mounting the var log crowdsec directory in the var log crowdsec directory for crowdsec. And crowdsec now has also the configuration in the crowdsec config folder that it should pass everything which is under the traefik folder in the crowdsec folder with the pattern anything.log and it labels it for the type traefik. So that means crowdsec is also aware of the complete access logs which are happening at traefik instance. And another cool thing is you can also add system logs for that. So for example, requests which are accessing SSH ports, like for example, port 22, because port 22 is not going through traefik, but it's going directly at system level. And crowdsec is also now able to parse that and is able to decide if it should do anything about it and this is like now completely only on the CLI visible and yeah i just started the services here and now we can also visualize that a bit better on our CLI and now we can also have a look at the metrics which the crowdsock container is writing and there we can see it's already parsing the var log of log file it has read 11 lines, four are parsed, seven lines are unparsed. Yeah, you can see crowdsec is working in the background and that shows us that crowdsec is doing its job. But now if we want to visualize that a bit more, like to have a better overview how that looks, we can even connect our crowdsec instance to the crowdsec console in the web. And for that, we have to go to the crowdsec web application and i would definitely recommend you to sign up an account for that and i just did that here and to show you what it could look like we have now another instance configured of me here like a few days ago and there we can see i have installed 35 scenarios which i want to check requests for and There are no alerts here currently most of the time, but we can get an overview like in the visualizer and there we can see that like a Source IP from Lithuania here like the one for one here. It is trying to do something I'm not quite sure here where it is exactly but we can Think we can filter for that keep only yeah So I just want to see what this IP has tried to do on my instance and then we can see we have a scenario here SSH minus BF minus user enum and I can click for details So there's no description here, but oh I can see there's another scenario here SSH slow BF that stands for brute force and if we now open some more details we can see that this ip is trying to ssh into our instance here with some kind of brute force or enum or enumeration method because of the configuration here i made with crowdseg i'm able to see these kind of attacks and be aware of that something is happening and like lots of people or lots of bots are trying to get access to your or to any instance there and yeah it's quite interesting to see how your instance is being attacked because like otherwise if you wouldn't have configured anything to see that there yeah you wouldn't be aware of and like I think most of the time you wouldn't notice if something is happening in your instance and now to guide you through how to connect your instance with the crowdsec console I am going back to instances here and to follow my guide we can see that we need to enroll our crowdsec instance into the crowdsec console and in order to do that we can go back to the browser and click the add instance button and there is a specific key which the console is now showing us and we need to add that and in order to do that we need to execute a command in the crowds container and therefore we have to enter docker exec crowdsec so the container name and we are using the cscli command which is the cli of crowdsec and we write console enroll and the key which the browser just showed us and if we execute that now we can see that it was successfully here and we need to visit the dashboard again to accept the enrollment of the instance after a little cut because my camera just switched off i hope everything is back as before i think like the image is a bit brighter now but yeah let's just continue where we stopped and now i'm here at my crowdsec console and i have to accept the enrollment of my instance on my server and i am accepting the enrollment here of the source IP of my server and now we can visualize what's happening in our instance. So i can see here now the roman has just been accepted and our instance is now available in the console. It needs a bit of time to visualize the attacks here and give me some alerts. I just opened the ssh slow brute force attack scenario here and let's see what it shows me and with that command here we can install this scenario explicitly but we can also browse some others here and because i installed some collections a collection contains some specific scenarios here so there might be some others here which are like probably well known there for example vault warden and Uptime, Gooma, traefik, Unify. It's quite interesting what you can find there in the community. And I like the tool very much because it's done by the community and also the attacks are reported by the community. And I think that makes crowdsec very useful or valuable. Yeah, and crowdsec is a nice tool to get use of it and prevent your servers or web applications out there from any attacks. All right, people, that was the tutorial on how to integrate crowdsec into your existing traefik environment. And I hope you guys enjoyed it and make good use of it. I'll make sure to link the written tutorial in my description to my GitHub there. And yeah, I'll be happy if you subscribe to my channel and give that video a thumbs up. I'll see you guys in the next one. Bye bye.
