Hacking Kubernetes: CTF Edition | Rawkode Live

[Music] so hello and welcome to today's episode of rockled live i'm your host rock hood today we're taking a look at hacking kubernetes playing some fun capture the flags organized for us by our friends at control plane before we get started there's just a little bit of housekeeping so first please subscribe to the youtube channel and click the bell this will get you alerts and notifications whenever we have new episodes of rockwood live next we have a very active discord server feel free to jump in there if you want to chat cloud native kubernetes and anything in between and lastly thank you to my employer equinix metal they allow me to do this on their time and leverage their hardware when required if you want to check out a bare metal cloud use the code raw code this will get you 200 of qq which is around 400 hours on one of our more modest instances all right that is the housekeeping today i am joined by andrew martin the ceo of control plane hello i just i've looked up and i've seen your big cheesy smell and i was like i wouldn't say anymore ed you want to tell us a little bit about yourself and then we'll get started on today's session sure yes hello it's a delight to be here um i'm andy i'm alcohol free at this point in time i am i promise and uh yeah hi i'm the ceo at control plane we are cloud native security which basically means uh we do devops and we try to do it well because devsecops and that layer above is just solid engineering so very proud to have a very smart bunch of colleagues doing interesting things we have plenty of positions open if people are interested of course and yeah my background is in low-level engineering and distributed systems i came into security as a product of getting access to these systems and realizing that there was less than perfect coverage of course 100 is never possible so i was just able to follow my interest and keep on going down rabbit holes and untangling balls of twine so that's how we find ourselves here today awesome thank you for sharing so what are you drinking it looks like a beer oh yeah i mean i i'm almost carbon neutral so i'm now trying to get alcohol neutral uh if they would like to sponsor me this is the most crisp and beautiful of the uh unfiltered lagers it is 0.5 percent nice uh it's a couple of really nice alcohol-free beers that i like actually the erdinger one is really really nice and uh the brewdog ones are particularly tasty too yeah the other silent magical beer is the uh the jupiler and from many years of drinking in belgium i've accidentally bought cases of non-alcoholic beer before so now at the other end of my existence um i'm happy to have to say it's actually really quite good as lagers go nice awesome so today we're taking a look at some of the ctf scenarios that you and your team are control playing put together is it the kubecon once yes indeed it is the scenarios from last week's epic cubecon frankly the the live streams that we did one of which you were a part of were just fantastic really thoroughly enjoyed doing them and we didn't have enough time to finish everything so we will run through them all end to end awesome well i actually didn't have a chance to partake in the ctf last week um except for what we did on the the live stream and between where we had a little play uh it definitely gave me every taster i should do more of this stuff so i'm really excited that we actually get to sit down together and do this together today so uh you have sent me some rather sketchy and dubious bash stuff to stick in my shell which i did without blinking an eye so there we go we're going to see how we get on today uh i've got this link from the ctf i mean can people still do this in their own time or is this something that they'd have to send you loads of money for there are two versions of course we are open core for all of this so there is an open source project called kubernetes simulator and that fundamentally is a provisioning engine aka terraform of course and a load of nefarious scripts that run through and misconfigure um i guess security contexts and the definitions of what's running in a pods and various various other things that miscreants may or may not take advantage of so that open source piece is used for the ctf uh it is beholden upon me to patch the changes back into that so that will be done next week because i had to take a breather and to actually run this at scale we've built a load of um distributed orchestration around it so we've run that for conferences we've run that privately as well but centrally all of the learning experience is available through the simulator and all the scenarios that we ran for uh kubecon 2020 north sorry 2020 eu no it was north america uh intercontinental confusion are available on the open source version so everything will be there but no one can cheat for the ctf so we don't publish them in advance got it perfect all right so this is the dodgy stuff you sent me er untires some configurations and ssh me and to something so i'm just going to cash play let it do its thing and we are in a shell either in a container on a vm and somewhere right is that is that all i need to know at this point in time that is a correct observation um each the the penultimate well the last line of text there where your cursor is tells us what the starting position is um actually you're you're kind of right about the layers of abstraction but yeah we only care that it's a pod right now all right okay so uh i'm starting in a hash jack pod which is in some name space it wants me to follow the captain and prove out his attack path to find a flag all right yeah there is this uh arched nemesis uh sort of archetypal sailor of the binary maelstrom and all this other nonsensical rubbish who is this character that i've constructed to write a book around essentially so that there's a book coming out in november called hacking kubernetes and in order to give kind of threat modeling and attacking stuff more teeth and something a bit more realistic this guy captain hash jack exists you will see on the fourth line up how much effort i put into the uh kind of utf-8 spelling of the word hashtag and joyful joys it renders in our terminals truly we are in the future uh yeah so this guy captain hash jack has done numerous nefarious things with his motley crew of scoundrels and the point of the ctf is to chase down what he's done and understand how he's performed these attacks in order to understand for ourselves how best we secure the cluster how we chop off those branches on the tactory and how we model the threat model in an abstract sense and of course in the concrete sense of actually putting these controls in place okay so should i just start pushing buttons yeah all right okay so i do see a process table that makes me think we're in a container uh confirmed this looks like a host desk all right it appeared to me that we can just mount the host disk that is entirely correct the uh the telltale signs here are the container runtime would normally mask dev and so not give us full access to everything that is available on the host because of course dev is a virtual kernel file system is provided because everything in linux is a file and in a container normally we would not see lots of ttys did we know david's not a big fan of unicode yeah there was a clustered episode where uh a fellow scotsman actually guy templeton broke the core dns config by swapping out the all for a unicode that looked very that much like it yeah that's the uh the original typo squatting i guess painful anyway yeah big time yeah so we're normally really restricted but but uh but yes you are you're bang on this is exactly where we want to be so i guess i mean i've got access to the desk i can just sure at this i'm assuming i've no uh kubernetes okay i've got the rootster that's always a good sign i've got ability to speak to the cubelet or even probably modify oh no thinking those daddy manifests all right okay so am i supposed to be looking for a flag or am i just trying to take the roots there it's like i'm looking for something right yes there are flags littered across all the clusters uh in this case there's only one the uh there's a vital piece of information which i should in the spirits of fair play share with you and that's that the flags have a well-known format so flag underscore ctf open curly parentheses a hex block and then close parentheses so it is possible if you have the time and inclination to find these flags some of them it takes a little bit longer to find so sniffing around in some places is uh is useful uh i mean i'll just check things that i would assume would probably be obvious uh so would you like i mean i've never done like a real ctf right so if i was partaking in this is there any hints or advice that you would give to people like what should they actually be doing like once you get access to the root disk there are common places paths commands they should be running things they should be looking out for that's a really good question the playing a ctf and avoiding or evading intrusion detection um are quite different things so if we were actually attacking a host system that we were uh we were red teaming we had legitimate reason to be uh attacking the system of course uh which is the disclaimer on on all of the the ctf stuff in that case we would probably be trying to tread quite carefully so um we wouldn't necessarily be grapping for stuff although that is frankly the best way to solve ctf a lot of the time uh we would try and perhaps depending upon our level of defense evasion i was just watching um we were doing a sans call earlier and there was a really great presentation on emerging threat actors and they popped up a solar wind slide the solar wind slide the attack began getting on for a year before they actually dropped the sunburst malware are the sunspots malware so they were super stealthy they were just doing one thing really cautiously at a time because i mean you don't want to trigger those alarm systems in a ctf uh i haven't played a ctf with ids i think it's it's a great idea and watch out for next cubecon ctf i guess um but yeah in this case speaking personally i go and look in places that uh i would drop things myself i guess uh the root directory is normally somewhere that an administrator if i've just got a snowflake server and uh as you and i can probably recall from the days before wide automation and devops being uh being a thing almost we used to have to go on servers and we used to have to move keys around we used to generate our certificates from the server itself and the slash root directory is it's a place where root trusts they only have access yeah so so something in here is um likely to cause of course problems for a a flag defender maybe all right well i love that i could see that carrying on with aws and i could have the metadata api that would be fun but i'm looking in this root thingy and you said you wouldn't grab why would you not grip in a ctf i would all right but on uh on a non um excellent work sir i hate that i don't have my art complete i know uh ctf dot star there was some there was something there there we go very good that is it i would look there's more more utf-8 watch the joy all right so if you just if you grep for the flag again yeah then the authorized keys fail wasn't it yeah you got it fat is bashing this thing there we go so the the flag is this nasty person has added ssh keys to the authorized keys so they have access to the host going forward that is exactly it yes and this is one of the one of the ways to achieve persistence if you are just mounting the host file system because of course without using any center we've only really technically charuted or entered the the host's mount namespace if you run a ps again here you'll see you'll see you're still in the container bash yeah the roots told you the oh the magic i think um yeah so yeah precisely so you're able to navigate that file system and pull all the things from it but but we're still yeah exactly as you see we're still in the process name space of the container so the only namespace we've managed to break into so far is the mount namespace and that means to actually break out onto the host we need to do one of a few things the simplest is well i say the simplest we can do what we've done here which is to drop an author a public key into the hosts dot ssh authorized keys for root if we wanted to use that attack route on a modern system we'd also probably have to enable root login um for uh for sshd because that's not always enabled anymore but we can do loads of things interacting with systemd units is a bit more difficult of course sorry to interrupt i was i was only going to say we have access to the etc kubernetes we have access to the cube in a static pod manifest so we i mean we could run a privileged pod in there which would get us access to the all the name spaces on the host yes if we had enercenter in that pod then we could just blaze into all the namespaces but because we've just because we're in a pod with that uh slash dev access which is normally because it's privileged anyway but that's the route we've taken but yeah exactly as you say we could just run another container we could just drop in a shadow pod that has the tooling that we want to break out onto the host we could drop in a cron job which fires off a reverse shell we could mess with some of the system d units although they might need a demon reload and that would take longer it's game over for multiple different games simultaneously nice uh super cool excellent work okay let us progress all right do i need any yes that shell is thoroughly popped and no longer very used to us so here we go and this highly advanced uh targzy based access provisioning mode is uh is how we run the the kubecon ctf as well and uh all credit to the attendees uh many of them did question what did we think we were doing just throwing around bits of bash and tables which is yeah why not just put it on a usb key and give it to them right it's coming in by pigeon uh all right so we're now in a second shell i'm assuming like what you know the new people they're joining cts right the first five commands you should run i'm assuming are look at the processes look at the mount points look at uh maybe you want to see if you've got a sophisticated network or no access to the network like what would you run beyond that yeah that's awesome um i i have some useful collateral that i will share in a second all right but but the first thing is just to exactly like you say check the mount points because they bleed so much of the abstraction of a container from there it's worth checking um we can see in the process table if uh if we are actually in a process namespace yeah precisely so all good um the network as well is super useful we should be we should have a pod well rather we should have a local ethernet adapter that is on something that looks like a private range it should probably also look like quite a small range because we assume it's a pod network if it's not then maybe we've got access to the host adapter um there's also a super useful bit of kernel jiggery pokery that we can do so because we're um i i check id as well just to see uh what uh what we are okay so if we now cat proc self status then we can see some stuff in here that also looks a bit like what am i contained brings us am i contained is the canonical internal container observability tool written by jess fraz who just shipped all the container security stuff right at the beginning but we do have to pull a binary into our pod if we want to run am i contained so proc self status will tell us quite a lot of things uh we can see that we have no set comp enabled on the like seventh or eighth line up there uh we can see some of the capabilities that we've got although broadly i don't decode those and then if we go up there is we can see the uid of course we can see that through id as well and it's some interesting information to take this to the next level the am i contained tool which will run in a later attack will also tell us it will take a best guess at which which container orchestrator we're in so let's try and guess which orchestrator we're in well that wouldn't be doing justice to clustered and it's obvious spelling hints perhaps but if we cat proc self c groups singular sorry then we can see that the name of the c groups that have been created also give us a bit of a clue and it's all this stuff around leaking abstractions that make containers at once super fast to spin up and beneficial and portable and useful but then they're not a perfect extraction abstraction okay so that's the first the first thing to do is really just to gather as much information as you can before trying to move towards the flag right yeah exactly all right i i'm going to show my naivety here i didn't see anything here to reveal maybe which cri implementation we're using i just see systemd and freezer does that mean we're using container d that's a really good point actually uh if these are stood up i mean yes is the answer um but i'm not i'm not convinced that i'm not sure that cube pods changes for different run times i haven't i can't actually tell you that offhand certainly if we were to run this via docker it would tell us that we were in docker there right so i i think you're you're definitely on the right path yeah that's how i handle most things i just say i don't know and guess so so i should just be prodding around right um i did see the amounts were uh it looks like we've maybe got access to the host again that way uh yeah maybe let's see oh no we don't okay that's how it should look all right so what do we have then not a whole lot on this one uh is there i don't know what system this is well let's see if i can wear that maybe that'll help me i'm not an lsb fail normally i'd like to go uh asterisk into asterisk yeah that's it oh but it's a bit different for different i mean older os's don't quite have it so that star rail always tends to work nice yeah i normally look for the lsb underscore release but i guess star rail would cover that too so handy all right so yeah when you started off you use the mount command which is super useful because it gives you all that extra context and uh and if we go right to the top i say right to the top just slightly further up then it also leaks the overlay first yeah exactly yeah that's the docker run team as well yeah okay and so that gives us a bit more um it's just something else that's leaking however this is super noisy because it tells us things about where c groups are mounted so i always run both mount and also df and that tends to scream a bit louder when there is a service count yeah we've got the secret been mounted in here and it's kind of most of the time people don't unmount the default service account which is obviously not a great look it also means that a service account may just be unable to do very much by default let's say and someone also kindly left me keep controlling this container yes sometimes thinking of the user does that mean i can run oh look at that bazinga oh at least there's some hardback going on oh we've got some uh crypto miners going on as well that's fun that is the badger and the context for this is that we've got unexpected workloads now notably just some background with this one the uh this is more about attacking kubernetes workloads and side effects rather than misconfigurations of the orchestrator it doesn't really mean very much does it okay um so i need to work out what my next move is right yes i mean i can start poking around um i can start executing into other pods looking for maybe better privileges like um in the interest of time yeah what should i do those pods should not be running and stealing our compute resource let's shoot them in the head oh wait we the good guys are the bad guys i thought i was trying to get rip we're the good guys following the bad guys and this is done there are a good number of root targeted flags but this is not one of them okay so we want to delete some pods i was going to spend up some more but whatever you've you've avoided the mistake that is easy to make which is we are in fact in the hashtag pod yes because my my first instinct was to do a delete pod dash dash all and i thought no i'm in this name space so uh although now that i've just realized i've deleted pods which is probably rather silly because they are tuples which would suggest there may be something else getting deployed deployments i don't have access to that but if you check the pods yeah they are coming back so not stiff that could be demon sets i don't have access to that you're very much in the right direction right okay so my goal is to get rid of all these pods yeah and between you and me they are demon sets no they're not that's instinctive i'm so sorry no they're deployments all right but i don't have the access to list the deployments on my current privileges but you could probably elucidate their names oh so i just don't have list access yeah that's right [Music] oh look at that all right incidentally i'm looking to launch all of those coins really i should have been a bit smarter and just says uh off can i list and i would have seen i can delete very nicely left as a hint there and i was just being rather silly but there we go so that should mean they're all gone at least now getting there is that it is it done almost at the point that they make themselves scarce uh something will probably turn up in secrets uh there was a hashtag secret i looked at it earlier oh there's a flag all right okay i think uh potentially this is managed by the world's gonna make a big claim the world's shortest uh operator perhaps i found my flag excellent excellent work okay let's keep on rolling here is your next scenario so in this scenario we have we're running internal container registers and the pirates have turned up taken the registry down and left the secret in one of the pods so this is a this is a kind of screaming backflip through a flaming hoop kind of exploits where again we're just using what kubernetes gives us in an unusual way so is it hacking well of other sorts [Music] i kind of want to run that but echo hashtag okay um is that important should i be looking at that or is that just there for the is that an easter egg uh yes it is not important uh i was hoping you were going to wreck rule me but um adjust your weights perhaps all right so we can see docker cri again we have our overlay we've got our secrets we've got some disks we have a service account no ip command all right so [Music] do we have control we do people are always nice leaving all these tools lying around it was uh yeah feedback from last time perhaps so they have to curl down the bainery in every single pod all right let's see what do i have access to here um well we can create deployments we can get logs um but we can get pods too okay let's see what else we have namespaced [Music] privateer one two and three we do not have the ability to exec into another pod so i'm assuming uh we're creating do you want me to create something is that why i have the ability to create create deployments yeah that's it there's the interesting thing here is the contents of the image we can't get to that image but we can run a pod using that image as the bass so it's then about and this is the uh the screaming backflip it's then about trying to it's almost a blind injection attack it's not quite it is blind because we can't see the file system contents it's kind of an injection but we are still just running keep control so that's perhaps an overly dramatic wording but if we if we have a look at what's running in those private earpods so dumping them as the ammo and gripping for image they're running excalibur that is the badger so the goal here is to run a pod using that image can you maintain your own registry this is the uh the secret internal registry for the purposes of this all right the answer is absolutely not uh sorry i cut you off but okay so that's so good right let's push me along a little bit here i'm not sure what my next move is so what we want to try is keep control run and set the image to be that crazy uh parasitical image and then yes nice uh we want do we want a terminal i'm not sure maybe just try double ha double hyphen i'm not sure if we yeah okay um and then i don't know if it'll work on the run but i figured i'd try it it's it's worth trying yeah what's there yeah i can't end that because i don't have pod exec which i kind of thought was going to happen so the way that we can do this semi-blind maybe it's kind of monocled half injection is to check the log outputs and the the the tool that i really love for logging is called stern from a company called worker the oracle boards like three years ago but this tool stern um el draft lincoln allows us to essentially specify a wild card that lets us grep across any existing or future pods in any or all name spaces we can't do that right now we'll just have to use keep control logs but we could but let's not bother and okay so where are we um i have to admit missing the previous command sorry i was running on logs against our hashtag too but i don't seem to be getting uh anything back out and then i just decided to check the logs or private tier one two and three but there's just sleeps oh yeah it's going round circles okay let's have a look at this injection command could you pull up the keep control run again please uh yep yeah this okay so um if we replace bash if in fact okay bash let me think about this uh if you can remove the it as well because i don't think we need an interactive terminal for what we'll do and then double hyphen and then we'll run bash again bash space hyphen c and then just put an id in there and see what happens all right okay gotcha and we've got a fun game here because we need a unique pod name of course every time we do an injection so yes apologies this is where stern is useful all right rich rick and rip cool okay so null has very helpfully posted uh another useful grip so if we replace id with the grep r flag ctf search then we will find and i think again for the purposes of speed searching in user share so up to please the badger and then um just knoll's last uh last chat in the that's the badger again and then if we use user share as instead of the root slash that looks good mate joyful choice and we have a flag very nice indeed interesting point to note here the rick roll will be from a different line of the song every time and if you actually and uh why under god's bright son would anybody ever create a directory name which is a comma who only knows but yes congratulations we are rocking through the flags and we go to the next one [Music] there we go thank you and we are in number four very nice now it occurs to me because of my exceptional timing i may uh this cluster may self-destruct around us can you just check this is an interesting experiment what does uptime say [Music] yes so this is this will go in a blaze of glory very shortly there's another one coming up so we'll be able to transition between clusters semi-seamlessly but let's uh service again again i'm assuming we're going to be following the same pattern here just go straight in there oh that's rude yeah lulled into a false sense of insecurity i'm afraid [Music] uh yeah the the back chat for this one is the supply chain is compromised and code has been merged into an application library that developers used the library runs in a pod attackers have used that reverse shell that the pod through to jump into the pod and find secrets mounted on the host so somehow we need to get out of this pod and and into a mount point okay cool and we have four minutes i have to say i again um load you somewhere because if you go back to the ps at the top now earlier on we both jumped into the same pod and that meant that we had two bash sessions oh the bash process is running actually what we've got here with two sleep sessions uh processes is potentially an indicator of misconfiguration okay oh there we go oops see you later okay what should we do we will have the next one up very shortly if you just okay yes so just looking at that we've um the first thing to do please is to scroll back up to the uh description and and then just slightly further down now just before we've typed uptime it says use cue control describe blah blah blah to see all of the containers in this pod and the line above that defaulting container name to audits that comes from cube control exec that is not part of the scenario description and probably in the spirit of fairness and goodwill i should put a dividing line between the two so that's clear right but concretely what that means is that there's more than one container in the pod cube control used to not tell you this information when uh when you jumped in or give you less information but now with kind of one two one yeah yeah it does you can set the default container now right yeah so it and we go toward it but actually the uh the container that we're looking for is in the same pod and somehow accessible and that's suitably vague to not be of any further use without a cluster well i wonder if we've got any more so if i just send you one that we've been let's uh let's reverse back to the last cluster we only need another couple of minutes before this comes back up again but at least we'll have something to poke around that mind you saying that they were probably all spun up at the same time this is where i miss lewis he never makes these kind of rudimentary mistakes uh all right so i'm deleting the old cluster and you want me to re-download the previous cluster which you've just sent me so i'll do that there that will be suspected that one may also go yes which is nice because i'm actually i'm wearing my marvin martian t-shirt today so thanksgiving is encouraged it's only up 19 minutes there we go so that's that what you expected yes because i was better at the beginning of this video than i am at the end so this might be an interesting time to pull am i contained fortunately jess does an awesome job on the releases page uh genuine tools am i contained that's the badger and if you go straight to releases she gives you a checksum validation i mean this is how software should be delivered right everybody who was running codecov and not checking and when i say everybody everybody even up to hashicorp the big boys so we've got a permission denied this is totally standard in containers we often can't hit the file system locations that we want but what we can do is yeah you got it we can write to temp and that's we can we can grep the mount points for the string rw and that will just tell us if we've got a mount if we've got a writable mount as long as we don't have node dev oh wait let's try to download oh there we go that's good um and the before as well twice so this kind of cat and mouse game of immutability on partitions is really not solvable applications need to drop pid files lock files they probably need temporary disk you've probably got access to dev shm shared memory segment really even gke which is super hardened and well configured by default you can still find places to write runnable files it becomes a lot easier to ids detect the thing but boom okay awesome so if we remember what we had in proc self status we've got a lot more visibility here we can see has namespaces pid yes that's a good thing usernamespace is no hardly anybody has username spaces right now because they're very complex they cut across a lot of other namespaces because everything is a file and every file must have discretionary access control users and groups and blah blah blah ken volcker doing a lot of work to bring username spaces to flatcar which is the spiritual successor of container linux or coreos as it used to be that would be super awesome when they ship that what else we have here app armor profile docker default ubuzi very rarely is app armor on by default in kubernetes so this is a very happy day and we can see our bounding set of capabilities this will give us an indication whether we're privileged to start off with but but also tell us things like we've got dac override that means we can change permissions on files and the same way we can with chone we can make nodes we can do two routes we we've got a lot going on in there and then yep sorry no no i was just agreeing yeah stuff thing apartment sc linux love all that stuff it's so easy to operate these days in 2021 tell me about it and this is where the vendors make their money yeah there's there are loads of things that i i know are just good practice but then you know it's hard um big time yeah there's there are tools that will extract runtime security profiles from running applications but unless those applications exercise the full set of production runtime behaviors they're not going to build the right security profile so there's this mysterious grey area wherever my security operates i suppose and often there's some awesome tooling that's come out to support this actually um the aqua tracy does some of this stuff red red hats have a but they have multiple things they have uh man i'm just tall override so there's that there's a tool that will use ebpf to trace system calls to generate profiles and i guess falco could maybe do bits of that as well if you've got that running on your cluster it's absolutely the same space yeah i guess falco reverse engineer yeah and said comp i think you know with the sitcom populator and all the work that uh dan and sasha are doing there it's it's getting a lot easier to run setcomp on kubernetes as well which is which is nice i just wish people would do that for app armor and se linux otherwise i'd maybe stop disabling them yes dan walsh cries every time someone disables sd linux that there is a great tool to extract se linux profiles from deny events uh similar to audit to our back that jordan leggett built i just can't remember what it's called i'm sure someone will find it i think normal has come in is it bean yeah but bain's great but it's actually a pre-processor for um for app armor i think he says not having used it like that so perhaps oh i think uh misty's first comment uh so i think it's also from jess rozelle and it's a tool for dynamically generated techcon profiles for a binary which is nice or unless i've just stitched two random comments together that aren't related it's it's in the right direction bain focuses on app armor and it's less of a reverse engineer it's more of a dsl so you can just so you don't have to write app armor basically you can write kind of higher order language and have it compiled down um but yeah i mean this is where container security vendors the next clusters with you by the way this is where container security vendors have been operating for the past like five seven what year is it eight years um and another great i'm just dropping stats that i heard in court earlier security operations centers that use open source tooling have a markedly higher talent retention rate people like open source yeah true it's awesome all right uh so we're getting close to the error and i know your calendar is always a nightmare so i don't want us to go over that ever so why don't you guide us through this one if we can do it in the next five minutes and then we'll do a quick wrap-up and then maybe i can schedule more of your time in the future to do a bit more ctfe fun but absolutely uh so etsy linux audit to allow is the one good spot we'll lead okay so what are we doing here we are so you've told me that we have a second container and this pod and we need to find a way to get access to another container right yes that is entirely correct no i know that pods share the net name space but i don't seem to see any way to confirm that i guess set off yeah you're totally in the right kind of area right okay go for it um because we've got those two sleep pods we want to just have a poke around and while ps shows us that we are in a process namespace the question is who else is in our process namespace by default kubernetes containers in a pod are in the same network namespace the same into process namespace but not the same mount namespace and not the same and they have their own container file systems of course and not the same process namespace in this case we have a shared process name space between containers in a pod so if we go cat proc 12 because 12 is the pit of the other sleep process and we could probably have found that by just doing an ls and seeing that we have a few numbers here so is that how you would have done it yes it should always correlate it would have shown me it here right you got it see sometimes i take the hard road i don't think right okay sorry we're in uh proc 12 cool cool okay so what we can do from here is see the root file system of the other process so if we uh let's start with cat and go into root please just uh yeah root within this directory all right and then just start auto completing so we've now got the root file system of the other container because the pid that is running in the other process namespace is now leaking that to us as well this is by design it's uh what it's necessary it's because we've shared the process name space between the pods the containers in the pod so i mean i would say i'm relatively familiar with the proc namespace and i had no idea you could navigate the root file system for each process in this way okay normally this is a highly i mean we're roots so we we have access to privileged system stuff and from here where we want to go this one you may want to look in temp so in the temp directory of the other container and then into temp again and then into logs again you might just want to also complete for a bit i should have just said no read them all out to me one by one yeah i feel bad that i've used the same file name twice uh directory name rather there we go excellent fantastic yet there are two more and there is a mysterious bounty flag hidden later so let's let's cut it off there and find a time to get the others through and then maybe run through some of last year's as well all right well thank you very much like i'm learning scary stuff which is always a wonderful thing to be doing like that director in the proc thing that that's wild i don't know why that's a thing but i'm gonna have to i'm going to have to look look that up for sure it's the thing because when the colonel and i'm i'm going to butcher this and please colonel devs correct me but when the kernel is managing processes it wants all of that information so that it can do its own introspection they're just sim links to places everything in linux is a file so it makes that all available through the proc virtual file system when we're root inside the container and here's the you know just say there's still a blur in the face we have access to all those privileged operations and so root inside the container can do an awful lot because we're still just using linux and that's why we want to elevate to a non-root user and would that be something that disappears with username spaces at least username spaces perform oh man it's a mouthful subordinate group and user mapping so where we've got kind of we used to have one to 64k we've now got one to two billion um possible user ids there's so many because you can then so you've got your roots username space you can then create another username space on top and that maps zero inside that namespace to 50 000 on the host and then you can do that again and you can do it to a point of brain melting complexity that we won't think about so does it go away with username spaces some of the privileged host mounting stuff does go away because we need that permission on the in the the root user name space what we did just now that's uh by design in inverted commas cool all right well i'll definitely be losing sleep and turning off all of my production infrastructure tonight you did a great job all right well again thank you for taking some time out of your day and walking us through this today it was great fun and definitely i'm going to get more involved with cts i think and just use them as a learning experience i think this is a really fun and engaging way to learn really lower level components of the kernel in containers that i've probably just been neglecting for a while so and it's always a pleasure but thanks again and have a great evening and i will speak to you soon cheers thoroughly enjoyed [Music] [Music] you

Channel: Rawkode Academy

Views: 607

Rating: 5 out of 5

Keywords:

Id: e1JQ3iZt76Q

Length: 59min 34sec (3574 seconds)

Published: Thu May 13 2021