Google CTF - BEGINNER Reverse Engineering w/ ANGR

hello everyone welcome back to the youtube video my name is john hammond and we're taking another look at another google capture the flag challenge from the recent google ctf that went on this past weekend again i bring this to you with transparency and honesty this is not a challenge that i solved during the competition and absolutely probably would not have been able to solve without looking at the write-ups after the fact so i used that to foot stomp and emphasize and highlight write-ups are incredible you should not have any shame in going to read those write-ups or going to try and learn those other solutions or seeing the tools and the tricks and techniques that other people use to really solve that challenge so let's dive in i want to try and showcase this beginner challenge and it is noted as an easy challenge and it's only worth 50 points it's kind of one of the lowest tier things that google ctf had to offer though google ctf is a hard ctf anyway this challenge says dust off the cobwebs let's reverse and there is a downloadable file so i will fire up my terminal and i created a folder that i have been working in when i worked through this live and in the moment i created a beginner directory for that challenge and i'm just going to clear everything out so you can get a fresh perspective because i probably had some dangling leftover files there so i had wget and downloaded that file which is a big long string and i'm just going to go ahead and move that to beginner or i suppose is really what the name of this challenge is oh and that's just a zip archive so actually i'll remove that to beginner to beginner.zip and i'll go ahead and unzip that so it extracted an a dot out which is typically the file name extension you'd get after compiling a binary and this is in fact a binary it is an elf 64-bit so linux program with dynamically linked interpreter not stripped okay so he might still have some debugging information let me go ahead and mark that as executable and run it so it asked me for a flag uh which i do not know off the top of my head so i'll type in please sub and weirdly enough that's not the flag so the obvious easy things that like we're kind of conditioned to do that you might see in a classic maybe okay cheesy beginner ctf which google ctf is not uh you would go ahead and run strings on that file and maybe you'll get lucky and you'll see the flag in plain text and there you go you win you get those points in this case i just see the beginning of the flag i see that ctf curly braces but nothing else we could do some other things i could try and run like ltrace or s trace to go ahead and see okay what library function calls what stuff is actually happening when i run this binary i can see it's prompting me again for some input as i run it and step through it and it does a string compare with these weird things so in the moment when i was trying this myself i thought like ah can i just like print this out i'll use printf so it'll try and actually use these raw values and bytes here i'll use that and i'll just pipe it into a dot out will that give me the flag no that fails okay so i thought like well will that happen if i l trace that with that input and that actually seems to start to do some other comparison which i thought is was weird and funky and just not doing what i would have expected so at that point with strings and l trace and grep or whatever all those kind of easy low hanging fruit out of the way then you'd want to go into like real actual reversing or debugging or trying to figure out what this program really does you can do that in gdb if you want to do some dynamic analysis if you want to do static analysis try and look at like the disassembly or the decompiled code that you might be able to pull out you could do that with hopper or you could do that with ida or you could do it with gidra and personally i like ghidra because it is free and that is a good thing so i have it installed and downloaded over in my op directory if you don't have gidra you can go ahead and download it simply googling guidra you can find the result here software engineering software reverse engineering suited tools developed at the nsa in support of the cyber security mission so they give you actually an installation guide it just is built off of java so you have to make sure you have a java runtime and a jvm setup but you should be able to just fire up gidra once you've got it all set up and prepared uh apparently that window is extremely tiny i hope you'll be able to see some of that and i'd already gone ahead and configured a project because i've been looking at this previously so it does probably already have a google ctf project already created if you don't have that you can just hit new project and then save a space and then you can press i on the keyboard to like import a new binary new program that you're going to work with so i've got a dot out that we've opened up and the code browser is already open so i don't need this other window or the new one that created now we're in guidra magic we'll go over to the symbol tree and because we know from the file command hey they don't have debugging symbols stripped out of this then we could actually maybe look for regular function names like the main function or like the start of where this program will begin so i'll go ahead and search for that filter in for it in that symbol tree window there and i can click on that main function i realize this text might be kind of small i don't know if i can amp up this size but hopefully you can kind of full screen and i don't think we'll spend a ton of time in here if anything let me take this code uh and i'll just copy it and i guess throw it in sublime text how about that subtle paste there we go let's set that syntax to see so we have a main function right and it is going to set up a couple variables here looks like it's going to display that message that we saw just simply prompting us for the flag and then read in from standard input with scanf looks like it stores that in local 38 so let's actually change all local 38 to be like our input there we go and what it's going to do is then p shuff b okay and it's going to shuffle maybe our input and then concatenate other things with bit shifts and oddities and weirdness and randomness oh and even has an xor thing kind of put in place so it does a lot of mangling and control over our input okay that makes things harder but that's obviously what we would want to reverse if we were doing some reverse engineering after that's all done we go ahead and do our comparison so we saw that string compare that was ran when we use ltrace or s trace and it's going to test our input with seemingly local 28 huh okay that must be agbar 3 is a portion of this i'm unsure what that's oh it's it's reading out 16 bytes in there or hex hex 10. i say 16 because that's kind of just as you would read it right if i were so fire up python 0x10 is going to be 16 in base 16 and hexadecimal write that 0x prefix and it's going to check if it matches the expected prefix at the first four so that must be the ctf open curly brace that we saw and it will go ahead and display okay return success probably because it's in the main function but it's going to go to these labels or these addresses if it is the correct flag so engidra can i actually click on that like expected prefix see what that is does it tell me i see a c here i think and i i'm not smart i'm trying to learn if i can like set the type i think it is or address this itself if i right click that can i specify what it actually is to change it yeah data and then terminated c string did that get it no it didn't how should that be done can i go to that okay clicking on that will bring me to ctf and i can see those bytes there good enough good enough obviously you can see me fumbling like uh this is not my strong shoot reverse engineering is not my forte binary exploitation i i still got a lot to learn right and i don't i don't hide that from anyone i'm still trying to get better too so when google ctf was over i wanted to take that opportunity take that time to go learn from some of the pros some of the guys that are really doing this and actually do a good job and when i can't even solve the most basic beginner easy peasy 50 point challenge that's a cool slap in the face so i'm over on ctftime ctftime.org and i see we do have new write-ups out and google ctf has plenty of write-ups that are available i see one that is referenced to this beginner challenge that i'm looking at so i'll click on that challenge specifically and i want to look at some of these other write-ups i've looked through a lot of these i've looked through all the other write-ups and a common trend that i found was people would solve this challenge with anger or angr and maybe you haven't heard of that maybe you have either way i do want to try and showcase it but keep in mind i am operating at like the edge of my understanding this is new territory for me i haven't played with anger before so i hope you don't mind me shooting from the hip here and hopefully we can still do some cool stuff and maybe learn something new so if you wanted to you could click through all of these write-ups and again i'm just trying to be honest showcasing this and exploring it but i found that this one is actually really really good in showcasing uh using anger and what we can do here they actually kind of follow the similar methodology that i do between using grep strings s trace p trace etc uh they don't get anywhere with that so they fire it up in guidro the same way that i did in that little demo and they can say hey this is scrambled we don't exactly know what it's doing uh we could reverse engineer it but that might be long and time consuming and boring and stupid and if we know a utility that can kind of automatically do this for us well let's give that a go or at least let's try it so anger anger is a python framework for analyzing binaries it combines static and dynamic or symbol analysis concolic i guess is the name of that or how you read that making it applicable to a variety of tasks so if you haven't seen it let me google anger python and you can see anger is a powerful and user-friendly binary analysis utility they have a github repository it's put together by the computer security lab at uc santa barbara or ceph common arizona state university and the associated ctf team shellfish and it's a big name right i'm sure you guys have kind of heard of those so it's a suite of python 3 libraries and i and i always look at this code and i've seen anger like write-ups that use this and they always have kind of funky like either variable names or things that i'm just not as familiar with so what i'm going to do is i'm going to get this set up and try and work with it work through it and showcase how we can build this using that write-up as a reference i hope you don't mind again i don't want this to be a stupid and boring video because hey you're just going over someone else's work uh i'm trying to learn and i hope we can all kind of learn from that so we're getting a little meta anyway uh it does say regularly the the short version of how to install anger is creating a little virtual environment and then just installing it with pip so originally i thought like oh okay sweet i don't i don't need to bother with a virtual environment then i'll just pip install anger and i think it like got it installed and then it was like not able to find some specific thing and cffi and then like pi c tables and weirdness so i kind of caved and was like you know what i should probably do what the instructions actually tell me to do so let me show you that mk virtual environment pk mk virtual environment i actually don't know is a command or if it should be a command or if it's something that i should set up whatever anyway they're going ahead and creating in a virtual environment we can do that we know how to do that i actually went ahead and put this in my just ctf directory so i have a tool and something that i can reach as needed so i'll use python i'll specify tac m to use a module and i'll use ven for vnv to access that virtual environment module uh when i type in python note that i am working off of python 3.8 because python 2 is dead and off the table and should no longer be in consideration so let's do mvemv with anger anger being the name of this folder or the virtual environment that we're creating and we're just using tacmvnv to go ahead and create that so after that's done i should have a directory here anger that i've created and that is the virtual environment so i need to go ahead and activate that so i will source anger bin activate you can see my pump prompt changes now i'm in anger and i'll go ahead and install it there we go that's going to take a little bit of time pull all that stuff down get it all configured for me but there we go we're in business with anger after a lot of blood and red on the screen it will actually be like yeah we did it so that's great i'll pause this and i'll get back to you as soon as it's done okay now we're back at it okay so let me hop into anger or actually let me go back to our google ctf directory where we had that beginner challenge because we have the file here and what i'm going to do is verify that i do have anger installed like can i import anger without a problem yes i get a regular python prompt back so i know that that's success i know anger is built off of claripie or clara p i don't know how you pronounce that so you guys are going to teach me here and that also runs just fine so let me take this beginner right up and kind of let that be a reference for me as i kind of walk you through this because what we're going to end up doing is we're going to end up creating a script let's go ahead and just subtle like uh winner dot pi or ape dot pi and we'll start as we always do with our shebang line but let's import anger and import clariby and as we know that should work totally just fine no issues with that great now we want to figure out and get kind of kind of see the the methodology and framework that we need to have in mind to solve this problem we're given a binary that does kind of peculiar and specific things all it does is just ask hey what's the flag tell me and i'll tell you whether you're right or wrong but because we can kind of see here given the source code and everything that we analyze it looks like it's asking for 16 bytes so probably it's it's sincere right it's compiled so maybe we're expecting a null byte at the very very end of the string so maybe that actual flag is just 15 bytes we know that and we know that we'll get a binary true or false yes or no success or failure depending on whether or not we got the right answer so we kind of want to know where in the binary is that because we'll let anger do some magic and brute force and hammer and do its symbolic execution to flow through and follow and find the correct input that this program needs to actually get to that segment of code to get into that code path where it is a successful input so we need to go determine the address of where those things are and where that happens i'm just going to go ahead and simply click on this put success line and you can see kind of over here in the disassembly we do have hex values as to where this procedure begins right i'm clicking on puts and i'm in this the middle of that code block that actually is let me go back there clicking on puts it highlights this one here so the very start of this looks like 1 0 1 1 1 d uh let me say that that is success address hex that thing and come back to guidro please if i were to check out what failure is that happens over here at 0 one zero one one zero zero great reading binary when it's actually hex so uh let me correct my typo there for success address and failure address that will also be hex i'll slap that in and we need to know where this binder is actually getting loaded or how it's starting or where all of this really takes place so it needs to know the base address and guidra can tell us that actually if we just literally scroll all the way up to the top the very very start here of all these instructions of everything that this program is going to do that's going to be the base address like right there so it'll kick start at 0 0 zero zero zero zero let me take note of that let's use base address and note that as that zero x value okay so now we've got kind of those ingredients and we know that we can work with those we also want to be cognizant of the length of the flag that we're actually going to end up getting so we mentioned okay we know the flag length they ask for input of 16 bytes and we're assuming that that very very last one is a null byte or maybe just hitting enter or whatever the case may be so let's just create a flag length variable and let's say that's going to end up being 15 bytes in length so then we'll do and i'll bring this in so you can see it because again i i'm being truthful here taking a look at some other code to see how they solve this and what they did here they're going to specify a project or an anger project with the binary and they'll pass in that base address that we're looking for so we could copy and paste that sure but it wouldn't be any good learning for us so let's try and do that ourselves let's create a project and that's going to be an anger capital project and we have imported anger over here so that's totally fine to use and we need the file or the binary that we're actually working with that's going to be that dot slash a dot out in our case and we'll need to specify some options or those main ops that you saw as a keyword argument we need to specify that base address or base addr that's how they reference it in kind of this dictionary um i had gone ahead and called that variable base address all spelt out there maybe for our consistency we can remove that and make it addr but regardless now we've gone ahead and created that project good showcasing this one more time now they do something interesting that i haven't seen before and i needed to do some research on so this is me learning and hopefully that'll help you they create this flag characters list and array you can see that by the the square braces here and they create this object claripie.bvs so i wanted to learn a little bit more about the relationship between anger and clarify and let me zoom in on that as i do my research anger's solver engine is called clarify that's how it's going to actually be doing its interesting magic and trying to determine theory and symbolic execution and determining and guessing and filling in information as it finds it uh and that's how that's all put together clarify is just the vessel and the name of that that concept and that idea but you do interesting things with that because you're going to fill in or at least create the placeholder for the information that you don't know so we don't know what the flag is and we want to know what the flag is that's our objective right so they create and you saw this in the syntax here they're using this claripie.bvs syntax and i'm like what the heck is bvs i don't know i've never heard of that or seen that before doing some of that research kind of diving into that documentation here a bv that is a bit vector whether symbolic like just with a name kind of like a variable or concrete or with a value and you specify the size there so looking at this example code they're specifying a 32-bit symbolic vector x you can see the syntax bvs so s for symbolic x will be the name there and it's going to be 32 bits so referring back to our code when they create a bvs or bit vector that's symbolic they're going to create a flag variable name for this and the percent d is actually going to be a little format specifier based off of i and we didn't see i anywhere else in this code yet it's actually carried on because this is going to do some list comprehension and it's going to be 8 bits so eight bits is one byte right so these are literally going to be all of the flag characters and it's it's using list comprehension to create all of this in a list and for i in range flag length so it's just it's creating the flag each character by character as a list that clarify and anger will be able to know how to use so let's steal that not not steel but you know flag characters can be a list comprehension based off of that clarify bvs or a symbolic bit vector and we'll call it flag you can call it character with the specific id whatever you want actually let me use an f string we'll do some interesting python three things and make sure that maybe that'll behave flag care i so i will be our iterator and we do want it to be eight bits in length so that's a full byte good enough for i in range of that flag length that we've defined perfect we have that list generated and created there now and they bring this together into a flag variable or object they concatenate i don't know why they used this asterisk here that was kind of an oddball to me but if you do that it's actually going to end up expanding out that list so it puts all those together and adds a another list object with the sim single element an entry and that's going to be a bv v so we recognize that we saw the bv earlier as a bit vector but now with a v so not symbolic but with an actual value that we specified so we're just going to specify a byte for a new line here we enter that in we just whack that so scanf or how the program is reading our input it'll actually be able to take that input it's like us hitting the enter key and it's just adding it in right so let's pull that into our code just as well we'll create a flag object and that's going to be that clarify class or module that we're calling it and we're going to concatenate our flag characters that we've created and we're going to add in that other single byte just inside this list here that's going to be a bit vector value with the b prefix to note that it's a byte and then a new line so you could have the comment here so centered in works or whatever so it actually accepts it so we hit that enter key okay then we go ahead and do some anger boilerplate stuff that i hadn't exactly seen before i just i'm just not familiar with that syntax yet right because i don't use this project all that all the time i don't use anger often but i want to get a little bit better at it i want to get smarter at it so we use that project object and we take the factory which i guess is just how it's going to access its run time or the module stuff to get it to do things and we'll run full init state with the arguments there and we're going to use anger options unicorn i don't know what that is list of state options so i'll google i'll just research these may be passed with mode equals to a state constructor are we starting a state yes we are we initialize a state unicorn options that enable the unicorn engine for executing on concrete data um i'll have to do a little bit more learning on that i think there's obviously a lot here like there's a whole kind of little online book to get started and learn a little bit more of the concepts and how it all works within anger simulation managers and program states though when we are letting anger flow through and work through and simulate all the potential inputs and work that maybe this will work with anyway let's go ahead and grab our code here to initialize that state we'll go ahead and create a state and we'll make that to be our project factory and it's full initialize or init if i can type state with these arguments here we'll specify args to be running our program dot slash a dot out and maybe we could have captured that if we needed to and the options that we supply and that was that weird thing i was trying to google and learn a little bit more about earlier the unicorn options so it can work and work with this data standard in what we are going to end up passing along in this program or we are going to give it to run with well that's going to be that flag object that we just created and because those are symbolic bit vectors anger is going to try and figure out and learn what the proper input there might actually be for us but we'll have to pass that in as the flag great okay now if i scroll down a little bit more here looks like we can go ahead and create a simulation manager or sim girl and we saw that as i was simply looking around and googling simulation managers let you wrangle multiple states in a very slick way states are organized into stashes that you can step forward filter emerge and move around as you wish that's very cool it allows you to control symbolic execution over groups of states simultaneously applying search strategies to explore a program state space so it'll try and work through and find all of these different code paths and it can step through each of them or do different things to move around in them i'm just gonna let anger go like we'll give it the the start position in the end or like the the directions and the the road signs that it needs because it's going to end up finding or avoiding a specific address but let's start to create that so sim gear equals proj.factory simulation manager and we pass in our state so i'll call it sim manager equals anger dot oh i'm sorry sorry i i need to i need to be able to see this i can't just balance that while trying to speak at the same time apparently it is a factory member called simulation manager and we pass in our state great now because we've already defined this find address and a void address or essentially like the success that we want to find will it'll give us the flag and the failure that we don't want to find this is like okay don't look over here but look everywhere else starting at this position so funnel your way through the maze until you find the goal in the end but don't run into any traps or spikes or bad things don't go to this address that will lead us to failure and not finding the flag so let's work with that looks like all we need to do is run that simulation manager tell it to explore with find equal the success address and avoid equals the failure address i'll bring that over just here simulation manager and it's called sim manager in my case let's go ahead and explore and we've defined these variables already ahead find this success address and then avoid this failure address great he's just gonna run now he's just gonna go so if we haven't found anything or if we have found anything excuse me if the length of what we found in this simulation manager is greater than zero well okay go ahead and display that out to us for every single thing that you've found we want to see that object and posix it sounds like i wonder if i can actually kind of drill down and find out what that is anger posix dumps emulated file system okay okay okay so we can actually zoom in on the process that we're working with right the file system socket pipes or terminals and a sim file which is interesting yeah just tell me how to do what i want to do they go over this interesting abstract example with the simulated file where is the notion of posix let me search for posix you may access the mapping from a file descriptor number to a file descriptor object in state.posix.filedescriptor or fd so we know that standard output is file descriptor zero so that's that's what we could be looking for and their code actually defines oh standard in centered in centered in a zero i'm sorry i misspoke standard out is the one and centered in is what it's actually going to be obviously entering into the program and that's what we're mangling that's what we're trying to uncover that is the flag itself so let me go ahead and and get that here if the length of the sim manager has found more than one thing so if if the sim imagers found anything what we want to do is we want to loop through each of those entries for found in sim manager found let's go ahead and in excuse me thank you let's go ahead and display what they found found is a state as we know can reading that documentation and posix will allow us to see specific file descriptors looks like dump will display an s i'm assuming could give us a string and 0 for our standard in file descriptor and because our standard input is what we're trying to mangle with a symbolic value here and it's going to solve that with clarify it'll figure that out maybe that will work for us so i think we've got some stuff here let's take a look let me run python winner slap that in see if anger will just run off the races proj is not defined oh excuse me i called it project proj this guy i'll do that just as well there you guys were probably already screaming your computer telling me that i was wrong thank you i appreciate it let's see if he gets anything now or if we have any other errors looks like he's running looks like he's going and he found something but not what we want ctf and a null byte apparently and other random bytes a couple printable letters okay so that doesn't work um well we can actually kind of get a sanity check here right because obviously we're typing in the flag on the keyboard ourselves and that's going to end up being real regular letters and numbers and things we can type like printable characters right so what we could do is we could kind of limit um what clarify or what anger is going to try and find and determine and we could we could set some bounds or some ranges to say look everything that you find i want you to make sure that's actually going to be within a certain ascii range like it's got to be greater than or less than uh certain amounts so it's in the printable character space and not these random wonky nullbytes or other non-printable characters and bytes so they actually showcased this in the section here in the code and i totally drove right by it actually purposefully uh so don't give me too much flack i did intend for us to skip over that because i wanted to kind of squeeze in that nugget and hopefully maybe that was a decent learning point it's like okay let's realize and know what we did wrong and maybe what we could come up with following that because these raw bites these null bites these random things that should not be in our flag so what they do is they take the flag characters like list here that we've created for all these big vector symbolics here these eight bits that we're trying to figure out this byte for every single character in the flag we need to add some limitations or constraints so we can say every single one of those characters in the solver that we're using inside of the state as we run this program we need to be able to say okay this is going to end up being greater than maybe the ordinal character of this and this exclamation point would probably tell me that's the very very start of the ascii table where the printable characters begin and then this tilde tells me that's got to be the very very end of the the ascii table so let me verify that and just so we all believe me there we go no bite snow bites no bites things does it show me non-printable characters here oh probably for convenience below are more compact tables and and hex so this is going to end up being the printable character range and set and looks like there are obviously ranges outside of that yeah yeah yeah all this stuff all this weirdness so the exclamation point is where we start to have regular normal things that we humans can read and understand just fine and then up to tilde obviously anything after 7f will be not a byte that we will be able to make sense out of as as people with eyes so let's use that let's use that code let's use that syntax it's cool that they use ord here to be able to get that like actual decimal value the data there so let me steal that stealing is not the right word but uh they do for an iterator in their flag characters all you see because that matches character a little bit more the state that we've defined that solver that's going to use we need to add a constraint that's going to be actually within the ascii character range that's printable so i'll use that exclamation point to note the ordinal value um or at least to note the beginning of the ascii table where the printable stuff starts same thing with the tilde to note the very very end and now we've specified that as some constraints for our state here good enough okay now let's try and run that see if he gets anything better or new or something interesting and peculiar there we go uh simdee for me ctf cindy for me well that's clearly very much a flag that looks like it's it's it is what it should be that looks like we got it we we did it with anger that was awesome very first time and we could go ahead and submit that if you wanted to we've got uh the google ctf up here make my face move out of the way so you can see the glory of this insert flag submit flag and then hopefully it'll give me that cool animation like you did it flag accepted all right flag captured nice great good easy except not obviously this is is new stuff for me i hope you were able to bear with me and i hope it was kind of fun and maybe some enjoyable to stuff to to walk through this but that was my first time using anger this is a little bit new to me so i am kind of going to reference the things that are out there in the world and i don't think there's any shame in that especially if you're learning and getting started on this but i think i wanted to boil it down to there were some key ingredients that we needed to know we needed to be able to kind of tell anger where to start or that base address and guidra can give us that the addresses of where we want to go and where we don't want to go where we can find the flag and where we know hey we failed at finding the flag and then obviously the length of what we're looking for so we have those we have we have those placeholders for the data that we need to determine and figure out and anger feels like dark magic it can just like crank through this and figure it out and solve it really really cool really really neat um i i'd love to be able to do more with this so i'm excited to be able to tinker with it but that is that um please go check out these write-ups please go check out other things and please do try and learn if this is new to you too it is for me but i hope this was hopefully a fun little safari ride to go on to uh showcase anger and solve one of the challenges another another easy baby beginner one from google ctf but i hope you guys enjoyed that so thank you so much for watching if you did like this video please do press that like button maybe leave me a comment i'd be super grateful if you were subscribed and i hope you guys are enjoying this series of videos lately i'm trying to keep it busy for us uh but feed that youtube algorithm you know you know alright thanks for watching everybody i love you i'll see you in the next video take care [Music] [Music] you

Channel: John Hammond

Views: 185,706

Keywords:

Id: RCgEIBfnTEI

Length: 39min 46sec (2386 seconds)

Published: Fri Aug 28 2020