debugging python segfaults with gdb (intermediate - advanced) anthony explains #208

hello and welcome to another video uh in this one we're gonna be talking about debugging python specifically c python uh with gdb uh which occasionally i'll run into the need for this if i'm running into something that's seg-faulting weirdly and i don't really have a good explanation for it and sometimes dropping into gdb and looking at the low-level c python stuff can be useful so i'm going to show you kind of a basic setup and how to debug it so let's jump into it uh we're specifically going to be looking at crash dumps today um and crashing crashing programs i think for most other situations normal python debugging should get you by um to demo this today i have set up a very simple c extension here um i fit on screen dang it okay well we'll look at it over here so um i just have a basic setup.pi which has a name and a version and i have set up one extension module and so the module is going to have this name and it's going to be implemented in this c file and i did another video on c extension so i'll link that in the description um which goes into more details here but this is just like a simple hello world program uh we'll i'll show you it running let's make a virtual environment and install dot and it builds and if we do import sample c extension and then sample c extension dot hello world uh you'll see that it ran this function which returned a string here cool that's all working great but now i want to show you an example of where it's broken so i'm going to make a null pointer here and reference it which should give us the segmentation fault um we're gonna try and set the value of of null to one and so if we pip install that now and we do python-c import sample c extension and then sample c extension dot hello world uh we should get a segmentation fault and uh the crash reporter kicked in but anyway so we should get a segmentation fault um and by default at least on my machine this doesn't produce it says it core dumped but it didn't actually produce a core dump um so that would have been the first part of actually figuring out how to debug this uh which is to get our cord up now if it says it core dumped and you don't have one there's kind of one or two things that could be happening either it didn't produce a core file at all uh or the core dump is in some location that is not the default location uh where is my core dump there's some setting for this uh let's go over that in a second proxy's kernel chord pattern process core pattern um okay so this is saying that it will take the core dump and it will put it it'll pipe it to app port which is the error reporting software on ubuntu um but we don't want to do that we want to have the core dump in our working directory so we're gonna do uh ulimit c unlimited uh which is gonna say don't limit the size of cordons i think uh so we should yeah cool so now we have this core file this is kind of like a crash dump when the when the program errors out and you can use this to do some debugging so let's say that you have a core file and you want to analyze it after the fact so you don't really have access to the running program but you do have access to a crashed up what you can do is use gdb and the executable that was running before and the core file and what this will do is it will drop you into gdb and in some cases it will have the full source here as well as the code that that trigger this this is kind of rare like it doesn't always have this um and i'm going to show you some ways later that make it more likely to have you know the actual source here this is kind of useful because it drops you into gdb i'm not going to go over all of gdb today i'm going to show you kind of two main things that i use for it one is bt which gives you a back trace it shows you the stack trace of all the things that called uh you can see here that a lot of stuff got optimized out so like we don't know we don't know what these functions are because they don't actually exist they don't exist as a named function in code but they are part of our stack trace here and you can see you know the actual original start of this but this is mostly the main eval loop in python itself of course i'm running an optimized build of python and so like a lot of a lot of the symbols will be gone a lot of the code will be gone so it'll be a little bit harder to understand what's going on um but anyway that's how you can go from a core file to running gdb uh the other thing i'll show you is q to quit but there's a whole there's a whole slew of things that you could learn for gdb i find that those two are the the most basics you can also like look at local variables and stuff inside this core file uh because it essentially contains a memory dump of when it crashed there sometimes you're in a better situation where you can actually run the code that you want so let's say we had you know a python file here that did import sample c extension and then did sample c extension extension dot hello world uh lowercase w and so you know running this triggers our sight fault here um but we want to actually analyze this as it's running so what we can do here instead is to gdb gdb python so this is similar to what we did above except we're not passing a core file this time so this will put it into sort of a execution mode where you can tell it to run a program and it will you know give you the outputs here so i want to run so we already have told that we're using python and we can use the gdb run command and this will allow us to pass any arguments that we want to python if we don't pass any arguments here it's just going to run a normal interpreter um you know a normal interactive interpreter and you could do your your stuff inside there but that's not what i wanted to show we're going to try and run tdot pi today t dot you'll see that it had it is run this particular program here and you know crashed in the same way so now we can look at you know the value of ptrp ptr yeah we can look at the value of ptr now so you can see that it's a it's an inch pointer that has the value zero for example um but that's how you can run a program um the the actual reason this came up cue yes uh the actual reason this came up is um someone had a particular case where import the sample c extension someone had a case where they their code was only crashing inside of pi test um and so it was kind of difficult to debug so i'm going to show you how to debug this test file using pi test and gdp so again we're going to do gdb python and we're going to use run in this case we're going to do dash m pi test this will make it so it invokes pi test using a module and then we're going to pass along our test files so that pi test runs this particular test file no module name pi test of course if install i test of course i forgot to install pi test uh and then we do run dash m pi test desktop pi and you can see that wait oh forgot to actually call the thing that crashed i was like wait why didn't it crash uh run edition cool so now you can see that even though we're inside of pi test we're able to do some debugging in here uh now the last thing that i want to show you is uh how we can potentially get more of these frames to be known and more of their code lines to be known um there's actually a lot of frames that are happening here um but you can see like all of these don't have source because uh we're not we're not running it in a debug mode uh quit yes and so what i'm going to do for this is i'm going to apt-install python3-dbg this is the debug build of python which often contains more debugging symbols and makes things a little bit easier to understand um type in my bad password there uh it's going to add a bunch of debug packages as it runs as it installs here and we're going to create a virtual environment using the python 3-dbg executable instead of the one that we had so we're going to do virtualm vf dvg and dash p python 3 dvg and this is our our debug interpreter here and so we look in here uh forget how you can tell that it's a debug interpreter sysdot uh i don't remember we'll just trust that it's a debug interpreter um but now if we activate this virtual app and we pip install dot um oh we'll be able to look at the c extension name yeah so cp38d that d means that it's a debug build and so now if we run this import sample c extension sample c extension dot hello world uh we again got a segmentation vault uh but let's do that with gdb gdb python um yeah and so we have a lot more information here uh because it's no longer optimized i mean there's still some stuff that's optimized due to vector call but we at least know all of these functions that occur here and so you can do a lot more debugging when you're using the debug build uh you can also so it lists the file names and their line numbers here so if you have a copy of the python source you can also get that out of these as well but anyway that's using g2b with python hopefully this was useful if you have additional things you want me to explain leave a comment below or reach out to me on the various platforms but thank you all for watching and i will see you in the next one you

Channel: anthonywritescode

Views: 1,066

Keywords:

Id: bXEXE6ywzSA

Length: 11min 17sec (677 seconds)

Published: Sat Jan 16 2021