How To Read from ESP32 - CLONE/BACKUP Everything

so just last week i was working on a customer project based on the esp32 and they had an existing prototype that they sent me so the software was done everything was good they just needed me to redesign the hardware and once i finished that up i needed a way to program the new prototypes with that same software from the existing prototype now instead of bothering them and asking them for the software what libraries all that stuff or even a bin file i decided to try out something different which was to take their existing prototype they sent me and read out from flash memory that bin file and then program the new prototypes and this worked and i thought it'd make kind of a cool video so this might be useful for that exact thing or maybe you know you you want to back up a design and because we're reading out the entire contents from flash memory that's also you know your wi-fi credentials the encryption keys all of that stuff this is also very useful for duplicating esp32 boards like you know for the trig boards here for example you might be installing like 20 of these all with the same wi-fi credentials and everything else so you could just take a backup of one and then duplicate them you know i don't know there's probably a lot of different use cases for this but there's also uh a major security concern here so in like for production this could be a major problem because you know somebody with the skills could read out your bin file and then you know copy your product or worse they could figure out the keys or the wi-fi credentials from that bin file so we'll probably in a future video talk about encrypting the flash memory on there and then try this thing again here where we're reading out the flash content but anyway you know how this youtube channel works you know when i do something that i think might be useful to you i make a video about it so let's go ahead and get started and i've made videos in the past on how to write to the flash memory directly from the command line using esp tool so this is very similar because in here is also a command to read flash so we're just going to do basically the same thing just in reverse and this can be kind of fun too because we may not know what the esp32 is specifically what memory is is installed you know like here on the husa board they actually have it printed right on the board it's a four megabyte flash and from the arduino ide you'll see we have several memory options here so if you're trying to copy the entire contents of the flash memory but you don't know the size you might not copy everything out so that's why it's important to know what the flash is and there's a way we can read that out as well so first though let's assume that you know what the flash size is and in order to do this you will need to have the esp32 core installed i'm assuming that you've been able to do this and program esp32s from the arduino ide so you have that installed meaning that if you went to tools here you would be able to select esp32 boards and we're going to use the arduino ide to cheat a little bit here to set things up so let's actually just go in here you can select the esp32 dev module and just pick a couple random settings here really doesn't matter for what we're doing and then i'm going to just take a usb to serial converter here plug it in without an esp32 installed because i want this to fail so what i'm going to do is just select that and let's refresh that go back down to port so i just plugged that one it in and it appeared right here we're going to hit upload obviously it's going to fail because there's nothing there and you see we've got the verbose output turned on which is important preferences and both compile and upload are clicked there which is important because we need this last line here so i'm going to take that last line copy it out and just open up a notepad and paste it in here because now we can take that and modify it so that it works with read flash here is the command so you see there in the read flash command we are calling up esp tool the port baud rate read flash and we need to know the memory size for this and like i said we're going to start with the adafruit huso board because this is known it's printed right on the board that it's a four megabyte flash and we'll be able to set that up and read out to a bin file so let's do that now so let's copy some stuff out here right there is where esp tool is okay and i'm going to copy out the port and baud rate obviously the husa will not use this yes this usb to serial converter so let's plug in the adafruit board now and let's look at what the port was on this so back up to tools we don't want to hit upload because that would defeat the purpose of this we would just overwrite the program we're trying to read out okay and the new one that popped up is this one down so let's type that in okay cool so we've got that now we can put the command in which is read flash and then we have to specify the address range so it'll be from 0 to 0 x which is in hexadecimal we're going to talk about this a lot here but for now for a four meg chip it's four with five zeros and then the bin file which we're just gonna call adafruit dot bin okay we're on a mac here so let's launch terminal and i'm gonna copy that line out paste it in and let her rip see what happens cool there it goes so you see here connected it knows it's the esp32 um and then the baud rate was changed it's reading it in and you're gonna possibly see some failures in here which would likely be due to the baud rate and actually for the sake of this video i'm right now just doing this quickly to get you up and running but what i'm gonna do after we do this with the user board is take um an actual esp32 that's loaded with something with a bunch of credentials that's doing something uh off of the trig board here and load it onto a brand new esp32 straight from espressif totally blank and prove to you that we're actually doing a full duplication of the flash memory here okay cool so we've read all of the flash memory out to that bin file let's go and see and you see that we're operating at this directory here kevin dara which is my home directory here and there it is adafruit.bin cool so now you have an exact copy of what's on this esp32 now let's write it back well actually we already know how to do that because we've copied it out up here so let's just take this entire line right here paste it down here and what's cool about this is that what we can do is we know we have to change the baud rate because we're just going to write this back to the same board here just for the sake of this video okay i'm going to delete a bunch of stuff out here and right over here is the command write flash okay with some parameters we don't care about too much but we're going to write it at address 0 because we're going to write over the entire flash memory because we copied out the entire flash memory so we're going to start at address 0 and then write the entire thing okay then where the bin file is i'm just gonna right click and on a mac we can hit option to copy out the path and then paste that right there and it's going to write that whole thing back to address zero i'm going to copy this out now paste it in and let her rip cool so there we have it so that is reading the flash memory out and then writing it back now this is all just to get you up and running quickly but let's actually do something cool here okay so now we're really going to get into the weeds of things we've got a trig board here let's pretend that we have no idea what the flash memory is here i don't think it's actually even written anywhere on the shield there on the case here of the esp32 so let's pretend we have no idea what it is and we're doing some forensics here we want to pull the flash memory from this board here let's get a battery so what we can do is actually go and read the flash id so let's do that next and read out what the flash memory size is okay so now we're using that usb to serial converter so we're back to that port and then we're going to put in the flash id command cool now we'll go ahead and put that into terminal here and the trig board we have to wake up so i'll press and hold the wake button here okay cool so detected flash size is 16 megabytes pretty cool now this is a little tricky here to understand and when you look at this and you know you see the two like okay two megabytes two okay that's easy four and four that seems to make sense but it's actually not all that straightforward so what you need to do with with these sort of things is always go to the data sheets but um we don't know we're doing forensics here now this is a quad you know a spy flash chip so we can go and pull some data sheets uh like i found this here and i'll have a link to it down below but this is the same kind of spy flash chip that's in the esp32 here what they use and we'll look at a data sheet let's first though look at the 32 megabit now they're showing it here in mega bits because even if i go to the tools menu i think they show it that way as well yeah you see here so four megabytes is 32 megabits right so just uh divided by eight 32 divided by eight is four right and uh you can go all the way straight down so 16 megabytes would be 128 megabits and that's the way these flash chips on this site here are shown so let's look at a 32 megabit which was similar to the one we just read out from the adafruit board just as an example because we've got that one done and we can look at it so let's just pick a random one here so you get a look at this data sheet you're gonna go down and look for something like the memory or yeah right here so here we've got sort of a memory map of the chip over here is where all the memory is and it's all divided into these blocks of memory with sectors and so on and we go all the way to the last address here in memory which is what three with five f's and i'll pull up a calculator so we can look at that okay so three with one two three four five and then we go over to this and we can see what the memory is so it's not quite four megabytes it's actually a little bit more than that in fact if we go now back over to terminal because we just completed that one from the adafruit side you see right here compressed 4194304 our last address is 4194303 why is it one less and that's because your zero location is also a byte okay so the total byte count is this 304 so that's why when we set this up we're actually look at this in hex it's that but that's not what we put in we went from zero to zero hex four with five zeros not this so let's add a zero to this or not add a zero but add a one to that and then we go to the hex and there it is and so basically what we're doing here when we do the read flash out is from zero to the address we want to go to up until right so so it's essentially the total number of bytes we're reading out in memory this is important to do because you know if you're doing a full copy of the flash memory you don't know how much of it is used up i and i guess it might work still if you did just read out four megabytes even but you don't know because you're doing a copy this is forensics and you want to do a full copy of the flash chip so i thought that was kind of cool to show now let's go to the 128 megabit which is the 16 megabyte flash chip we have here and again just pick a random one okay so now we're reading up until six uh f's there in hex so we'll throw that into the calculator one two three four five six okay and then we'll add one to that because we're going to read up until that address there copy that but we'll go over here and there you have it so would be one with six zeros in hexadecimal is what we're gonna put in and read this out okay so let's go now back to the cheat sheet okay so now let's create our um full command here to read from the trig board we're going to just copy that we'll use that port same baud rate but instead of from zero to four we're going to do one with six zeros do i got that right one two okay we'll call this trig board back up bin okay let's let this rip okay so this is interesting and i knew this was gonna happen fatal error has occurred corrupt data interesting so for whatever reason this is not working worked fine on the adafruit but not on trigboard with the external usb to serial converter and what i found is that this is due to the baud rate so what we're going to do is slow this down and what speeds are compatible well i always go to the arduino ide for this so let's go to tools and you see upload speeds we've got 92 1 600 and then we also have 460 800 so let's try 460 800 next okay here we go fatal error no good okay what was the next speed uh 230 400 is the next speed let's try that here we go there we go so not only is this flash memory size four times larger than the adafruit but also we have to read it at a much slower baud rate so this is going to take forever and the code size might not even be close to the full memory size that we have on the esp32 there but we want to read out every byte out of memory to create this bin file so this is going to take a long time but keep in mind that this is only a one-time kind of thing when you're doing a backup so we'll let this go and we'll see you back in a few minutes oh and just while that's going i just i'll show you what that trig board here that we're copying is actually doing so this was part of a little experiment i had with modified base firmware to wake up every 10 seconds connect to the wi-fi network and log the rssi and total on time to a google sheet here and the idea was to see if there is some correlation to total on time to the rssi so you know the closer a trig board might be to your router you know the better the signal strength maybe the faster it could get a connection so i have you know a whole bunch of trig boards doing this and setting this up trying to gather this data but we're going to copy this out which means we're going to copy out the base firmware that does this as well as the wi-fi settings and the encryption keys to get connected to this google sheet so with the factory trig board here straight from the factory the esp32 module is straight from espressif totally blank we're going to duplicate the one you see here and with any luck we should see new entries here once we write that back so total proof that we are able to duplicate this esp32 so let's see how we're doing after all that rambling yeah we're going to be here for a while well we're still reading flash there but i wanted to tell you another little story here so while i was doing this i thought it would be interesting to try and read out from a memory location way higher than what's even possible here so what i did was i put in a 2 here instead of a 1 and this worked on this chip so what happened was it actually froze right at because you see here it's actually showing you the byte count and what was cool is that it actually froze right at this memory location um or the this total byte count and it just stuck there so i was thinking that would be kind of cool like if you had no idea you couldn't read it out you didn't know what spy flash chip was used or the the way the blocks and sectors worked in the chip so you had no idea maybe you could just throw in some huge number and just let it freeze up there and see where it froze at but then i tried it on the adafruit board and it just kept reading because some of these spy flash chips i think wrap around so it just kept reading and it read out the huge number and i had this huge bin file of course you know that wouldn't work to program back to it so it was kind of useless but i don't know anyway i thought that was kind of cool so i'm just sitting here watching the screen and i was thinking about this this customer project it's going to be funny when i send them the boards and they're already programmed they're probably going to be like how did you program the boards did you get the software from because they never gave it to me we'll see what happens there all right well that's still going and i was just thinking about you know how a lot of processors that you might work on it's kind of nice for production you can do the the lockout built in it's very easy to do but in this case with the esp32 the spy flash chip if you look under the metal shield in fact let's do that just because i'm getting kind of bored of this okay images there we go okay there we go that's a good picture so under that metal can there you can see the actual esp32 chip is right there man there we go is right there this is what we're reading from the spy flash chip it's a separate device so it's not as easy to just put in read protectors flip some bit so you can't read out of it because if you did that then the esp32 itself can't read from it so encrypting this is is a little bit more challenging and i think that would be make a very cool uh part two to this whole thing is enabling the spy flash encryption so that the esp32 here can read from it but we cannot or if we can read from it it's all just garbage data and we can't write it to a new esp32 so that might be kind of fun to do okay just about there come on there we go all right just about there okay there we go so there you have it let's pull up the calculator again and there is the total byte count read out matches up nicely with the calculator there so we're good and there's the bin file right there so now instead of writing it back to the same esp32 we just read out let's write it to a fresh esp32 straight from espressif totally blank there we go this is the way i receive the trig boards and let's do this okay so let's create our command now to write this out so just like before when we were writing to the adafruit board we'll just copy that out of course the port is this usb to serial we can let the baud rate rip at the fastest speed all of this stuff looks good let's grab the path to the new file option copy that out paste it right there and let's see what happens but before i do that for total proof let's delete everything out of this sheet here there we go so if this works we should see a new row pop up here here we go letter rip at address location zero the entire bin file boom okay that's done let's hit reset on it okay well i screwed that up so i got to do it again i hit reset before we got a response a successful response back from the esp32 so that's no big deal we'll just do it again okay that took a little while but after it wrote it we had to wait for everything to be verified and then we get the hard resetting via rts pin i always wait for this before i do anything don't remove power or anything i just did that up here and i got a fatal error has occurred invalid head of packet so i wait for everything to complete we should be good now i can hit the reset button led turns on which is good it's running some code is it connecting to the wi-fi network is it connecting to if this then that is it connecting to google sheets i don't know let's go check oh yeah it is there we go every 10 seconds so it's reporting again this is the rssi and total connection time and uh it's working pretty good so there you go okay i think that was a kind of a long video but um hopefully you found that interesting lots of good stuff i think again a part two would be to lock it down so you can't read out the flash memory or at least if you do read it out you can't do anything with it because it's all encrypted garbage so anyway thanks for watching

Channel: Kevin Darrah

Views: 48,344

Keywords: esp32, copy, duplicate, read flash, backup

Id: 2GwzbBn7uRw

Length: 23min 56sec (1436 seconds)

Published: Sat Feb 13 2021