Disobey 2018 - Reverse Engineering fine details of Game Boy hardware - Joonas Javanainen

[Music] hi my name is Yanis and i'm here today to talk about my favorite topic Gameboys i'm sure many of you recognize this little device it's an original Gameboy from 1989 and this little device started a handheld game console family that was very popular even today if you think about Nintendo's handheld console line like these consoles they don't have Game Boy in the name anymore but there are definitely descendants of this device so the Gameboy console family started by this device in total there were 11 models released one way to categorize these models is to split them into three like hardware platform categories one is the original Gameboy hardware platform that content well the original Hardware the original instruction set many people think that Gameboy used as a deity but this is actually not true the Gameboy used a custom 8-bit processor that is in many ways similar to the AIT and also similar to 8080 but it's not either one of those ok so after the first hardware platform we got the gameboy color which was more like an incremental upgrade we got color and also some hardware features quite minor things and Game Boy Color platform can be backwards compatible so gameboy color devices can run original Game Boy games and also Game Boy Color games can choose whether they are only for Game Boy Color consoles or if they work on the original game as well then there's Game Boy Advance as well which is a completely separate thing completely new platform instruction set architecture based on arm and Game Boy Advance games on backwards-compatible in any way you can't put a Game Boy Advance game into an old Game Boy so Game Boy is in 2018 are they still relevant they are still very popular among emulator developers so it's quite typical that somebody who wants to get into emulation goes to some internet forum and asks which platform should I emulate usually people say first okay to chip eight that's the easiest one but what after that many people recommend the Game Boy and yeah if you go to github this a lot of game by emulators there's also an active home rule and modding scene so even last year there were some Game Boy game releases and I'm not talking about just random roms on the internet but like physical cartridges that you could order the modding scene is also very active so this original game boys didn't have any kind of backward in the screen so adding a backlight that's a that's a very cool thing to do so in 30 years a lot of hardware research has been done I think we had the first Gameboy emulators in the late 90s or something like that and after that even more accurate emulators have have been developed so the hardware details are quite well understood so I said quite well we can ask do we have a complete understanding of the Gameboy hardware this question can be also refreshed in some different ways can we build an emulator that behaves just like real hardware or can we build an emulator which cannot be detected with a test ROM so that yeah a Game Boy program couldn't detect that it's running in an emulator and the answer is no we can't build an emulator like this at least not with public documentation so munna GB it's my gameboy emulator it doesn't support the Game Boy Color games or Game Boy Advance games but I'm focusing just on the original hardware platform at this point in 2014 I wanted to get into emulation in some way and I owned a Gameboy pocket since 1997 so I thought yeah why not do Game Boy it's supposed to be easy but eventually I found that the documentation was lacking and I had more questions than answers so I shifted my focus to hardware research and today munna GB has very difficult but detailed hardware tests and all this is based on real hardware research and validation so I don't trust any documentation just because it's been available for a long time and right now there is no emulator that passes all my tests yes they work on real hardware but no emulators can pass that okay some people think that you can run pokemon so what's the point well I think all official games should be runnable even maybe unofficial ones yeah like homebrew games we really would like to run homebrew things in emulators and not just the existing ones but also the support homebrew development so if you've spent $100 on some nice homebrew game and then you find out that on one specific Game Boy Color version real hardware it just crashes completely that's quite annoying if you if the emulator hasn't given you any help finding these kind of problems then the speedrunners who care about accuracy pokemon speedrunning is a really serious thing and preservation also there will be a day when all the hardware will stop working and we would like to understand the hardware better this kind of research very deeply research is difficult but for me at least it's fun that's also a very important reason and one one thing I like about the Game Boy is that I can imagine like a 100% completion for my research it's not necessarily an infinite project ok yeah that's a very typical case example about emulator versus real hardware pinball deluxe a nice simple original Gameboy game but if you run it in most emulators today you see the result so after the game starts it just crashes after a while and that's it on real hardware it of course works it's an officially released game if it works and real about hardware yeah they don't care if there's some edge cases that the emulators can't can't emulate so just very quickly yes game by models you can see that there's a lot of handheld devices but also the super Gameboy and super Gameboy - which are meant for Super Nintendo so you can play Game Boy games by using these on a Super Nintendo and a TV and also there's the Gameboy player third one from the bottom left that can be used to play Game Boy games on GameCube all right so a bit more technical stuff than Game Boy CPU chips one nice thing about the Game Boy is that the central CPU chip handles pretty much everything so instead of like early Super Nintendo versions where you had the CPU you had the pixel processing unit one and pixel processing unit - all separate chips on Gameboy it's one chip it's basically a system on a chip and if we just focus on the original Game Boy and Game Boy Color platforms it's same instruction set all in all all the models and the same basic hardware is the same so we can make this kind of assumption that the behavior of the system depends on just this one chip so just by looking at this one chip we should be able to figure out the behavior behind the entire system this is not completely true there's some cases some things that affect it can also affect behavior in theory so there's mainboard versions and individual unit differences and this might affect things because there's things like analog behavior many people think that the original Gameboy had the best base the Buddhist base so I'm not sure it's about the CPU chip I think it might be they're like the supporting chips that affect this there's also like semi random behavior static Ram initial values they are not truly random more like certain bits tend to be ones and certain to be zeros but yeah it's not really deterministic but we'll just ignore this and that's a very nice assumption to have okay I like to invite the CPUs in the generations they are QFP chips and if you look at the pinout you you see you can group these different CPU versions so there's like five generations later generations a bit more pins and so on and actually there's a generation six as well but that uses a BGA chip and this particular chip doesn't it's not backwards compatible so I'll just ignore that okay so let's say you build an emulator and you use x16 documentation which CPU version are you actually emulating is it original Gameboy CPU or the next one that has a bit different silkscreen or the a or the B or the glob top version or the sea or the SDP or the mg B sub-2 or gameboy color chips is it one of these or gameboy advance chips and yeah the last one is the BGA chip so we'll just ignore that so internal we have around 20 versions depending on on what you assume that for example the two chips that have slightly different labels like are they the same chip or not okay around 20 versions now and just to illustrate that this matters in some edge cases there's a very popular audio test ROM it's a very good test from but if you run it on the first and second generation CPUs it works on the first one doesn't work on the second one then it works for the next three versions and it stops working for the next three so if you're basing your emulator on this audio test ROM which chip are you emulating and we can also ask which version should we emulate what's the best one is it the most popular one or the latest version with of course all the hardware bugs fixed and real hardware has backwards compatibility issues as well so this is an example with game boy : so there's a game Donkey Kong lambda3 and if you run it on gameboy color or later devices the sprites have this weird horizontal glitch lines there's a timing difference in Game Boy Color and earlier devices so you can't run this without glitches except on the old hardware and a proper emulator would also emulate these glitches because otherwise you would not be emulating gameboy color if that's your target okay so the solution is just research everything so here's a photo with most of my game boys and so I showed you all the CPU versions only one of them is missing and all mouna GB tests are verified on all Hardware versions I own so every time I write a new test I test it 19 times and I would like to test it 20 times if I had lost CPU ok let's move on to some hardware research tools how do I find information about the game boy the first thing is that I can use a flash card and some test runs so here's two flashcards the first one has a USB connection so you just plug a cable there flush the ROM take the cable out and run it on a real Hardware that's nice but the only thing is that you can't keep the cable connected so then I decide my own PCB my own flash card the GP like 32 which also has a USB connection would you just keep it plugged in all the time with one command from some random dashes script or something you can just flash it and reboot the system no cable you don't need to detach any cables so that's nice we could also probe the CPU solder some stuff to CPU pins and for this purpose I also have some custom boards I've got these breakout boards the M and F and together you can have this kind of test passed through where you can have a real cartridge connected to a real device and have access to all the 32 cartridge bus pins so that's quite nice on the right you see a Super Famicom so a Japanese Super Nintendo basically a super Gameboy these two breakout cards parts logic analyzer a lot of wires and a duck never forget the duck another option is to just desalter the CPU and put it on some customer test board so I also have a board for this custom board so this 1gb bench g1 it's a custom test board for first-generation Gameboy CPUs and yeah a custom four layer PCB there's a complete first generation Gameboy own on the board so it has two CPU the RAM chips the cartridge connector it doesn't have a joypad an LCD or a speaker but all the signals are there it's completely functional in addition to that I also have two lattice CPLDs that give me full bi-directional access to all the CPU pins all the digital pins at least and this port alone doesn't do much it needs a control board that connects with a parallel control bus so I've got a teensy 3.6 controller board as well and on the software side I've got these automated tests written with a lisp so I can just test this entire system quite easily and because I see all the signals that are in the CPU pins I can capture a CSV file a value change dump file and because the LCD signals are there I can capture a PNG just all the frames that it would be seen on the game boy and this is all possible to do in an automated way all right so let's do a demo okay so I've got the main GP bench board here and then there's the controller board with the chains on top and I've got two games here I've got the original Tetris let's start with that one so I'm just going to connect the tetris game to the board and then I've got this lisp test that basically does upset a sequence of steps first of all we reset the system we let CPU run until it's ready to execute the first instruction of the ROM then respawn some writers we want to collect some data and finally we arm some triggers this is basically just a way to sort of quickly wait until certain signals are visible so I know that Tetris is almost ready to go to the title screen when this address is being read so I'm just get the ARMA trigger there and wait at most 20 seconds and then we step 400,000 clock edges or full hardware samples all right this will take a while one of the annoying things about Tetris is that it takes forever to get to the title screen we should be yeah okay we see that some lights are blinking it's aligned to the title screen and collecting the four hundred thousand a thousand samples all right we're done and now we can take a look yeah there's some signals there all right the raw signals are a bit boring so let's do something different I've got this nice Python script that looks at the CSV file that was just collected and it extracts from the LCD signals the frames so it found two frames that were identical and what's the frame oh it's Tetris [Applause] and let's just do quickly another one we've got the Super Mario Land and for this one this one I have a different trigger point and that's it so let's run the Python script again and we've got Super Mario Land and there was another frame with the mushroom sprite yeah that's how it works okay so I can use existing cartridges and test them but one of the important things about this board is also that I can run I can create and run automated tests that just don't use any ROM because I've got control of all the signals I can pretend to be wrong and I've got this test set of low-level Hardware tests it's just running stuff there and this verifies my understanding of the game boy all tests passed yes and I can also run the same test set on a different version a different CPU I just need a different board I've got three of these so yeah all right that's it for the demo let's go back to the hardware research tools and methods one way to find him for information about a CPU chip is to decap it so this photo is from the silicon Tron website we only have at the moment a photo of the top layer so it's not possible to reverse-engineer the entire transistor structure in detail you might have heard of a visual 6502 that's a project that did therefore reverse engineering based on these kind of photos but yeah we only have the top layer but even with the top layer we can see where how things are connected and maybe we can gain some information from that okay so here's the chip let's fade it a bit so we can highlight things a bit easier and all the pin labels there's quite a lot of pins there well technically these are bonding pads which in in the package actually have tiny wires going to the pins let's add the supply units so we've got the ground and power and they form this kind of rectangle around the chip and then just at all all the pins so we see where things are connected and well I've also done a bit more so if we also had some internal signals yeah that's quite a lot of stuff now it's a mess there's a huge amount of signals and stuff it might be better to just focus on some of them for example we can just see where the video RAM address pins go and the LCD pins and all these go to the bottom left corner of the device in the end of this big pile of logic and transistors so that's most likely the PPU the people use on the same chip and the PPU cares about these signals another thing we can see is if we highlight all the data pins and at least some of the control pins and those things go to the top left and also to the top middle in a bit bigger area so these areas are most likely some kind of bus control timing related things so just by looking at how things are connected we can make some guesses okay we can also see that this place this area here contains some kind of mask ROM and I know that it's the boot ROM this chip doesn't really have any other big roms and we if we zoom in a bit this is what the boot from looks like and you can see those tiny white dots those are the individual bits in the boot ROM and one way to dump the boot from is to read out the bits I don't actually know what the pattern is it's not the kind of obvious left to right top to the top down but yeah it's possible now speaking of boot ROM let's talk about a little bit more the boot ROM is the piece of code that executes when the Gameboy starts and it's responsible for the scrolling Nintendo logo that you might know boot rooms have been dumped quite a lot there's several versions the first one was dumped in 2003 and that was done with visual inspection I've dumped to boot rooms that were previously not dumped at all so CPU the CPU starts from address zero after reset after reset the boot rom is mapped to the first 256 bytes in the address space the external romo is mapped to the first half of the address space so there's overlap but the boot rom always has priority when it's enabled so basically when the CPU starts executing instructions it's going to execute the boot rom and the external Rama doesn't get any chance to execute anything until the boot ROM has done its things and the last boot from instruction actually our Maps the boot rom so when we have control the boot from is not readable anymore it's gone too late so we need a way to stop the unwrapping prevent the boot ROM from basically killing itself so what kind of attack vectors could we have for this kind of dumping well we can't really use flashcards and test roms because like I said the external room doesn't get any control until it's too late we could do some stuff with probes and solder some stuff yeah there's some ways to dump it we could also use a test board where we have control of all the pins there's a slightly different technique for that but my board can also dump their boat from and yes we can use some visual methods - yeah we can pick up and inspect the entire CPU and read out the boot rom okay so time for another demo so I'm gonna do a Dumpty boot rom and I'm going to use this clock glitching method which I believe was first discovered by Ben van and this requires no soldering all I need is one wire and well a flash card that has a special boot dumper ROM so the dumper ROM basically just copies the first 256 bytes to the cartridge sabra so we can then access it from the computer alright where's the why I'm alright let's see how this goes so first I'm just going to start the Gameboy without doing anything not using the one wire so we see into the locker scrolling and there's a sad face the dumper program didn't detect a boot rom okay yeah I didn't do anything special so it's obvious that yeah the boot from killed itself and it's gone but this time I might need to try this a couple of times I've got a couple of seconds while the Nintendo logger is crawling to put this wire into some places so one point one end of the wire is going to ground connection and with the other other end I'm just going to gently scratch one of the clock pins and this should lead to the clock so that it jumps somewhere and the dumper Rama basically has a lot of knobs and a jump in the end so if the CPU happens to jump into some place controlled by me it will end up in the main dumper program alright let's see nope Oh happy face so now we have a copy of the bootrom on the cartridge Sabra I'm not going to verify it that takes time but but yeah it works trust me all you need is one wire okay that's it for the demo let's continue so the title of my talk promised some fine details so I'm going to give you some fine details about the instruction decoding and interrupts on the Gameboy this will be technical and very quick part of the presentation but yeah some of the information I'm presenting here you can't find it on any public documentation even my own documentation because I'm not sure about some of this yet okay so we have this game where a program that consists of three instructions we have an OP instruction that does nothing then we have load instruction to a certain memory address that just copies the value of the accumulator to that address and finally we have incremented actually major instruction and the first instruction starts from address 1 2 3 4 in hex and the actual data byte is 0 0 that's an op on Gameboy and the second instruction actually consists of three bytes because there's the opcode and two bytes for the address and the last instruction is another one byte instruction ok some basics about instruction timing on Gameboy first instruction is always have a duration of the duration of instructions is always a multiple of four clock cycles and we call this four clock cycle like chunk a machine cycle and a good rule of thumb is data a machine cycle is required for each memory access so by looking at an instruction and thinking about it how many memory reads does it - how many memory writes does it - you can have a pretty good guess how long the instruction is in machine cycles an example is pop PC which pops a 16-bit value from the stack and puts it in in the PC register pair so first of all we have the opcode read the OP code itself then we need to pop the low byte and copy to see register see that's another memory read and also we pop the high bytes and that goes to register B so three machine cycles simple so it's a good rule of thumb it's not always correct but most of the time this is how it goes okay let's go back to the program we had and now we need to take a look at some timing diagrams we know that the system has around four megahertz clock signal and if we look at all the CPU pins and well actually just the cartridge bus pins we find that there's another clock signal there that is one megahertz single one megahertz clock signal so machine cycles where multiple machine cycles where four clock cycles each so this looks like maybe this one megahertz clock is a machine cycle clock yeah what make sense we also can see some addresses on the address bus we see each address for each other each of the individual bytes in the program and also the target address that we are going to write in the load instruction if we look at the data bus we see once again all the data that correspond to the bytes in the instructions and also we see the valid value of a whatever that is during the bright cycle then we also have some control signals these are active blow signals so for example Reed is active in most of the time and it goes hi just for a bit when the right happens the chip select actually there's multiple chips alert signals but it depends on which device you are accessing but there's a chip select signal and that pulse is low in each of the Machine cycles where we need to access device which is now an external room basically and finally we have the right signal which is high most of the time but it goes low in the end of the right machine cycle okay this seems quite simple and yeah and we can then just have a lucky guess that maybe the instruction boundaries so where where the instructions start and end could be like something like this that would align with one megahertz clock yeah what makes sense now if we run this program on the test bench and we are pretending to be a ROM we don't actually have to care about the chip select we can just have rubbish data on the bus all the time except for that one clock edge when the Gameboy actually cares about the data bus and if we do this we find out that there's just one clock edge in the middle where the data bus contents matter okay so we've identified the clock edge where this data bus is sampled simple now we don't want to actually stop here because there's another way to investigate how the Gameboy executes instructions we can abuse serial i/o a bit there's two pins on the CPU that we care about there's a clock pin and enable pin for the Zorilla there's two registers there's a control register and a byte register and the control register we can use that to start a serial transfer and also choose which side of the serial connection is the master that controls Tupac and because we are using a test bench of course we want to be the master so we'll just have the game bar in a slave mode and that means that when we pulse the clock pin we can feed in bits to the serial byte register which is basically just a simple shift register so by having control of these two pins and starting this kind of transfer we can just choose the value of SB basically and one thing about the serial i/o is that it's completely asynchronous to the main clock we can stop the Gameboy clock we can shift in seven bits if we want and then continue with the clock and the Gameboy she's a modified SP value and this gives us a lot of control and possibilities to do things so in order to see when data is sampled let's first set up a serial transfer and then jump to SB the game boy doesn't care where the instructions are you can jump to any address jumping into hardware registers is my favorite that because you can do a lot of things with that and by controlling the value of SB we can then see what the instruction was actually executed okay so this is a simple diagram showing this case so we see that the game boy is reading the SB address and in this case where the SP has an internal initial value of F F and we are not modifying it the corresponding instruction for FF is rst 38 which is just a quick function call to address 38 and I also have the vertical line in the diagram that was the point of time that the clock edge where the sampling happened in the previous example okay so now that we have control of SP we can choose at some point of time where we change the value so what if we just feed zeros 8 0 bits to SB right before this sampling edge ok well then the game will actually execute an OP which is 0 0 so we've changed what instruction it executes even after the instruction read has started ok simple we can also move the time a bit so let's add one cycle ok but now we are past the edge clock edge where we thought sampling happened and aim didn't so clearly the sampling edge is wrong because at that point it would have seen FF but it executed not ok we can keep doing this and eventually we find that the last point when we can feed 0 0 and half the game boy executes 0 0 is at that point and this is a bit weird because that clock edge based on our instruction boundaries that we have in the diagram is supposed to be the first clock edge of the next opcode so this may be something wrong here so looking at the original program what we had was this the sampling happened basically in the falling edge of the machine clock but the reality actually is something like this okay so it's a bit weird that the first rising edge would be of the next instruction would be the time when the previously start for instruction is actually executed okay and we can shift our guess about where the instruction starts and stops and because that's just something we've we use to understand the system everything else is real signals but the instruction boundaries are something that we just made up so this is an open question still where does the instruction start and where does it end we see all the signals but what's the beginning okay so maybe it's wrong machine cycle boundaries maybe there's some pipelining even the 6502 a very popular processor as well had a very simple pipelining as far as I know where the fetch of the next instruction overlaps the execution of the current instruction so this would also explain things but it's an open question okay another example this is the last thing interrupt handling interrupts on I only checked between instructions otherwise you could have an interrupt in the middle of an instruction that does multiple things and you would only see some side effects for example you push a 16-bit value you would only see rated the first push that wouldn't make any sense sorry interrupts like retain their instruction boundaries interrupt handling takes machine cycles on Gameboy this is testable with some test drums and you can find this in some documentation there's five possible interrupt sources that can cause interrupts and they have a priority if to interrupts happen and then the interrupts are checked after some instruction only the high priority highest priority is interrupts is actually handled okay so more timing diagrams this time we are just executing knobs to keep it simple so if the interrupts are checked between is instructions we would assume that those red vertical lines would be the points where the interrupts object so if an interrupt happened in this blue line we would expect all the knobs to be ignored and instead we have interrupt dispatching so the five machines cycles and after that we have the first instruction of the actually interrupt handler the code okay this is what we would expect now to study interrupts we can once again abuse serial i/o because serial transfers are one of the interrupts AUSA's so because the serial i/o is asynchronous in this case we can just trigger eight clock pulses to trigger an interrupt at any point we want between clock edges we can stop the time of the Gameboy and just feed an interrupt anywhere so if we do that we actually find that it's more like this so there was a like a two clock cycle offset so now if an interrupt happens where the blue line is then we could interrupt this button and if it happens one clock edge later it doesn't do this interrupt dispatching okay this doesn't quite match what we expected this also means that the Gameboy CPU starts reading stuff and then it decides much later if it wants to actually sample the data or not so it basically just cancels the read and starts doing interrupts instead okay there's another clock edge that's meaningful so it turns out that the first clock edge the redline is just a check should we do some interrupts dispatching but when the game boy chooses which interrupts to serve that happens at a different time and that's the second clock edge the redline so so now there's a blue line and an orange line if a low priority interrupt trigger to interrupt dispatch at the blue line and another high priority interrupt happened at the orange line it would not serve the low priority interrupt at all if we just do the high priority so there's a conflict okay so interrupts are checked at the falling machine clock edge which is not what you would expect based on the information we have and the address where we are jumping is checked much much later so the different different times for should we jump and wait to jump and higher priority interrupt can completely take over and I don't know if you remember pinball deluxe that was I I showed some screens from that game I understand changing emulators to match this behavior was one of the keys to almost fixing pinnacle deluxe it doesn't work yet on my emulator about this one emulator called same boy that emulates pinball deluxe today correctly after making changes based on these findings okay final words so even after 30 years there's still work we still have open questions I didn't even I showed you all the different CPU versions but I didn't explain any differences because I just don't know it's still unclear and for me it's really fun to do this kind of very deep hardware research reverse engineering and research and even very core concepts that you would think that we would understand now like instruction execution and and interrupt handling we would understand this but they still have surprises all right that's all there's a couple of links for many Gameboy things I've done I think we have some time for questions if there are any all right if you have any questions later just find me and yeah I'm happy to chat about game boys thank you [Applause]

Channel: Disobey

Views: 9,004

Keywords:

Id: GBYwjch6oEE

Length: 43min 1sec (2581 seconds)

Published: Sun Feb 04 2018

Reddit Comments

[u/corvusfan23]

Thanks for the share. Found this pretty interesting! Know of any more videos like this?

[u/binjimint]

Great talk, gekkio! I haven't seen your gb-bench before, looks really cool. Looking forward to your uncovering more secrets about the gameboy :-)