How I implemented MegaTextures on real Nintendo 64 hardware

foreign [Music] what you are seeing is being rendered in real time on Nintendo 64 Hardware this scene contains about 40 megabytes of texture information but runs on four megabytes of RAM this is possible using a technique based on its software's Mega textures Mega textures was developed by ID for the game rage and allowed for textures up to through 2000 by 32 000 pixels however Omega texture for the Nintendo 64 can only go up to 1024 by 1024 pixels which is regular size textured by modern standards but is far larger than we typically see for the console so how is this done before I explain how Mega textures work I'm going to explain how regular textures work for the Nintendo 64. to start there is a coordinate system called the texture image space it ranges from negative 1024 to 1024 in both the X and Y Direction positive X is to the right positive Y is down you can place an image called a tile inside the texture image space the origin 4 tile is on the top left a single tile must fit in the four kilobyte tile cache this really limits the size of a tile which is why tiles typically are around 32 by 32 pixels to make use of the entire texture image space you can offset a tile scale it by a power of two or repeat the tile across the entire texture image space when you define geometry you specify our texture coordinate in the texture image space for each vertex this Maps the texture image space onto the triangle this means you can specify a texture coordinate halfway through a pixel now with that out of the way let's explain how Mega textures work because the texture cache is so small you can't use very large textures this is why so many Nintendo 64 games look so blurry one workaround is to use multiple smaller tiles combined into a larger image each portion of the image needs to be copied into the tile cache then to the screen with this technique the limiting factor to the image size is Cycles needed to copy texture information and available Ram to hold the image copying many tiles also becomes a problem when large image scales down and is drawn to a small portion of the screen you still need to take the time to copy each tile into the texture cache even if the tile ends up taking a small portion of the screen if each image tried to render each pixel it contained frame rates would come to a crawl the solution is to create MIP maps for each image this involves creating multiple lower resolution versions of each texture each level of detail has half the width and height of the previous level of detail lip levels are created until the lowest level of detail can fit into the tile cache now when drawing a surface you can choose the MIP level that gives the best ratio of image texels to screen pixels when the image is close to the camera use a more detailed MIP level when it is far you use a less detailed level resulting in less time wasted copying data into the tile cache the next problem to solve is getting around the small amount of ram now this demo has about 40 megabytes of texture information and yet it can run on an N64 that has about four megabytes of ram this is done by only loading the tiles from the cartridge needed for the current frame since tiles needed for one frame are likely needed for the next frame most tiles can be reused from frame to frame only loading a few newly needed tiles on screen you can see this happening in real time the grid of tiles are tiles that are loaded into RAM the 3D view next to it is rendered using these tiles notice as the view gets close to a window you can see the tiles being replaced with more detailed window tiles there are some frames where more new tiles are needed from the cartridge than can be copied in a single frame when this limit is reached new tiles are skipped and a lower detailed replacement for the tile is found the engine tries to find the highest detail replacement available to ensure there is something available for any tile the lowest two levels of detail are always available and are never replaced when new tiles are requested this tile limit is why some parts of the screen are blurry as you quickly rotate before the detail shows up in the tile view you can see the first few tiles never change these are the permanently loaded low level of detail tiles if I modify the tile renderer you can see the tile boundaries and what level of detail is being used red tiles are full detail the Hue shifts to Yellow then green then blue for each lower level of detail this also gives a better idea of how many tiles are visible on the screen at any given time determining which level of detail is pretty simple for images that face the camera I simply determine the size of a text along the screen if the textual is too small a lower level of detail is used to compensate the equation on screen is how I determine which level of detail to use based on the size of a text cell on the screen for images that don't face the screen directly multiple samples are taken one sample for the level of detail for the Texel closest to the camera one at the furthest and one in the Middle with this information the boundary between each level of detail can be determined each level of detail is drawn with a dividing plane separating them preserving a nice clean boundary between the MIP levels it's also important not to waste time or memory on tiles that aren't even visible on the screen it would be wasteful to copy a tile into the tile cache if it is never rendered onto the screen to prevent drawing off-screen tiles I use the camera frustrum for tile calling the camera fresh trim is the shape that contains all visible objects in a scene if any tile is outside the first trim it can be skipped entirely if it intersects the thrust drum it must be drawn if a face is partially outside the frustrum and partially inside then only the tiles on that face that are inside the fresh trim are drawn this is done by determining the shape of the overlap between the image and the freshstrum in texture image space any tiles intersecting the shape are drawn any that aren't are skipped one last optimization I make is disable the Z buffer the N64 is pretty limited when it comes to memory bandwidth so disabling the Z buffer reduces the amount of memory I need to read and write to Ram per pixel with the Ziva for enabled each frame and first need to clear the Z buffer which is 16 bits per pixel written to it and then every time I draw it to a pixel I first need to read 16 bits of information for the Z buffer and if the pixel is visible right back the 16 bits to the Z buffer and another 16 for the color buffer that's a total of 48 bits written and 16 bits red by disabling the Z buffer I only need to write 16 bits per pixel this does come with the downside that I need to sort all of my faces and draw them from back to front and when a face overlaps another I may be redrawing the same pixel over and over again which is wasteful but overall it saves quite a bit of time not having to copy memory to and from the Z buffer with all this the final results are better than I expected when starting this project I was able to use the highest resolution the Nintendo 64 supports with more texture data than expected and higher frame rates than expected of course this is a single small room without a lot of geometry that leads to the question could you make a game out of this and yeah you probably could but there are some trade-offs there's a lot of texture information I have about 40 megabytes in this small scene here and the limit is about 64 megabytes so you have to find ways to either reuse textures across levels and scenes make a game that's fun in just a small number of rooms or potentially just use smaller textures I don't think you need to have the full texture quality I have in this demo to really to have a scene that looks good with the right balance of texture detail and level size and an artist much better than me I bet you could make a pretty compelling game that has high resolution textures that look out of place on the Nintendo 64. it comes with the trade-off of less geometry detail because every face takes a little more processing than you typically would see but yeah I think it's totally doable now you may be wondering do I plan on using this for portal 64 and probably not as it's implemented I may use it for the posters and drawings of ratman on the wall but it's not going to be implemented exactly the same way in any widespread use would probably lower the frame rate to a point that's just not playable and not worth using so you're probably not going to see high resolution Textures in Portal 64. and that is all for this Tech demo I hope you enjoyed it and learned something and with that I'll see you next time but

Channel: James Lambert

Views: 325,016

Keywords:

Id: Sf036fO-ZUk

Length: 8min 51sec (531 seconds)

Published: Sat Sep 16 2023