I Made A Fully Ray Traced Game

hello everyone my first ever game Jam has recently concluded to great success if you don't know what a game Jam is it's where you try to make an entire game within a certain time limit usually with a given theme for my jam everybody had 14 days and the theme was aberration meaning an unexpected change that's usually not good essentially an unwanted subversion of expectations on the final day a total of 992 games were submitted to the jam almost but not quite surpassing 1,000 we games ranging from beautiful Shader exhibitions to fun mini Rog likes demonstrating strong cruci of concept but one of these submissions had a dark secret it was actually mine the whole time I secretly spent the same 14 days as everyone else tirelessly working on my game in between IMAX showings of Dune 2 while secretly laring on my Discord alt account in the end I produced Philo Calia a fully Ray traced solitary confinement simulator that runs in real time to drive a realistic sun cycle a date of death has been randomly generated for the character in this room which transforms our meditative experience into a death game who will die first you or the prisoner I wrote my own compute Ray Tracer for this figured out how to calculate the position of the sun from date and time made all the materials and I wrote all the music so today I'll be going over how I did all that I didn't have the idea to secretly participate in my own game Jam until like the day before when one of the judges said it would be really funny if I did and that's all the convincing I need to do a lot of Labor since the theme is all about subverting expectations I can technically do anything I want because my presence in the jam alone satisfies that condition so I went digging in my project idea backlog and picked one I've thought about a lot a game where you sit in an empty room with a single window that runs in real time and you are effectively sitting and waiting to die in game and by extension in real life if this doesn't sound very exciting to you then you'd love other ideas from my project backlog such as watching grass grow simulator or navigating an endless Labyrinth with no exit or a game where you play as a Roomba despite the simple concept there's actually a very large space of opportunity here for graphics it would be the perfect context to demonstrate bounce lighting from the singular window light source the low scope of the room itself allows for really expensive shading effects because there isn't a lot to render and since it's based on real world time there's plenty of interesting Skybox simulation opportunities for stuff like sunrises and sunsets or different weather my full vision for the game involved having a door that would have daily time-based events such as being served food at the same time every day or perhaps being able to interface with different mechanics through the door such as being handed a sponge to clean your dirty ass room or something unfortunately that was heavily beyond the scope of the jam so I cut it pretty much immedi immediately without the door the scope is small enough that I didn't really need to do any formal planning I could just keep it all in my brain as it stands the game is composed of two major components the ray Tracer and the time wizard which drives the sun cycle and the music I'm sure most of you are here for graphics so let's start with the ray tracing or how my game can run at 500 frames per second despite being driven by one of the most expensive techniques in the field am I goated or am I just lying to you but first something little different this video has been sponsored by brilliant brilliant is the best way to learn math data science and computer science interactively brilliant offers thousands of lessons from basic to Advanced so if you're just getting started learning math or are brushing up on your linear algebra for graphics programming brilliant has something for you even if you don't know where to start that's not a problem brilliant customizes its content to fit what you need just take a quick quiz when you sign up and Brilliant will match you with lessons that fit your interest and level of expertise I personally use brilliant whenever I need a quick and accessible refresher on math concepts I haven't worked with in a while such as complex numbers or some aspect of calculus I appreciate that brilliant makes it easy to find lessons on topics you're looking for be sure to try out everything brilliant has to offer with a free 30-day trial and 20% off an annual plan when you visit brilliant.org Rola or click the link in the description thank you so much to brilliant for sponsoring this video you might be wondering wondering why did I want to make a raay trace game instead of just like a normal game mainly cuz I don't actually have a complete Ray Tracer right now in fact I don't have a whole ton of experience with ra tracing beyond my old CPU R Tracer and a simple compute one from a tutorial I followed a few years ago also since I want this project to be a good demo for bounce lighting I'll need to write a ray Tracer for that anyways thankfully Ray tracing is really easy to understand imagine you're in a dark room you can't see anything because there's no light suddenly a light source appears and you realize you found yourself in a room full of distinctly different cardboard cutouts of Molly Ranken you hear a voice from behind say that you're not supposed to be here with the light turned on rays are being cast out in all directions bouncing around off surfaces till they run out of energy many of them reflect into your eyes enabling you to see the cardboard cutouts you could easily simulate this by generating a million rays of light in random directions from the light source and then whichever ones happen to bounce into the camera get rendered but that seems silly as 99.9% of the Rays would be wasted thankfully we can make our lives much easier by taking advantage of a principle proposed by Stokes and Herman helmholtz in the 1850s that describes how a ray of light and its reverse Ray experience the same Adventure this is known as the helmholtz reciprocity principle and it means that if we were to take one of these rays that bounces into the camera and cast a ray from the camera in that same direction we would end up with identical results because those two rays would experience the same exact reflection events so instead of doing our tracing from the light source to the camera we do it from the camera to the light source that way we only do work for the Rays that actually matter this magical optimization is not a new idea it goes all the way back to the 7s when Turner witted demonstrated recursive Ray tracing for the first time ever giving us the name witted style Ray tracing anyways that's the abstract but how do we actually implement these ideas obviously if we want to trace a scene we need to figure out how we actually represent the individual objects mathematically this generally comes in the form of Ray geometry intersection formulas such as Ray sphere Ray Cube or Ray triangle when we cast AR Ray we want to determine if it intersects with any objects by checking it against every object's intersection formula in the scene some of you may have had a terrible terrible realization just now because if all we have is a ray triangle intersection formula and complex 3D models are composed of individual triangles that does unfortunately mean we check against every single triangle of the mesh this is just one of the many reasons why Ray tracing is generally not very performant if we want a real-time Ray Tracer we're going to have to keep the scene geometry as simple as possible so I decided to limit my game to just cubes let's say we have a cube of side length one positioned at the origin and our camera positioned 3 m away is pointing towards the cube we can see on the graph that the camera is 2.5 M from the cube which we can use array Cube intersection formula to verify when we cast a ray from the camera we take the reciprocal of our Ray's Direction multiply it with the Ray's origin and negate it then we take the absolute value of the earlier reciprocal and multiply it with half the box size we add and subtract this to the earlier calculation to create the two potential intersection distances on each axis and take the Max and Min of each this first distance is what we want as it represents the distance from the ray to the nearest intersection of the cube if the near distance is greater than the far distance then there was no intersection coincidentally this value we have produced is equal to 2.5 the same distance we observed on our graph earlier this is pretty much all we need to Ray trce a scene of Cubes the intersection function tells us if we intersected an object in the scene and if we did it also tells us the distance of the intersection from the ray the reason we need to check every object in the scene is because we want to ensure we get the closest intersection instead of just the first intersection now that we figured out how to interact with our scene mathematically all that's left to do is draw the rest of the owl and do the lighting if you remember from other videos all we need for basic diffus lighting is a normal vector and the direction of the light source currently we have no normal Vector just the distance of intersection so we need to extend our intersection function to calculate that thankfully a cube is a pretty simple shape and only has six possible normal noral vectors three of which are just opposites of the others so it's only a problem of detecting which face of the cube we intersected with and we don't need to do any cringe calculus whichever axis we intersected the cube on we multiply with the negative sign of array Direction and we get the normal our lighting is then as simple as taking the dotproduct of the direction of the light source and the normal Vector at the intersection point and if we do that for every pixel of our screen we get a ray traced Cube we can easily add a Skybox by sampling a Cube map with our Ray direction if our Ray doesn't intersect with anything and adding more objects is as easy as checking for more Cube intersections buty Rolla this is supposed to be Ray traced where are all the cool Reflections at the moment we are only modeling diffuse surfaces but when a surface is shiny or reflective a light Ray that hits it is going to reflect off the surface and collide with another object or maybe go off into Infinity in which it effectively collides with the Skybox this is where the actual tracing of the ray comes in if we intersect it with a reflective surface then we want to calculate the reflected ray Direction and do all of our Ray tracing work again but from the intersection point in the direction of the reflection instead of from the camera each time we intersect with something we add the lighting at that point to a cumulative sum which will be the final output color as you can see all of our cubes have transformed into perfect mirrors in reality not everything is a perfect mirror and a a small amount of light will be absorbed by a surface each time a ray bounces off it so if we have our Ray keep track of its energy we can multiply it with the absorption of the surface each bounce to control the strength of the reflections or tint our objects towards certain metallics such as silver or gold our scene still doesn't look complete because there's no Shadows thankfully Shadows are pretty easy when we intersect with an object we can check if that point is in Shadow by casting a ray towards the light source and if we find an intersection we know an object is in between that point and the light source thus casting a shadow on that intersection Point that's about all there is to a simple Ray Tracer you might be confused what about soft Shadows what about global illumination and bounce lighting what about emissive surfaces unfortunately in reality things are much more complicated while the recursive style witted Ray tracing gives amazing Reflections it's far from photorealistic because unfortunately in the real world light bounces in more than one direction it bounces in many directions depending on the surface in order to get more photorealistic renders we have to delve into path tracing which involves calculating the light contribution at a single point for the entire hemisphere of possible incoming light directions to better approximate the rendering equation what's the rendering equation I don't know as you can see though when most people say Ray tracing they actually mean path tracing and because that involves doing lots and lots of recursive indefinite integrals it's clearly going to be far less performant than our simple Ray Tracer but because we've chosen to go back to basics our game renders very fast and all that's left to do is Implement Textures in order to texture our cubes we need UV coordinates which are what we use to map textures to the surface of 3D models currently we only have the distance of intersection and the normal Vector so we need to extend our intersection function to also calculate the UV coordinates of the cube at the point of intersection thankfully a cube is a pretty simple object so the UV coordinates are just the normalized coordinates of the intersection Point within the local bounds of the intersected face you have to change it up a little bit based on the axis of intersection but I'm sure you can figure it out with UV coordinates we can now do any of the usual texture things to get per pixel detail such as color textures or more importantly normal Maps which will enable us to make scenes look much nicer for instance this mud material turns our flat Cube surface into a convincingly 3D muddy plain and since it's Ray traced the normal map makes the reflections look very cool you might be wondering how people even make these materials everyone just seems to magically have a bunch of textures lying around the main tool used in the industry is Adobe substance painter which is a handy tool for layering a bunch of different procedural noise functions to satisfy any of your general surfacing needs I've never used Adobe substance painter before so it was one of my goals of the jam to use the tool to make all my own materials for whatever environment I ended up making so let's go over how I did all the art for my game since my goal is effectively a solitary confinement simulator I thought looking at some real world examples would be a good idea I originally planned for a concrete cell but I quickly realized it would be hard to make decent looking concrete materials since a lot of the appeal of concrete comes in degradation which if the geometry is totally flat it'll be hard to get that to look good with just normal Maps so I decided to go with a wooden cell which reminded me of a story of this japanesey guy who escaped from prison like four times despite the incredibly high security of each location his name was Yoshi shiratori and he was a career Gambler I went looking for reference pictures of the cells he escaped from and found a nice photo from the third jail he broke out of abashiri prison it's pretty much exactly what I was looking for a small wooden room with a single window before I made any materials in substance painter I wanted to block out the scene in unity first but there's kind of one major problem our Ray traced cubes technically don't exist exist and are hard coated into the Shader Unity has no idea about them in order to fix this I restructured the ray Tracer to operate on a buffer of cube positions sent over from the CPU and now we can take advantage of unity's level editor and existing Cube mesh to block out our scene and have it transferred to the ray Tracer with ease using the reference photo I crafted a little room and window with flat color cubes which looks quite boring at the moment but some textures will fix that over in substance painter the name of the game is layering different noise pattern patterns to create something visually interesting Adobe has a wood pattern noise function already and we can layer the brick pattern preset to create individual planks other than that it's just various dirt noise and color blends to produce some mild wear and tear on the wood as well as make it look old and slightly moldy I made three different variations one for the floor walls and smaller more processed pieces of wood we have yet another problem though how do we have each Cube know what texture to sample taking a look at the memory layout for each Cube it has a position size albo and specular component to support its base colors unfortunately we can't just tack on Textures to the struct because that's not how gpus work this analysis brings up an inefficiency though as a lot of cubes are going to have the same albo and specular components it's really just the position and size that differs to fix this we want to separate the material information from the cubes and instead have a dedicated material buffer that contains the unique materials in order to avoid tons of duplicates attached to each individual Cube then instead of having all of that info in the cube struct we can replace it with a material index that is then used to get the corresponding data out of the material buffer if the index is zero then it's the wood flooring and we sample the wood floor textures if the index is one then it's the wall textures and so on this isn't a huge optimization for our small scene but consider the case where there is thousands of cubes with the same material data after giving each Cube its proper material our solitary confinement cell is complete it's really dark though in the real world the Sun's light would be bouncing all over the room Illuminating it but since this is not a path Tracer we don't get to see that instead I just put a point light in the room to fake it this wouldn't be an ace roller game if it wasn't heavily post-processed so I also made use of my shaders from other videos to add some Bloom color correction tone mapping sharpness and anti-aliasing that I have yet to make a video on the art of our game is finished but that was still only half the project now we've got to figure out how to use the real world time to calculate the position of the sun to give us a realistic daily sun cycle after lots of reading and researching I Came Upon a research paper with perhaps the worst title in history a solar asmith formula that renders circumstantial treatment unnecessary without compromising mathematical rigor mathematical setup application and extension of a formula based on the subsolar point and aan 2 function here's a better title this is actually a great paper that explains everything pretty well but I'll give a general overview in order to get the position of the sun we need a time and position on the earth in terms of latitude and longitude while I could very suspiciously ask the player to turn on location data so the position is accurate to you personally that would be a little weird so I decided to choose where I live to be the Canon location of the game this is so that I could verify my code works by just going outside and looking at the sun myself since time zones exist all of this math is based on universal time or UTC which is which is the base reference used for all other time zones since I am on the west coast we need to take my local time and convert it to UTC before doing anything else which is as easy as adding 7 hours from this we calculate the number of days of terrestrial time from j2000 terrestrial time is an astronomical time standard for observations made from the surface of the Earth and j2000 is the current standard Epoch which essentially refers to January 1st 2000 so this is the number of days from the year 2000 accounting for leap years in such then we calculate the longitude of the Sun the mean anomaly of the Sun the ecliptic longitude the obliquity of the ecliptic the right Ascension of the Sun the declination of the Sun the distance of the sun from the earth and the equation of time what is the equation of time I don't know man Google it all of these pre-calculations enable us to finally calculate the latitude and longitude of the subsolar point which is the current position on Earth directly underneath the sun then we take our local latitude and longitude from earlier do some trig and we have the X Y and Z components of our sun Direction Vector this is the position of the sun in the sky at noon on January 1st 2024 in Bend Oregon I know this because I don't have to actually use my current local time I can use any time I want for debugging purposes using my handy dandy time wizard we can see that the sun rises and Falls as the hours go by and in the summertime the sun gets higher into the sky than it did in January one thing I really like is that every year the Ang angle of the sun changes just a little bit meaning that because we're using real time every single frame of our game is basically unique and can never be returned to as we March ever farther into the future since I didn't have time to model the moon I faked a sunset and sunrise by blending between a day and night Skybox creating perhaps the best Sunset effect in the industry with this the visuals are finished and all that's left is the music this may come as a surprise I was a band kid unsurprisingly I was also Al a theater kid a very unfortunate combination I played the flute for years but gave it up when I entered College recently I've gotten back into music in my free time and I've been taking piano lessons since November last [Music] year for the jam I wanted to apply what I've learned in lessons so far to try and compose some simple background music instead of defaulting to White Noise like I did for my other game so let's learn a little bit of music theory a note is one key on the piano for instance c a chord is composed of multiple notes there are unpleasant chords and pleasant chords there are lots of formulas for how chords are put together but I won't really bother with explaining that the main basic idea is major and minor chords which convey different feelings for instance the c major chord is composed of c e and G to get C minor we flatten the middle note meaning we move it a half step down to E flat in this case keys on a piano repeat every eight keys so if we count up eight from this first C we return to C but one octave higher meaning that if we play these same notes of C major at different octaves we're still playing the c major chord even though it's six notes instead of three moving on an arpeggio is when you play the notes of a chord individually instead of all at once the arpeggiated c major sounds like this but having only three notes isn't very exciting so we apply the idea from earlier and usually tack on the root from the next octave like so lastly a suspended chord is when we make use of the two notes a half step away from the two notes that form the major and minor chords for instance a c sus chord could sound like this or like this suspended chords are nice as a transitional chord into the main major or minor chords since the middle note is closer to the other notes there's a slight dissonance and tension that makes the next major or minor chord sound better because of its more proper Harmony and release of the implied tension from the dissonance anyways that's all the theory my favorite chord is at the moment is B minor I just think it's neat B minor is the central chord of songs like Arya of the soul from Persona the main Melody plays along a suspended B minor before resolving I wanted the background music for my game to be kind of droning and reinforc the theme of time so I took the idea of a clock alternating between two sounds and recorded a b and d octave to swap between each second since we're swapping between two of the three notes of the B minor chord it doesn't really resolve so to speak until the third note F SHP is played so playing a chord without resolution gives us that droning feel then to make use of our real-time mechanics I wanted a little Motif to play every minute and every hour so using ideas from Arya of the soul I made use of the sus chord for the minute Motif and finally at the turn of the hour I resolve the repetition and we go right back into the droning tick of the clock it's not very complex but hey this is my first composition on a time limit please cut me some slack with the ray tracing and realtime systems driving the sun and the music my goals for the jam were complete and all that's left was to decide on a name I'm not religious but I sure love the Aesthetics so I thought a lot about my game and the themes I was going for and I was reminded of the religious text filocalia the religious text known as filocalia is the foundational text of hesm meaning quietness Stillness or solitary life since my game is all about Solitude and Stillness it seemed like quite the fitting name with all that said and done I submitted my game to the jam on an alternate account so let's take a look at the feedback along with my submission I also proposed a bit of an ARG challenge to players if they could figure out where on Earth the room is from just the sun alone they'd win a special prize someone did figure it out congratulations to xyzt for determining it's an Oregon first they surprisingly are an astop Buist themselves but they used none of that knowledge because it turns out they just live in Oregon too my submission ultimately went mostly unnoticed since I didn't try to advertise it or push it at all Beyond sneaking it into one of my highlight lists but I got some lovely feedback from others such as not bad from Gyer Z I don't really see the gameplay value from TF Phoenix and this note left on the judging spreadsheet thanks guys this is understandable because if you don't know what makes my submission interesting then it's really just a fancy wooden great simulator thankfully I have spent this entire video explaining to you why it's cool actually and not super lame and boring if you're interested in the full judging stream as well as the trailers for the top 10 and honorable mentions they are all linked below or can be found in the Ace roller Jam playlist thank you so much to all who participated I'm sorry that I can't give everyone the recognition they deserve but I hope you continue to create new things and better your skills in preparation for the grand ace roll of jam 2025 and with all that in [Music] mind I am so excited to announce the Channel's first ever merch to celebrate the conclusion of the first ever Ace Rolla Jam from the artist who brought you the chibi Ace Rolla comes an exclusive poster design this event took about 4 months of work and I currently owe a lot of money to the government so if you would be so kind to buy a poster I would appreciate it very much as usual a huge thank you to all of my current patrons without your support I wouldn't be able to spend all of your money on Dune 2 IMAX tickets anyways that's all from me I hope you have a great rest of your day and I'll see you next time

Channel: Acerola

Views: 158,974

Keywords: Programming, Creative Coding, Educational, Computer Science, Algorithms, Essay, Video Essay, Acerola, Learning, Tutoring, Art, Generative Art, Technical Art, Physics, Simulation, How I Made, Shader, Shader Programming, Shader Code, HLSL, Game Development, Unity, Ray Tracing, Acerola Jam

Id: s-9UHZFBH08

Length: 26min 17sec (1577 seconds)

Published: Sun Apr 21 2024