An introduction to Raymarching

[Music] since the beginning of 3D computer Graphics one particular shape has been used as the building block for rendering complex scenes and objects triangles these three-sided shapes are at the core of the vast majority of 3D video games due to their many interesting properties it's the shape that can form a polygon with the lowest amount of vertices they are both computationally and memory efficient and all modern graphics cards are specifically designed to process triangles in parallel generally the more triangles you can process at a time the more detailed your 3D models and environments will be rendering all of these triangles on the screen is done through a process called rasterization which comprises three main stages in the first place a Vertex Shader will transform the vertices of each triangle from 3D World space coordinates into their 2D screen space projections using the current camera position and orientation the actual rasterization stage happens after this step and takes these screen space vertices in order to calculate which pixels of the screen also called fragments are actually part of the current triangle finally the fragment Shader processes each of these fragments to calculate their final color displayed on the screen this part generally uses informations from the vertices and external light sources to shade the pixel accordingly this process is done for each triangle and for each fragment that needs to be rendered to produce a single frame of course this is an oversimplification of the 3D rendering process there actually exist many more stages in the rendering Pipeline and numerous optimization techniques have emerged to minimize the number of triangles and fragments processed at each frame since the Advent of rasterization the objective has always been to draw more and more triangles on the screen simultaneously for larger and more detailed environments however alternative 3D rendering techniques also exist ones that don't necessarily rely on millions of Tri triangles to create beautiful and complex visuals this scene is currently being rendered in real time using a technique called Ray marching all of the objects are generated procedurally employing not millions or thousands but merely two triangles let's count them here's the first one and here's the second one using this technique the vertex Shader and rasterization stages become nearly insignificant in terms of performance given the mere two triangles and four vertices processed per frame now everything from the camera to the scene is calculated in the fragment Shader stage for every pixel of the screen but how to illustrate it I'll use this amazing Shader made by Canada that explains how Ray marching Works in shaders using Ray marching I've also used it as a base to create a lot of custom animations that you'll see throughout this video so in Ray marching the 2D screen will be projected into a 3D World directly inside a fragment Shader for each pixel we Cal calculate the direction of a ray that originates from a camera and that passes through the canvas we can use these Rays to calculate the intersection with the scene and colorize each pixel based on what they hit the term Ray marching gets its name from the way it calculates the intersection between array and the scene in contrast to Ray tracing which employs intersection functions to precisely determine where a ray hits a shape Ray marching uses distance functions to gradually step and March into the scene until an object is hit we can use them to calculate the distance to every object in the scene given a position in Space the distance to the closest one is used to safely March the ray in its current direction while being sure not to overshoot any object this process repeats until the closest distance becomes arbitrarily small indicating that we hit an object or until the ray extends too far out indicating that we didn't hit anything this simple process allows the rendering of any scene or object that can be defined by a distance function and it led to the creation of stunning worlds and effects that showcase the potential of this technique however despite its power Ray marching is rarely used as the main rendering method in large-scale games even if we're rendering only two triangles we still have to procedurally recalculate and render the entire world at each frame for millions of pixels and the computations can quickly become expensive Ray marching is more commonly used in video games on top of rasterization to render effects such as volumetric clouds or Global illumination this technique is also well suited for artistic purposes and mathematical renderings it can lead to infinite creative possibilities often in a few lines of code and in real time in this video I will be explaining how I created the following animation step by step while introducing some important concepts related to Ray marching so you can start rendering your own 3D scenes I will also share some useful resources at the end of of the video if you want to learn more this tutorial will be the building block for many of my next videos as I will go deeper into techniques related to Ray marching such as lighting and shadows Neons volumetric rendering terrain generation 3D fractals path tracing 3D path Trac fractals and every other amazing technique that I'll get to learn through my journey in the Shader realm and remember that this is only the ray marching branch of shaders there exist many more Concepts to play with I couldn't emphasize enough how much anything's possible within shaders from cellular automatas to neural networks you can find a remake of Minecraft or a functional Apple 1 emulator it really is a creative Playground now I will go to the Shader toy website and I'll create a new Shader I'll remove the default code and replace it to display the UV coordinates in clip space as explained in my first video I highly encourage you to watch it as I explain the very basics of Shader programming and the glsl language and won't go into as much detail for the steps that I've already covered now let's delve into the Practical steps of Ray Marching In traditional rasterization we usually Define the camera and objects beforehand and send them as inputs to the rendering pipeline but here we have neither a camera nor objects as we're directly working inside a fragment Shader so we'll have to create them for each pixel the very first step consists of creating array origin and direction for each pixel in order to do so let's enumerate some constants first that will help us represent objects in our world we will use three component vectors to denote points in 3D space and the world's origin is the Point located at 00 0 we also need to Define three orthogonal axes that will form the basis of our Vector space for Simplicity let's define the X and Y AIS of our world so that they align with the canvas's X and Y AIS represented by our UV Vector consequently the additional Z axxis will represent the depth from the the canvas note that we don't actually need to Define our origin or these vectors in code we only need to visualize them and know what they are to allow for a more intuitive way to build and interact with the scene with that in mind let's define the ray origin Vector it corresponds to the camera's current position in our world for our initial setup let's place it three units behind the origin along the negative Za axis let's now create a vector to represent the current pixel's Ray Direction initially pointing straight in the Z Direction since all Rays originate from the same point giving them the same direction would result in them following identical paths and having the same color to make each pixel's Ray unique we can leverage the UV Vector which is in clip space directly as the X and Y components of our Ray Direction this works because we've decided earlier to align our world's X and Y AIS with our UV coordinates this modification will make each pixel have a unique Ray Direction spreading out from the center of the screen and effectively creating a virtual canvas to project the world to finally we need to normalize the ray Direction normalization scales the vectors to ensure they all have a length of one which is crucial for accurate distance calculations in the next step now let's introduce a float variable to keep track of the current distance our Point has traveled from the camera's origin this distance will increment with each step starting from zero and will ultimately store the accumulated distance traveled by the ray across the world creating array origin Direction and travel distance can be seen as the initialization process and I'll indicate it with a comment the next part will involve the actual Ray marching we are going to Define one step of the algorithm that will then be iterated using a loop until an object is hit now our primary interest lies in determining the position of the current marching Point rather than just its distance from the camera luckily calculating the position at each step is straightforward using our current variables we'll Define a new Point called P that will be located at the Ray's origin and add to it the ray Direction times the current distance traveled this gives us a 3D Point representing the current position along the ray based on its distance from the Ray's origin we can then use this point to compute the distance to the closest object in the scene note that it actually starts at the ray origin as T is initialized to zero to do so we will use sign distance functions also called SDF these functions can be us used to obtain the distance from a given point in space to a particular shape in the scene I would highly recommend you to check out this article written by the master of shaders in Nigo quiles that provides some useful distance functions and tricks related to Ray marching so to define the objects in our scene we will create a function that takes a 3D Point as input and Returns the distance to the nearest object in the scene we'll come back to it later and for now it will only return the distance to a single sphere of radius one located ated at the origin we can then use this function to get the distance to the scene given our current position along the ray which I will store in a new variable this value represents the safe distance the point can travel to in any direction without overstepping an object finally we need to add this distance to the variable that keeps track of the total distance traveled from the Ray's origin this is the step where we March the ray we can repeat these steps by encapsulating this code within a for loop with the variable I representing the current iteration or Ray marching step for now I'll set it to a maximum of 80 iterations this number affects both the quality of the result and the performance too few iterations might result in Rays not having enough iterations to reach an object while too many can significantly impact Shader performance typically you'll need to adjust this parameter to suit your specific scene in summary for each pixel we start by initializing array origin array Direction and a travel distance distance these parameters help us calculate an intermediate 3D Point representing the current position along the ray since the distance is initially set to zero the point starts at the Ray's origin which is the camera's position from this position we calculate the distance to the closest object in the scene which is used to safely March the ray forward we then repeat this process until we either hit an object or exceed the number of Ray marching steps we've set and with this code we're just one line away from displaying our first 3D object on the screen I will create a new Vector that will contain the current pixel's color and make it the new output of our Shader instead of displaying the UV coordinates now the easiest way to visualize something in a ray marched scene is to colorize pixels based on the total distance traveled by the ray which is stored in the variable T I will assign this value to the red green and blue components of our color Vector to create a grayscale image as a reminder the shaders output color ranges from 0 to 1 therefore for as soon as our Ray travels a distance greater than one it will be displayed as white and we won't see any variation which is currently happening since the camera is 3 units away from the origin to address this we can either move the camera closer or divide the distance variable by an arbitrary number to start seeing details further away and there you have it the unit sphere that we defined in the map function is currently being rendered using Ray marching what we're observing can also be referred to as a zbuffer or de buffer as each pixel's color represents the distance from the camera to the scene in grayscale value before we continue let's optimize this code to prevent our graphics cards from performing hundreds of unnecessary calculations which they'll very enthusiastically do if not constrained for instance each Ray currently goes through the entire 80 iterations but in reality it can take far fewer iterations before the distance increase for any consecutive Ray becomes too small to see a difference let's visualize it instead of assigning the output color based on the distance traveled I'll display the total number of iterations that happened I'll divide it by 80 in order to map the value to the 01 range from black to white as expected all pixels display a white color meaning that they went through the entire 80 iterations so let's add an early exit condition to the ray marching process this condition instructs the loop to stop if the current distance to the scene becomes smaller than a certain threshold let's say 0. 1 you can experiment with different values for this parameter but if you set it too high the ray may stop too far away from an object's surface leading to less precise shapes on the other hand if it's set too low the performance gain may become negligible with this modification we can now see that the rays which actually hit the sphere don't go through the entire 80 iterations but a way smaller count given their shade however the other rays will continue through the full 80 iterations extending dramatically far into the distance to put the things in perspective I've written a small JavaScript function that calculates for each step the distance traveled by a ray that barely misses a sphere for the first 14 steps it doesn't go further than 10 m away from its starting point but then things start to take an exponential turn as only eight steps later we're going past the kilometer Mark and once we've reached the 40th iteration the Ray has nearly traveled the distance that separates earth from the moon finally at the end of its trip 80 iterations and 36,00 light years later the Ray would have traveled a third of the Milky Way at speeds that general relativity cannot comprehend it's highly unlikely that we'll need to visualize objects at such extreme distances so let's add another exit condition that halts the loop if the current distance traveled by the ray exceeds a certain value we don't need to calculate this distance precisely given how fast the ray exits the scene instead we can set it reasonably far away to ensure that objects in our scene won't get clipped or cut off prematurely and as a result we can observe a significant reduction in the iteration count for rays that don't hit the sphere let's now group these two optimizations in one single line and go back to showing the depth buffer instead of the iteration count we finally reached the point where we have our fundamental Ray marching algorithm which can be very similar to a blank canvas in a 2d case it serves as the starting point from which I typically begin to create explore and experiment with Ray marching and you can find this Shader in the description there are many Concepts techniques or hacks that can be used together to create mesmerizing scenes and I'll demonstrate a few our attention will now be directed to the map function that we defined earlier to make things clearer I will create a new variable containing the distance to our sphere and call a function that computes its sign distance let's start by exploring translation we have the ability to manipulate the positions of objects in our scene by adjusting the input position that we send to the distance functions let's introduce a new Vector that represents the sphere's position in our 3D space initially set to zero I will subtract this Vector from the input position before sending it to the SDF now it is possible to move the sphere in the scene in all three axis that we've defined earlier by adjusting this Vector for example let's move the sphere 3 units in the positive X Direction This calculation may seem a bit counterintuitive as we actually need to subtract the vector in order to move in the positive direction to understand why let's visualize what happens to the Ray's Direction during a translation when we perform the subtraction as described we're essentially Shifting The Ray three units in the negative Direction resulting in a movement to the left relative to the world but if we replay the same animation relative to the ray itself you'll notice that it seems as if the scene is moving in the opposite direction from left to right this explains why translation and other transforms will will usually appear inverted in shaders finally it's very easy to add realtime movements by including the current time in the object's position in this case I use a sign function of time to make the sphere's exposition fluctuate betweenus 3 and 3 as a quick note the Distortion we're observing when the sphere approaches the edges of the screen is inherent to prospective projection which we're implicitly using this Distortion is connected to the camera's field of view which indicates the total angle covered by the Rays a larger field of view lets you see more of the scene simultaneously but also intensifies the Distortion if you'd like to control the field of view you can multiply the X and Y AIS of the ray Direction before normalization to influence how the rays are spread now let's add another object into the scene I will Define a cube of side 75 located at the origin in order to combine these shapes we simply have to return the minimum between the two distances using the Min function you can think of this line as an operation that unites the two shapes also called the union operator if you want to get creative you can experiment with other operators like the differ or intersection operators to combine shapes in more interesting ways additionally there are smoother versions of these operators that allow for a gradual blending between the shapes I'll use the smooth minimum function which includes an additional parameter for adjusting the blending allowing us to create a smooth Union between these shapes finally I will quickly add a ground by creating a new variable that holds the signed distance function to the XZ plane which is quite easy to calculate let's visualize one Ray coming out of the camera the distance between any point along this Ray in the ground is directly determined by the Ray's current y position which represents its height above the ground I will also push the ground further down by adding an offset of 75 to its distance otherwise we won't see anything as the camera would be at the exact same height as the ground and intersecting with it once again we add a positive number to move the ground in the negative y direction it's also possible to smoothly Union the ground with the other shapes this time I'll use a lower blending amount another transformation that we can easily apply is scaling while these two shapes can actually be resized using their input parameters scaling remains a valuable operation for more complex or grouped objects to execute the scaling operation you simply need to divide or multiply the input position much like the translation operation the resulting size change of an object is inversely proportional to the scaling Factor however scaling actually distorts the metrics and distances within our scene unlike translation which can introduce artifacts in the rendering to address this we need to counteract the scaling effect this involves multiplying the final result of our distance function by the inverse of the scaling factor or in simpler terms dividing the output by the scaling Factor this adjustment ensures that distances in our our scene remain accurate and removes artifacts as a final note here I've made the scaling uniform but you could also multiply the position by a vector in order to apply a different scale factor for each component I'll remove the scaling on the cube and now I'll introduce the last basic transform that we can apply which is rotation this one generally involves the call to an external function as it requires a bit more math to rotate a point in space you need to multiply it by a 2x2 rotation Matrix in the 2D case or a 3X3 rotation Matrix in three dimensions as you can see there's a gap in the complexity of the computations when adding another dimension fortunately in the 3D case there exists a simpler alternative called the Rodriguez rotation formula which achieves the same result without using matrices I won't go into the math Behind These functions but I'll link in the description some resources if you want to get a better understanding about vector rotation let's start with the 3D scenario this function takes as input a point in space an axis and an angle of rotation in radians the axis is a vector representing the 3D line around which the object will rotate and it must be normalized we can use it to rotate objects in any direction but I often prefer using 2D rotation functions even in 3D scenes the main reason being their speed and simplicity however this function only operates on vectors of two components in order to use it we'll need to extract two components of our 3D Point using swizzling syntax this constrains the rotation to be around the three fundamental axes only but it can be chained around multiple axes to allow for a greater range of rotation once you can visualize the three primary axes in space using this function to rotate objects becomes quite straightforward in the swizzling syntax the component that is omitted represents the axis around which our object will rotate to demonstrate it I will create a copy of the input position in order to rotate that copy only and pass it to the Box distance function if if I wanted to rotate the Box around the z-axis I would exclude the Z component in the swizzling syntax and write Q doxy I can also use time to animate this rotation now one aspect that is crucial in Ray marching is the order of operations if you translate first and then rotate you'll get a different outcome compared to rotating first and then translating this rule applies to most transformations in Ray marching so if something isn't behaving as expected double check the order in which you apply the operations it's also important to understand that when we apply a transformation to the input point it maintains that transformation for all future distance functions this is why I've created a new variable to rotate only the box if I were to remove this temporary variable you'd notice that the ground also starts rotating however the sphere which is defined before we apply rotation retains its original movement now it would be even better if we could add Mouse control for the camera to view the scene from different angles this can easily be done using the 2D rotation function I introduced along with the current position of the mouse which is accessible on Shader toy through the imouse global variable it's a four component Vector where the first two components represent the current pixel coordinates of the mouse much like the frag chord input parameter the last two components contain information about the starting position and state of the mouse but we're only interested in the mouse position here which we can extract using thexy syntax to get the mouse position in clip space I'll create a new vector by applying the same transformation as our UV coordinates ensuring they are in the same range this new Vector will have a value of Min -1 minus one when the mouse is at the bottom left 1 one at the top right and it will be zero when the mouse is centered next let's rotate both the Rays origin and Direction around the Y AIS the rotation angle will be determined by the current exp position of the mouse Vector which needs to be negated if you now click on the C canvas and move your mouse you'll observe that you can rotate the camera around the scene horizontally to grasp the mechanics behind this let's visualize what happens when we rotate the ray origin only you can see that the camera indeed rotates around the center of the scene but it's always pointing in the same direction this is why we also need to apply this rotation to the ray Direction when combined together it will make the camera rotate around the scene while always pointing at the origin for a complete 360° rotation around our scene we can apply the same technique along the x-axis this time using the vertical position of the mouse the order of operations matters and it's important to place it before the horizontal rotation you can also scale the mouse Vector to control how sensitive the rotation is now let me introduce the concept of space repetition which is an extremely powerful tool in Ray marching which allows you to render an infinite amount of shapes while defining just one similar to my previous explanation of of space repetition we can employ the fra function to replicate objects in space this function Returns the fractional part of the input for each component keeping the result between 0 and one to visualize what happens when we pass the ray through this function let's consider a 2d scene defined by the SDF of a box the white line represents array as we previously defined it here it's starting very close to the box but not intersecting with it the yellow line represents the same Ray after going through the fra function while the original Ray remains unaffected by this transformation the input to the SDF is always constrained within the unit cell at the origin from the Rays perspective this makes the scene appear to contain an infinite number of Cubes each one unit apart from the other to achieve more granular control it's possible to use the mod function instead which takes an additional parameter that can be used to control the spacing between each repetition fract being a special case of the mod function note that the current VIs visualization is missing an important step as the cube is originally located at zero while the center of repetition is located at 0.5 this makes every Cube extend further than the repeated cell boundaries and it results in them being clipped this is why we also need to subtract 0.5 after applying the space repetition function this translates each Cube to the center of its cell and fixes clipping issues finally I'll scale down the cube because it currently has a bigger side length than the distance of repetition which makes it fill the entire space and using this simple Technique we have achieved Infinite Space repetition in all three axes the render is only limited by the maximum iteration count and distance limit set earlier note that it would also be possible to apply space repetition to individual axes instead of all three of them moreover we can lower the intensity of the depth value to see further away as the colors are still clipped above one we could also add an upward movement to every Box by off setting the Y position of our point before applying space repetition once again if we were to add it afterwards the boxes would clip out of their repetition and now that we've covered all of these topics we can start to get creative with our scene I will start by removing everything from the map function apart from the repeated box to which I will add a movement in the z-axis towards the camera so far we've seen how to make AR Ray marching algorithm and how to play with objects in our scene but it's still colorized using the depth to the scene which is a bit boring the next easiest way to colorize objects is to pass the depth value as the input to a color gradient function that can convert this linearly increasing value to a color Vector I introduced a handy one in my previous video that can be tweaked to produce many different gradients so I'll use it again let's go back to the colorization part I'll call the pallet function to calculate the output color and pass the depth as input and with this one simple line our previously monochromatic scene takes a whole new dimension to it I will now replace the Box distance function to the one of an octahedron because I find this shape more interesting and it's also a bit faster to compute I will also decrease the spacing in the Z direction to make each shape closer together only in the axis that is parallel to the camera for that I'll decompose the space repetition in two parts the X and Y axis will be repeated using fract as before and I'll use the mod function with a spacing of 25 for the Z axis I also need to offset it by half of the repetition spacing which is125 in this case now you may notice that the current colorization technique makes the objects appear very flat as we didn't Implement any lighting or occlusion effect a quick solution would be to incorporate the iteration count from the loop into the pallet function this value actually contains information about the scene as it will increase with distance but also increase When approaching edges of objects as it will need more step steps to travel the scene if we extract it from the loop and display it directly after normalizing it we can see how much information it contains this is only a hack because the iteration count is not smooth as opposed to the depth buffer the lower the iteration count the more visible the artifacts will be this is why I will multiply it with a very tiny number so the addition stays subtle I also need to use the float keyword to convert it from integer type to floating Point type the change is Tiny But adds a lot to the depth of the scene in my opinion to add a dynamic touch let's start to distort the scene using the sign function like we've seen earlier we can apply transformation to the ray itself in order to affect how objects appear in our scene in this case I will add a sinusoidal offset to the Y component of the ray which will distort the scene in a unique fashion I'll add this effect in the main Loop before the map function to indicate that it's a scene transformation and not object specific so I'll add the sign function to the Y component of the position along the Ray and the frequency will be determined by the total distance traveled so far I'll also scale down the amplitude of the waves for a cleaner effect it's starting to look very interesting and remember that our scene is actually still defined by a single undistorted octahedron located at the origin it's just our rays that were given superpowers to completely ignore the rules of 3D cartisian space and somewhat hallucinate new geometry next I'll completely remove the camera rotation code as I found it more interesting to control the frequency of The Wiggles using the vertical component of the mouse I want it to be going from 0 to 1 so I need to remap the Y component accordingly this allows me to change the frequency of the oscillations by moving the mouse up and down now I will use the same Distortion principle this time not to control the offset but to control the rotation of the ray position I'll go back to the previous visualization and this time I'll draw four Rays instead of of just one I want these Rays to start rotating around the z-axis the further from the camera they go this can be done using the previous 2D rotation function like I explained earlier as I want the rotation to be around the z-axis I will extract the X and Y components of the point the rotation increases with distance producing this stunning spiral effect that demonstrates once again how mesmerizing yet simple mathematics can sometimes be note that these two Transformations actually break the distance calculations similar to the scaling operator if you distort the space too much visual artifacts will start to appear finally I can use the X component of the mouse to control the frequency of this rotation I won't remap it here so it ranges between minus1 and one from left to right allowing me to change the direction of rotation when combined with the vertical movement of the mouse it's possible to explore various and interesting twists for the scene as a last step I found it to be particularly interesting when moving the mouse in a circle around the canvas with this setup so when the mouse is not pressed I'll add a default behavior that simulates the mouse going in a circle motion this is straightforward using trigonometry the position of a point along a circle given a radius and an angle is the following as we're in clip space the radius is one and I can simply use time to have an infinite looping rotation let's add it to The Code by overwriting the value of the mouse Vector when the mouse is not pressed which we can detect by checking if the Z component of IM Mouse is negative this addition will fake an endless circular motion of the mouse around the canvas to give a final touch of dynamism to the scene I believe that now would be the appropriate time to sit back take a break from the code and appreciate the journey that started from a blank canvas and led to this intricate world we can now immerse ourselves in this mesmerizing scene as the different parameters evolve and synchronize in real time to produce beautiful geometric patterns there's so much more to cover but I hope this gave you enough foundations to start playing with Ray marching and become a ray Bender yourself the door is now open to many interesting 3D rendering techniques that I will cover in some of my next videos if you want to support these tutorials I've created a patreon that I will use to share updates about what I'm working on and ask you for what you'd want the next video to be about in the meantime if you'd like to learn more about Ray marching I would suggest exploring the art of codes Channel specifically his Ray marching tips and tricks video lets you discover alternative methods for twisting space and how to address some common ray marching artifacts moreover if you happen to have some free time watching the replays of enigo quil's live coding sessions can be incredibly enriching too I gained valuable insights from these sessions that I believe can serve as a fantastic Learning Resource for you as well happy creative coding and see you in the next [Music] one [Music]

Channel: kishimisu

Views: 125,634

Keywords: ray, march, marching, marcher, raymarching, ray-marching, vertex, shader, rasterization, space, transform, transformation, repetition, glsl, coding, code, tutorial, beginner, advanced, shadertoy, fragment, triangle, triangles, procedural, generation, art, world, 3d, mathematics, manim, camera, pipeline, render, rendering, computer, graphics, video, game, realistic

Id: khblXafu7iA

Length: 34min 2sec (2042 seconds)

Published: Wed Dec 20 2023