Rendering Water With Sine Waves

this video has been sponsored by brilliant hello everyone well I'm all out of video ideas so let's consult the tones [Music] I guess that works water everyone loves water especially me I love water 23 years ago I was born in the dregs of the United States also known as Portland Oregon Portland receives lots of rainfall so the aesthetic of rainy weather and flooding streets is something my daydreaming mind reaches for most often when trying to visually construct an interesting scene three years ago when I was a fledgling Graphics programmer I dreamed of creating my own real-time fluid simulator with legitimate fluid accumulation from falling rain and accurate rain splash physics then I realized that partial differential equations really do have hands and that real-time fluid simulator is a bit of an oxymoron unfortunately for me I'll probably die before large-scale real-time fluid simulation is possible but if fluid simulation is too expensive for games then how is everyone rendering their water if you went to school then you know what a sine wave is but many people lack an intuitive understanding of them and if you're an aspiring game developer you should start learning to love the sine wave thankfully they're very simple sine waves are oscillators they repeatedly move back and forth between two different values this is useful since we know for sure that no matter what point in time we sample our wave it will always be somewhere in between its Peaks the peak value of the wave is known as its amplitude since the amplitude of a basic wave is 1 we can easily change it to whatever we want by multiplying The Wave by the maximum value we desire the distance between two peaks of a wave is known as its wavelength the shorter the wavelength the faster the signal will repeat in order to get a desired wavelength we need to convert it to frequency which is the rate at which a wave repeats every second thankfully we can get our waves frequency by taking the quotient of 2 over the desired wavelength and multiplying it with our positional coordinate with a wavelength of one our wave now completes a full repetition in between zero and one and if we consider the x-axis to be time then our wave repeats every second this is our core wave equation but it's not very useful to us game developers without one more crucial component the phase at the moment we can't animate anything with our wave because it's not going anywhere in order to move our wave forward through time we need to add time as a second variable and add it to the inside of our wave function we can then control the speed of our Wave by expressing speed as a phase constant we multiply our desired speed by 2 over the wavelength and then multiply it with our time variable since the waves frequency is also equal to 2 over the wavelength then our speed parameter is in terms of frequency it is with this basic wave equation that the secrets of water rendering are revealed to us in chapter 1 of the first edition of GPU gems making use of a technique called the sum of signs GPU gems tells us that one wave is boring but many waves now we're getting somewhere the sum of signs is fairly self-explanatory it's a it's a sum of sine waves if we add two of the same wave together nothing that interesting happens but if we add a wave to a wave with different amplitude frequency and speed we begin to see some more interesting patterns the more waves we add together the more detailed of an animation we get with just four waves we can already see some convincing fluid-like movements in 2D but what about 3D over in unity I have a simple plane composed of many vertices in order to see our waves we need to displace the vertices according to the sum of signs the easiest way to do this would be to have our CPU iterate through every vertex of the mesh updating its position every frame but for a plan with this many vertices that's perhaps the worst idea you could possibly have instead we'll Leverage The immense power of the GPU to do our displacement for us in the first programmable stage of the rendering pipeline the vertex Shader when the CPU tells the GPU to draw an object the object's vertex Shader is executed on every single vertex of its mesh the vertex Shader is responsible for finalizing per vertex data that will be sent to the next stages of the render pipeline which includes finalizing the position of our vertices this is the functionality we're interested in for the water we have our vertex Shader calculate the sum of signs on each vertex displace it accordingly and then send it off to the rest of the render pipeline as you can see our mesh is now animated with the vertex Shader doing all of the work for us and it's very fast too especially when you compare it with the CPU approach but Ace roller it doesn't look like water at all that's because we still need to implement the second major stage of the rendering pipeline the pixel Shader the pixel Shader is responsible for outputting the pixel color on the screen right now our pixel Shader is returning white for every pixel of our water resulting in this formless white blob to give our water form and detail we need to calculate how the light interacts with any given point on the surface of our water there are many intimidating equations I could show on the screen right now but let's start with the basic calculation that is the core of every physically based Shader the lamb inversion diffuse imagine we have a sphere with a light shining on it if I ask you to draw in the shading then it would probably look something like this the points on the sphere that point towards the light are fully lit the points that point somewhat away are less lit and the points that point opposite to the light are fully in Shadow it's from this that we could Intuit that there is a relationship between the direction of the light source and the direction of the surface also known as the surface normal a vector that is orthogonal to the surface we can see that the greater the angle between these two vectors the less the light is scattered off the surface conveniently the cosine function will give us a value of 1 when the angle between the vectors is zero and will give us negative one if the vectors are pointing opposite from each other for simplicity's sake we clamp the cosine value to zero because negative light doesn't exist but how do we get the angle between the vectors if we look at the equation for the dot product of two vectors we can see that the dot product is equal to the product of the vectors magnitudes times the cosine of the angle between the two vectors since our light Direction and surface normal are normalized vectors their magnitudes are both equal to one so all that's left is the cosine of the angle this means that if we take the dot product of our light Direction and surface normal we are given a value between 0 and 1 that describes how bright the point on the surface of the sphere should be and we have successfully derived what's known as Lambert's cosine law this concept was introduced back in 1760 by Johann Lambert and his book photometria hence why we call it the lambertian diffuse we've already gotten ahead of ourselves though because if we go back to our water you might start asking the question where are the normal vectors like I said earlier a normal Vector is a vector that points orthogonal to the surface at any given point in order to calculate a normal Vector we need two other vectors the tangent and the binormal the tangent Vector is the tangent line of the point on the x-axis and the binormal is the tangent line on the Z axis the cross product of these two vectors produces the normal Vector to get our tangent and binormal vectors we need to find the slope of our point in the x and z directions we can easily do this the old-fashioned way by sampling neighboring points in our sum of sine's height map and then calculating the slopes with the usual rise over run formula that you learned in the third grade this method is known as calculating normals through Central difference it's a very common technique because most of the time you can't do anything better Central difference has a number of drawbacks though for one the accuracy of our normals is limited by the distance of our samples another drawback is that it's expensive if we aren't pre-computing our sum of signs into a texture then in order to calculate our Central difference we need to recalculate the sum of signs four extra times to get our neighboring samples so basically we're doing a lot of extra work to get a normal Vector that's an approximation and doesn't always look good normally I'd say this is the best we can do but for once that's not the case in fact we can get pixel perfect normals by making use of the power of calculus as a quick recap we are creating our normal vectors through Central difference which means we are estimating the slope along the x and z axes but why would we estimate the slope when we could just calculate it so far I have been using the word slope to avoid the use of the word partial derivative but that's exactly what we've been approximating the partial derivatives of our Point since the surface of our water is defined by an exact function for once this means we can use calculus to find the exact partial derivatives of our water for a single wave the derivative of sine is cosine to get the partial derivative we multiply the cosine by the direction of the wave and the axis we want then we construct our tangent and binormal vectors and take the cross product to create our Pixel Perfect normal vectors now all we have to do if you remember from earlier is take the dot product of our normal Vector with the direction of our sun and we have successfully calculated the lamp version diffuse but Mr what about the normal vectors for the sum of signs what a great question it turns out that the derivative of a sum is the sum of the derivatives so we just calculate the derivative of each wave and add it to a sum as we go along to get our partial derivatives with Calculus our normals are now exact not an approximation and we're doing half the work so we are using less resources it's a total win-win we can change the color of the Water by multiplying the diffuse term with any color we want like blue wow look at that beautiful water you know many Ace roller viewers ask the question how do I become a graphics programmer well hold your horses partner before you learn to program a graphics you've got to learn how to program in general which is why I'm excited to tell you about a free and easy way to get started on your educational Journey brilliant is the best way to learn math data science and computer science interactively brilliant offers thousands of lessons from basic to Advanced so if you're just getting started learning math or brushing up on your linear algebra for shaders brilliant has something for you don't know what your skill level is brilliant customizes content to fit what you need just take a quick quiz when you sign up and Brilliant will match you with lessons that fit your interests and level of expertise I was privileged enough to have a formal math education but my last math class was four years ago so I've been taking advantage of Brilliance math lessons to re-familiarize myself with the basics of calculus there are lesson plans let you go at your own pace and I'm the kind of person that has to read something 20 times over to understand it so I appreciate that a lot be sure to try out everything brilliant has to offer with a free 30-day trial and 20 off an annual plan when you visit brilliant.org forward slash Ace roller or click the link in the description thanks so much to brilliant for sponsoring this video now let's make the water actually look like water the lambertian diffuse is just one piece of the lighting puzzle it models the light that is scattered in all sorts of directions after hitting a surface but there's one more important thing that happens when the light Ray hits a surface some of it reflects light rays from a source that reflect directly into your eyes are not just annoying they are also referred to formally as specular Reflections so how would we add them to our watersheder imagine a point on our water we know it's normal Vector we know the direction of our light source and we also know the direction we are viewing the water from since these are reflections then intuitively the specular highlight should be strongest when our view Vector aligns with the reflection Vector from the light source we can very easily tell if this is the case by calculating a halfway Vector between the view and light vectors if we then take the dot product of the halfway Vector with the surface normal we'll be given our specular highlights but why does this work if you recall from earlier the dot product of two normalized vectors is the cosine of the angle between them and if that angle is zero then cosine outputs a one if the view direction is the same as the reflection Vector of the light Direction then their halfway Vector will be equal to the surface normal giving us a DOT product output of 1 meaning a full strength reflection we can then control the size of our highlights by putting an exponent on our DOT product this model for specular highlights is the most well-known and most common method known as the blinfong specular proposed in 1977 in fact the combination of the lambertian diffuse and the blinfong specular along with a constant ambient light value is the most basic and introductory lighting model for physically based rendering which video games used for a very very long time with specular highlights our water is one step closer to looking like actual water but at this point we're no longer held back by our lighting instead we need to bring our water animation to the modern era the sum of science method presented in GPU gems is a great Baseline it's perfect for small bodies of water that are not that turbulent where too much detail would look unnatural the basic sine wave works great here but if we want to model waves with sharper Peaks like that of real-life waves we need to make some modifications to the sine wave the sum of signs in reality is just a sum of any kind of wave we could use cosine if we wanted to it really doesn't matter so any modification we make to our sine wave the behavior Remains the Same we create a bunch of them and then add them all together there's infinitely many variations of the sine wave to give it sharper Peaks the one you are probably most familiar with is the gersner wave but I'm personally not a fan of them as they are really hard to work with they're overly complicated and the parameters of the gersner wave are very finicky meaning it's easy to create a wave that curls in on on itself which completely ruins the visuals instead I'll be making use of a much simpler variation of the sine wave where the wave is the exponent on Euler's number as you can see we have sharper Peaks and wider troughs like you would expect a water wave to have and we can easily control the size of the Peaks without fear of it accidentally ruining the visuals like gerstner waves this does complicate our derivative a bit more but that's to be expected with our new wave equation the sharper Peaks are immediately pretty obvious and add some nice detail to the water but our sum of signs is suffering from one other major issue if we want more detailed water then we need to add more waves but adding more waves leads to some very ugly visuals without compensating by reducing the amplitudes of every wave in order to fix this we're going to make use of one of the most important algorithms in graphics fractional Brownian motion Brownian motion is the term we use to describe a movement that is completely random at the moment our sum of signs could be considered brownie in motion since we are summing up bunch of random signals to create what is effectively white noise that happens to vaguely look like fluid movement fractional Brownian motion on the other hand is very similar but instead of being entirely random there's some memory to the process it's kind of hard to explain so let's just get into how it works we start with a wave with an amplitude and frequency of one and a random direction we sample our wave add it to a sum calculate the derivative and add it to a different sum just like we've been doing for our sum of signs but now unlike before we are going to multiply the current frequency by a number greater than one like 1.18 for example and multiply the current amplitude by a number less than one like 0.82 then on the next iteration of our Loop we use the new frequency and amplitude for the next wave that has a new random direction we can do this for as many waves as we want but eventually the amplitude is going to approach zero so adding more waves won't do much that's precisely the point though we start with a big wave and we are iteratively adding a wave of higher frequency but with a smaller amplitude each time these higher frequencies are going to be the source of our finer details but will have less and less of an impact at higher wave counts in order to prevent complete chaos like with our normal sum of signs the fractional Brownian motion allows us to easily control how detailed we want our water to be at lower wave counts we have a more stylized look that's similar to the basic sum of signs but at higher wave counts we have very fine details with precise normals that makes for a convincingly realistic looking surface is it actually realistic no not at all but people are very bad at identifying realism myself included which is why I spend 75 percent of my time staring at random light interactions around me and asking myself the question if I saw it in a game what I think it's realistic and the answer is usually no anyways the last thing we can do to improve our water animation is something called domain warping while we're summing our waves we can push the position we are sampling our wave at by the derivative of the previous wave that way it's almost like the waves are pushing each other around it's a very simple change but it makes a huge difference our water asset is finished now it's time to make it look cool a large body of water requires a large plane so let's heavily increase the size of our water we set the ambient light diffused light and specular highlights to appealing colors then we spend the next hour fiddling with the water parameters to get a nice animation The Horizon cutoff on the water looks ugly we fix it by applying a distance fog post process we attenuate the distance fog based on height to make it look more like atmospheric scattering these specular highlights look unnatural because there's no sun to go with them we can create a fake Sun by taking the dot product of our view Vector with the sun Direction and additively blending it with the source then we can find a Sky Box off the internet to get a better looking Sky the Skybox is going to do wonders for our water because of the beautiful environment Reflections wait a second I forgot the reflections the appearance of water especially still water is almost entirely composed of Reflections without them our water won't really look like water when it comes to real-time Graphics we have three General options for Reflections the first is Ray tracing Ray tracing gives perfect Reflections but it's extremely expensive and I'm still working on a 1660 which doesn't have any Ray tracing Hardware so clearly this option is not valid the second option is screen space Reflections Gamers love to complain about Bloom motion blur and other effects but what they should really be complaining about is screen space Reflections this is by far the worst image effect that is commonly used in most games today screen space Reflections are a complete and total sham they are extremely unrealistic and if you know how to break them and it's really easy to break them it immediately destroys the visuals of the game this is because screen space Reflections can only reflect what is currently on screen so if something that is being reflected moves out of view the reflection disappears here's an example from Final Fantasy 16 which came out just a few weeks ago the only reason they are used is because they're the cheapest option by a very wide margin and if you don't know any better then they look convincing so I'm sorry for ruining it for all of you the third and final option is image based Reflections where we make use of cube maps to store environment data which we then sample from to determine our Reflections wait what is a cube map a cube map is a combination of six different textures projected onto the sides of a cube instead of using two-dimensional UV coordinates like normal textures we use a directional Vector which points at the pixel we want from the cube map our Skybox is a cube map composed of these six textures we are using the camera's view direction to sample it we can then very easily render Reflections on the Water by calculating the reflected Vector from the camera and using that to sample our Skybox this immediately makes our water look much better but it's not quite correct yet because we aren't taking fresnel into account fresnel describes the relationship between the angle of incidence and the strength of the reflection essentially surfaces exhibit much higher reflectance when viewed at near grazing angles this is something you observe every day of your life but now when you see it you can point and say wow fresnel a decent approximation of fresnel for real time is the Slick fresnel which looks like this I'm not going to explain the math but basically if you're looking down on something then it exhibits very little reflectance but if you are looking at something at a grazing angle the reflectance is very high this phenomenon also applies to specular Reflections so make sure you're multiplying your speculars by fresnel okay where was I the Skybox is going to do wonders for our water because of the beautiful environment Reflections the static Skybox looks weird we can animate the UVS and blend the Skybox with itself in order to create the illusion of moving clouds the sun is supposed to be bright but it doesn't feel bright so let's add a bloom pass the bloom pass pushes our render into HDR we can use a cinematic tone mapper to bring our values back down to get some beautiful contrast at the same time we finish up our effects with a subtle and yet and complete the scene with some calming ocean ambiance in the end the sum of signs or a fractional Brownian motion is the secret behind the water rendering of most of the games that you play because it's cheap and effective you might ask the question how can you say something like that with such certainty well if I lift the camera up above the water then all of you will go oh the unfortunate truth of this technique is that It suffers from the dreaded tiling patterns that we all hate to see in games like Pokemon Legends Arceus and even Elden ring we can remedy the repetition by adding more waves but because waves are oscillators the patterns will always repeat eventually this means that you should only use this approach for small bodies of water or Coastal views where the player is unable to see the repetition as easily like I have so expertly done in my demonstration other games like sea of Thieves have remedied this issue with much more sophisticated fluid simulation techniques let's for another time in another video if you would like to support me and keep the channel going check out my patreon all my patrons get to vote on the next video topic and while the next video topic has already been decided you'll be able to vote on the next one as always a huge thank you to all of my existing patrons without your support I would not be able to eat Taco Bell 21 times a week also I'm streaming right now over on my twitch Channel I'm building a Lego typewriter if you have any questions about the video come hang out and chat anyways that's all from me I hope you have a great rest of your day and I'll see you next time

Channel: Acerola

Views: 166,701

Keywords: Programming, Creative Coding, Educational, Computer Science, Algorithms, Essay, Video Essay, Acerola, Learning, Tutoring, Art, Generative Art, Technical Art, Physics, Simulation, How I Made, Shader, Shader Programming, Shader Code, HLSL, Game Development, Optimization, Performant, Post Processing, Unity, Water, Water Rendering, Fractional Brownian Motion

Id: PH9q0HNBjT4

Length: 22min 52sec (1372 seconds)

Published: Mon Jul 24 2023