Live Coding "Greek Temple"

today actually it's a bit of a special string because we have some guests which have not our usual you know computer graphics shader toy crowd we have some people from the Harvard University from the Graduate School of Design or GSD as they call it so I have a friend there professor Jose Luis Garcia and he his students are learning how to generate I believe mathematical structures for architecture based on you know like nerves and polygons and these kind of primitives so he thought it would be a great idea for them to have a look to how we do things in this other side of the fence in the game development community but especially those who are doing procedural content in shader toy and systems like that like the machine and other communities of people who are creating procedural content with maths but without necessarily using polygons and traditional computer graphics tools so welcome to them I would say actually for them as an introduction that what we are going to be doing today is different to what they usually do in that what they do I think I would call it explicit procedural isn't where you define rules in the computer that tell the computer where to place different elements in space how to place those elements like we have triangles or control points of a Bezier surface with an herb and you say okay put this point here in these coordinates put this other point here and those two points will connect with a line and maybe a third point connects that's a triangle or a fourth point connects as a patch or something but basically the rules and the procedural techniques defines the pipes that were to place elements while what we do here what we are going to be doing today is still using mathematics and still use procedural techniques and programming but not necessarily to place elements but to define volumes in an implicit way so that's why I was saying explicit versus implicit so what we are going to be doing is defining formulas like the formula of a sphere the formula of a box and then let the computer guess where those elements are in space based on that more abstract representation of the geometry so once we have some geometry going we will see some of the techniques I use to make the pictures more pretty and beautiful and realistic which well I am keep calling it making them pretty but it's not only about making them look beautiful in terms of coral colors and Composition but also to improve the readability or how easily your eyes understand what the scene is how it is composed and what are the elements in the scene so depending how you place your shadows your lights your ring lights your highlights you can sometimes either make the image more confusing and busy because it's just too much detail or you might you can make it cleaner and easier to read to the eye which becomes important when you know when you are making a film and a shot is five seconds long and the audience just came from another shot there is a cut and you are presented with a new image and you have like two seconds one second perhaps to quickly understand what you are saying without having to spend ten seconds analyzing everything and making a mental picture what you are saying should be immediate and easy and in the case of a movie where there is a story point and probably a character the lighting would also be helping you guide your attention to that one character or whatever but we don't have characters here we are going to do various minimal things but we are going to be playing with lighting to help read the shapes and the objects in the scene okay let's go so what we are going to be doing today is let me show you it's this little guy here I hope you can see this this is a Greek temple well it is not like I have no idea what a real Greek temple looks like actually there was some from Greece in the chat Dimitri's was it so apologize apologies Dimitri's I don't know how a Greek temple looks like I've never been there my parents have a few times but I haven't so I don't know if the proportions are correct or they are not but it's something inspired by a Greek temple that's what a Greek temple looks like in my mind so for those at Harvard what you are seeing here this picture here which I'm going to stop playing but this image here is made just with maths there are no polygons there are no 3d models you know there is no rendering software there is no you know miles or photo shops or even processing all that we have is this code here on the right side and these lines of code of which we have how many we have 600 something those 600 lines are implementing different formulas and equations which combined together give us the shape of the Greek temple and the background terrain and the ocean there are little clouds and even the lighting and the composition it's made through those equations and formulas what you are seeing here is shader toy so this is a website that I brought with my friend Paul Jeremias and this is basically a community where it's like a bit of like YouTube basically where actually let me show it to you if I go here where is my OBS yeah so what you are seeing here is the main landing page of shader toy where you can browse for different mathematical creations which usually are pretty amazing and I have an account of course and I'm using the website not only to share my mathematical creations but to actually make them as well to create so this is a live coding environment much like processing that I know you guys producing where you type things here on the right side of the screen where the text goes then I compile and the left side of the image responds to the code and create an image accordingly so without going into much detail for now because I want to jump straight to the coding of the modeling but just to give you a very very quick sense of how things work basically this program on the right that says my main it's running for every pixel of the screen so there are a million pixels in the screen or two million depending your resolution and for every pixel this this program will run and we will take the x and y coordinates of that pixel you know we're starting from zero zero and that bottom left corner to pixel number eight hundred and four hundred fifty on the right side for X&Y coordinates because the campus in this case is 800 by 450 so for every X&Y coordinate the program runs and takes x and y which is two numbers it does math on them like having them together multiplying them together doing a square root of one minus the other something and then those numbers that you get out of those formulas get interpreted as colors so basically we are gonna inject those numbers into the red green and blue channels of that pixel in this case I don't have any formula but I'm going to just inject numbers directly so if I put one seed of 0 which is this triplet of numbers here then we are saying telling the computer to inject full red and no green and no blue intensities in that pixel and then we get red if I put 0 1 0 then we get green and if I put 0 0 1 we can see the percent of red see the percent of green 100% of blue and then we get a blue pixel and we can combine them we can do cyan if we mix you know blue and green if we put a lot of red but a little bit of green that we can kind of orange or yellow or whatever so that's the idea and I will do just one more thing which is I'm going to modulate the whole color by zero to get black or white 0.5 to get dark brown or 0.85 to get a bit brighter Brown the higher the value the brighter the colors right so now what I could do is to take take the X&Y coordinates and use those to modulate the color so if I called P the pixel coordinates if I take the X component of the pixel and I multiply that with a base brown color that we had because coordinates on the left side of the screen and a small we are multiplying the brown color or the orange color with small values make them pretty much black and as the coordinates increase to the right side of the screen up to the point where they reach one point zero then we will be multiplying the orange color with one point zero which anything by one multiplied by one stays the same so we get the original orange color on the right side of the screen so that's what we are getting now I did this same explanation few months ago with another of my life streams and I am aware that what I'm going to do now it's like a crazy jump in complexity because I'm going to go from this very simple explanation to something that suddenly looks 3d but you have to trust me that I don't want to stop on the next on the phone on the internals of how to go from A to D Brown gradient to a 3d sphere because that's technical it's more programming that it is mathematics and it is more related to how the computer works and have 3d perspective works than it is related to modeling and design and using math to create shapes so I'm going to skip over that I will do another stream if you guys find this interesting you know you can join us in the next live streams and we might touch on those topics but for now I'm going to go straight to these little blue sphere here which sorry yellow sphere which is a 3d sphere but it is generated just in the same fashion where for every pixel we do some computations and then we get a color and if the color is blue our I will interpret this as the background or the sky but if I place a color which is not blue but kind of skin color or yellow or white then we can make a sphere floating in the middle of the blue sky and this maths that go from the 2d pixels of the screen to a 3d environment where you can define a 3d sphere those equations as you can imagine have to do with perspective and how things which are further away look smaller and things which are closer to the to the avi were look bigger so it's kind of those equations by reversed and these some maths going on there but what I want to focus today on is let me make the text bigger so everyone can read is actually using maths to play and shape things so here we are this line of code 160 what it says the equals length of Q minus 0.9 is the one responsible for making that sphere actually very quickly if I do is let me show you right so as fear as you might know is something like you know x squared plus y squared C squared equals radius square that would be a sphere centered at the origin and in this case all the points that satisfy this equation are on the surface of a sphere something you can do is to take a square root in both sides of this equation and then you would get something where you get something like that and that still represents the same surface the same points that satisfies this equation are on the surface of a sphere however this time this quantity here it's a distance or it's basically the length of a given vector of coordinates XY and C so basically what we are doing is you know for a point XY and Z in space which has for this vector here which has this coordinates we are defining we are measuring this distance basically and when that distance equals R when this equality happens then that point is in a sphere so we are getting you know a sphere that goes through that point so this equation here if we isolate it and we equal it to zero and we do something like square root of x square plus y square by C square minus R this becomes zero is still the same equation down here I just moved the R to the other side this quantity now becomes a distance it becomes a distance from any point in space for example that point here Q to the surface of the sphere so this quantity here the square root would be the distance to the origin and then we are subtracting the radius of the sphere such that a point that is at the surface becomes makes this equation equal to zero this equation is measuring distances and this is something we can use to define geometry in space and the algorithm that I have in shaded so it's going to be using that distance to kind of find point in space and colorize them as blue for background or yellow for the foreground for the sphere itself so what you are seeing here this line of code 160 that says length of Q minus 0.9 is implementing exactly this equation I just showed you with the square root this guy actually I could write it as Q dot x times Q the text at Q those who I times Q dot y plus Q dot see Q dot C so this is taking the X component of Q squaring it taking the Y component of Q squaring it taking the AC component of U is coordinate putting on them together making the square root subtracting R the radius which is 0.9 so this can be shortened in programming in this programming language that we are using which is GLSL as lens just the length of that vector Q and 0.9 is the radius and of course if you know I make the radius bigger like 1.9 then the sphere becomes bigger and I can make it 4.9 and suddenly you know every sphere is really big there's a shadow there and a few lighting that I won't explain today but the lighting and the shadows are made with maths as well but for now this is what I want to show you of course if I make the radius a variable radius equals 0.5 and now we add a sine wave to it let's say sine sine wave of okay that was too much you know this is a time where you can play and you can start modifying the the radius of the sphere and instead of making it constant you can add a sine wave that runs along the x axis and in that case well the radius itself goes up and down just as the sine wave does there are some artifacts so we could fix them by multiplying the distance by a small number which tricks the algorithm which is underneath the rendering all of these to kind of fix the artifact but you could have it the idea that I can also multiply by C and I can multiply by Y and add some sine waves everywhere and suddenly I have this very funny shape that looks more like an I don't know like like something organic like a something you would find on the cell of a human body or something so these are very creative use of math in that it's not like I was sitting down and you know like doing equations until I found what was the perfect function that I needed to use to create an undulation of a given shape I would just drop a sine wave there and see what it looks like and if I don't like I would go and change the parameters and find another sine wave that looks better and sometimes there is this kind of creative process where there is not so much hardcore maths but more playful and very pragmatic use of mathematics and that way actually by playing like that it's very easy to gain intuitions about things like frequency and amplitude and face a sine wave for example and sometimes we use Exponential's and what not so we very quickly developed intuitions about what functions to use for shaping things in a given way that doesn't mean that sometimes we don't do actual hardcore math sometimes we also do integrals and we find what's the exact function that we need but sometimes this was a lot of playfulness like this and the last thing I'm going to show you before going to modeling is that all of this is real time I seen you know I just added time as face like a face off set for the sine wave run on the X component and now the X component of those bumps is moving basically with time so the longer we wait the more the sine wave is going to be shifted in a given direction and that makes the whole thing move and that's great because by virtue of having everything run in real time we can iterate very quickly and that process of you know developing intuitions and finding what sine waves look good and which ones don't we don't have to spend days trying numbers and waiting for the computer to render we can treat life and very quickly understand which values look good anyway I'm going to go back and start modeling there the Greek temple so we want to make columns you know I'm gonna start doing those columns that create that hold the roof in the temple and basically those are cylinders we are going to start with cylinders and then we will shape them a bit but cylinders are pretty much kind of two-dimensional spheres they are like a circle that got through it so it's like you have a 3d sphere and you kind of forgot about one of the components and kept it constant and just you know took the circle that makes the profile of this fear and just extruded it and that's very easy to do because we can take just two components of our point in space Q which is a three-time three-dimensional point and instead of considering X Y and C in our equation let me go back to the maths so here we were just in X Y and C to represent our sphere but now if we just use X and C and we forget Y then we are basically ignoring Y and it means that no matter how high you are in space we are gonna be creating a circle at that height in space and because we are considering considering all points in space so every height or altitude in space will run this formula then we are going to be stuck in a lot of 2d circles basically and that makes a column so this equation here is basically going to create a cylinder like that it's going to be an infinite cylinder though so actually let's try that so if I go here and I just put the X and the y component or x and z components then I get an infinite cylinder so the problem is that we don't meet an infinite cylinder because you know temple is only that high so I'm going to clip it or create kind of a plane or a cap for it and let me copy these equations I think I think I used a six here there we go so what I did now is having two more lines of code I was able to cap the cylinder and normally there are better ways to do these this is not how I recommend doing it this is not as efficient as doing as using the equation of a cylinder and that's one of those cases where I would go in paper and compute the exact equation of a capped cylinder but this is an approximation and runs good enough for other purposes today so what I'm doing here the max equation here it has two components to it so first one first there is the max function which takes the maximum of two values you know the max of six and seven is seven and the max of negative two and positive three it's three so then we are taking the value that we have for the distance to the column cut to the infinite cylinder and pass it to the max function with some other distance here which I can I could call d - if you want and we are going to take the maximum of both and the second distance D 2 is the distance to a plane so I created a horizontal plane that goes to an altitude of 6 units and using arbitrary units here it could be meters perhaps and again Dmitry's can tell us how high how tall one of these temples is I'm sure there are more than 6 meters that let's say they are 6 meters I'm gonna claim a clamp the column at 6 meters by creating a plane infinite plane that is at 6 meters and the equation of a plane again for a for an infinite plane you know something like that would be as simple as what hide it is so y equals 6 for example minus 6 or something and that's the question we are using so then because we are clamping let me try this again because we have a cylinder and then it's an infinite cylinder but basically I'm clamping it through a plane we are generating this other cylinder here that is constrained to the plane so that's what we are doing here this is the question you know B equals 6 but I equated to 0 by passing the 6 to the other side of the question with a negative sign that becomes Y minus y minus 6 and that's what I input here to create a clamp in and I'm gonna also clamp in clamp it in the bottom side of the screen with another plane I don't know if you can see the difference there at the bottom it's no longer touching the floor it's floating a little bit above it and that's fine I have to reverse the sign of Y because now the plane is pointing down the previous plane was facing up this one is facing down so we get caps the cylinder that makes sense to the viewer you know if you have the surface normals of this shape the cylinder is facing us all the normals and the surface are pointing outside of the cylinder at the cap at the top cap we want the normal to point up and at the bottom we want a normal to point down so there is a reverse of the normal y direction for the caps that's why it is a negative sign there so okay let's keep model in this a little bit I'm them a little bit more irregular if you was looking at some pictures of real temples and normally the base of the columns is a bit thicker than the top so we can do that where you know this is the radius here and we can check you can we can make the radius known constant that's what we deal with as we leave with a sine wave to create those bumps on the sphere now I can create a known constant radius to shape the the column so I'm going to add something like something like that q dot Y so what I did is you know if point 9 was the radius and by making it smaller I can get the column to be thinner then I'm gonna subtract not a constant amount to make it thinner but an amount that is bigger at the top so we are going to subtract more radius at the top to make it thinner so I can't just take the Y component of the point and and subtract a given amount so I could make it more dramatic you know by making a subtracting 0.2 or still the point 1 but I found that 0.05 makes for a good shape in and again this is a bit of a hack I seen this cone has also a mathematical equation which describes it said it's shaped perfectly and that the question is not this so this is an approximation and I have the questions in shader toy and in my website where that you can use the create exact sine difference field exact formula for a cone that has the shape but we are not using that here just for the sake of having a good time moving fast and and getting something that is approximate because we are going to be doing so many things anyways that we are going to that break the maths that having one more mistake here doesn't really change things too much anyway let's make this a little bit more pretty by you know what I'm gonna call this thing radius so we don't create a super long line of code and I'm going to keep modifying the radios I'm gonna break these in steps actually I normally like breaking my code in steps that make sense and by that I mean that you know this is the base size and then we shape thinner at the top and now we can't create more operations on top and stack operations and this is pretty much how an artist would work in Photoshop right or in any program where you start with something basic and by layer in detail and detail you can make the image as complex as you want and as beautiful as you want and this is pretty much similar we start with a simple shape and we start adding and layering little equations into those little brush strokes we add literal equations one by one and then we we get more and more detail so I'm gonna use a sine wave of let me see this should be C I suppose and then let me write it and then I will explain that but actually let me explain it right away so I have created a quantity here that Wow okay you know what do it first and then I will explain it okay so what did I do here I modified the radius of this cylinder by an amount of Cedar Point one if that was just constant the cylinder will become smaller again but I'm modulating that curve in effect at zero point one carving with a sine wave again but instead of running the sine wave along X and raining the sine wave along the perimeter of the circumference of the of the column so okay let me draw it mass so top view of the column would be like a disk like that a circle so what I'm doing is I'm measuring the angle of any point in the surface I'm measuring that angle alpha here okay let me make it bigger so you can see it I measure in alpha and alpha for a given point X and C is gonna be there are tangent arc tangent of X over C so this is X and this is C actually it's gonna be C over X doesn't matter really it's just an offset but basically we are measuring in the circumference all the way from 0 you know 2 pi to 2 pi or tau if you want to call it that way and that measurement that distance along the circumference we are going to fill it in a sine wave and as you know we saw earlier the sine wave is going to do that and then what I'm going to do is to compress the sine wave many times I think I did compress it 16 times so that sine wave that goes from 0 to 2 pi is suddenly going to become you know 16 of those sine waves from 0 to 2 pi and those then through the arctangent they get wrapped around the circumference of the sphere so that's going to become basically a sine wave that does something like that geometrically in space and then I make it small because the same wave as you know goes from negative 1 to positive 1 I multiply it by Cedar Point whine 0.1 as you sow so that makes it smaller and then I add it to the column so what used to be just a column like that suddenly it's gonna become a column like that so that's what we are doing basically let me go back here so we have the sine wave here I just you know instead of going from negative 1 to 1 I always like my signals as we call them back all these pieces of information to go from Nick from 0 to 1 instead so to make something go from negative 1 to 1 and remap it into 0 to 1 range we have to multiply it by 1/2 and add 1/2 to it as well so that's what I'm doing here and once I have that then you know I can multiply by 0.1 and subtract it from the radius so we can carve a bit of a shape one thing that we often do when creating these mathematical shapes is to take a signal that like this sine wave that we had and change its shape and control how spicy it is how smooth it is have rounded it is and that's something we can do with power functions you know like x squared X cube X fourth or x2 1/2 which becomes a square root or X to the negative one-third or whatever so these powers that we use can change the shape of a function so in this case if we had a sine wave like this which goes this is zero and it is one we had a sine wave that was going from zero to one now thanks to the biasing if I square it and I make sine squared you know like sine times sine then you have something that is more like spike here more like a comp and that's what I want because I want my my columns the profile of my columns not to be like you know like a smooth flower shape but I want more that something that is more like more spiky like that I don't know how to drive it but probably get the idea I want something to spike here so I have to square the sine wave and that's what I did here what I'm about to do I'm taking the whole thing and squaring it and now I have something that looks better more like I want it's a small detail but all the small details count in the end and then something else we can do is to create the fake sense of bricks having you know like this column would normally not be made of a single chunk of stone it would be make relief of smaller blocks of marble or stone or some sort of material and they will be stacked on top of each other so we can simulate that by taking the radius of the column again and pushing it inside and carving it carving little you know like planes into it or little ring that happened at periodic intervals as we move vertically on the on the column and another way to create periodic signals is by using a sine wave because sine waves are periodic on their own so it's not to be very similar but instead of using the arctangent to measure positions along the circumference of the column and gonna be measuring positions vertically in the column to create those carvings periodically and that's just the y component of the of the column and it can change the frequency and now there you can start seeing some of the let me make this bigger some of the shaping I'm getting but so you can definitely see the sine wave from the profile of the column there maybe you can bring the camera closer let me quickly bring the camera slightly closer here okay there we go so sine wave they are very periodic but I want something that is more like straight and then very quickly curves in to make to simulate the gap between two little two blocks of stone so I'm gonna use the pal function again instead of squaring it I'm going to use a power that is smaller than one so 0.5 makes it a square root maybe like what about I let me see should I make it positive yeah but then I can subtract it when 15 again okay okay so basically I am [Music] I think I can do this no I'm gonna put it back to what it was I think I got it right so this is the point we can provide him that I will get things right and sometimes I won't get them right so this is the difference with and without so what I did is to take the sine wave and pass it through a pal function with a very very small exponent so basically you know the comp shaping that we made before is that to the extreme where basically the sine wave is staying high and only in a very few locations goes down and very quickly very rapidly comes back to be 1 again so it's like a sine wave but you push it up the point which are at the very bottom stick to the bottom they don't move but as soon as something is not at the 0 level it very quickly goes up and basically that makes the whole shape go down very rapidly and recover very rapidly as well and stay up and that shape is exactly what I need for the profile of the column to create those fake bricks all right so let me move the camera back to where it was which I think it was 25 so now let's add more detail to this we are gonna add two boxes or a box at the base of the column and another rectangle or a box at the top and I'm gonna do that I will take a small Q&A so we can have a bit of time to talk about other things and I think that my stream is gonna be longer than two hours for sure but we see how long it goes at least we will cover all the modeling that I want to cover and then we will delight in at the end for the brave ones who stay alive after all of these okay so book says I don't if I should explain this protein I'm gonna skip it and go straight to the equation of a box let me show you the equation of a box already created for us I don't know what's a question sighs should be improvised little bit again okay that's a box can you guys see that box there it's a tiny one so let me multiply that by three okay there we go a box the same way we had the equation of a sphere we which was you know the square root of x squared plus y squared plus Z square minus radius we also have the equation of a rectangle or a box of arbitrary dimensions and actually have a whole youtube video yes on that if you look up for my name in this same channel that you came to see the stream if you browse the videos the second to last video should be one about how to create the equation of a box and it also involves one square root and a few coordinate manipulations it should have been x squared and all that there is a few max functions and a few other things that go on but basically it's like that just like a truncated sphere in a way so I already encoded that function here this is the equation of a box I won't explain it but it's just a couple of lines of code it is not a lot so basically it's not a very expensive operation but you can think of this like a square root of coordinates and that creates a box so what we have to do now is to combine the box with the column and that's something that we can do by using the mean operator so earlier we use the max operator to the max function to clip the column we sorry we have an infinite cylinder and we use the max to clamp it and create a top cap to it this is similar but instead of cutting we are going to combine two things it's not you know it's not subtractive it's like additive combination of things so in that case I take the mean and and the use that the reason to use mean is the way the the technique or the algorithm or the equations that we are using to project all these shapes into the screen the way that algorithm works is by measuring distances and by combining two distances in this case that of the column with that of the box with a mean operation we are placing the algorithm we are doing something that is compatible with that algorithm and we ensure that that algorithm will find either the box or the column whichever is closest to the camera technical details we could go through it another time maybe but basically with the main function we can combine these two shapes so yeah let's go ahead and start you know modifying the location of these sphere sorry of this column when I call a new coordinate QQ just gonna be just Q and I'm gonna subtract some amount from it actually you know I'm gonna call it when i reconstructed from scratch and just copy in the values X Y C to a new component to a new vector and put a label here okay yeah point three okay let me see five point three point six maybe yeah and now we can make this thing smaller I think I will do one point four I think okay so we have one there I couldn't move it up a little bit something there now in theory I could create a second one and I could put it somewhere down there like there let's say so that would do we have two rectangles will make the base on the top of the column but one of the advantages of using mathematics as I was saying earlier is that you can populate huge amounts of detail and you know vast vast vast amounts of space without having to explore the memory of the computer and in this case one box because when no one line of code two lines of code two boxes now became twice as many lines of code so that means that the complexity of the code and the memory used and the computational expense of all of these it just doubled because we have doubled the amount of boxes so thus okay but not ideal because it's a linear cost but there is a trick we can use with math which in the spirit of doing lot of stuff in little amount of memory a tree that we can use that avoids this linear increase of complexity and we can use one single box to model both the top and the bottom so this is the trick when you are when the algorithm is kind of rendering or evaluating the maths for the column and the at the point of evaluation is kind of at the top of the column the only of the two boxes that we care about is the top box the one at the top the one below doesn't care it doesn't impact neither the colors nor the shadows not the shape of the column is fine us from the top of the column that the bottom one doesn't affect in any way and it's not just an approximation it really doesn't affect because the other one the other box is closest and similarly when you work when we are at the bottom of the column the top box doesn't really impact in any way so what we could do is something like if we are at the top of the column or if we are above the middle point of the column you know center of mass if we are above it just evaluate the top box and if we are below it evaluate the bottom box so we could do any statement you know if key without y bigger than 0.5 times the size of the column then these box otherwise the other box that would safe but still is not ideal because computer is not always like conditional branching like that and if statements basically but there's another trick that we can use which is which is symmetry which is using that idea that when you are in the top half space of this column you only care about the one at the top and when you're in the bottom we're going to care the one in the bottom so what we can't do is something like you know take the absolute value of y and let me see if this will work ok there we go so look at that it's a single line and I got two boxes even though we are only evaluating sorry Oh screw up what are we here we only have one box but the QQ coordinate is different from Q in that now anything that is in the negative health space of the problem anything that is below the center of the column which would make for a negative Y component for Q Q del Y will be negative now we are passing it through the ABS function which takes the absolute value of a number meaning negative 3 becomes 3 and therefore every coordinate which was below let's say I've got a position negative 1.5 suddenly through the ABS it's equivalent as if the point was in the 1.5 coordinate of space so basically anything an evaluation that is happening below the center of mass of the column is going to be automatically be moved to the top part so it like we are mirroring in space we are doing a symmetry basically so well you know that you know you have a function of X a polynomial and exponential a combination of things whatever and you replace the whole formula the whole equation and when it says X you do absolute value of X suddenly the whole function gets mirrored in there no negative side of the x-axis so this is the same we are mirroring the whole volumetric because this is a volumetric function is not just a function of X like you normally graph it's a 3d function of XY and C but we are mirroring it vertically by taking the absolute value and now what we need to do is to shift the whole problem a little bit and do something like that okay these are details now I just was shifting the mirroring point before the mirroring point was zero but that okay let me go back that was you know the two boxes I'm not exactly Center the the one above needs to go a bit up so I can fix that by shifting the mirroring plane by 0.3 units and then I can make the box a bit further away from the origin something like that ok so we got that we got a small column let me move the camera so you can see it in all its glory and actually the communist moving very slow let me move it a little bit faster there we go so there it is come on and I think I'm gonna take Q&A now very quickly someone is asking how to turn an SDF and export it like a mesh data is there a way to bake the results of an SDS volume to something that can be used in game ok I missed that one but the answer is there are many answers one is yes you can export data often SDF in multiple representations you can export it as polygons we don't have that function in shader toy but we could add it and I'm just need myself in some of my demos and things where I take the SDF and I make polygons out of it for rendering in my own engine or game engine whatever you could use marching cubes or surface nets or matching data Hedra there is a whole sooo of techniques and algorithms that you can use you will always of course lose some detail unless you go with a super super high res mesh you would think of doing something adaptive where on the high curvature areas get more polygons and areas which are more flat get less polygons but it's complicated and yeah not so interesting I think another approach is to render the SDF directly in a game engine there are a few folks who made a plugin for unity I believe that can render SDF directly and I made my own for the Unreal Engine three years ago just as an experiment I never published or pushed it too far but it does work basically you implement a remark which is the technique I'm using under the hood in Shaler toy here to explore there's DFS and splat them into the screen and you implement that on in a shader inside unity or unreal you know you can you can put like a 3d box like a normal polygon box in the scene and then on the shader of that box and the material you write the remark sure and there you evaluate or there's DFS and you take the color that you get from the Ray marching and you put that on the shader and if you you very much it doesn't find any intersection you discard the pixel to create a transparency mask and if you do find the intersection you split the color but also you take the depths of the intersection and you write down to the death buffer you have to do a conversion there between a true distance which is a square root based to a linear distance which is what projections my projection matrix is use when you go this buffer based 3d rendering that's a small conversion just one division and then you can compensate normal polygons things with very much things and then you get the flexibility of both things we have a game engine so you can alter the rest of the game and script your game as usual and you even have shadows from all these very much things into non very much things but you still have the flexibility to do funny effects like morph blending distortions and we are things which prolly are what you are after when you use as DFS because those are things that said that are difficult to create with polygons other ways to do it is to export there's the earth as it is in BD v is it like a bonding oh what's the name is the open source library that mean works released for volumetric data that they use for rendering clouds and other 3d effects for their movies so you can't represent SDS and bake them into a data structure so you don't have to reevaluate all the equations and formulas every time but you have just the SDF back in voxels basically is there an example of the setup script available on your website I think I have an article where I explain how to blend the depth buffer of a 3d engine with the distance to a pixel that you get in a very merger I have that I that's all I have I have examples of I don't if I should show it and hopefully one day I will finish a real-time for shield I'll demo that I'm making and there I basically took some of my experiment for now I took some of my shader toys that I have like this one here let me show you to you so there is this shader toy that I made as well it's another for those in Harvard this is just a very much see just like like the temple we are building but it's more organic so I took this guy backed it in polygons with surface nets make some lighting of the vertices and then in the 3d engine I just blended it with you know 32 times multi sampling and the same shader that runs in shader toy for the materials I'd run it as a fragment shader for the materials of the mesh and I get exactly the same image running you know like super multi sampled beautifully and I think it was running like 900 FPS on my previous graphics card which was a seventy I think so yeah there are wasted it's not too complex I think the surface nets technique i I borrowed it from somewhere and I customized it and I improved it and I made my own version of it but it was like three or four years ago and basically it's based on scanning the volume in stacks so it's like a 3d printer but kind of the opposite you have this DF and you start slice by slice going through points which are regularly spread in space and for every point you you evaluate the SDF and if it's negative you know you start creating that to the polygons in it but then you start extruding it up and you start connecting the 2d slices together anyway yes there are things you can do open V DB says Garcia de castilla yes yeah open V DB does the one which stands probably for volume description something had an old EP is rebounding me distance bounce I don't know anyway let's keep going we were here I think step number two so repetition yes someone was mentioning we are gonna be using the mod function to create repetition because we want many many columns and again for the price of one the idea here is that we don't want to explode memory or complexity in the code we want to keep seed machines simple and have one single column so in what we need to do basically is to take the domain where we are representing the column and make it periodic so let's do it I'm gonna move fast because many of the guys here already know this we have touched this many times so and I start accelerating a bit so we can get to the lighting which is the fun part I think for some of the folks here but let's see how we do this okay let's take X dot X dot X and I'm gonna pass it through a something like that okay so let me explain what I did well first of all before I even explain it this is awesome one line of code infinite many infinitely many columns the screen doesn't show but this line of columns goes to infinity and negative infinity like literally goes to infinity mathematically infinity so this is something that in a game engine would be complex to do or you will have to start having instancing and scripts and things and technically inside you would have to be running in the same thing many times with bounding volumes because you don't want to render them all you have to do frustum : you get weird very quickly here just one line of code so let me explain what it does takes X the X component and the mod operation basically is taking X and whenever it goes beyond four which is the second argument here if whenever the value goes beyond four it subtract four from it so you know five becomes one and five point two returns 1.1 1.2 and the function as well if something goes beyond eight it does subtract eight so eight point one becomes zero point one and the same with you know twelve it subtracts twelve so every four units it kind of resets the whole number as if it really started the game from zero so from zero one two three four oh it's fine but when cross is four it goes back to zero so basically what that is doing is you know taking the whole space which is full of coordinates from negative infinity to positive infinity in a Quincy and basically focusing only on the zero to four range and repeating it forever anything that is quicker than four will be as if it had been rejected at the origin and that means that because we are you know describing our column in terms of the coordinates in terms of the origin of the space you know like an the box was about plus five units but it was like zero units left and right and serial units in depth and the cylinder was centered at the origin what we are doing that is that any point beyond four with it gets rejected at the origin a new column pops up pops in that new location so again if the algorithm is evaluating something at position 12.5 this mod operation make sure that the algorithm things that we are evaluating evaluating the function at 0.5 well we do have a column so when you column shows up in the new location so there is that there is the 4.0 which tells us the repetition space you know if I made it 7 the columns will be spread apart a bit bigger but 4.0 is good for this case and then I have a plus 2 and a negative 2 here which all they are doing is making sure that the repetition range is not from 0 to 4 as I was explaining but actually from negative 2 to 2 so 2 point 1 will be rejected into negative 1 point 9 right so we are just shifting the the region of interest we are resent or need recenter in it at 0 which is where we have the column so there we go easy now we can do the same not only for the X but for the Z component and and now we have so many columns that we cannot even see them let me move the camera a little bit farther away ok ok look at that we have so many columns because now we are repeating it in X and C so this goes to infinity only that I have kind of clamped the whole problem to a small area in space so we can well small volume so we can't render quicker but yeah we have so many that that is filling the whole space so let's fix that and for this I have a function that I already prepared that takes the repetition and instead of making it infinite it makes sure that we only repeat it a given amount of times you know I was saying like like twenty point one will become point one again because of the 0 to 4 repetition but we can clamp that effect to say repeat it four times and from that point on preserve the numbers so I have that function already coded I think I called it operation repeat limit and we pass it the point we pass it the repetition period and then we pass it how many times we want to repeat in each coordinate so something like that let me zoom back and then I can go back to the code to show it to you closure okay so again recap we are repeating the domain with a spacing of four and we are within repeating it only for x in x and two times in Y so we should have you see one two three four one two three four and the one in the middle so that's the this is clearly the X component the X direction and here in the Y which will have five copies and you cannot count but let's see one two three four and five columns and that's the central one plus two to the right plus two to the left this is the tool that is controlling that if you're curious I have an article about this in my website about how these works that is based on the mode operation but then we clamp the effect of it basically right so what else yes I'm gonna have remove the columns in the middle if you remember the original temple there are no columns inside so let's do that and I'm gonna do another curve in operation pretty much like we did you know clamping the column with the top and bottom planes to create a limited cylinder I'm gonna do the same by carving a box from the inside of the temple to remove all those columns which are inside there are probably better ways to do this but you know let's do that for now there's the box of P and I don't know start with the numbers here and see what happens maybe display I still see columns inside five okay okay look we remove the ones in the inside but I still do see the basis of them so don't make this higher six I still have a row of columns here so I'm gonna make this one bigger okay I'm gonna have you up in the floor so this is a copy of the same there was another safe point that I made for myself in case I would screw up things a lot but basically is the same as we have I think the camera is slightly different and the colors are different too actually you know what I don't like the colors of much let's go back with this one I think you can see better what's going on okay let's add the floor floor if we go to the original one that I made the floor is made of boxes so let's go ahead and make boxes super you see we already made boxes so we know how to make those I'm gonna call these floor we are gonna combine them with a mean function we are gonna make the floor what what is the values I have here you see much smaller to and to see the point one okay it's during the inside you know what let's very quickly remove the outside so we can see the inside so that's our tile I'm gonna make it bigger okay I'm gonna put a scope here and what about we repeat the domain as well what is my repetition domain function that I just used okay here so I'm gonna take P actually let's call this one P here I'm like three P alright so we are gonna repeat this one how many times this one slightly smaller okay like that okay we have a lot of times they're gonna make them higher something like that [Music] okay but let's put it back with the thing let's push them down Q is my VP but we are gonna move P down by some amount I don't know there we go sorry for that so we are interested only on the SD FS which less DF values which have a value of zero and by shifting that condition a little bit to understand that is not zero but 0.3 suddenly we are changing the whole geometry to the next isosurface that is defined by the volumetric function and these are so these isosurfaces get rounded and rounder the further away you move from the pseudo isosurface what's that video you will see why that is the case but all we need to do here is to just shift the isosurface a little bit so this is the repetition operation for those who I don't know if the screen was not casting the right thing to you I was in broadcasting the right thing when I was talking about that that this is the repetition function and this is the box equation and is the mean that combines the boxes with and we take the SDF of the box and we are going to round it and all we need to do is so first of all yeah let's subtract an amount from it zero point one and that makes it run there so now someone was asking in the chat and this is something I forgot to mention in my previous video which is rounding shapes makes them also bigger and thicker basically we have a shape at a I see right to surface by switching to the next types of surface with a subtraction we are making it bigger and bigger and bigger and bigger and rounder as well we are inflating it and it's loosing definition but it's becoming bigger getting volume so the way to compensate for it is to make the inner box from which we are growing the rounded box smaller so we take the core of the object the isosurface we push it inwards and then by changing the dimensions of the base object and then we inflate it back by the same amount add words and then that rounds the shape does it make sense so you have a box if you just round it it becomes bigger what did you first make it a smaller and still make it a box and then round it then you can get the same value of that volume that you had but with rounded corners so in this case actually let me put the numbers here to not make this any more confusing that it needs to be so this is our box of dimensions one point nine zero point four and one point nine let's say I want to round them by an amount of 0.2 I subtract zero point two but I also subtract it from the dimensions themselves so I make this box smaller by zero point two units and then I inflate inflate it back by zero point unit zero point two units and then you get a running effect without increase of volume so look at this I'm gonna go back and forth there there there and there so yeah great actually this is a bit too much so maybe we can close the gaps a little bit this is up to you and you know what I think we should round also the the base and the bottom of the column which is here so we are gonna take that guy and run them a little bit I know that's not the one that's the clicking one I'm sorry this is the yeah a little bit more what's already around it bad like that something like that maybe yeah whatever that looks good you know okay okay there we go rounded tiles for the floor much better than not rounded ok let's create another layer so what we need to do now is to create another layer of floor tiles so I'm gonna copy the whole thing I'm gonna push them down by some amount let's say 7 and now we are going to okay so I want them to be not aligned but offsets from each other so I'm going to take the X component Y I'm shifted horizontally by half a tile on a tile right now it's 4 units right so let's shift it by 2 units and that didn't work at all oh it's because I'm using P so we should be using Q here there we go actually I'm gonna shift it in Q and C okay so I received it that way okay let me shift it the other way there we go so now we have the second set of column of tiles let me move the camera a bit away from the thing so I can better see what's going on how much and also yeah okay so let me go back to the code and yeah so I'm gonna have another parameter here which controls how much okay so I made another version you ever have another II over written written who you pronounce that we have a second version of this function operate limb which allows me to choose how much the repetition extends not only in you know left and right but left and right individually so I can repeat something in this case three times to the left and two times to the right and I'm using that to make the whole shape symmetric so there we go and also I would like to push it forward like that so I can also shift in see by its way okay look at that we have a second layer now let's do a third layer which we don't shift anymore but we push down by another unit eight now we need to create more repetitions so three and four we need five and see there we go okay that's our base pretty and let's see what time it is because it is 12:35 we did start at 11:10 sauce it's been an hour and 20 minutes I'm going to copy paste a few things now and with that then we can move on to lighting I guess okay let's up that roof quickly I'm gonna copy paste this and I think I wrote it down somewhere what is it okay roof okay I'm gonna write here roof and I'm gonna copy paste the whole block of code here which is unfair but there we go okay that's gonna be our roof it's not connected yet but so what this is is a basically is the equation of a rhombus you know we had the question of a sphere we made it into the question of a cylinder we use the equation of a box there is an equation of a rhombus which is kind of round you know like if you take a box from two of the corners and you press in so the question is very similar to that of a box so I guess I just used it in this case I made it very big and rotated it by 45 degrees to place it at the roof so nothing too difficult there I also let me see to the floor yeah and also what we need to do is to add a few more boxes so before you know the gap between the rhombus and the ceiling or the roof and the temple is because I made some space there for putting some of these boxes like that which is again pretty similar to the floor you know just just one box which has been repeated multiple times and then just for the sake of having a bit more detail I also added a second block of boxes that you can sit on and off here that kind of complete the cylinder there we go something important that I've not been talking about is that you [Music] know all these elements have the same colors and materials and that is not very realistic so let me switch to a more complete version of these where of the the temple where every okay but I'm moving too fast hurry for that let me let me do a few things so this is the temple that we just made but if you can't see here every column has a different shade of grey he has a different color and that's because I created not only a distance through the distance fields but every point in space also has a unique identifier a number that identifies which piece of geometry falls into that point in a space and these numbers are arbitrary numbers like you know the first tile on the floor is number three and the next tile is number twenty and the next time is number the next tile is number 17 it doesn't matter really but basically the same code that creates the repetition of the domain you know the zero to four endless repetition to create an infinite amount of columns that's in code keeps track of which one of those repetitions we are in you know point the point eight point one will become point one to the repetition every four units but it will remember okay we were in the second repetition first one being from zero to four the second one sorry well the zero repetition been the identity the co2 for the first repetition being four to eight and the second repetition being from eight to twelve so eight point one falls in the second repetition number two and you do the same for X Y and C and you get you know the second repetition in X and we are also in the third repetition why and the first repetition in C for example and that gives you a triplet of numbers which you can multiply together and do some operation to it to get a unique number that tells you which overall repetition in space you are and with that number you can go back to the code that generates colors and modify or affect the color with with that so yeah there we go so you can create individual colors for each piece of geometry it's not too difficult something I also like doing and that's the thing I commented out here and I was earlier talking about making things look old and aged and more realistic that's something you can do in a multitude of ways in my latest video and YouTube I talked about one of them which is modulating the roundness with some random functions in this case when I made this one I didn't think of that technique yet so I just used a different idea which is not that good but it kind of works which is taking the overall mathematical function of this shape which is a temple and now so you can imagine of these you know like a bigger question we had the question of the cylinder the arctangent the sine waves to make the column more detail we have boxes combined together with a mean we repeated the period with a mode function we combined them again with a mean we clip the inside of the colors with a max all that combined is a huge monster equation you could write it down you know like X square minus the square root / you could write it down so we take the whole equation and then we had a plus at the end and then we add some random functions there and by random functions I mean something we call noise in computer graphics which is you can think of a sine wave which is still periodic as it goes through zero continuously it goes up and down but the height of each one of the valleys and Peaks it's one of those oscillations it's random the amount of the height of the oscillation is random and then there is some constraints about you know if you have two oscillations of different sizes you have to connect them continuously with continuous derivative something so listen constraints there but basically a noise function you can think of it as a randomized sine wave and actually it randomized in sine waves is one way to generate early noise and noise and things like that not the ideal one but it can be used so that's what I'm going to do here she's taking all those random sine waves and combining them together and adding it to the monster equation of the temple and there you can see in the screen when I enable and disable it I can't create those kind of holes or I don't know how to call it like you know 18 it makes the whole thing look older like you can imagine a temple after two thousand and five hundred years been exposed to weather and wind and peopled and the rocks lose a bit their shape and yeah so you can do that a few times there are some artifacts there you can see some black pixels at the time I didn't know how to fix them so I let in there these days I could do that without having to have those artifacts but at this point I would call the modeling done again if the camera was closer to the thing we would have to add more detail and I'm not even sure these are correct proportions but the point is that what do we use we used one cylinder two sine waves one box for the top and the bottom of the column three boxes only to do the whole floor because we use repetition two boxes for the ceiling and one I think two rumbles so we are talking of like twelve primitives twelve equations involving what twelve square roots and a few sine waves so this is relatively a small amount of computation if you had these in you know in polygons we would be talking here even through instances we will be talking of hundreds of thousands if not hundreds of thousands of of vertices of numbers floating in the memory plus a lot of processing power in order to consume all that and project it to the screen and light it and whatnot so these are a bit the power of doing things with maths and it does have uses in real life you filmmaking and things like that I want to spoil a bit that talk that I gave but basically the I guess the one-liner is that when when when you are in a new when you are thrown to a new challenge like let's say you are an engineer or a technical artist which is more what I am more than an engineer and there is a new film or a new game or a new whatever that you need to do and you don't know how to do it because they are asking you to do something difficult like we want to render a forest and we never made a forest before our machines you know our computers cannot handle more than this amount of geometry and how do how are we going to make a forest or or how are we going to make shadows for a character in VR when you only have like you know 11 milliseconds to render and only three of them are for the character and and you have challenges like that and they are know if if it's early in the technological cycle of the technology that you are using the same VR like 4 years ago or when I was a picture we have very like 4 gigabyte machines and that was what we had at the time 2010 and they could not render big stuff so there's a limitation on technology then because the technology cannot give you what you need with standard methods you know by using the Mayas and unreal engines and whatever then you have to find solutions which need to be implemented without tools and without without the Mayas and reals and even without pipelines or without the concepts maybe just in polygons is not the right solution in those super constrained situations and suddenly in those situations is when being able to create content just from maths like skipping or bypassing the technology let's say the the processes the bypassing the pipelines and the tools and all those things going straight from math to pixels as we are doing here it's a value it brings lots of value because suddenly you can find workarounds and invent or tune techniques that exist without having to be constrained by the processes that are in place which are insufficient in this case so there is a very real juice of doing with mouse know that this is how you want things to be working on the long run because I can't you want artists to be part of the creative process when making a film or I came and therefore you need to give them tools but for a moment in time for a small window of time when new technologies show up I think people who can create things with maths have a place and can't bring lot of value and be the first content creators before anyone else can create content before the miles and the Adobe's and you know the Autodesk's and the epics go into the field and start developing tools anyway these are really used for this but it is also super cool it's super fun to do and that's why I think most people in shader to who are there right guys we are they're just doing it for the fun because it's great to for once not being the technician who helps artists but being the artist makes me not that this temple is like something you would put in the big screen or in a game but you know we can have some joy and pleasure finding pretty images and so on speaking of pretty images this image right now it's pretty ugly so I'm going to fast forward like a lot and show you this other image instead which still not pretty enough but it is better it does show the temple that we just had after I added a landscape to it so I missed at this point when I was making this so making this temple took me a day I think and the day probably is like let me scene from five to ten hours depending if it was before or after baby before baby my days were ten hours long now they are 5 hours long so having 4 hours in the process or whatever and I'm here I have a composition you know I have the landscape I have that the temple I have framed the thing I have moved the camera a little bit and I know this is the image I'm going for and I'm happy with it I have modeled everything I needed based on this one camera so I'm not model in the back side of the temple I'm not model in the inside the terrain is just as simple as it can be to kind of sell the image and gives you the sense of perspective and the setting of the of the picture but now is the time to make it pretty and most of the like 70% of making an image good-looking I think it's lighting so lighting is the process of you know the having the shadows from the right angle and making sure the silhouettes pop breaking the right colors so you have good color balance and things like that so I'm I'm slowly getting better at lighting I'm still not like super super good at it but what I have learned to do a bit is to avoid ugly images and more than creating pretty one so I tried things until I detect I'm outside of the ugly spot kind of same kind of thing so in this case yes by looking at this image I already notice right away that I mean I'm not happy let me look at the picture of it actually this one it doesn't look bad let me see what my lighting is yeah it is not that bad but I feel that the temple could get some more lighting and I could get more shaping out of it so I'm going to modify it so I have this sunlight thing here let's see what it is define so this is the direction of the key light of the Sun okay you know what let me show you first of the lighting components one by one so we know what we are talking about so let me remove all lighting and light in our equations again in this case actually unlike the modeling process that we went through which was defining shapes through equations which is very unconventional and different to how computer graphics is done normally which is polygons on the equations we use for lighting are similar - the ones juiced in CG normally they are not that different at least not too different to how CG was done five to ten years ago where colors would be added almost by hand maybe the pictures look pretty and we will see I still do a lot of painting by hand I seen placing lights in different spots and we will see that but the key lighting components are not different to what you would do in a traditional 3d engine are not too different so first of all I always start with the key light so this is the key light key because it's there it's key to the look of the image this is a term borrowed from Hollywood and there what did you call it three light rigging system whatever the three-point light system where you would only lead they will always lead the character first with a strong light from the front to read the face of the character and then they would put a second ambient light you can feel the shadows and then a third light behind to create a rim or kind of edge lighting to separate the character or the person for the front back from the background so some of these terms that I'm gonna be using are borrowed from that but basically the key light in this case is the Sun you always need a key light in a scene sometimes it can be a lamp fit and indoors scene or a fireplace or whatever but when you are outdoors the natural key light is the Sun and that's what I have here and it is coming from a given direction that I chose to kind of give me satellite I didn't want to light the too late the temple from the front because it looks very flat it's like when you you know what you could take a photograph with a flash and you are lighting your subjects straight from the front and then you don't have shadows in the scene and everything looks super weird and flat and boring and not interesting it's always nice to move the light a bit to the side and in films that you normally put put it at 45 degrees instead of zero in this case I pushed it all the way 135 I think all the way to the back to get long shadows coming to the camera but there's a key light there is the ambient light which is kind of light coming from everywhere in this case the ambient light which fills the shadow so actually let me bring like the key light and all the black spots here will automatically be filled with ambient light there we go and it brings detail to work otherwise wouldn't be any detail so the ambient light on isolation here is basically a blue light which is motivated by the fact that the sky is blue and that's kind of what feels the shallows the light coming from the sky it's a lot less intense than the key light you can see here that the key light is about five units in strength the ambient light is one unit but even less because it's multiplied by little very small number so probably normally the key light is about ten times stronger than the ambient light the key light is yellowish here you can see point 9.50 5.35 that's a very orange color actually sorry to simulate sunset and the ambient light is bluish because the sky is blue so you get point fifteen point twenty five point thirty five that's RGB so the good thing of having these two orange and blue colors is that they are complementary to each other you look in the color wheel you know red is in front of green purple is in front of orange and blue is in front of yellow and because they are complementary we kind of like it that's what you see in real life as well if you ever go a skin and you see super white snow which is the perfect reflector you will see that the shadows look super blue so I have that that's the basics so this is two key two lights and then I have always a back light which illuminates a bit from the opposite side of the Sun so the idea here is that you know the Sun hits surfaces and the surfaces bounce light back and the bouncing is going to be on average the opposite direction of the light that came to the surface so if the Sun is coming from here the light will bounce and to come back so if you have a white surface some light will be reflected and illuminate the things here you can see this very often if you have you know a light source or something and you have a saturated color object like something that's super green or super red it will create like a red and green hallow of color hollow of colors around it because like this bouncing so normally these days you do these things with global illumination and other rendering techniques but in this case because it's real time and it's old-school I just put the bounce light by hand the backlight and I make it the opposite direction of the Sun and I modulate with I modulated with occlusion and fill the things and then I add some specular speaking of occlusion well I would touch on a question later because I don't compute occlusion I use a paint occlusion by hand but basically yeah the back light illuminates slightly a bit more here and makes the thing look great but going to the key light I'm not super happy with how the temple looks because this area here it's super flat there is nothing of interest here I have lot of bright spots here which is great it's super early asked by the way I will fix that later and then there is this which looks great because I can see the landscape that rain here with a nice long shadows I mean detail here so I want to lick some light there so if I go search for where the light source is this is the light detection this is where I can't change where the writing is coming from so if I make that negative you can see that now the lighting is coming from the other side which doesn't look bad actually it looks pretty cool I think stay where we were if I mean this negative you can light it from the front it looks really ugly because I tweaked everything for camera and this is the problem with making things not physically based but painterly okay I always treat these things as a painting not as a as an engine or a scene that we should navigate it's more like making a painting I'm gonna make it look good from this camera anyways yes I play with it if I make the Sun Direction a bit higher then I get this very nice light leaking inside of the temple and it eliminates some of the silhouettes of the columns which it's great because when they are just flat it gets a bit difficult to read which that these are columns that they are a different depths that there is stuff going on in there it's not a good read the geometry but when I put point three in the white component of the light direction which means I made the Sun a bit higher in the sky more likely it was noon or something then some of the light is coming through the roof which has a big hole in it and it's hitting those columns and I love it I love those things so that's great the problem with this is that now the background looks really ugly like it's totally bright we lost the long beautiful sunset shadows not to speak of how bright it is and how much it is competing now with a temple like you know as I was saying before if you have two seconds to read this image and you're presented with these your eyes don't know where to focus do they focus on the bright spot on their right side of the image or should I focus on the left side of the image because both are super bright they are calling for attention so we have to fix that but again if I go here the image is more balanced but then I do is my beautiful light in there so what do we do about this I'm going to be doing an ugly trick which let me see it's an ugly trick which you should never ever do in a game engine or in a film and what not in a film I think you can but not like an engine I think if you do you will be fired so don't follow this one advice unless you are in shader toy or making a film in which case everything goes everything is fine but what we are going to do is to have two sons one illuminating the temple the way we want which was by moving the Sun a bit up and another one even mean it in the background the way it is now so we are going to you know take the Sun Direction which is what we want I'm gonna normalize it which we need for the maths to work and that should change nothing but now here there's a pseudo point two factor that we want to use only for the foreground I'm not for the background so what we are gonna do is add it and modulate it with a mask we are gonna call it I don't know sand mask and now what we need is a signal that you know zero means what we have one means the improved temple lighting so we need a signal that goes that has value zero in the background and one in the temple of course we want to smoothly transition between those two lighting setups because we don't want to see like a weird lighting scene or anything we sydney the temple to sit properly with the terrain so we couldn't do something where terrain if terrain just this light if temple just does lighting because then in the areas where the terrain and the temple coexist then things would look weird things wouldn't match necessarily and that can be a problem so it doesn't have the transition between one lighting setup and the other line rick can't be based on geometry or materials it has to be a volumetric effect right so that's what we are going to do so first we are going to measure how far away we are from the origin and the temple is centered at the origin and now we are going to mask some areas with it okay we got it so this 30 and 80 values i remembered them but basically I think the temple is about what was it when we make when we were making the floor the floor tiles we said repeat it like from negative 5 to put it be 5 but each tie was 4 units long so the temple is about like 20 units why 20 meters or whatever so what I said is everything close the origin than 30 units you know keep it this is what this move step do is doing anything closer than 30 units keep the old lighting rig well the new one and anything farther than that smoothly over the curves of the curves of 50 units from 30 to 80 smoothly transition to the old lighting rig so in this case basically it's like the same direction was changing in space what are we like we had some weird relativistic effect where the light bends but instead of being based on speed or something it's based on location it's like you know it's like the temple had was a black hole or something and it was bending the right the light rates in a given direction and again if I was in these in the wrong context I would fire myself for doing something like this but this is the right context I'm just painting all these equations the time using our brush strokes they are not meant to be generic code they are not meant to be reusable they are not meant to be you know something that yeah anything professional it's just a painting I'm making it look good and the same way in a canvas you could put a color and if you're not happy you would go on overwrite it with another color and you keep putting splashes of color and strokes this is the same we go one by one until the image looks good at which point we publish it in Shaler type speaking of splashes of color I'm gonna go for another half an hour I think I want to finish something and show you my philosophy when I do light in so I showed you one already when decision I took where I'm so when you say in there and there is a flag to amplified and color yes yeah there lot of tricks here I won't go through those necessarily but there's one I want to touch which is besides fixing the conflict I had with the colors where the background and the foreground were competing potentially when I was doing the right thing for the temple and when I wasn't doing the right thing for the temple the background was great all those things there's another thing that also pops to me that that looks bad here something that screams for a change which is the interior of the temple looks too dark and too boring and bad it's not an inviting picture right now I don't know if it's my screen could be my screen as well everything looks gray and dirty and ugly in a way it's not a temple I'm invited to go in so I'm gonna fix that by faking some bounce lighting basically some indirect lighting so I'm going to add yet another bounce light here and I'm gonna make it pretty red so I'm gonna colorize the inside of the temple so when choosing a color you have to think what will be the color of the bounce light inside of the temple and you could you know write a path tracer and it would figure figure it out for you but you can't reason about it like if the Sun is generous and the temple is kind of sand color which is orange yellow then the incoming light which is yellow will be convolved or multiplied with the color of the temple which is orange and that's the color of the result in light go now this orange times yellow what that is is a supersaturated orange or yellow you know yellow times yellow is super yellow and yellow times orange is a very orange thing but intense way more saturated than the source color of the Sun and more saturated at the color of the temple so I'm gonna use a supersaturated orange color I won't put any glue on it actually maybe just a little bit because you never want blacks in the color to always want to lick a slightly bit of color everywhere just in case you need to color correct and bring information because if you put zeros in a pixel you cannot do anything with it once it's zero you cannot manipulate it but if this is slightly bit of something in there you can still multiply it by something but recover some color or some intensity you can shift it so it's always good not to put zero but I'm gonna put it's almost zero in the blue Channel almost black I'm gonna leave it there I don't know how intense it should be and I may get three and then we can make it less so now I need to create a signal which is the bounce light so I put it everywhere I get the super orange color everywhere but this is not what we want so let me look around see what I did I think we are gonna just you know the same way I used a volumetric mask to separate the background from the foreground meaning the landscape from the weather terrain from the temple it was just a smooth step to fade too great a signal that smoothly changed from zero to one and that helped me to create a soft select kind of thing I'm gonna use the same here to select where the orange color goes so I'm gonna use a smooth step of some values I think I have some values here let's say length of pause - 2 D as well and I'm gonna put some numbers and then we will find what those are okay let me break it into line so everyone can see what I'm writing we need this thing a bit bigger oops there okay alright so there you can see I have a blob of orange inside the temple but it's just at the wrong place because if okay let me remove it quickly these patches of light that are coming through the columns and hitting the floor they would most likely illuminate and bounce back into those columns in at the back not the ones in the front so we have to move the blob the orange blob to those columns in the back and that's gonna be I don't know I'm gonna try pink things until we hit the right spot okay that looks better is this what I want done yeah four maybe five is good five five something like that okay okay that is clearly too much though like we would never have that amount of illumination but we are in the right spot something we can do first of all before lowering down well actually you know what let's make it smaller so we've done so it looks a bit better okay that might be a bit too much to the point five okay maybe 0.6 now this is where your eye has to play a role at picking the right numbers okay yeah look at that awesome actually let's make the signal zero and disable the bounce light back there okay boring somehow better boring somehow better still looks like too uniform so now I would start doing things were not necessarily physically based or physically inspired even I would just start having variations to that lighting by using whatever I think one of my favorites is just to use the y-component you know of the normal we could be using the X component you just use the X component yes because okay so I'm modulating the amount of lighting by the normal yes because in real life you would expect different orientations of the surfaces to pick to pick up different amounts of lighting and since we don't really have a light source I mean we could actually you know decide that the bounce light source is done there on the floor in the center of the temple backward down where the light of patches are we could do the average of that location and considered that to be to be in the the source of the light of the bounce light and that's how actually all school virtual point lights would work in order to fake global illumination and then with that light source we could compute the dot product of the normal to that light source direction and do some stuff that you know that this is a quick drawing a quick painting anything is okay so in this case I'm happy with that and yeah I'm gonna I mean the original shader what I that I made has a better lighting than that it has more better looking colors and stuff but this shows the point where I just created basically you can take of this blob of color that I did like a volumetric brush where you know there was a painting I didn't like and I took my brush and put there a blob of color to make the thing look better and it's done it's just a smooth step and a square root like it costs nothing okay all the things we can do is add some you know like overexposed the image a little bit on the right side because the Sun is coming from there and those pixels on the right are super bright so you could expect that the camera the film is a bit overexposed there and that tell us a bit of - in the image in that area so that's something I do a lot and I'll need to that you need to do basically is just to measure the list you know how far the the view direction for that one pixel is compared to the Sun Direction what did we call the Sun we called it sound League something like oh okay that's too much so we shape it a little bit again by raising it to a power maybe eight and you know this is giving us an indication that the Sun is around that area so now we can take that again the power function just like in the same wave makes things in the sine wave the power function was helping us make things spike here in this case as well we can take this signal that indicates it's brighter the closer we are looking at the Sun we can take that and sharpen it a bit with a pal function by making it rising it to a high power like eight and then we can you know we make it a bit orange because the sine is kind of currency and then we lowered a bit intensity something like that and now that I see it I want it why there's actually so power for but now I lower it again in amplitude something like that and now I think it's too much so it's plus four we did seek it wasn't enough for is too much so we make it six this is kind of a binary search there we go great the next thing would be to convey the sunset you know quality or nature of this scene by making a red horizon and let's see this is the sky here but you know what because we don't have that much time I'm going to go to the final image and do it there it doesn't make sense for me to copy and paste too much when the code is here so this is the final shader which also have some projection based anti-aliasing which it's a very technical thing that helps basically smooth out the pixels and blur the image a little bit in a meaningful way not just blur in everything but really bring in a geometry a worth way so you can soften the edges anti-aliasing is actually super sampling but with my sky sky color here so actually just look at the sky step by step so this is what we had so that's the sky is blue as you can see the first thing I did was to add some clouds for a hint of clouds that's done by ray tracing a plane which is high in the sky and then texture in with some noise in this case I'm using just a texture normally I would create a mathematical cloud that in this case I was moving quick in the creation of this picture when I made it and yeah 2017 I definitely had a baby then so I just used a texture just to make my life easier and yeah so then the next thing is having the Sun set so this is a vertical gradient which I normally use to create the horizon but then the closer you are to the Sun Direction the longer the gradient becomes so I think I'm just an exponential to make the you can see Academy if you look at this line of code you can see that the sky is just blue in the background and then I have this line which mixes the blue with some level of grain based on this exponential of the Y component of the right direction meaning the closer the the more horizontal the Ray is the more horizontal a pixel is or the closer the pixel is to the horizon the closer to zero the Y component of the Ray is the bigger exponential becomes because exponential of negative x is highest at x equals 0 and then the more we are going to be blending the original sky color with this other white color normally the sky horizon is it's brighter than the sky and it's also more desaturate it so it becomes very disorienting you can look at we have pictures and you would notice that so that's fine and that great that horizon but then I use the same to inject some red color but the follow-up of the exponential which is 3/4 the background when it comes to the red color I make that fall-off smaller when the Ray is closer to the Sun so I also measure I again measure the distance of the Ray or the pixel to the Sun with a dot product I take that and I used that to decrease the fall-off of the exponential and that naturally makes the horizon color gradient to go from thin to thick again and then thin again as you move away from the Sun so that creates a nice horizon similarly those colors have to you know do spectrum colors for the sky have to match the color so let me show you this is what the sorry this is what the landscape looks without fog fog is or haze is just the the phenomenon where the farther away something is not only the smaller it is which helps you understand depth but also the blurrier or the fuzzier it is in color so listening on planet earth where because there is no air and oxygen and and molecules in the air the farther away something is the more of those molecules that are between your eye and the object and the more they light from the sky and the Sun are going to be bouncing around those particles and scattering light and basically the further away the more light the earth touches that's how I see it the air is transparent but it's not complete transparent just like water when you go underwater things which are far away they get you know blurry and they lose contrast and you don't get to see much so it's the same for the earth just happens that the air is thinner than the water and it takes a longer distance to make the colors blurry but the effect happens and there are two effects happening one is extinction and the other one is in scattering these are physical terms for how light behaves but basically the more molecules that are between you and something the less light will make it to your eye from that something because there is that light is hitting more and more particles and it's become like small shadow caster so there's an extinction factor which is this one here that I can show you the difference on and off it multiplies basically just fades things to black because there's less light coming and then I inject light back through the in scatter in which inject normally blue color which is the scattered color of the sky and that blue color this is the point I wanted to make has to be in sync with the background color of the sky so I'll both blend together and the same way the sky color gets more red the more closer for the closer a picture is to the Sun the color also gets more red the closer the pixel is to the Sun so instead of using just a constant blue color for the fog always good idea to colorize it I think that's it I think that completes the thing oh maybe just a bonus for whoever is left 20 people still good let me show you how I painted occlusion occlusion is always one of those things that still tricky but is very important so let me remove all you know all the post-processing and everything and just for the sake of let's make this guy white alright so this is our occlusion pass this is what we use for modulating all the lights except for the key light very important don't do that mistake I have seen people especially people in the WebGL community who are not that much into rendering and usually copy-paste shaders and from other places and I have seen that people they are doing you know multiplying the key light in their PBR shaders with occlusion and that is wrong because the Sun tequila is basically they deduct lighting right they delay the direct lighting component before things bounce around of the key light which is a control light source a small disc flatterers maybe a half a degree of a cone of light in the sky does the sound of the moon then basically that thing is in shadow or it's not you just cast a shadow array or what a few if you want to have some shadows or you do a shadow map and the key light is in shadow or it is not but occlusion is measuring something different occlusion is measuring the average exposure of a surface to the sky so don't use the clusion to modulate the key light because it doesn't make sense and all it does is to create a very unrealistic and cartoony and all video gaming look to your scene and in the old days sometimes they had to do because they had to do that because you know the lighting was computed in one pass and then ambient occlusion or the screen space ambient occlusion was computed in a different pass and once you have both informations the whole light buffer and the occlusion pass by then you already had lost the information of the different light components you had the only had accumulated lighting of the scene including the key light so all you could do is to multiply both and hope for the best hope for the best it wasn't a good idea because you were effectively multiplying the key light with the closure which is bad these days you can do differently so occlusion so I was an intrusion which is this thing that we are seeing as you can see modulates all the signals or the lighting components because I'm faking global illumination and all these lights except for the key light are capturing or mimicking or emulating bouncing lights that come from everywhere kind of but not a key light the key light doesn't take the occlusion signal but everything else is the backlight the rebounds light and the ambient light this is run of course because yes because the amp in life is the sky what the ambient light actually is the one that makes more sense to use occlusion but for the backlight and the bounce light ideally you should be casting shadow rays in the direction on the cone where the bounce light is coming from but you know occlusion since we already have it computed we can reuse it and kind of mimics a little bit what we are trying to do and if we do remove it and the backlight looks very flat for sure as long as we do the writing for the key light which brings you know 80% of the lighting information as we said is 10 times stronger than the ambient lights at least we are doing 80% the right thing and if there are 20% we are in the wrong thing but it helps to make it look better than 20% bad only so let's look at that occlusion signal because it is less straightforward than you might think it is not just computed by the usual SDF trick where you measure a few as the examples around the pixel that you are shading and then you kind of compare how far the objects are according to that as the F measure compared to how far you actually placed the sampling point from the surface and you look at that discrepancy and from there you win you derive an estimation of how concave or convex your pixel is so instead of you know I mean all of that is going on as well but that's not enough so this is what you get when you do that let me show you you get that so this is what you do with the cheap and well known now as DF ambient occlusion approximation and it is okay it does bring some detail it captures some nice details on the terrain it captures the contact between the tiles on the floor and the floor and the columns and the base of the columns with the body of the columns and then even the internal you know the little cracks that we carved inside the column to fake the stacking of bricks so all that gets captured and also it captures the concavity in the ceiling where we have that triangle that is curved from the base shape but it is not enough especially because it's not it's not a long you know the the sampling of the SDF is not big enough to capture the audio occlusion that would happen inside of the temple due to the columns that are there so I would expect the inside of the temple to be pretty dark in terms of occlusion and if I make it if I change the code that is computing this occlusion to make it wider then I use the small details and then I could have two occlusions and multiplied or multiply them together or do a mean function but then I'm paying twice the cost and the compilation time probably so not a good idea but something I do a lot and I did use this in films and is that because I'm in control of these shapes I made them I can create an occlusion signal that matches the geometry properly and let me show you this is the F stands for fake occlusion this is the occlusion signal that I am creating my hand to complement the computed ambient occlusion approximation it's not proximation self approximations but thanks to this extra occlusion I get that extra darkening in the inside of the temple I get some extra darkening in the inside of the triangle here I get extra darkening here then as you can see this small gradient of color here where the bottom will the top part of these bricks because they are closest to the ceiling they get darker and also the interior of the columns the part of the column that are facing inwards are darker than those which are facing outwards so all those signals that fake occlusion is made by hand and when multiplied by the computer pollution it brings the kind of you know put it all together it kind of makes the occlusion good enough to be a good source of modulation for the backlight ambient light under the bounce lights so how that one is done the fake occlusion let me see it's pretty much painted by hand I did I do start with conclusion of one meaning white meaning don't modify the computer clusion and then I'm gonna start adding terms where I make a signal darker and darker according to different criteria so for example where is my I think these are this is my architecture isn't it yeah okay they start looking yeah so this one H oh that's the one I don't know why I call it h Oh to be honest at the time it made sense for me Roley maybe horizon horizontal occlusion I think that's what it is horizontal occlusion clearly still remember three years two years after so that basically is doing what I was saying it's measuring the angle between the normal surface normal of the temple comparing to the vector goes from the point that is been shaped to the center of the of the temple basically basically it's like you can't think of it as having a negative light you know and light that sucks light in that sucks light that is placed at the center of the temple so you know if you had a positive light source you have my normal dead first in the middle of the center all the parts of the temple which are facing the light which are fighting the inside could get brighter in this case they get darker so all that is going on is a dot product I guess between the normal but I am only considering the X and C components as the horizontal like we had a two dimensional world and I'm doing lighting in a 2d world so that's why horizontal occlusion I guess and then I'm doing the normal dot product and then I am adding north at Y x dot north at Y probably this is to make to bend a bit the light source to the bottom on the top I don't know at the time it makes sense to me so that the first component and as you can see that helps take the columns which otherwise would be super bright in the inside and make sure that everything that is look facing inside is darker while the things which are facing the exterior are still as bright as they were before there's another signal here which is measuring oh yeah that is the dark blob for the floor so the floor also needs to get darker in the center because again they sent the more in the center view out of the temple the more columns you have around versus the parts on the outside so all I did is just to create an ellipsoid I took I take the length of the position of that pixel I am multiplying it by this amount here to make it an ellipsoid I'm stretching it twice in the component and four times in the C component that's the point five and you know you could consider you can think of this as 1 over 2 and 1 over 4 so I'm stretching the measurement by 2 and by 4 in x and y and x + C or Y in this case and using a smooth step to again create a selection mask and then I'm biasing it by 0.4 really we could do point to and get more dark inside there we go darker now and maybe we could even stretch it a bit farther and make it look better I don't know up to take probably probably didn't spend too much time but look without without I think it helps a lot and then I have two other modulations here let's see what they are I think they are related to vertical gradients probably yes so this is without when I add this one I get I don't if you can see the difference in the columns at the top of the columns they get darker and that's to simulate the fact that the top I mean the center of the column and the base they are very well exposed to the environment right there for catching a flight probably not very occluded but the closer we are to the ceiling the more percentage of the view the ceiling takes and therefore the more occlusion you will have in those areas so the higher you go in the columns the darker they should be and this is what these vertical gradient is doing and all it is is a smooth step of the position - um number and bracketed between - some numbers of these controls how wide the transition band is and the three point four computes how high the run starts where it starts and again that 0.4 0.6 just are by us to control the contrast so I can make it more contrast II like that and maybe like that maybe it looks better I don't know maybe that's too much a pollution Oh 505 anyway you can see that I just painted the pollution vertically and this one over here is probably doing the same guys doing the same for this area where this part of the ceiling because it's coming out it will potentially it will probably include part of the lighting coming to those points down there so that this another ramp there that is also done with a smooth step very very similarly it's at a higher altitude because this thing is higher and it's a shorter ear and smaller yeah it's a shorter more sharp transition so basically that is painting the pollution for the temple and as you can see is very manual the same way I painted the Internet light by hand by putting a blob I painted the pollution by hand some of it to create a strappy tail so that's it this is the final picture with all the elements in place carefully placed with lot of love for a whole day and we didn't talk about the terrain but yeah there is some color gradients there and the phone which are all tricks of course very different shades of blue but I think that's it I don't know if I had anything else I'm gonna open oh I forgot to do Q&A let's do one Q&A before we call it a day and see if people have any questions and I think I've been streaming ok I'm always terrified that I haven't been streaming for an hour or something okay let's see if people want to ask any questions where's my OBS here all right let's see if not I will drink some water minou young guess is from Korea maybe China I think it's Korea right could you cover the AAA anti-aliasing yes I can what caused the black dots features what could fix it okay so there are few questions so much effort yes it is a lot of effort but it's fun I think it's worth it I enjoy and sometimes we invent little techniques right so that's great anti-aliasing yeah so anti aliasing is a super simplified version of temporal anti-aliasing which is this technique that became popular three years ago I think because then real engine started using it and then also fueler games used it which is based on the idea that you know one way to help but not fix anti-alias a aliasing is to super simple which means to consider a pixel not a point at which you inject a color but an area and therefore that area in the screen when receiving information from the 3d scene will receive information all the area not just in a given point between the pixel what that means basically is that computing lighting becomes an integral and and frustrates rasterization becomes an integral and one way to simulate all those integrals and kind of approximate an area with current rendering techniques is just to have more pixels or have sub pixel if you want so you could think of it as render in a very high resolution and down sample it or yes having many sub pixels now doing that is expensive because it has you are you know it's more memory more depth buffer that you have to keep in memory more color buffer more bandwidth so you are using the GPUs if anything are good at computations but not so much as to add bandwidth and read in memory so it gets expensive very quickly so that's why I came cannot afford it if they want to keep a 60fps frame rate so one way to do it is multi sampling which I won't cover it which is doing it in super something only for the depth and not for the color but another technique that they invented that epoch was to kind of we use the rendering that you did in the previous frame of the animation and considered that information as valid information for the current frame so what you are doing is every friend you jitter a little bit your sampling points basically the pixels move a little bit by half a pixel every frame and if the camera wasn't moving and you were taking all those friends that you're jittering and just accumulating them over time you would get after some amount of time you would get an image that is equivalent to super sampling but when the camera is moving that becomes a problem because the jitter in basically doesn't guarantee that the thing you are sampling is the same thing you were sampling the previous frame especially the camera is moving fast or the object is close to camera so what these temporal anti-aliasing technique is doing is still do it still reuse the pixels from the previous frame but look up where that pixel what the current pixel was in the previous frame it's a bit similar to what video codecs do where do you want to compress color by taking differences across frames to encode only the deltas but they need to know where the previous pixel where the picture was in the previous frame so they extract motion factors and things the good thing is that in games you don't need to extract motion vectors like video codecs do you can generate them on the fly from the same vertex shader that rasterizes the geometry so basically you render a velocity vector field and then you use that to reproject the current pixel in the previous frame pick the color from there and accumulate it in the current pixel however in this case because the objects don't move and all the movement in the image is due to the camera movement I don't even need to compute velocity vectors because I can simply transform the pixels with the camera matrix of the previous frame and transform it again with a current camera frame and take the difference and that gives me a velocity vector on the fly on the spot without having to accumulate or store velocity vectors so this is what is done at the very end so this is the shader the main shader this is were we saw this where our lighting is overcome correction where color processing and normally this is what you would just do you know flat color equals call and that would be end of the shader and but instead this is the projection technique where I fetch the old camera matrix and explain I I can explain how I do that but I extract the old camera matrix from the previous frame and then I used that to generate the I take the point that we intersect it in that one pixel so we have the right Audrey Marx found an intersection that's a point in space and take that point I project it back I converted it to come out of space through the camera matrix I this is this line here come in a space then I project it to NDC or normalized device coordinates which is you know the your traditional negative one to one coordinate system into D by doing the perspective division which is how GPUs would do it as well I convert to screen space or viewport space by normalizing from zero to one and do the I undo the leader which is a technical thing but basically then I go to rest rest pace which is texture spaced from the previous the texture that holds the previous frame and then its sample it and I sample it here that's our color and then basically done here I do the cumulation I blend the current color with the color with red from the previous frame and yeah this I mean this one trick here where I'd use a make sure that I don't blend across objects so I take the depth buffer which I also stored in the previous frame in the same texture assign store the colors take the depths I compared to the depth of the current pixel and the current frame and if it has too big you know if there's a big gap in depth between the pixel and what was at that pixel before then it means that probably these two pieces of information belong to different objects and then I've done plans but that's pretty much what it is and then the trick to read from the previous matrix to read the previous matrix the previous camera matrix is to I store the matrix the 4x4 values the 16 values in four pixels in the color buffer so if you look down here I don't know if this will make it through the streaming techniques will mp4 and all that but here at the very bottom left corner you should be able to see a very bright pixel and that's because those four pixels down there are storing the coordinates of the camera so that's why I'm here and make sure that if we are in the you know in the first row of the pixels and first four columns then I stored the Camilla matrix in those four pixels so that's how it works that the rank and you cover it shortly yes well also in something you said that now you could avoid the artifacts from the FBM on the column distances how would you do it now I would do it differently I wouldn't just add noise to the whole thing well I think the scholar does those black pixels are due to two issues one is the fact that I wasn't using always the correct clamped repetition function you know how I use repetition but then I made a function of clamps the repetition so it doesn't happen forever in some of those floor tiles I actually did have an infinite repetition which then I clipped with a max function and that creates problems of the edges and those problems change with the natural problems of displacement in HDFS the problem I think so I think that yes by model by using the right repetition function I could get rid of many of those problems still if I had any do to the FBN then I would use the technique which I explained in the latest video in youtube which is modulating the curvature of the rounding modulating the rounding of the boxes instead of blindly displacing the whole surface up and down that would create a very nice look and if that still didn't fix and I really wanted those bumps inside of the of the of the tiles I think I would do some more constraint thing where I dream arch the whole thing without displacement and then based on the material I add another layer of displacement or something I don't know I would have to be willing to do it and try but I think I know a lot more things now that I did back then two years ago and I think I should be able to have a clean pretty image without artifacts anyway we'll do it sometimes it's not right for me to revisit shader that I brought three four years ago and tweak them again and fixed and fix things and improve things I've done it quite a bit so it's funny because sometimes I see people copying my shaders and they I they show up in unity plugins and places like that and when I see them I can tell when they copied it because some of them don't have the latest fixes that I did for the shader shader play so maybe one day I will revisit this one but yeah I'm confident I should be able to create a clean image for this one at this point yeah you're welcome min-jung can you cover the terrain yes the terrain this one doesn't have much actually normally as I said I would create a procedure a terrain with you know noise and fbms and a few other techniques but in this case I believe it's straight texture I think that displacement I mean the the height map itself so the terrain is just yeah here we go terrain let's look it up if I remember well you just yeah it's texture samples so well first of all oh what is it very sorry for it is for all this okay so hmm first of all this terrain is a height map which means the SDF it is not correct and actually just a week ago while I was at SIGGRAPH Asia I play with some techniques that allowed me to do a terrain with cheese and SDF which is a true sign distance field or a bound to it which means you can rematch it without artifacts like any other that's DF like a sphere and the shader is there online it didn't end up end up looking very good but the basics are there you can look it up it's my latest shader in Seattle however this is not that this is just a you know a plane the distance to a plane and then that get shifted up and down with a height map which means you get the wrong SDF especially if the terrain is very step if you have high slopes very like yeah big slopes then test EF is very broken and that's why I'm here somewhere I'm taking the SDF of the terrain and then multiplying it by 0.35 so I'm slowing down the remarks are potentially in those areas by a third just because my SDF is not correct I'm sure if I remove it I will have artifacts otherwise I wouldn't have put it there yeah but actually but yeah it's just a displacement so in this case the displacement is let's see it's made of layers let's explore what I did I forgot what I did so the first thing is a base layer height which is based on channel number two which is this texture so I'm using this texture it's a base elevation map you might notice this is texture good instead of texture so I could use just texture the normal you know the normal texture mapping of the GPU and then you can see there are these artifacts you can see squares you can see this very flat polygons kind of thing and that's because the hardware is using bilinear filtering to to sample the texture which means the landscape you are generating is just made of be linear patches and there is a discontinuity between the texels so I did write this other function texture good which I wrote and it's up here and it is smoothing out the bi-linear nature so it's not jump into my cubic interpolation still bilinear in that only four samples are taken as you can see here but I do disable texture filtering by fetching always from what is it yeah from fetching always from the mid level zero by using texture LOD and now when I have those four samples sample without any interpolation I do my own invalid interpolation by using a cubic interpolant visca it's kind of smooth it's a smooth step function which is this one nine one forty eight so basically instead of linearly transitioning from one color to another color for a given edge of the patch it kind of stays for a longer time with the first color before very rapidly transitioning to the second color and then taking a long time before staying on the last color so basically it's kind of a smooth enough a be linear interpolation and I have an article that talks about that I think it's called better better texturing if you go to my website which is here articles it called someone here interpolation maybe no yeah it improves however interpolation maybe no it's not that one okay I should do a better job naming my articles but there is an article here that talks about this exact issue but basically yeah it does that and if I bring it back you can see that now the polygon shapes are gone so that's great and what's my terrain here terrain so does the first layer let's see what the second layer your test oh yeah this is important so the second layer is the hill so oh no he'll heal and that's done yes by you know I placed at the rain really low and then I sculpted the the hill up by using a smooth step so the closer you are to the origin in terms of X and C into D because the the temple is centered at the origin the closer you are to the temple basically the more I push the terrain up and I just do it by just by mixing the current altitude to an altitude of negative seven which is where the base of the temple is and the blending between the original terrain and this bump which is a cubic function which is a smooth step it's based on the move step yeah then there's a third layer which is having a higher frequency again I use the same texture channel number two with a higher frequency so this was divided by 10,000 as you can see here 0.001 that's the coordinate divided by 10,000 here I'm dividing it by 5,000 I think or 500 or something it's a higher frequency version of the terrain again added to itself with a smaller amplitude so the first base layer is creating variations of you know from 0 to 90 meters and the second layer is having details of 7 meters height so it's a lot less people and this is how nature works right like the more detail struck more detailed structures you have the smaller they are as well so that's the principle of fbms and fractional Brownian motions and terrain something like that and then the last one I am having yet one more layer of high-frequency detail should be nice but instead of noise I think I just used texture 0 to do it more structure so it's again even higher frequency it has smaller amplitude so it's again the same principle big things give you the overall shape higher frequency which means smaller wavelength and smaller amplitude add more detail and even a smaller wavelength and smaller amplitude populate the thing with even more detail so the last one I didn't even bother to do the the smooth filtering I just want to try to no filter in at all actually well just baleen our filtering yeah that's it what is the USD box used for in modeling the roof Oh so USD box stands for unsigned okay that's funny because s d stands for sign distance to a box and the U is meant to be here unsigned but I didn't remove the S so it should happy you D unsigned distance that is USD box maybe I don't know what is it used for let me see it's probably too kind of clip the roof I guess we are talking of line 246 one let's see yeah I think it's clipping so the there rhombus the SDRAM pose which is the the Roku's side that makes the triangle shape in the roof is just a 2d shape which has been extruded to infinity and again in the my website I explain how to extrude things to this shape to a limited amount I call it elongation and there is an article about that and a few shaders but and that creates a very good SDF this thing that I'm doing here with USD box into 1592 46 it's a bad practice don't do it but it was probably the fastest thing to get me going again I had one day to do it so it was running but basically it's taking the 2d shape which otherwise it's infinitely big and chopping it and clipping it to a given region to make sure it doesn't extend forever and stays within the boundaries of the term of the temple so that's what it is doing and it's a unsigned because the implementation of an unsigned distance to a box meaning the distance to a box which just returns zeros instead of the box instead of negative values it's a bit cheaper cheaper like more efficient to compute you can escape a few max operations we can check it out what is it rhombus okay here we go so you can see that the USD box is like SD box it has this part here but it doesn't have all these min and Max part which means it's unaware of negative distances in the interior but for some operation that's fine and in this case it was actually if I were to review this shader as I was saying before and fix some of the black glitches I would also fix that and gonna break it down actually for myself fix black glitches fix us the box of rhombus with a long cutting operation yeah is there a way to add global illumination in ray margin yeah yeah yeah there is I think there are a few shaders doing it in shader toy I mean ray marching and global illumination are orthogonal concepts meaning that they don't limit or affect each other so yes so basically remarking if anything is a substitute for Ray casting ray casting being the core operation in a ray tracer so ray tracing is the idea of casting Ray's in this space as you know and following the light sources and shadows and the basic operation the primitive operation is intersector that's a ray cast operation some people think ray cast is just when you get in 2d like in a doom game that pre cast in general is cast in array in space and finding an intersection that you normally do with when you have triangles you can do that by traversing a KD tree or a BVH or something and there is all these technologies and remarking is a substitute for that so that you can rematch objects which are described not through polygons but through mathematical equations that's all it pretty much is from that point on and we didn't show it but that's just just it's just a for loop that slowly marches through space therefore very much in until the intersection is found and that's all it is at that point that's the inner core of the red tracer anything that happens in the outer shells of the of the renderer is independent so you could have a red tracer implemented with Ray matching inside or you could have a ray tracer with normal recast implementations you could have a podracer with normal ray cast operations which is what most people do or you could have a path tracer with an inner rematching core and once you have intersections you could do your regular light sampling Russian Roulettes Monte Carlo integration your you know your multiple important sampling all those things with very much no problem and I have a few images of my own that I have done with ray path tracing here and I have fully not the best because it's not like I use it so much but then I go to basic Monte Carlo this thing here is reemerged global illumination it doesn't have specular reflections so it doesn't look very realistic but you can see bounce lighting and things which were not painted by me my hands this is coming from a path tracer if the terrain is a height map where to the color where does its color come from colors of the of the terrain probably very simple probably is just a brown mixed with green based on height on slope let me check we close all of this and let's check it out right away [Music] okay so this is the terrain I think [Music] yeah so there is a brown base layer let's show that first actually you know what let me multiply this by something low so we can see it okay there we go so it's a brown layer nothing on it nothing special just well sorry I'm lying this would be a constant brown color and actually we should multiply it by three okay this is our base color it's kind of a pretty white from what I see 95 9 85 just that constant color then I multiply it with a texture which is texture number zero which is this thing here and I guess I used that to bring out of color variation so you can see now there are some darker Browns versus lighter ones or grace I would say just to bring a bit of color variation on the base layer and it's always about layer in detail just like in modeling so the next thing here is is the shoreline wet rock line you will see what I mean you see that line that showed up where the terrain contacts the water so it's gone it's back so that's to trick the brain into thinking that those rocks because of the tides you know high tide and low tide those rocks are sometimes underwater and therefore the growth of vegetation and maybe they have mussels and shells in them and they are wet so they are darker so just asta line which I just take did you know the height of the pixel in world space and world coordinates and the closer it is to the sea level the darker and make the pixels so that's what it is going on here it's a smooth step you know so the sea level is negative 25 meters for about one meter from negative 25 to negative 24 all that area is gonna be darkened by 0.25 so everything above is gonna be left unchanged and everything below that line 1 meter line is gonna be x 1/4 making it darker so that helps sell the idea that there is a different material there and then what else is going on there's a bump map to bring some extra detail which I think we could do without to be honest and then there is the crash in a year which yeah normally the grass layers are based on the slope so you can see the green they're showing up no green green so basically as soon as the normal of the surface of the terrain is fitting up meaning we are in a kind of horizontal surface we tint and green to get rest the idea is that grass doesn't grow in areas whichever is sloped like cliffs and things like that that gives a very nice look the other thing important thing is that I also removed specular contributions where the grass is normally rocks are shinier than the grass the grass is more you know it has like scattering and it's blades of grass are facing random directions as well and that kind of diffuses the specular reflection so normally not only a tint in green I also reduce the specularity and that's it just you know like 3 like base color times the texture dark line next to the ocean grass when it's facing that which is just green and that makes it there's a lot of trick in the brain into thinking that there are more things than there are that's something that we use a lot in films as well it's about tricking the brain more than trying to be photo real or realistic or having complex rules to grow grass based on I got it in some things this is good green and it works Paulo Vikas days Thank You Man yeah he says he learns a lot every time he watches one of these videos thank you I love your shade you're one of the very first shader to users as well remember your shader within black and white spheres okay guys thank you so much thanks for being here it's been fun see ya

Channel: Inigo Quilez

Views: 28,285

Rating: 4.9886684 out of 5

Keywords:

Id: -pdSjBPH3zM

Length: 160min 6sec (9606 seconds)

Published: Sun Apr 05 2020