Introduction to GLSL - Markus Heckmann

let me see if this is really recording yes it is welcome to the introduction to attach to to GLSL sorry I'm should drink some clerk motive over here to wake me up we are in the intro to GLSL workshop thank you very much for all joining it's pretty packed if it's getting too warm let us know I think we can open up more windows I'm Markus I've worked with derivative and yes I'll give you basically the lowdown of well the basic load out of how you can use shaders in touch designer be it for simple 2d things or be it for some three-dimensional manipulation of pixels and such so let's get right started by finding touch designer and open up touch designer there it is and we have multiple nodes in touch designer that are connected to shaders one of them being and we'll start with that oh I meant to mention this actually if many many people point to the book of shaders come as an excellent learning resource too once like to browse through and look at simple examples - more and more complex examples when you're trying to wrap your head head around certain techniques in GLSL so it's a really good URL to remember and it looks like it's being still completed and there's more examples coming online so and so often so it's a very good place to check for everybody who just came before I started handing out the sorry after I started handing out the USB keys there is a USB 3 USB keys going around and if you cannot get to this URL because the Internet is rather none existed then on this USP use P Keys you will find a folder called simple particle system and please copy that will not need it right away so just whenever the USB key comes across all right thank you so once you have touch designer open like this let's have a look at what we have for GL SL operators one of the things actually where you can already find cremate GLSL things is if you're checking through some of the image filters in the palette so on the left side when you open up touch designer by default you have a palette that opens up and there you find the image filters component and just dragging in the Cartesian to polar component which is the first one it's a very non doesn't do much right it's basically converting the texture coordinate system from a Cartesian to a polar system and you can see that this is done with the GLSL topic and the GLSL top because you're you are in the top world and the texture operator world means that all the operations done on here are on the pixel shader side while you can address a vertex shader here I think you literally just have one polygon that you're drawing on it I actually never used the vertex shader on this just the pixel shader anyway a little example where you can find something like this so let's put down GLSL top and for that open up the op create dialogue can't how many people of you actually are not novices like not total novices in touch designer so everybody but everybody knows how to place an operator how to find the yeah if you want me to if if you need help with things that i might jump over or just assume that everybody knows please let us know we'll be happy to explain so when you open up the upgrade dialog you find in the top family you will find two nodes relevant to GLSL which is the GLSL top and the GLSL multi top so let's start by putting down the GLSL top the whole thing comes with and adapt attached to it and it gives you a starting point to how a shader works or what you can do inside a shader currently you see that the output is a nice white square so it's not doing anything but if you explore the duct a little bit you can see that there's something here out back for frak color and then so there's some definition of the four-item vector that's called frac color and then there's something that looks like the loop or a function it's a function and the main function here where a new color variable that has four that has the size of four like a four item vector is being assigned with something with a number 1.0 and if i change this year to 0.5 the thing that you would expect that happens happens it turns gray and this whole thing this color here is then assigned to what was earlier defined as an out like out but it's what it's writing to essentially frac color and then it uses this TD outputs missile and just just take it as what it is the TD outputs missile for now we'll look at that a little bit later and yes and it's a Veck for ya because color would be would be expressed as an RGB of a value correct yes no because so that's the next step here what you can do in an GLSL is is the way how you can assign values to variables and while we hear declared a color variable that's of the Beck for type a vector with four items then what you can do now you have access to each of these items by what they called missiles so you can type color dot and then you could control for example R and you can say R is 0.9 and then never forget the semicolon and you get a beautiful flesh color skin color and you can do this with all these missile types so we also have color dot G for example and you could say 0.2 and make that more pink and color dot B which you could assign to 1.0 don't forget the semicolon and so you see we are changing we keep changing the color and last but not least with a color that a set to 0.2 the Alpha is changed but I have to do this because I have my preferences I'm not sure the preferences when you open touch designer by default come as a checkerboard background for the Alpha and I always have this as black but I'm going to change that to check a port and you see basically now this color dot a is controlling the Alpha yeah oh sure sorry yes of course it references dip dip dip dip sighs No I never did that before right I can know that it's all 15 right yeah let me just also fix that at content you and what and it will be mainly for now I'll try to do it like this so these results there's a there's a multiple of these results are available you have rgba for and they can be used interchangeably like you don't have it doesn't mean that if you say color dot are that you have to define color that are basically means it's the first part of the BEC for so I could also say color dot X and set that to zero point one and now it's affecting the first part of the color vector so those results that are available are totally up to you to use in which way you would like they are they're more for you to also know what you're actually what you're actually adjusting so we have dot are the G dot B dot a dot X dot y dot that dot W and dot s dot T that you know not that you I actually don't know you have you oh you VST ok thank you obviously I never used those I'm sticking with with I'm sticking with those but that's perfect another way to - I showed you how you can address one single or we looked at how to address the whole vector by one by one value here we addressed single parts of the vector but you also can do things like color dot RG equals back to because now I'm only talking to two parts of it zero point two comma zero point five control s and you have control over two parts and you also can do square brackets I believe yes it is literally it's literally the way that you want to name them so you know what you're actually what what you're doing right so by default the if you look at texture operators in touch designer then on the common page you have an option or you have a menu entry called pixel format and the pixel format pretty much defines or sets the value range that you have that you can if you have an 8-bit if you have an 8-bit which it is by default an 8-bit color value then you cannot go outside the range of 0 to 1 but if you increase that then you can go outside of that range as well but in general you will have if you're just dealing with pixels and you're using the pixels to look at things then your value ranges 0.021 if you're starting to use which is totally which is it use of shaders if you start to use pixels to store information or whatever you can store in there whatever values you like setting the right pixel format so what happens when you forget one of these semicolons or you do something else that's illegal tap shows this blue red checkered board here and alerts you to in to a compile error and it tells you to use the info dot to see details so it's usually a good idea actually to put down an info dot because yes and the info dot has a operator parameter and you drag the GLSL node into it onto the operator parameter and now you can see ok the pixel shader reports an error syntax error unexpected identifiers blah blah blah online 18 and you can basically know well you basically have to work back and try to see you have a you have an error that shows up on line 18 as an invalid identifier but the last what was actually happening because of the semicolon missing here it tries to go further in the shader and alerts you to this error so the info dot is really a very useful thing to have connected to a GLSL top at all times or to a GLSL material at all times to see what what errors are happening there sorry that a passive that is if if values in because you can attach any kind of operator on to the info dot and if if you're attaching operators that you've cooked every frame that are force cooking every frame then the passive I believe on only updates the info dot when you look at it and I'm looking at Jared - yeah yeah my sublime language in context yes you can see if it's already installed of you have to download it with the package control I'm not sure on Windows it's ctrl shift P how to get there [Music] yes yeah that will be all posted eventually so one more thing about defining values here because it's just all these combinations of swizzles and whatnot you can also do things like this color our yes our G equals now let's do this float red so I'm defining a single valued float here float red is 0.8 and now I could say color RG is Beck - because I'm describing I'm setting two values on this color vector and I could say red and some other value 0.1 so inside of these vector descriptions you can also then use other variables that you defined earlier so the Oh frack color and color so okay the the frack color is is a definition that's what you can do in touch is you can you basically tell it to write to an output buffer so you can you can tell it okay take this pixel shader and then I would put output what the output of the pixel shader is or your calculation to the screen or to some other buffer that you're defining and we'll get to this in a bit and to define all these output buffers you can write out the for output and then beg for which would be because it's an RGB a color buffer and then a name and frac color is just the name that I've given this thing what I can do here is I could learn lycée my output and now rename this to my output and I have the same thing so it's basically a definition of an output name essentially the the buffers will be named yes it's by buffers because you can have many buffers that you can address and it becomes important that you name them properly so that you know where you're writing to I'll get to the buffers in a bit because yeah I'll show you how that works in a second the next step the next thing that is already in a shader when you just put down a GLSL is this mech for here I'm just gonna comment out you see commenting outlines is by two slashes and the next part that is here is the that comes in the shader by default is something where the color is defined by a texture called so it's texture which calls a texture that's a GLSL function and then there is this STD 2d input 0 which is how you describe STD 2d inputs is an array with all the inputs to the operator to the GLSL top and then 0 or 1 or 2 the defines which input you're picking and then after that comes vu v dot st v UV being the the texture coordinates and this is basically the pixel that you're operating on and this is already defined inside touch you don't have to define that anywhere so what this texture call does is it looks at the first input and then picks a pixel at a position that it's currently operating on assigns that to the back for color that is the fact that's being defined here and then it's being output at the end so you can try that to try that would be adding a movie file in top and you can plug that into the first input of the GLSL top and now by taking these two slashes out you get a banana here's to be asked one sorry oh yeah that was thick press me before and I didn't answer that yet because of the yet but then we actually came up you need to define that okay I have some better information to the TD output Swizzle a function that I generally just put down but Malcolm's going to give you a short closer explanation to that yes basically you just always want to have whenever you put a color from any of our changes both the materials and is tops you always want to output it through that function and the reason is is because mac and windows stores stores textures differently for example on mac there's no such thing as a alpha only texture so internally I need the stories of reading only texture so this essentially just takes the value that you carry also and that moves over to the red channel on that so essentially it just allows your failures to be portable between Mac and Windows that work the same on both of them we're going to despite their differences in the way they use OpenGL so really you just just always use it really is in the end it's funny to do okay okay back to the textures we have one input here and just to show you the second input and I think you can repeat that for yourselves quite well copy and paste the movie file in and pick a different image or sample image and plug it into the second input and now you can define the second color back for color two is texture and now the STD 2d inputs 1 comma V u V dot st and a semicolon so now I have a second color here and I can do something with those two colors I could for example now say color equals I think actually that works to cover two so I'm chef struck I'm subtracting the second texture from the first texture here and what I get is banana with the sky subtracted from it so basically you can get to multiple textures that you're inputting and then you can combine them or manipulate them in any way one thing and priam us that earlier actually was that one use of GLSL textures or GLSL materials j GLSL tops specifically can be if you're running systems that where you're running into problems with top chains you have lots of tops that are taking up too much cycles on your GPU or too much memory if then sometimes it is advisable to combine them into one shader and that's most of them if you take like a level a composite and a level or something like that and those are these ways how you you write your composite function yourself and then say your level function would be color x equals 0.4 and you have your own simple level right there so it makes really sense to do these when you're running into trouble that's too many too much pressure on your GPU yeah yeah and it does it would be a one-to-one pixel match because it takes the first input as a resolution and then you have the UV is stepping through the UVs or through the zero to one in each direction of the first texture so you're not sampling every you're basically squishing or stretching the second input yeah so if you take a little let's just try that right here if I change the resolution of my second movie file in to an eight nothing really happens to my output but the GLSL is still the GLSL top has still the resolution of the first input and it's sampling with those values so in this case you would be a sampling multiple times in the second texture turn up slack before doing workshops there we go all right now now we looked at getting a textures and we looked at the different inputs to it GLSL top and we have also this vuv dot s T which is the texture coordinates that we want to look at that we want to sample from the texture and obviously we can also take this vuv which is given by us from touch designer and change it so we can make our own back to text court and just say V U V dot s oh sorry I'm in the wrong there we go bu v dot s T like this and now we could say for the now we want to offset one of the like we want to offset the whole thing in X for example so text court dot well let's take s plus equals zero point three and now we're shifting the whole texture a little bit and now we can use we can use this new texture coordinate that we just made and use it to pick the color the pixels from the first input which was our banana and our banana shifts because for the first four there you or poor yeah for the first pixel so to say that its operating on it's actually looking zero point three units into the texture holds the pixel from there and places it when it's currently operating pixel um Didem okay similarly you can also stretch you can also stretch these coordinates so if you want to do scaling in a shader then by multiplying the texture coordinates with a certain value you can squish the image together or scale it up but that's probably there scaling the whole thing now the scaling happens from the first like from the left side because that's the origin of your coordinate system there okay now we already talked about all these different buffers that we can output to but we never have done it actually so let's have a look at the GLSL top and oh sorry no we should do something else before the u-visa passed them by touch designer itself so that's you're just getting that it basically it's screen space so yeah that's correct and this is so in the so there's a difference between the GLSL top and the GLSL material we're in the GLSL top it's given to you and in the material then you actually have to get it from the texture coordinates itself I heard something like a plaque or something sorry yeah is it is it does it have power I heard yeah we can maybe there was the first one that I defined as an output yeah yeah let's have a look while the projector is all the projectors coming back by itself that's good if you're if you're having a look on the parameters of the GLSL top itself then on the first page the GLSL page it has a number of color buffers at the yes excellent at the bottom of the GLSL parameter page you have a number of color buffers and let's just do that right away we have it's set to 1 by default so let's output two color buffers here and previously we changed already that name here the default name was frat color and I changed it to my output but I can just define another one here out back for my other output and then further down I can just assign to my other output another color and I'll do the TD outputs visual function as well and then I'll just assign to my second color buffer I will just assign the color to so the second input that I had there now nothing changed on the output of the GLSL itself because the GLSL top itself is only it's only possible to look at one color buffer at a time but we do have these we do have these render select tops and when you put down the render select it doesn't just take render tops but it also takes GLSL tops any top basically that allows you to write to multiple buffers and so let's drag the GLSL top onto the top parameter of the render select one and they call a buffer index parameter off the render select has by default one but now you can switch between the two color buffers zero was the is the output of my GLSL where i do the color manipulation and the texture coordinate manipulation and the output buffer one was just the second input is there this if you want to look at a good example for lots of color buffers and where you need this and we're getting to this later in the workshop under tools you can find a component that's called particles GPU and when you drag the particles GPU in and look inside the component itself then and we'll get to this later but this really a good example for it the did the GLSL top that calculates the particle position cannot or does not only have to calculate the particle position but it has to store much more information than this it has to store rotation of the particles it has to store life how old those particles are and you only have four color channels though per buffer so all this information you need to store in more buffers which are then like here's a velocity which it's hard to see let me find something so here we have for example the position for all these particles stored in a separate color buffer and yeah life doesn't look too exciting but you see it kind of the red dots going away there so that's a perfect example for color buffers where you actually need them to where they don't go ahead where they're not optional anymore okay is it uh Oh if you go in order not even an order it guarantees it okay so in this case what you want to do is you would define those as by one name like frak color which is the default and then assign it a array size so if you want to have well we just have to color buffers no oh oh right sorry yeah thank you and then to output to those and I should be doing this in here so everybody can see it thread color zero so you have a Ray access to that and track color one and go back so that's a good point I really like naming them but I guess I just been lucky because I'm running on my computer okay yeah I have it here somewhere but I can type it out yep this would be out Oh to the right we can also find that on the hub page here nope we don't have internet do you have it in your yeah obviously already like for and the name third out outputs with let's just give it a default number or like a gray okay and increase the color buffer to three and we have gray on the third output okay yes so definitely yeah if you want to name them then we need this layout qualifier on at the start you only can when it comes to textures you only can reference what's what comes via the inputs but how do you get information into the shader one of the things for example that you might want to do is I have been typing in all these values here but those values are usually things that you want to control from the outsides they're not fixed and this is when we want to start defining uniforms so if we want to have control over this scaling factor that I have defined here we can define outside of the main loop a uniform so we'll type uniform and we'll make it a float because it's just a single value and we'll call it scale X and instead of 20.7 I'll use scale X here save it and now we need to define this exact uniform that we defined in the shader we also have to create it on the node itself and that's what these vector pages are for so on the vector one page if we type scale X comes right away back because the default value is 0.5 and now you can control this scaling factor from the node itself so now exporting onto it with an LFO chopped or if you need if you need to bring those things inside and these uniforms can be as you see here all these uniform names have these uniform values have again up to four characters or up to four items here and therefore you can define uniforms that are of the type back for as well or back to or whatever you need so in fact if I don't want to just scale in X but I also want to scale and y then let's rename this to scale and I'll make this effect to scale and I'll be transforming all of my texture coordinates here was the scale parameter then now you have control over the X scale the horizontal scale via the LFO and I just gonna turn that off zoom in a little bit and Y scale with your second value here so you can use this to define or to bring in colors or any other values to control your shader any questions on this nope okay one thing that's really nice to do in touch designer and I'm not sure if you all know about this sometimes you make little examples and you want to package them up and just yeah have them in a component itself you don't have to copy and paste into another component you can select all the nodes that you that you want to have inside a component and then right-click onto a pane and say collapse select it which is pretty low down in the menu here it collapses it and you can call it 2d GLSL examples all right and I'm gonna close this and the next step will be looking a little bit at the GLSL material so to use the GLSL material what we would need is what we need a scene let's read the 3d scene to render out and the easiest to do is probably let's create a box so open up the create a lock and find a box up now usually what I do to set up a render scene you need a geometry component so what I do is right-click on the output of the box up and get the geometry component here and we need a camera and we need often light so we got a light here although let's skip the light let's just take bender top and sorry there's a digital component and the last thing we need is a material and in this case we'll take a GLSL material so in the up create dialogue go into the material page and find the GLSL material and let's reference the when you put on a material other than the Fong know always we need to reference the material on the geometry component itself so click on the geometry component going the parameters to the render page and then drag the GLSL material onto the material parameter there even though it has an error and we can check what the error is we don't have a we don't have a vertex or pixel shader defined so to start with vertex and pixel shader let's put down two text that's text and we'll call this the vertex shader and another text that which will be our pixel shader and we'll reference those two shaders in the in the appropriate parameters in the GLSL mat so when selecting the GLSL mat on the load page you find the vertex shader parameter so we'll drag the vertex that into it and you find the pixel shader parameter so we'll drag that into this as well okay so we need these two elements one is the vertex shader which basically acts on each vertex of your incoming geometry and you use what you do in it is you describe how this vertex is brought from the object space to the camera space so that you can display it through the camera in your render top and the screen space yes thank you and the a very simple vertex shader or all what you need for it basically for the simplest one is similar to what we had previously in the pixel shader in the GLSL top will need a main function and then we'll use a call which is GL underscore position and this once this basically wants all the vertex positions taken from the world space into the screen space projection space sorry oh it's right altgr there's no all tr site for it spacing in the space yeah just a projection yeah with a pre given function TD world to projection and then insight here will take a TD d form function and inside that will take P and let's take that a little bit apart so the TD world to projection is a function attached designer shader function which converts from world to projection space the TD deform function will take your vertex position attribute which is peep which comes in by default and do any deforms on it that are through bones and instancing there's if you go to the page if you go to the page how to write a GLSL material it lists the it lists pretty much the single parts as well because you also give the single parts that the TD deform does if you need them right yeah of it correct and the other thing that TDD does it is also converts your soft space positions and was that given we can also write a very simple pixel shader and we'll start again with the Outback for track color from previously and we'll need our main loop and we'll need to define output color and we'll give it a constant color whose color and I'm slightly confused why I don't see this as a white box sorry Oh huh thank you I probably should have started with deleting the default Network here because the render top picks up the render top by default picks up all the all the geometry as the geometry parameter as a star in it so okay so we're rendering a white box here now let's go back to the vertex shader and let's see what we can do with the vertex shader and for example what we can do with these positional attributes that are coming in by default which is this P we could describe our we could get our positions or take the positions and write them into a custom variable so let's say well take back three my position equals P yeah oh he's sure yeah it vote yeah that's probably a good idea to put down again as I showed you before you have the info dot always connected to your to your shader so that you can get back if there's errors but you're guessing the obvious here with taking these positional attributes and writing them to our own variable vector three variable we have the chance now to actually transform the position of each vertex so taking my pause and D the x-coordinate of it and setting it to a specific value or actually adding to it so plus equals zero point three we're moving the box my position that y plus equals 0.3 or 0.5 you're moving the box up describing this as a uniform as we did previously where we can then use outside parameters to control these values here we can define uniforms in the vertex shader as well by typing uniform we'll take it back to four X Y and call it translate we could also let's make it a back three because we'll just be able to translate it into all three directions and then say my paths class equals translate and now in the same on the vectors page of the GLSL mat we can define that uniform call it translate and because by default the values are 1 1 1 it shifts the box up into the to the right hand corner but we can move it now via these parameters around perhaps even again use an LFO chopped and in an LFO drop you can define multiple channels at once on the channel page via the channel name so t square bracket XYZ will give you three channels here and we can export that onto the translate uniform that we defined I just gonna offset the face here a little which with an expression which is me dot sorry you can create three channels on the channel page by saying a toq square bracket XYZ or you can type it out I mean you don't really save that much here T X T YT Zed and then we can create an offset or the phase offset by typing something like 0.1 x mito-chan index and you basically offset each channel by the phase zero point one depending on the channel number times the channel number now the next thing that you can do in in shaders especially if you use the GLSL material is you probably want to put some kind of texture onto the geometry an outside image that you pull up onto the geometry itself and for that you need texture coordinates now most young most geometries that are generated in touch you can define the texture coordinate as it that it's created with and you can also double-check when your middle mouse click on to an operator onto a sock then it tells you what kind of attributes are attached to that operator so here you can see we have a point attribute end which is the normals and we have a vertex attribute UV which is the textures middle mouse click on the box yes then oh oh this one sorry yes so this is called exporting so to export values from channels on to parameters you select the operator that you want to where you want to control the values so in my case that's the GLSL material and you go to the page where those values are and then on the on the top here click the active flag your active flag which is the right corner so the whole border disappears of that node and now when you roll over a channel it turns green and now you can take this channel drag it onto the value that you want to export it to and let go and the little menu pops up where you choose export shop and you can do this with all three to this now to add a texture on to this cube we first need to have that image so let's put down a movie file in top yeah yes so we do have for the external editor yeah it depends on how it's being opened and we do have in the in the text that there's a there's an option here where you can say edit view extension and by default that says from language which would be Python or T script but you can select which language it actually should should be so what I did I didn't set that I just set the coloring in my in the editor to it but you can if you pick GLSL here for these two then they always will open up as GLSL and be recognized as a so that's on the comment page edit view extension GLS out yeah [Music] of GLSL there's the list of functions on the example yeah yeah yeah it's a long wiki page that explains it it's not a just a list it's an article so it goes through all of the all of the sections but it's yeah yeah it's it's it's an article which basically has all the explanations right yeah that would be them yeah I don't know for sublime either all right does everybody have a movie file in top so let's pick an image that's not the banana maybe and what we need to do here is we need to a read in the texture and we did that previously in the pixel shader we'll probably do that again in the pixel shader now but in the when we use the GLSL top we were given the texture coordinates by touch designer view VST now we have to get them from the or we have to take them from the geometry and move them to the to the pixel shader stage so first let's get this image here that I have on my movie file in and attach it so that we can access it in the GLSL mat and this can be done on the samplers page so if you click on samplers one you can give this sampler a name and let's call this I don't know I'll call it a photo and reference the operator just gonna attach a null here for prettiness what this means now is giving a sampler name that means that inside our shader code this this texture will be known under photo not under the top name but under this name that we have given it here and now in the pixel shader we can already get to the we can already define our incoming sampler and we'll do this by typing uniform and then this is a sampler 2d and then the name photo but again what's missing here is now previously we had movie we need to get from the vertex shader stage we need to move this into the pixel shader stage so in the vertex shader let's define the texture coordinates so well yeah to send those over to the pixel shader they'll have to type out and then that's a vector to for the texture coordinates okay [Music] and now we have to define what does text chords are so inside the main loop we'll write text chords equals UV and this is given to us by touch designer and that's an array because you can have multiple texture coordinates on an object so there's the first one dot st so we're sending texts courts out and now on the pixel shader site we need to receive it so go to your pixel shader and in the pixel shader type in back to text courts I think I called it text chords and now we can do what we did before we can take we can describe the out color or assign to the out color the color values that are picked from the texture photo from the sampler to the photo so we'll type out color in the pixel shader equals texture was the call to get access to the texture photo is the name of the sampler 2d and then text chords texture is the GLSL function so what we did here is essentially built a very simple shader that takes the geometry and assigns the material to it the vertex shader yeah and I'll package that up again into a component here collapse elected and call that material SL example so the reason for GLSL why I mean it's the processing of GLSL is essentially much faster obviously on the GPU than doing a lot of the calls on the CPU and therefore things that can be done is if you have to do operations on a huge amount of points for example and you want to do you want to offset points then you don't necessarily want to do that on a soft level but you want to do this on the graphic cards level a good example for that would be let's place down a grid a grid sup sorry okay it's going in the stages of the of how the in the shader stages so first you have the vertex shader stage and this is then going into the pixel shader stage in between there you have the geometry shader but if you only have the vertex and pixel then the out will send it to the pixel and then out on the pixel sense it to the party that doesn't work exactly because it does first the vertex and then the pixel [Music] known outside there that's correct too so so what does if you let's take for example the grits up and let's try to offset this grits up by a noise top oh sure I mean I can this basically an example that you don't need to work with because we'll be replacing it with GLSL so maybe I'll just move on here we have a grid and we want to move the points of the grid in the z direction by the values taken from a noise top and a way to do this would be a pure node way would be to convert the grit into a into a Chuck so we'll take a sub to chop it up to get this and now we want to convert the noise into a chop as well so we get the colors so we'll take a top to chop and reference the noise on it and now the first input to the top to chop is UV sample values and we can use the the grit here we have to manipulate it a little bit because UV sampling values are always between 0 & 1 while the grit has its Center at 0 itself so we'll take a math here and just offset that by 0.5 but again you don't need to do this with me and now I've got all my noise values here and we just need our because it's a monochrome noise and we'll use the r2 with chop to stop to offset the values in dude up okay and we can compute normalcy as well that's great so yes that's basically the regular workflow now this is all fine and dandy if you have a low amount of points or you just have a static image but if you start animating if you start animating the whole thing then pretty quickly depending on the number of points that you're acting on here you will see that your framerate starts dropping I'm pretty impressed actually how long that takes here who needs shaders 50 frames per second nobody realizes anyway so you have this eventually you will have hits performance issues here because maybe you won't do something else than just that so we already did parts of this because we used we created this shader where we offset the box vertexes by some values that we were exporting onto it so we can recreate this whole thing in a simple shader Network and I'll just delete all this now I'll keep the noise here okay delete this we donate the Chop to and instead I going to set up a little render network as we did before so I right-click on the output of the quit shop and find a geometry component and add a new camera just in case you didn't do that before I did package the previous example with the material into its own component so that it doesn't so we don't get confusing not multiple cameras and the render picks up only one geometry etc okay so I'll create a render top here as well and this time I'll also create a light and I'll create a song that because what we'll do is we'll use the phone mat as a starting point for this example here so the phone needs to be referenced in the by the geometry on the render page you find the material parameter so drag the phone onto there and now what you can do is with the phone here the phone has a little pulse button here where it says output shader so this can create a prepared function up for you when you hit this output it creates a phone one GLSL and the vertex and a pixel shader and the vertex and pixel shader as they are are exactly what the funk shader would be I just use the default that is selected there and click OK you can do two-sided if you it's not necessary I think we'll just look at it from the front doesn't matter for our example yeah so all I did was I place the phone and I referenced it to the geometry material and then what it if you click on the phone and bring up the parameters on the first page right at the bottom it says output shader and when you click that just use the defaults here click OK and it creates the phone one GLSL and now we can reference the phone one GLSL that was just created on the material page of the geometry component nothing changes as it's exactly the same so far and I'll delete the from one so next thing that we want to do is we'll take the noise here and we'll use the noise values to offset each vertex so first thing that we need to do is reference the noise as a sampler on the GLSL material so on the samplers one page will call the sampler offset and plug in the null or the noise [Music] [Music] all right sorry about that what we did previously here was that we defined the we defined the sampler and then we used the sampler in the pixel shader to attach it onto this cube but what we would want to do now is we want to bring in this sampler this noise into the vertex shader and offset the Z offset it in Z so let's edit the vertex shader right-click on it and say edit and I'll shrink that together again and it's probably a good point also in the class here to look at what actually this output shaders created here we have a bunch of uniforms that correlate with the color parameters that we have in the phone mat by default so those are all in here and you have something called an out vertex it's a it's it's a struct structure exactly where you can pass multiple variables as one structure over to the next chain ER stage which is the pixel shader here in our case flat int now this is something you have to send integers as flats and I actually don't know why [Music] would you would you just get illegal values when you do just int in error okay it's you don't you don't necessarily need it right away here it's just because we created it from a phone and yeah so it gives you an error you have to define an int that you sent over as a flat end and further down the first thing that's happening here then it says first deform the vertex in normal and then return the values in in world space and this is being given to the GL position so what we want to do is we again one of monkey around with this positional attribute here so first let's get in the image the sampler so we'll have to define another uniform here uniform I'm blanking I'm sorry sampler 2d and we called it offset and now we can fetch this we can fetch this upset texture in here by defining of well actually we just need one value because it's a monochrome texture anyway so we'll take a float offset value or Val equals texture and it's called offset comma and then the texture coordinate would be you b0 s T and what did I do Estie is basically the swivels so you could say it's which part of the variable which could be a vector and then which part of the vector you're pulling out so previously we had the color vector with the RGB a and so for any vector you have to or if you want to just have parts of that vector then you define it by a swivel and st would be one of those swivels yeah I can write those out they are R so we had a swivel RG be a then there is XY that W and it's UV st right it's st peeking okay yes yeah you can you literally use them as decorators or not that great but something that makes it legible for you alright let's check the info that oh right obviously so what I did here is the info dot gives me the error implicit narrowing of type back for to float so what did I do in my vertex shader here I said float offset value I defined it as a float but then I'm looking at a texture which will return an RGB a value so I have to narrow this down to just look at one of the components and it's all good again and now make myself a new an offset for the position and I say float my offset P equals no not float sorry back three I need the pole position is P because P is not something I can change I have to assign P to my own variable and then I can say my offset P dot that equals offset up so I have created here my own offset position and this I now need to pass into the TDD form function and then towards the widgets then pass to the GL position so let's take my offset position here and what I get is a quickly changing just going to lower the frequency a little bit very flatly shaded but offsetting grid so again there's a there's a couple other things in the default function that we don't necessarily require which is for example you don't necessarily have to have a camera index if you only have one camera but given that you output it from the function or then all of these things are by default connect supported same was instance coloring etc this is not necessarily required now but the next step what we would want to do is we actually would like this to be properly shaded and we would need to calculate the normals for this whole thing after transforming the position so to calculate the normals we need to give it the normals yeah yeah we give it exactly the grit itself it's just a plane right yeah giving us this kind of thing so to do the normal shading now or the the shading of this all grid we would have to pass in the normals as a texture as well and we can do this by adding as a new branch and adding new branches in touch middle mouse click on the output of the noise that you pluck down first and find the normal top normal map top and pass that into the shader into the function as well I'll call this sampler normal map and reference this now to to it there probably is but yeah I mean that there definitely is but since there is an operator I'll take that yes okay we've got that connected and the next thing we need we can apply it or we can recalculate certainly the normals of this grid but we also need to do this kind of mapping we also need tangents for the geometry because currently we also have texture coordinates and normals but we also need to add a tangent attribute to the whole thing so let's add onto the grid after the grits up let's add an attribute create sup and enable compute engines okay and now let's grab the normal map in here in the vertex shader again by saying the uniform sampler 2d normal map okay and we can fetch those normals by defining a vector3 and we'll call it normal text equals texture normal map comma UV 0 dot s t-that RGB so I said that we needed the tangents the tangents attribute and by default we get attributes like the positional value here or the texture coordinates but something that's not available by default to us is for example the tangents so we have to get in the tangents here outside the main loop so we can get attributes that we define and those can also be custom attributes that we define by in that's a Veck that's like for thank you the tangents yeah because it also has the is it called a hand eNOS and eNOS yeah so in fact for T capital T and the next thing that we need to do is we need to convert the we basically need to convert the the space from tangent into world object space or world space to conversion the TBN that's from into okay and luckily to do this malcolm added a function for us in very recently where we can convert this the tangents into this t v-- n matrix which will be required in the pixel shader stage to recalculate the normals and you can define matrices in touch designer as well so let's do a matte three and we'll call it t BN met equals and now the function is TD create TVN matrix and what this TV n matrix requires is normals as the first value to this function then the tangent the first three parts of the tangent vector T XYZ and as the third argument to this function requires the handiness so T dot W so it's X Y Z is the oh yeah exercise it yeah X Y that's how the how this works right yeah that's by default the like the GLSL convention because you have X Y Z I guess and then they add nothing else to go to okay in this TV n matrix we need to send over to the pixel shader which we can do with the struct that we earlier set we can that was defined there's the D verts tract struct and so in here we'll also add a map 3 and we'll call it TBN I already used Matt well just let's call it TBN Matt sure and in our function call actually let's say not we don't declare it as a Matt 3 but we just say we vert dot t be and Matt which means that now we are declaring this heart this variable that's part of the structure which was already declared here so we don't need to declare it again sorry yes that was added I mean it's in the it's in the latest official oh it's you don't have to lead it's official there is let me give you the room so what you can do also it's basically it was a convenience function so what you can do is you can calculate the by tangent here with this function take the cross-product from the normals and the XYZ components of the tangent vector times the handiness that's no I'm you should sorry I'm showing if you if you're not on the latest official built so the this sorry yeah and then you can build up the TP n matrix yourself it has the component vectors for again the tangent X Y Z then the by tangent and then the normals and this is a different shader it's just the TBN the TD of TV and matrix function that we just added was something that will has been added in the latest official so if you don't have the latest official sorry yeah if this is and we should be done with this yes am at three TBN net and the whole thing is airing again because the because there's a mismatch in this structure that we're sending across so currently we defined a new matrix in the vertex shader in the out portion and now the in portion gets something that is not or the pixel shader receives something that it's not expecting so we have to go to the pixel shader here which we haven't opened yet yeah right you have to sorry I called it I called it a matte three TBN matte and here it was caught is a matte three TV and it's basically just a variable name so I'm in the I'm in the I'm in the shader currently for everybody who's not updated to the latest official and I'll switch back to the shader which is in the latest official yeah [Music] yeah okay okay I'll weld Malcolm helps you out look again at the error that I'm getting here which is that there is a vertex mismatch between the shader stages and as I said we defined something in the out structure here but we haven't defined it in the in structure on the pixel shader site so let's open up the pixel shader and by default you have your pixel shader attributes here the uniforms that are coming in the colors from as by the parameters of the phone map and now here we'll have to declare the matte three in the instructor TBN matte okay yes yes if you didn't get the mismatch then did you define are you sending over the sorry let me just close some stuff here as a map three yeah and you're not getting at you saved it no mismatch knockin thanks pixel shader TBN max [Music] at this if you if we miss something yeah the the pixel shader is already there it's already prepared for you when we created it from the phone that sorry okay so I'm in the I went to the pixel shader yes sorry yeah and in the in vertex to find TV and okay I went to the vertex shader which where which part did you get so the last step in this example here is now that we need to we have this t VN matrix where we can recalculate our normals with in the pixel shader to have the correct shading depending on the lights depending on the normal map that's incoming here no we still have to we only created this matrix that we can use to now recalculate the normals in the in the pixel shader stage sorry yeah you would have to correct yeah you would have to since the normal is a is a three three-part vector then as well you could bring in the noise and multiply it by the direction of the normal so by the normal on the plane surface that's correct did at the door this shouldn't be too complicated if we go on the vertex stage well basically we have the normals defined as n already in the shader so now we can say that my offset P is P and is offset bow times end that's what I would have thought all pointing forward let me just see - the one [Music] no it's not just times normal it's the it's plus equals you want to take you want to take your pee wees which is the position of your vertex C vertice and you want to add the offset value times to normal for that echo displacement yeah I had it I just had an equals there before which obviously and XY just squished it together to nothing because the normals are zero there does that work for you no problem okay so what we in the vertex shader yeah so when you define the TD create TVN matrix it's called fever TB and mapped and then in the out vertex structure it should say bad three TB and Matt and to follow right okay so I think we we all have we are all at this point here so let's switch to the pixel shader again where we defined the TBN mat as an incoming mat three here in the vertex shader struct and to recalculate the normals we need the normal map that we had defined earlier as a texture to the as a texture sampler to the GLSL mat so let's bring it in as a uniform here as a uniform sampler 2d and I had called this normal map okay and we'll try to get to the we'll try to get to this normal map via the texture coordinates now the funny thing is that we actually don't have any texture coordinates being passed in here right now at the current point so we will have to set that up in the vertex shader as we did before in the first example of the year sorry yeah we didn't we didn't define em they didn't come in through the foam mat though because so what happened this when you when you output the shader from the phone mat and you don't have a sampler a color map or something defined then it cuts cuts out that part so I going to switch back to the GL GLSL vertex shader and we'll just have to create this vector for the texture coordinates so in the out Burrtec structure to find the back to text chords that's the name correct well this structure can carry multiple types so you can send stuff across that has various types no vertex is the that's a good point actually in vertex you know it's out vertex vertex is the the name that it can be referenced in on the other side Val V vert is your internal variable name for the for the offset value so that he can use any geometry you see it as flat yeah we're not we're I have to define the texture coordinates in the vertex shader because I hadn't I hadn't defined a color map on the phone when we initially output the shader and by not defining the color map it doesn't actually create the texture coordinates as a part of the rebirth structure that's being sent over so we have to do that first now and this was the Veck to that I define Tex courts and then let's do B vert text chords equals and that's the UV 0 dot St and once you have that you get an error again because you have to find you have to define those texture coordinates on the incoming side of the pixel shader again so here on the pixel shader it will be back to text cords and we've got these texture coordinates here which means now we can actually fetch from the normal map sampler that we had earlier declared okay so yeah in the vertex or the pixel shader you v-0 because of the ability of having multiple texture coordinates assigned to a geometry and then the parts st okay and in the pixel on the pixel shader site on the instruct back to text cords and since we want to recalculate these normals here May will start adding stuff and before the BEC 3 on the pixel shader site before the back 3 normal call here the declaration will read in the normal map values from the uniform sampler to the normal map here so normal map valve and we have to define what that is we'll take this as a vector e normal map Val equals texture normal map comma get the texture coordinates from the the V vert structure and this would return a four part vector because RGB a but we only defined it as a three part so you would have to add the swivel it's RGB to it and we get our normal map red in now the normal map comes in as a bit of value range between zero and one and what we need to do though is as they could point Doris that could point in two different directions we need to normalize that to a value range from 0 to 1 to minus 1 to 1 so let's do that and this would be just reassigning normal map Val equals normal map tau times 2 sorry 2 times normal map valve and then minus 0.5 okay I need to yes thank you and we now can multiply those normal values that were reading from the texture with that TBN much matrices that we brought in from earlier so normal map Val equals the normal map Val times B vert TBN Matt oh sorry and we need to do this that's important which direction that goes it's actually the TBN Matt times the normal map Val sorry about that it's not important this way no shouldn't is it the is to direct the often it's important if the matrix is left multiplied or right multiplied and the normals that we have now need to be basically normalized into unit vectors with the norm normalized function from GLSL so that they all have the lengths of one size of one so my normal map Val equals normalize normal map now okay and now these what was previously here the normal which is then being used further down in the lighting call of this shader that already has been created cost we have to replace this normalized fevered world space normals which came in previously viable world space with our own just recreated normal map out so the back three normal let's change that to MEK three normal equals normal math valve oh thank you and what you get is basically now properly I shouldn't move the camera obviously what are you getting now is a properly normal eye or properly shaded deformed grid so we jumped a bit back and forth but just to repeat the the four steps that we literally took here was that we offset we offset each vertex by the amount of the noise stopped coming in and we added the thing that we can have multiplying the offset by the initial normals of the geometry then we created this tangent transform matrix with the TD crate TBN matrix function called and sent the result over to the pixel shader stage and on the pixel shader stage we read in the normal map then rearrange the normal map values from 0 to 1 to minus 1 to 1 multiply it this with the TBN matrix or mode again have the matrix on the left side the normal values on the right side to transform it from the tangent space into the hubble space finally normalize this and pass it on into the lighting call that's further down in the shader so the it's not too many steps essentially you can get there fairly easily we did a little bit of jumping around by adding texture coordinates and such but it's not too difficult you can simplify this shader a little bit as well the normal map atop the normal map map top has the option to add the height map in the Alpha Channel what this means is that we could just read in one sampler and reach the offset value of the Alpha Channel of that sampler instead of passing in the noise and the normal map separately you could just have those two combined so for the next for the next portion of this workshop for for the remaining time I think what I'll do is we'll go into instancing and I'll show you how to use instance geometry inside of a shader this is in part by since instancing is such a huge part of touch designer it's really important to know how that works in shaders the the project files that we're on the on the drive that I handed out or that you downloaded kind of extend that whole example to also show how to build a little particle system out of the whole thing we'll see how far we get but I also would like to give you an example of how to take a shader toy example and convert that into touch designer because it's also very interesting so I might skip the particle system part just to get just to get through but it's on the it's on the files that I have handed out and the I mean I can just share the instructions as well so that you then can recreate that so the files that I handed out it's called simple simple particle system I think there's a yeah simple particle system underscore one two three those are the three parts of that file which will just probably will do the first part and then okay gonna I gonna collapse my network again to push it into a separate thing so that it's not interfering with anything that I will build now select all of that collapse selected and call this displace shader 3d displays it doesn't really matter what it's called but that's fine the first step would be that on the drive that I shared with you or in the Dropbox here's a little tio X file which is called connect texture 1280 by 720 and please drag that into your network all that is it's basically it's two images in there that are locked from output from the Kinect the first output here is the is the Kinect point cloud as it comes in from the Kinect camera so the point cloud here saved all the X values of each pixel are safe than the red Channel the Y values are safe than the green Channel and the depths are saved in the blue Channel and the second image here is the color map from the Kinect and our task will be to take this color map at the depth map create with instances create a geometry out of that or not geometry but instance particles in space and as the last then apply a color map to it so let's start by adding the do you'll have to connect texture 1280 by 720 so let's add a couple null tops on the output of it so to now so that we can always see the the two different outputs from the Kinect camera here and since we will we'll see every point that the Kinect point cloud gives us will render as a particle as well and to render particles that you instance you can create a simple particle system by using an adds up and the X up lets you create multiple points essentially we just need one point in the center and then on the polygons page we'll have to convert this one point into so that it's recognized as a primitive and polygon so on the polygons page put star into the polygon field and then turn closed on and now use a converts up and convert this whole thing to particles and the particle type yeah so just if you get a warning make sure that on the points page you have this checkbox turned on to create at least one point so we have we have created a little particle system now where it's exactly one particle all you need for happiness apparently and we'll set up a render network now so at geometry comp I always do this way some people create the geometry insight the geometry component I just create usually the geometry outside and then edit you'll come to it just to keep it simpler and will again take a camera and we'll use the oh actually no don't take a camera delete the camera and from the palette wrong button from the palette grab under tools the art ball camera art wall camera yeah so the art ball camera comes as a little component which is basically control an interface control interface that you can rotate by mouse rotate the the Kuban and if you go inside that arc ball camera so it comes as a full example but all we are interested in is actually this cam one inside of it and this will allow us to rotate around this whole thing that we're setting up so inside arc ball camera just copy cam one and go a level up and we can delete the arc ball camera and place the cam one and now at a render top and a tan out top to this render okay sorry the arc ball camera or so I I drag then the arc ball camera and then went inside the arc ball camera and copied the cam one component and then in pasted it outside of the arc ball camera and removed our quote camera so the artful camera or this camera here if you if you look at the parameters you see it has a custom a custom parameter page which lets you define which panel component it should look at to enable the interactive tumbling and panning after yeah all these yeah exactly and it's if you look inside of that you will see that there's just a bunch of text that's mainly and it outputs it comes back with a Maitre see which is used in the on the view page not on the view page on the transform pre transform page to change the transform of the camera so this camera basically you need to put that inside a container and since we're inside project one it will be referencing that okay and let's also take material phone material here and reference that to the Geo and we'll we won't do the same mistake again last time we had to manually create or define not to find manually declare the texture coordinates and everything let's just do that by default here by dragging the color map on to the column a parameter of the phone mat and then hit output shader and will again reference the phone one GLSL on the material page of the Geo one component and I'll delete the phone yeah so the next step what we should do is maybe let's remove any lighting because for this example wheeler-lea we don't need any lighting we'll have a flat shaded point cloud it's going to be shaded with the color from the camera itself from the color map yeah exactly so in the vertex let's open up the vertex shader and what we have here is we have if you look at all these things here diffuse color ambient color specular shininess shadow strings all of that can go I'll delete it and we don't even need a diffuse say if that and now open up the pixel shader I'm just gonna close a couple of other shaders here let's open up the pixel shader as well oh okay sorry yeah all the uniforms yeah all the uniforms are gone but we'll keep the vertex structure here the believe we need yeah in the vertex shader will remove all the uniforms but in the pixel shader we keep the color map so when you when you now switch to the pixel shader remove all the diffuse ambient specular color shininess shadow strengths and shadow color but yes keep the color map and looking further through this pixel shader here we can also remove a lot of these functions we have diffuse sum and specular some something we don't need we don't need any normal calculation in here so we can remove this as well diffuse some specular sum and normal delete that yeah yeah that's expected the next place is this whole loop here that steps through every light that's used in for the scene potentially this whole loop can be removed as well we can remove all these diffuse contributions final specular out color RGB is gone out color x equals color map color is gone we can remove the fog the alpha calculation don't need differing don't need alpha tests basically everything yes okay okay that's a very fair point so will will remove everything except for the TD alpha test and the dithering so we'll go redefine alpha ourselves here the duct to defuse them you will get an elephant an error now because alpha is not defined so let's just define an alpha here float alpha equals 1 and save that and this should get rid of all errors you still you should be getting as before like a very black black and bleak okay oh I already mentioned that we currently have a particle system with one single particle and what we want to do is we want to actually instance this one particle as many times as we have points in our point cloud from the Kinect yeah I removed it all I removed that as well and I did I did yeah because we as Malcolm said when we keep the dithering and the Alpha tests then you already have it in place in case you need it later on so it's good to keep that as a and then because alpha was defined earlier but I removed it I added float alpha and set it to one yeah okay so if you want to create if you want to create now the amount of instances as we have points and our point cloud image here first we need the resolution of that image and you get to this resolution by using an info shop pretty easily so let's place down an info shop and the info shop expects an operator which will be our the image was the point cloud in it and now the info shop has a scope parameter and in the scope parameter we'll just we just want to pull out the channels res X and r sy now to get the total number of pixels at a math chop and multiply the two channels math combined channels multiplied just gonna move some stuff a little bit around here give myself a little space and I'll add a null at the end of it and now we'll use this number to create instances of our single particle so select the geometry component and on the instance page turn on instancing and then there's this parameter instance count and on the instance count we can either get the lengths by as probably many of you have done before by getting the length of the channel so you can have like a hundred samples in the channel and that will create a hundred instances but we just have one number and we want that number to be the determinate ER here so we'll change the instant count to manual and then export from that null three that is the total number of pixels by activating the view active flag and exporting onto the numb instances parameter okay so we now have created about 900,000 particles to which we have access in the vertex shader and in the vertex shader now we need to offset those particles by the value given by the point cloud but currently the point cloud image here is not actually used as an input in our funky LSL material so select the GLSL mat called from one GLSL and on the samplers page add something that we can call as point cloud and this is perhaps a standard that I don't add here too often enough the naming of your variables that you're passing into the shader like the samplers if you prefix this prefix this with a little less than you know oh that's a sampler that I've defined outside if you take uniforms put a little U on it in front then you know where they're coming from so it's then it's better for red ability definitely and this s point cloud sampler name that we have created here the reference type for that is the now one for me should be probably not one for everybody with the RGB point cloud from the Kinect okay our next our next task in this will be to figure out which instance we are currently working on like what's the what's the particle that's currently being worked on in the vertex shader and then fetch the correct pixel from the point cloud image and apply it to the position of this particle so let's edit the vertex shader here again I'm switching to the vertex shader and we do have our P position attribute which we'll adjust but first again let's try to figure out if you want to know which instance is currently worked on it's it's passed into it's passed into touch designer and is called is it the GL instance ID no it was renamed it's the TN TD instance ID TD instance ID so previously we had GL instance idea but instances are worked on in batches as well Malcolm right okay so for this we'll use TD instanceid which gives us a pre year a number where we don't have to jump through different hoops to actually get to the instance and this is an integer so let's define that as an INT instance equals TD instance ID okay now how do we get to the correct pixel that we need to work on to to get to this we would need the resolution of the of that sampler and that resolution is given to us know that resolution has to be passed into by a uniform that we define so let's go on the GLSL mat to the vectors page and let's create a uniform called you press for example and the two values for this you rest should be the resolution the channels that we grabbed with the info chop here so I'm just gonna make a new branch by middle mouse clicking on the output and export as x-rays Y onto the us parameter export and export exporting again we did that with the activate the Bureau flag and then drag the channels onto those two parameters now in the pixel shading the vertex shader here we can grab that uniform by defining it as a uniform mech to us and now my back to curve pixel I'll just store it goes into a back to type and I'll set that to zero for start whoops back to obviously and now to figure out which pixel it is in a row by saying cur pixel dot X equals and this is TV is instance modulated by the X resolution so that's you res dot X and my curved pixel dot y so I know which sorry so we we we now know which column it is curved pixel got X and now for my curve pixel that Y I'll do instance divided by us got Y X it's instance modulo us dot X and then instance / us dot X Ont the second one the us thought why definitely needs to we need to floor that or use int to get to the actual row number which is fine yeah we just need the it's a it's a back to so a float for that and only for for the since it's a modulo it will give us but for the row number we need to put the end around it which gives us an error let's check that oh the us that X needs to be an integer itself yes I BEC - that's clever that saves that some brackets yes boys it's very nice the tea I don't have to walk to him he's right there and we'll use it currently we have to pixel as a integer number but if we want to now use that to look into the texture we'll have to divide it by the resolution making it a float again so that's the idea for this so yeah our next step would be to divide the curb pixel by the resolution because Kerr pixel currently gives us a literal row column number but we need to have a UV kind of scale zero to one kind of scale so [Music] no we don't have pee-wee's because we just have a single a single particle yeah there's nothing there's nothing there each each in each instance if it would have UV would have its own separate basis yes Soaker pixel divided equals you rest now one thing that happens as well is that the what we're now having is if you look at the first pixel that we're working on what's happening here is that we are looking at the border of the first pixel because we take zero the first pixel is zero zero and so our UV would be zero zero looking at the edge of the pixel which is not necessarily something you want to do because that edge is kind of not well-defined point you're in between pixels so we should be adding half a pixel to this whole thing offset this UV by half a pixel and this we can do by saying curl pixel plus equals one so we have to calculate the size of one pixel which is one divided by you rest and we'll put brackets around it this is one pixel we are offsetting by one pixel now but if we divide this by two we're now looking at the center of every pixel yeah postback - yeah and we previously have to find the point cloud and as a sampler so that we should be bringing this in as a uniform into the vertex shader here so let's do that uniform sampler 2d s point cloud and now we can say back three of my point plus is texture because we're going to look into that sampler as point cloud comma cur pixel and instead of the P I mean the position of all these particles is zero anyway but I don't really need that P we can just replace the P and the TDD form function with the my point plus and begin there mm-hmm Oh casting yes yeah that's because I used the texture which returns again RGBA and it should be RGB since we are just interested in the three positional arguments that are safe than there this is actually a good place where you probably don't want to use RGB but because you're just grabbing position from it you could use X Y Z's here red ability again that's in the lluvia before texture coordinates I wouldn't be able to tell you that it's a it's a good question as well Malcolm I never I never asked that we use you use usually we say UV coordinates but in GLSL it's dot s dot T why is it not that you don't me okay perfect yeah so the next step the next step would be we have to point position and it's a little bit difficult to see that here because it's all black but what we already have defined an our output structure is the V vert color which is basically the probably we could best describe it as the about the vertex color of that point of that particle so we just use that we will just use this V vert color and look into the color texture with the same current pixel coordinates fevered color equals texture obviously we have to sorry I forgot to define the texture up here as an incoming uniform so we should do that first uniform sampler 2d s color map and then again once we have that get the color from it as color map from our curve pixel and since this V vertov color is a BEC for now we have to write its both back for by deleting everything previously in the shader in the pixel shader what has happened is that we have this mech for color there will switch over there in a second and back for color was set to zero like to a null vector and now we have to use the de Libertad color and assign that to this color vector that is currently defined as black inside the pixel shader so once everybody looks like we're ready so let's go to the pixel shader and the color here if you look down it's called it's called outcall so let's do that right under here we'll go under the leg for out call and say out call equals revert dot color and everybody should see something like this oh sorry the black background yes of course the black background for me I just switched that back what probably most of you see is this maybe with the checkerboard sure yeah trying to understand why that is glass I wonder if you bothered to see what happens one second just whenever so chambers left home fighting an average right so so for changing the position we have to change the camera right and that's why we brought in this art ball camera yeah yeah yeah to be hospital yes which once I know the yeah you can already you can you can toggle around the view by opening up the viewer for your for the network that you're working in as we added the artful camera to it now there would be one more changer to do which is on the we would have to set currently if in the viewer of your container that you're working in you can only tumble your view you cannot use the right mouse button or the middle mouse button and you can change that by right-clicking into the network and opening up the parent parameters so those are the parameters for project one and just turn on the middle and right mouse UV buttons on the panel page [Music] you can it's my play [Music] good I'm not sure which will be aligned because you have to use the connector line yeah it won't probably be line you know absolutely well because you want us to stay like this well because you're friends because I'm ignoring that alpha Nora yeah who could be you mean yeah yeah you don't necessarily and why isn't that working yeah so there's another way so further down in the shader there was there's another spot where beaver duck color was getting set which was overriding where we were setting it up earlier so yeah coming out this line down here isn't the vertex shader yeah [Music] in this particular case yeah pretty much the picture state is just passing the color theorem that's all yeah it's not doing any actual shading yeah it's just passing any colors through but you always need a big enough yeah you always need something so he's so easy give me a second let me answer your question so well so these are so you try to run these it's like then you go so they're trying to run this Texas I find they're strong huh which is why you see those [Music] there's a write-up on the record many others 80s for there's not the my of two cases it's not rendering anything with just single points but they also compared this is an Intel that this is a GTX 960 the if you can just raise your hands force one point rendering yeah so given the risk yes [Music] yeah you want to do as much as you can on the GPU yeah that's correct anything animating if you're trying to animate lots of things that you can do it with instances and that's perfect do it with instances there's if it's in general for your geometry animations or yeah for the time being I mean you can get pretty far with ups I find you can push it quite a bit before you run to bottlenecks but if you're looking at hundreds of thousands then there is no way around it currently no we we only have we only have one we only have one particle that we sent to the GPU the GPU is instancing and we're offsetting then there sister yeah there we have the [Music] [Music] more or less you don't have you don't have connections between the so you would send in a great yeah right static grid and then offset it with yeah you could do that definitely there's one thing about optimization here which should be mentioned currently we are sending in one particle and we're instancing it 900,000 times this can be scaled up to whatever an HD image or things eventually you you might you will run into performance problems with instancing as well just one second so the so what what you can do in this case is you could you could just take multiple particles like if you have two particles you only have to instance half as many times those two particles and so the scales you have a little bit more leeway there to scale higher all you have to change here in in the vertex shader is the calculation for your current pixel currently the current pixel is the instance modulated by the US X what you want to do then is get the vertex ID which is the which is GL underscore vertex Aidid and that class instance plus yet so you take the GL vertex ID plus your TD instance ID and the TD instance ID multiply it with a uniform value that says how many particles occur in since you're passing in so both of them were of course are you sure fight like button to break or something to do we give it like I guess yeah yeah on that for instant I don't know where the hip is no I don't know that is any times when there is communication between the CPU and the GPU when you're uploading or downloading data yeah so the next thing I'll just show you and you have this file if you open up the simple particle system three file yeah at vertex u butyou can only you can only do this in the geometry on the in the geometry no in the geometry shader in the geometry but that's way too many points for a geometry shader it's not fast enough it would be you would you would be bottlenecking very quickly without it you cannot create as many points on the geometry shader stage okay so I just I'll just give you a quick walkthrough here this is basically the same thing except all these points that were there have been transformed by a little particle system and the the transformation is happening in the well before the point cloud is given to this little render system so your particle system is not built in the GLSL material here it's built in a 2d GLSL top and what it's being done here I'll show you the pixel shader for this is that you define a number of outputs number of buffers you need the position for your for your particles and you need the velocity for your particles and this is for this simple example we'll we just have position and velocity and then you feed those into a feedback network into a feedback talk so that you get the previous iteration of this yeah you while you get the previous position and the previous or and the velocity that you calculate and then you just add it then you add to the velocity you add turbulence with a certain factor just to adjust how much is applied to it and you add that velocity to the position and you had a little bit of drag to the velocity so that it's lowering over time so that it's not static and you output the new point position in the velocity to the two buffers now the the feedback point position here I used also and fed it into a noise component that you can find in the tools section of your pallet in in touch designer and this is the shader that not sure actually this is from Brian sharp it dot wordpress.com and what he has he has a number of different noise functions but the most important thing here is that it can give you the the slope between each noise point and this is necessary in a 3d noise so that when the particles are moving through 3d noise that you can apply the slope as turbulence on to the velocity to get a proper turbulence movement so we don't have that currently we don't have that function in our noise top that's why this noise noise component that's in the pallet under the tool sector under the generator section sorry generators noise you can grab it from there now we can just quickly reset this whole thing so you see what's happening do it we have to reset those two particle systems and then you can kind of follow what's happening here it's being displaced in all different directions by the velocity and the turbulence and the velocity the initial velocity is given through a UV map that I rescaled to be so that they they drift apart basically all the points but it was from 0 to 1 to minus 0.4 to 0.4 but that's all in the it was whatever yeah just to normalize it basically and I multiplied that with a noise the noise stop has this function make an out where you can multiply the noise with the input so I used that to kind of make it a little bit irregular a full version of of this particle system here and I wanted to mention that again is in the tool section under the GPU particles particles GPU now if you download the experimental version of touch designer or what's currently experimental the there's a slight change in this system from before we're using here two modes to create particles until now we had to create all the particles at once because in a particle in a in a you cannot keep count in a GLSL shader in the official version since the official version we added in or not not all of us but derivative did add in the atomic counter functions for shaders which gives you the ability to actually count every for every pixel operation that you do in the pixel shader you can increment or decrement these counters and this gives you the possibility to actually count how many pixels you have worked on and in that case you can count how many particles have been born and abort operations or just pass through operations if enough particles have been created so yeah this is interesting to look at it and another thing that we won't cover now because the last ten minutes I want to convert this shader toy thing is to have a look at ya inside this component here at the geometry stage of the particle system and where is it you know we have the GLSL so what this system works exactly the same as the previous that we have a single point that's being instanced and then we add we add a texture to it but we're rotating every single we're rotating every single particle here now this is not something you could theoretically do with a single particle you cannot rotate that thing because just literally a point and that's where the geometry stage comes in so in these in the middle geometry shader here what this is doing is per particle it's creating four vertices and there's a couple rotation functions here and then you can see that for each it creates a vertice at the left bottom and then emits this vertice it creates one at left top emits it right bottom right top and then emits the whole perimeter so in this stage in the geometry shape stage you see how it's creating four different vertices and then emitting them as a particle and as a as a primitive and with the functions here because there's rotate functions for each [Music] no the atomic drop the atomic counters are and this one is also in the regular official version the geometry it's just good to check out and you can kind of follow how it works with the geometry a simple example of geometry shaders in touch much more detailed stuff probably tomorrow at 11:00 a.m. when Stanislav is going to show his GPU geometry shaders yes definitely something not to miss if you're not in the workshop ok so for the last 10 minutes and luckily this should be fairly quick I'm just gonna close this down quit I have this here okay I gonna collapse again my whole thing into a component collapse elected into and call this connect point cloud more of these pixel operations sorry I have to mention that as well there there are there are more of these GLSL shaders things here that render basically particles for these color scopes that we added it's the 3d scope the histogram as well as the waveform monitor all of these actually use the same system of taking taking particle or taking points vertices and placing them by looking at a different texture as a transform input worthwhile checking those as well okay so now let's have a look at this in the same folder that I gave you there is a shader toy example the GLSL and you can just drag that in and it should create a text dot it's an example by Inigo acutely I'll work on that to his his examples on shader toy are very very very useful as website with techniques as well in general and I copied this one and it's a Voronoi example and I would like to get this running inside touch designer so the first thing for this to do would be if let's create constant top and set it to 1280 by 720 or something it's not really that important for this and create a GLSL top and now instead of the default GLSL pixel shader that's coming here just use the text 1 dot that you just dragged in from the outside and replace the pixel shader parameter with it so the pixel shader parameter of your GLSL top should now be pointing to this one that we got off shader toy and you should get a nice error that's very expected and the first thing to do is check the info dot so to place an info dot and drag the GLSL top on the operator parameter and have a look what the errors are so it's complaining about four different errors here one is I channel 0 then I time that's actually yeah twice so I time that's the same error here and then I resolution and so let's work on this i channel 0 we can check actually what's the definition is of all these variables that are causing errors by going to shader toy and I should have probably but we can find it quickly the tip although the internet is so badly oh no that's this is the one that I download it here so if you're downloading from shader toy then a good thing to check out is this shader inputs the left side should be I don't know why it's adding my left GL hit us neck but it does matter the left side is the output which should be usually rendering running and then on the right side you usually have the code but also you have this list of values or variables here and so the eye resolution tells you it's the viewport resolution in pixels so we need that it's our texture resolution then I time is playback time in seconds so that's something we have to pass in as a uniform and what was the other thing oh and I channel 0 well that's actually the those are the textures that are being passed into into the shader in this example it's a noise it's a random noise texture so what we can do here we can replace the constant 1 with a noise stop so let's create a noise top and set the type to random and choose the random GPU type so we can nicely animate it and I'll change my resolution of this noise also to 1280 by 720 and then plug it into the first input yeah yeah noise type random GPU and now let's open up that shader and right away at the top where it's creating there's a function called hash - it has it has a return texture level of detail and then an eye channel zero so the texture level of detail I had to look that up it's and from what I read it's usually only useful on for mipmapping purposes in the vertex shader stage but we don't need that right now come oh so you can control actually the MIT math level yeah okay in a in a yeah so it in the math it would be can be important in the GLSL top we don't need that and to get rid of to get rid of the whole thing it is a it does have an extra attribute at the end of the texture a third a third part of the texture call which is here to 0.0 so we can remove that as well and then replace the I channel 0 but I our s and that's from all the start from the very start of our class as TV inputs 0 as T d2d thank you to the input 0 okay first thing on so let's create a nighttime eye time we don't we note that this is play time in seconds so we'll define this as a uniform here.i time and then go pass in time in seconds which if we we just want to be really quick we'll use Python function apps time dot seconds and then in the shader we have to bring this in as a uniform so uniform float I time when we have that what's left is the I resolution line here everybody should have approximately the same and let's try to find our resolution so here I resolution brings actually something else we suddenly see something else that's unfamiliar we have a function that's called void main image we are mostly used to void main so let's fix that let's kill the image part and then the other thing is that we have out back for frac color and in back to fret court defined inside this function call it's also something we previously have seen how this is defined outside of that main function so out back for track color and the in vector for frac color we do fret court we don't need we need to figure out what fret court actually is and for fret court I had to Google a little bit before I found it what it means the gets it's not that you or me it's the UVF as integer pixel yeah so [Music] yeah as integers yeah looking at the center of the pixel yeah so I divide it by the resolution here which is basically our UV our own UVs that we get from such designer so this mech to P which is the integer pixel position divided by the resolution is actually our view V so we can replace this whole thing here track part divided by I resolution dot with the UV dot s T ctrl s and we'll remove this and you should have the same shader now running in touch designer if you don't have arrows if you have errors what would be the errors let me know

Channel: TouchDesigner

Views: 33,043

Keywords: GLSL, Particles, Pointclouds, Pixel Shaders, Shadertoy

Id: FImsLtt4Ab4

Length: 222min 47sec (13367 seconds)

Published: Wed Feb 21 2018