Intro to Graphics 07 - GPU Pipeline

all right well thank you all for tuning in for another um lecture for introduction to computer graphics last time we talked about transformations this is the week where everything is going to sort of come together we're gonna form the whole pipeline that will involve rendering and the gpu and so today's topic is going to be gpu pipeline that is hardware accelerated rendering uh this is a a very very important topic so a lot of things that we did was sort of building up to this topic this gpu pipeline this is how we have computers that look like the you know the devices we have today now we have cell phones and you know everything looks so much nicer than than than it used to so what do i mean when i say gpu a lot of people when i say gpu they think of uh whoa this is big yeah this guy this is not a gpu all right this contains a gpu but this is not a gpu this is a video card right now gpu is you know obviously an important part of it but you know this is not quite what i need when i say gpu uh so what does this guy have this guy has you know a gpu in it that the graphics processing unit what else it has some dram right graphics dram memory that is gpu memory what else um some huge fans right because we kind of need to cool it down right and some heat pipes and yup that's pretty much it and the rest is just uh you know connections to all sorts of different different pieces you know the main components here i i would say the gpu the memory and the stuff to cool them down that's about it so when i say gpu this is not what i mean what i mean is is this guy right so the actual uh processor that that's what i mean and that prediction is is sort of important because some of you may think that i'm talking about oh it's just trivial stuff but you know this distinction is gonna play a role in the way that we understand how how this thing works right i'm gonna talk about today what i'm gonna talk about today is the gpu pipeline but by gpu pipeline i i don't mean all these like connections in here all these uh connections between different parts and stuff i can actually spend uh quite a bit of time trying to explain you what these little colored bits are and why are they like that and whatnot so this actual hardware details are not stuff that we're going to be concerned with this is this is not too important because what we're going to have is going to be a like a simplified view of what this guy actually is and that's going to be the important part we're going to have a functional understanding of what this gpu is and what it does and that's going to be from the perspective of the graphics api that's going to be given to us so and this is very very important because gpu has changed a lot over the years but the graphics api we're using well that changed too but not nearly as much uh and the graphics api it just generates code that works on pretty much any gpu like all gpus new gpus of course their their differences the new gpus can do stuff that all gpus couldn't do that's not what i'm talking about but the whole high-level view of what this device does and how it operates that is the important bit and that part did not change as much of course if you think about the last 30 years yeah there were some changes i would say but yeah so that that distinction is also important so we're not going to be concerned with exactly how this thing is designed and what the connections are we're going to be looking at the gpu in a more high level perspective all right so in a high level perspective i look at this and say this is gpu what does it do what does the graphics processing unit what does what does it do it does um it it produces images that's what it does it's a vaster image making machine right so what it does is it produces these things and we typically display them like we would put them on the screen but we can do other things with them as well but it's it's fundamental job is doing this now of course there are used gpus have some other functionalities that could be useful but it's my main job is is this right and how does it do that i mean it needs to take something to produce this image right if this is the image that i want to produce and i'm using my gpu for it i need to have something some kind of like scene data some description of what my scene is so and the gpu will consume that and and from that it's going to give me this output right so this is the concept of what a gpu does from the perspective of compute graphics and when i when i look at the scene data um what is it now it could be whole sort of different things but for the time being let's assume that it's just some some collection of triangles right c data is a bit more complicated but for the time being let's assume that it's just a bunch of triangles uh and other stuff that's going to define the image that i would like to produce all right good just a simplified overview now we're going to have to dive a little bit deeper instead of looking at gpu as this image making machine we're going to have to go a little bit deeper and look at what we have inside of this guy all right so uh i'm just rearranging everything here i didn't do any triangles come in here my gpu does something and i get an image out of that and what i'm going to do with that image is that i'm probably going to put it on my display and i'm i'm going to take a look at it on my display right i can do other things with that as well but that's the primary purpose of why i'm producing those images and so this guy this scene data the bunch of triangles they're gonna have to come from somewhere now where are they coming from well you can say they're coming from the desk or something not though right before the gpu remember gpu is the processor so this stuff has to come from the gpu memory all right and you know this is important because this means that before i can do anything with my gpu i need to put my scene data into my gpu memory right that's very very important this is where we start so we kind of start with putting everything inside the gpu memory and then we're going to tell the gpu to to render that that data and we're going to get this final image right okay how does stuff show up in my gpu memory well for stuff to show up in my gpu memory they should be in my cpu memory first okay now this is a bit more familiar you know if you haven't done any gpu programming i'm i'm guessing that you've done some cpu programming well if you're taking this class i'm pretty sure you've done some cpu program so you know how to put stuff in cpu memory right you uh you know those stuff from this you just have your areas your classes or whatever data structure you you know how to put stuff in gpu in cpu memory so what we're going to have to learn is how to take that data and put it in the gpu memory and then from from there we're gonna learn how to pass it through the rendering pipeline on the gpu such that it produces the the output all right so this is a high level perspective of how the gpu uh pipeline operates how we do rendering using the gpu all right let's dive in a little bit deep deeper i think i think we're in we're ready so this is my gpu what's what do we do what do i have inside my gpu well okay i have lots of gpu cores and whatnot but that's not what i'm talking about i'm not talking about the actual hardware that operates um conceptually what kind of operations go on inside that gpu pipeline so we're gonna something is gonna take our scene description and do something to it and in the end we're gonna get this image so you can think about it this way this is like my my my high level perspective of how i see a a gpu in an operation so it takes your scene data and it applies some transformations first transformations like we like we talked about last time it applies some transformations and and and moves our scene data into the canonical view volume right so applying transformations and canonical people and then there's this green guy here which is rasterization if you haven't heard this term before it means exactly what you would guess it means it will take some vector data which is over here and it will convert it into a raster image data right so basically it's going to take my my triangles and it's going to convert them into a bunch of pixels that correspond to each triangle and then for each one of those pixels i am going to do some operations that will compute the color of those pencils and that is going to form my output all right so this is a very simplified overview the scene data it's not just transformed there are a whole bunch of things going on here the the vector part of the pipeline does all sorts of operations to the primitives but basically its main job is to take the scene and transform it into this canonical view volume and then there's going to be this rasterizer this rash riser is a hardware unit that that generates this these pixels or actually we call them fragments because you can think about this as like fragments of my primitives like i take a triangle and i take that triangle and i chop it off into little pixel size bits and each one of them is a fragment and then i'm gonna apply some coloring to it i'm going to set the frag colors of each fragment well of course there's there's a bit more stuff that's going on from here to form the image like each triangle is forming some fragments and i'm coloring those fragments of course i need to put all these together there's alpha blending going on we've done all the blending right so that's going to take place right after here after i compute the colors and all that uh so there's the stuff that goes on here that's a bit more complicated but i would like you to have this rather simplified view of the graphics pipeline right so if you dive a little deeper just to understand what these components are we can think of the uh the first part the transformation part as the vertex shader shader is a funny term i'm going to talk about why it's called shader but let's not worry about that for the time being so there's vertex shader his job is to take care of all these transformation stuff and i have my rasterizer his job is rasterization and i have my fragment shader its job is computing the color of each fragment okay so this is the general gpu pipeline now all these three pieces two of them are going to be programmable i'm gonna have a vertex shader and a fragment shader these two components will run my shader codes this one is going to run my directx shader code and this one is going to run my fragment shader code so i have total control over uh what a vertex heater does and what a fragmentator does rasterizer on the other hand it expects a certain type of input from my vertex shader and it's going to get that input and it produces a certain type of output into my fragment shader and it's going to produce that output rasterizer is not a programmable unit this this part is implemented in hardware there's actual physical hardware in on our gpus that that do this rasterization for us and it does that fairly efficiently so that's going to be the hardware unit that's doing all this stuff and we're going to be controlling the vertex shader and the fragment shader now as i said there is stuff after the fragment shader that happens and that all happens on hardware we can we have some control over that but it's uh it's yeah our actual control is in the fragment shader and there's stuff that is before the rasterization that is going to be our vertex shader and and again there's a little more stuff that goes on here before and after i'll actually tell you a little bit about it but i i don't want to show you a complicated picture because i want you to retain this this simplified view of what's going on inside the gpu so one important thing to to notice from my visualization here from my drawing here when i when i'm representing it triangle here i didn't just draw a triangle i draw i draw a triangle with vertices the three vertices of that triangle highlighted and that's going to be important because my vertex shader is that my vertical code is going to look at one vertex at a time so it's only going to process one vertex so this is going to be a parallel machine right so it's going to uh my scene has a whole bunch of triangles they all have three vertices uh so they're all these vertices will be sent to my vertex shading stage and my vertex shaders my vertex is going to run in parallel and each execution each instance is going to look at one of my vertices right and its job is going to is going to be taking that vertex just one vertex and transform it into the canonical view space that's its main job and there's going to be some hardware unit in between here that's going to take these transformed vertices and it's going to form the triangle and the rasterizer will get the triangle and its transformed vertices that that we call this primitive assembly so over here we split our triangles into vertices and our vertex shaders vertically to get those vertices at like one one vertex at a time and then over here right after it's done the rasterizer uh before the rasterizer we have primitive assembly that forms the triangles back again and gives it to the rasterizer uh now this is the the interface we have uh we're gonna have using javascript and webgl but actual gpus this this vertex processing part of the pipeline can get a bit more complicated so um we have additional possible shader stages applied here after the vertex shader we can have what we call tessellation shader so you can take primitives and then take each triangle or quad and split it into smaller and smaller triangles and quads that's the tessellation shader and then we have optionally geometry shader geometry can take that data and do whatever it wants with it so it doesn't pass the data along to the rasterizer instead jump machinery gets that data right before the rasterizer takes that data and it outputs whatever it wants to the rasterizer it's a super powerful shader that runs our code and you can do a lot of interesting things with it but these tessellation shaders and geometries those are optional shaders and we will not have access to it uh to we will not have access to those shaders when we're using webgl with webgl we will only have this this required vertex shader and fragment sharing as other more complicated stuff that actually exists in the gpu rendering pipeline we won't have access to those uh and you know that will be a bit more advanced topics we don't necessarily need to worry about them in the introduction to computer graphics course but i want you to know that there is more complexity here in fact more recently we have what's called a mesh shader that takes all this complexity and replace it with one holistic shader that does everything so it takes your our entire primitives and then you can output whatever whatever it wants it's a you know there's this concept of mesh shader that does some just uh one shader stage but sort of replaces that complicated multi-shader stage uh pipeline if you if you look at the graphics apis you will see that this whole thing can be quite complicated and if you you know check out various forms of gpu rendering pipeline what you will see is going to be quite a bit complicated if they want to include all of these additional stages as well but i want you to again i'm not showing you all this stuff intentionally because i want you to retain this simplified picture and because that's what's important this is the required part of the rendering pipeline gpu rendering pipeline and this is the part that you must really understand well and this is it really this summarizes what goes on here i have my vertex here that does the transformations i have my rasterizer that converts the vector data into raster image data and then i have my fragment shader that computes the colors all right okay oh good so how am i going to get access to all this stuff well i'm going to control all of this stuff from my cp application so i'm going to run some application that's going to control all this stuff now this application the cpu application you can you can run that you can write code for this application you can implement it using whatever programming language that you like and we typically use c plus plus in computer graphics but you don't have to you can you know other you can use other uh languages as well and this this and in this course actually we're using javascript for that and the javascript is going to be running on our cpu it's not going to be running on the gpu we're going to be controlling what the gpu does from our javascript application right so it's important to understand that javascript is in here and nowhere in the on the gpu it's all in here all right so from here i'm gonna have to send data to my gpu right so that one important part is that my cpa application will need to send data to my gpu then the scene data and the way that we're going to send is you know that's just like the picture i showed you earlier the javascript will create data on the cpu memory and that that data on the cpu memory will be pushed to the gpu memory and now when we're ready to run the rendering gpu rendering pipeline we're going to tell the gpu to start consuming data from the stuff getting that start getting data from the gpu memory right that's how it's going to work uh what happens if there's dedicated hardware to run webgl well then then our cpu is going to simulate this pipeline so there's going to be a software fallback layer that's going to look very much like this from application developers perspective but this whole stuff is not going to run on the gpu it's actually going to run on some code written on written for cpu if we don't actually have dedicated gpu another thing to consider is now on your a lot of your laptops you can have integrated gpus so you can have a gpu and a cpu on the same die right so there's going to be theoretically separate devices but they could be on the same dies so they're gonna be like connected to each other uh physically at least and also on a device like on an integrated device like this you're going to have your cpu memory and gpu memory on the same physical hardware uh but there is still distinction between the cpu memory and gpu memory even on those devices so you're going to take that that memory space and split it into and say this part belongs to the gpu and this part belongs to the cpu and that distinction is going to be important because the way cpus access memory is actually quite different than the way gpus access memory so gpus will use the memory very differently than quite a bit differently than how cpus use memory so even even if the gpu memory and cpu memory that it's physically the same thing virtually they will be they will be split into a gpu piece and cpu piece and there might be in some new devices there might be some shared space where cpu can have access to data that is controlled by the gpu but it's it's it's good to think about them as gpu having its own memory and cpu having its own memory even if they're physically on the same same draft also you know when we have gpu and cpu on the same die it's it's a good idea to think about that as having a separate cpu and a separate gpu that that perspective is actually very helpful all right okay so this is the important perspective of what's going on here my cp application my cp application is going to be sending data to my gpu right uh and we're going to talk about how to do that another thing my that my cp application is going to do is that it's going to set the code that's going to run on my vertex shader and my fragment shader of course if i have other shaders in a more complicated pipeline it will set those as well but let's think about them as you have only had vertices on a fragment shooter so by cheap my cpu application will set the code that's going to run on these devices on these vertex shaders and primary shears now when we're developing cpu application oftentimes we think about your write code and compile right same thing is going to happen here but because my gpu can be physically some different device my gpu code should be able to run on different devices with different properties what we're going to do is that oftentimes we're not going to pre-compile this code we're going to send this code to gpu or we'll give it to the our graphics driver and our graphic driver will get the source code from us and it will compile that source code as we're giving it and that's so we're going to be compiling our shader codes as we're running our code well when we're using javascript when our application is running javascript well guess what javascript is not compiled either so well actually javascript is not compiled at all but this stuff that goes to the gpu it is going to be compiled but it's going to be compiled at one time okay it's going to be compiled as our application sets the the shaders now our application is also going to set the parameters that go into these vertex shader and fragment shader now my implementation of my vertex shader and fragment shader it may require some parameters because i want some parameters i don't want to compile every time i change something in my code maybe maybe there's a knob that user can tweak and based on what that knob is doing i'm going to be setting some parameters here for my vertices and fragments here i'm not changing my scene data i'm just changing what exactly my vertex shader and fragment shader is going to do to that scene data so there might be some some parameters we call them uniform variables uniform in the sense that i'm going to set them set those parameters and they're going to be applied to my entire scene so they're uniform in the context of rendering my my entire cd or at least rendering the part of the scene that i'm interested in rendering so i'm going to have my cpu code is going to set those as well so it's going to set the the actual code and it's going to send the variables that work on that code well how are you going to do that how are we going to set all that all that stuff well we're going to be relying on the the graphics api to do this for us so the graphics api is actually you can think of this as the cpu cpu code or functions that our cp code will have access to and those functions will will do this stuff for us so we're going to be in our cp code we're going to be calling functions of our graphics api and the graphics api will directly communicate to the graphics driver and the graphics driver will actually directly communicate to our gpu hardware okay so that's the uh that's the perspective so a lot of times people like drawing these layered cake models like i have my cpu hardware gpu hardware i have my drivers on top of that i have my apis and then i have my cpu sorry my application i don't quite like these uh you know layered models too much i i kind of like this sort of this sort of visualization makes a lot more sense to me but note that all these arrows are connected they originate from the graphics api so in my cp application i'm going to be connect communicating with the graphics api and the graphics api is going to be doing all this stuff for me so it's going to be very important for us to understand graphics api and how that works now just like having just like the fact that we have different programming languages for writing cp code we also have apis developed for different languages and we have different apis for let's say that our application is c plus plus there are multiple apis that are uh available to us if you want to use c plus plus we can use opengl opengl is a fairly actually very old one probably uh the oldest api uh especially if you think about gl before it became opengl gl was an sgi library it's gl because it's graphics library and it's a c live it was a c library not even c plus and opengl came out of that and that is also a c library you can use it from c plus plus that a lot of people do but it's actually a c library and we have alternatives to opengl is for example we have directx directx only works on windows devices and um i guess you know all other devices including windows supported opengl windows also supported directx uh and directx was an api that trailed behind opengl for many years and then it caught up and met actually yeah it got you know today drag text and some features that don't exist in opengl so finally it went ahead of opengl and i guess apple got jealous that microsoft had their own api so now apple has metal available on apple devices only there's also a relatively new api graphics api that is balkan vulcan is a a much more lower level lower level language than opengl so it's a much harder to do stuff with vulcan than it is with with opengl but because it's a lower level graphics api it allows you to squeeze out more performance out of your gpus by really carefully writing your code the api that we're going to be using is going to be nandos the api that we're going to be using is going to be webgl because that's the api that's available for us when our application is javascript code [Music] and webgl is uh very much like opengl it's like a web version of opengl it's like an a version of opengl and it's a relatively simplified version of opengl it has some extra functionality but it does also have some some limited functionality but it's uh the nice thing about webgl is that it's a pretty wide standard and works on pretty much all modern browsers today so it's it's something that exists on your browsers and it gives you access to all of your gpu capabilities within a web page so it's super powerful in that sense it might be a wonderful api for starting to learn about graphics programming and gp programming so that's the api that we're going to be using in this course so in the remainder of our time today i'm going to dive in and i'm going to try to explain how this thing works by showing you step-by-step examples about including web gl code webgl and javascript code and we're going to be learning about all of these components and and how everything comes together i'm not expecting that we're going to be able to complete a full application today we're going to go as far as we can today and when we run out of time we're going to stop i'm going to continue next time from where we left off okay and you guys are ready i'm diving in slowly all right so webgl the webgl is a javascript graphics api ah so we're going to talk about all the stuff but in the context of mgl now remember the when i showed you the the first diagram we said that the graphics api the gpu is generating uh an image it's a raster image making machine and we're going to be putting that image on our display right so we kind of need to figure out a way to get that image from the the gpu and put it on our display and this is how we're going to do that using webgl so in webgl we need something on our web pages that will show the image that our gpu is producing that our gpu is rendering and that's going to be our canvas this is where we're going to see what the image that our gpu is producing right that's going to be canvas and that's actual html code that will create a canvas for us with this id so i'm going to show you bits and pieces of almost a full application of probably 90 something percent of a full webgl javascript application that's going to be a web page and we're going to go over them step by step and and then i'm also going to uh give you that that entire web page so you you'll you'll see what this this whole thing looks like right but the important bit is that this is an html container that will show our image right so just like any html container we can set its size and place it wherever we want on our web page okay so the first thing we're going to have to do is that we're going to have to initialize webgl we're going to have to take that canvas you know this was a canvas our canvas we're gonna have to get that container and we're gonna create a webgl context in it and this get get context is gonna create a webgl context if the webgl context has already been created it's going to return that context if it hasn't been created it's going to create it for us so this is pretty much all the initialization we have to do to have access to webgl to have access to our gpu this is great because if you uh look at the kind of code we will need to do the kind of code we'll need to write for for any other graphics api it's going to be quite a bit longer than just just one line just one one call although there are higher level graphics uh libraries that sort of simplify the step quite a bit and makes it and they make it look very much like this but even though it may not exactly look like this but very much like this so this is going to create our webgl context so there's a question now what exactly is a context well think about this think about this as like a class where we're going to have a whole bunch of functions in it and we're going to be calling those functions to control our gpu to to do everything that we need our graphics library to do but probably calling it a class i mean from an application perspective from javascript perspective it looks very much like a class but it is called a context more generally that's the term that we use for other graphics api as well it actually means that there is a corresponding component to it within our gpus and our driver as well so it's not just one class that lives on the cpu it actually comes with with everything that connects our cpu code to our gpu uh through the gpu driver so it's a bit more than just just that just the class you know it's a class object but from from our applications point of view it will look like the class a class object so this will contain everything about our applications control applications access to the gpu so all the gpu data we put into the gpu memory or all the shaders shader code that we write everything will will go into into here or we'll go through this this this context all right so you know this is fairly simple right i got my html container then the canvas and and i just created a context bias and get context is webgl now this this function takes a few more parameters a few more optional parameters so you can set it it's it's parameters if you don't want to get the default context you can actually set some some parameters uh but i'm not gonna go into it just yet so we can we can talk about those details later on uh i'm just letting you know that you know there are a few other variables that you you can send here no you can set an area of properties so you get the the context that you want well one of the important things that is missing here is yeah my gpu is going to generate an image but and i'm going to put it and and that canvas is going to display it on that canvas actually automatically but i want to know the resolution of that image it's a raster image right so i kind of need to set its width and height in pixels so you know i i want to be able to do that and it's fairly easy to do i'm just going to set the width and height of my canvas with an analytic properties this is not the dimensions of my canvas in that as it appears on your web page the dimensions that it appears in pixels is going to be this client width and client height the width and height properties of the canvas will define the output image resolution okay so just setting the width and heights is not going to tell my gpu to generate an image with that resolution because it's possible to get a canvas and let's say if my this entire screen is my canvas i can render an image over here and then render another image over over here and then render another image over there so i can use my gpu to render multiple image pieces so i'm going to have to set my viewport this is my viewport and i'm setting here as like this is the starting position x and y of my viewport and this is the size of my v port width and height so the way what this function call does is that it sets my v port as the as something that contains the covers the entire canvas typically like most often that's what we would want to do right we're going to be rendering one thing in our canvas and we would like our viewport to cover that entire canvas so that's typically what we're going to do there's one bit missing here uh that is decline width and client height they're typically given in pixels but not really because depending on which device your web page you're looking at your web page that device may have some pixel ratio uh device pixel ratio uh settings so you remember uh for example in your operating system you can set how big your text is going to be you can you can set your on windows 10 for example you can set your scaling factor and that scaling factor will also scale all of your all of the web pages and everything though so in the old days the the scaling factor was one-to-one basically there was no scaling factor and pixel in the context of a web page meant a pixel in your display but today we have display scaling so a pixel on a web page may be larger than a pixel on your device if you have if i have 4k display and everything looking really really tiny i mean i may want to scale everything up a little bit and that that's why we ended up having that because we have quite high resolution this place today so what i'm going to do is that i'm going to add that to my javascript code here i'm going to um i'm going to get this pixel aspect ratio if it exists it doesn't exist it's going to be one and i'm going to multiply this with height with the pixel aspect ratio now this is going to give the actual pixel size in your display pixels of my canvas and as it's displayed on on your web page right and and this is the weapon height that i'm going to be using for rendering my images because i don't necessarily want my images to be scaled because i'm rendering them at runtime i might as well render it at native resolution that doesn't make sense this is an extra bit of complexity uh added and if you don't add it everything works fine it's just that you're you're going to be rendering images in a global resolution and then your operating system might be scaling them to your display resolution right so this way you're rendering them to the native resolution that will actually be used in your display um so there are a whole bunch of other other initialization that we can do um for example one of the things that we're going to be doing is that we're going to set the the clear color what does that mean so before i start rendering my steam data i'm going to clear the screen when i clear the screen you know that my application with my gpu would like to know what color to use to clear the screen the clear color i'm here i'm setting rgba values it's going to be one one one zero that is white with zero alpha so that's gonna be invisible white so it's not gonna be visible so if you set it to another color and keep this part zero you're not going to see anything but so if you want to see your background color all rendered you need to set this alpha to to something that's non-zero so you can set all sorts of properties here for example when you're drawing lines with webgl you would like maybe you'll want your lines to be thinner than one pixel so you can you know set the line width you can't be larger than one pixel unfortunately webgl won't allow us but you know you can you can set that and go to a whole bunch of things you can set you can initialize your uh your things our initialization is not done but at least we have a webgl context and we set our output image resolution and you know this is the we set our background color which is going to be invisible white invisible because alpha zero and this is actually not important it's not even necessary for the application i just put it there just to show you that you can put also some initializations right here now of course these you can't call these before you call this guy but that's sort of obvious because i'm going to get gl here i'm going to call the gl function so yeah i i need to get that first now an important bit here that i spin the code but i it's in the code but i didn't say it out loud i'm going to have to do this after i have my canvas so what i'm doing here is that i am putting this inside windows window on load function so this this function is going to run after my web page is loaded if you don't put that if you just put this whole code inside a javascript container and that code is going to run in as soon as your browser sees it and and it may show up if you put it in the top part of your html file it's going to start running before before your you have your canvas container somewhere your canvas your canvas container appears later on in the html page it may your script may run before you load your whole page and so before you have a canvas and you don't want that right so you want your whole page to be loaded and then you initialize your ngo right that's why it's put here in the onload function okay so by doing so i have my webgl context now i'm ready to do anything that i want so i use my graphics api that's that's webgl so i'm gonna be uh doing all of these things using webgl step by step so the first thing i'm going to do is that i'm going to set my scene data so what is scene data from webgl's perspective my scene data could be a whole bunch of different things it could be a a bunch of points and lines line straps and line loops and a bunch of triangles and triangle strips triangle fan it's just some data but whatever it is whatever it is all of these different types of primitives that i can use for defining my scene they will all be defined by a set of vertices so if i have a triangle a triangle will be formed by a number of vertices so for my scene is going to be defined by a primitive type i'm going to have different types of primitives maybe it might be just one perimeter type it's okay and i'm going to have a list of vertices that's going to define that primitive or multiple primitives so if my primitive type is triangles i'm going to have a list of vertices that will define the triangles so every three vertices will mean one triangle so what do i mean by vertices like how do i what's the vertex what does that what does that even mean well obviously you can think that you know lists of vertices will have vertex positions like 3d positions of a vertex that kind of makes sense right what else let's see over here i color them so maybe i'm putting some colors on my vertices why not for some applications i may need colors we call them vertex colors i may need them actually there are a whole bunch of different things that i can add here about my vertices so vertex is you can think of this as like a data container you can yeah i can put whatever data i want in a vertex and three of those vertices will form a triangle if i'm rendering triangles right so and these are all sorts of properties of the vertices and we call them we call them attributes vertex attributes talk about the history of this briefly in the old days when we had fixed function rendering pipelines we had some very uh specific types of attributes today with programmable gpus and programmable gpu pipelines these are things that my application will specify so i'm going to have vertex positions vertex colors vertex whatever as my vertex attributes and this is going to be my scene data and i'm going to give and i'm going to give this to my gpu and i'm going to say this is my scene that uh you hold it on your gpu memory and when i'm rendering the scene when i'm ready to render the scene the gpu will get that data and it will pass that data to my vertex shader now this was my data and my data is sent to my vertex shader so i am free to interpret the meanings of these these this data however i want so gpu will just do the transform translation for me it's just going to take that data and send it to my vertex shader and because i'm writing the vertex shader code and i'm defining my scene data i have total flexibility here right there's um so the things that i'm going to talk about this data is completely under your control for example uh one of the vertex attributes could be vertex positions that kind of makes sense what is my vertex position well it's going to be x y and z coordinates of each vertex that kind of makes sense right do i need the z coordinate do i not need the z coordinates if i'm rendering something 2d i probably don't need the z coordinate if i'm rendering something something 3d i probably need the z coordinate or maybe i don't i mean this is completely up to me what what what this is right it's completely 100 customizable oh yeah maybe not 100 customizable customizable within the limits of what i can do with webgl all right so i'm going to say that this is my position data and i wrote this in like a 2d array kind of form it's xyz going this way and you know number of vertices going this way but this is typically going to be converted as a 1d array so you start with x y z x y z x y z like in the scan line for a scan line order it's going to be like one d data of x y z x y z xyz xyz right so very much like color data so if i had this vertex color data it could be like rgba rgba rgb igb rgb so that's that's the data that i'm providing for my for my gpu and i'm going to send it to my gpu and say this is going to be this is going to be my data for example one way that i could write it is like this i have a variable i call it positions and i put some numbers into it those numbers are the xyz coordinates well i know what they mean my gpu has no idea what they mean it just knows that it's an array of values okay i understand what they are but gpu doesn't quite know what what it is yet actually this has nothing to do with gpu this javascript code this is going to generate an array in my cpu memory right so this is on generating the data in my cpu memory now i color coded these so that you can see which xyz value corresponds to which vertex i'm dividing so these six vertices these six vertex positions define two triangles uh those two triangles forming a quad so as you can see the first triangle is going to be this this upper triangle over here and these two vertices this red one and the blue one they are repeated over here right they are repeated do we need to be concerned with the winding order of vertices yes uh so in which order you're you're you're providing them clockwise or counterclockwise yes we won't need to be concerned with that but not necessarily it really depends on your render settings and in some in some cases you don't need them it's done because it doesn't matter in some cases they do matter uh so depending on which order they are uh when you look from camera they'll be front facing or back facing and if you want if you tell the gpu to treat front facing triangles differently than back facing triangles then it will be important if you don't care then it won't be important so in this case the winding order that i'm using is clockwise actually i don't remember if it means front facing or back facing in the context of webgl but for this example it doesn't matter anyway all right so this is my vertex position data if i wanted to create vertex color data i can do that prince vertex color data rgba rgb rgb values these are the colors that i associated with my vertices right so again you know these two are duplicated right they appear twice in the in the array so this is not an array of six values this is an array of four times six values that's the area of 24 values right this is an array of three times six values and the area of 18 values but conceptually i know that this is you know x y z coordinates and i know that this is rgba values i know that nobody else knows that just yet so we're going to have to tell our gpu what this is in a little bit so the first thing we're going to have to do is that we're going to have to allocate some data on our gpu memory and now we're going to pass that data uh we're going to write that that data so the way we allocate data on gpu is different than the way we allocate data allocate memory on on cpu memory so with with gpu i'm going to just create a buffer a buffer will contain some data some data that i know what it is my gpu doesn't quite know what it is just some buffer that contains data right and i haven't really specified the size of that buffer i just said create a buffer for me and it tells me then the index of that buffer some name that's associated with that buffer name meaning some number number associated with that buffer and then and then i can bind that buffer and we're going to be doing this binding a lot actually this is the opengl way of handling things i bind that buffer and then i say you know remember these positions already converted to a 32-bit floating point area of values and put it onto that buffer that i just i did bind here right so i first find that i first bind the the buffer and then i send the buffer and then send the data to that buffer right so i'm not i'm not telling which buffer to send here it's going to be the buffer that i just just binding over here right so this is how i took my my positions positions data over here that i created in my cpu memory and i send it to my gpu memory and it's in my gpu memory right now and i have this static draw here it's not too important it basically means that i'm not planning to modify that data very often it's just a little bit of performance optimization there are a few more details associated with that but it's not important for the time being so this is how i send my position data pay attention here that i haven't specified the size of this array uh because hey this javascript positions wasn't was an array javascript only knows how big it is so like you just takes that entire array and puts it into uh the gpu buffer i can do the same thing for my color data i take the color data create a buffer for the color data i bind that buffer and then i push that color data into my gpu memory so at any time i can only have one thing one one buffer that is bound right so i if i find this buffer it's going to be that buffer when i find the color buffer here that bubble is now is gone now now now this is the one that's fine and this is going to happen a lot actually uh for for for a lot of things we're going to be binding things and then the upcoming commands after that we'll be using using that object whatever that is with in the context of webgl and opengl you don't directly allocate memory on the gpu so over here i created it and i didn't specify a size so this actually didn't do any memory allocation over here i said bind it again it did not do any memory allocation on the gpu over here i'm saying oh here's the data you're supposed to put inside that buffer this is when the memory allocation happens so the gpu will will take that data and allocate enough space for it and copy that data onto it and that's how my position and color data ends up in my gpu memory so we finished the first steps we took our scene data and put it in our gp memory and that and also we sort of ran out of time today as i thought we would so the next thing we're going to do is that we'll talk about this this this shader program and we're going to talk about how to write the vertex shader and and the fragment shader so we're going to talk about we're going to talk about this stuff and then we're going to talk about how to send data and how to how to do the rendering and in the end we're going to have a full program that uses the gpu rendering pipeline but for today i'm going to stop right here we're going to continue from here next time as i said we did this we're going to do this and the rest next time thank you thank you

Channel: Cem Yuksel

Views: 2,510

Keywords:

Id: UzlnprHSbUw

Length: 59min 10sec (3550 seconds)

Published: Thu Feb 18 2021