Let's code 3D Engine in Python. OpenGL Pygame Tutorial

this video is a tutorial for beginners to learn opengl using the python language we will analyze the basics and fundamental concepts go from rendering a simple triangle to more complex scenes with custom models in simple words we will create our own 3d graphics engine step by step so let's start with the fact that opengl is a specification that defines an application programming interface to a video card for creating applications that use 2d and 3d graphics we will work with the so-called modern opengl this concept arose in 2010 when the version of opengl 3.3 was presented and from that moment this api uses a programmable pipeline with the obligatory use of shaders and to work with modern opengl i will use the modern gl module because when compared with the standard pi opengl module it has a number of advantages that allow you to develop applications much faster and more conveniently so let's start in practice to understand what opengl is and first we need to install the pi game modern gl and numpy modules using the pip install command and now let's import the pi game modern gl and system modules then write a class with the loud name graphics engine let the size of our window be the input parameter now we initialize the pygame modules take the value of the window size into the attribute and set the attributes for opengl we need to specify which major and minor version of opengl we will use here we assign version 3 to 0.3 we also set an attribute for the profile mask we use the name profile core which means that the deprecated functionality will not be used then using the set mode method specifying the window size we will create an opengl context for this we use the opengl and double buffer flags now we need to detect this context using the modern gl module it will automatically detect the created context using pygame and we will also need an instance of the clock class to set the frame rate and delta time next we need a method for checking events where we will monitor the event for closing the window and also pressing the escape key and when such an event occurs we will gracefully exit our application let's write the render method here we clear the frame buffer that is fill the screen with the selected color and then swap the buffers using the flip method it remains for us to write the run method to start our application here there will be a main loop in which we will call methods to check events the render method and set the frame rate to 60 frames per second and then let's create an instance of our application and call the run method let's run the program and we can now observe the set resolution window which is filled with the color we have chosen well as you can see creating an application class in an opengl context is quite simple and now we have everything prepared to implement more interesting things so now i propose to create a new file and call it model and using the example of creating a triangle model we will look at how the opengl pipeline works then let's create a class of the same name and let it have access to the application instance into our opengl context and let's take a look at the simplified rendering scheme of the opengl pipeline the pipeline receives information about the vertices of the object then for each of the vertices the vertex shader is launched while let's say that it processes the vertices and passes it to the primitive assembly stage this assembly is carried out based on information about vertex connectivity so then comes the rasterization stage where the primitive is divided into fragments and now for each fragment a fragment shader is launched with which the color of the fragment is determined then there are various tests after which the fragment becomes a pixel and output to the frame buffer and as you will see later to get the desired result the vertex and fragment shaders have access to uniform variables that we define ourselves well as you just saw we first need to get the vertex data for our model and for this we will write the appropriate method where we define the coordinates of the triangle vertices taking into account the fact that opengl uses a right-handed coordinate system and here it is important that the vertex data was a numpy array and the data type was float32 so when we have vertex data we need to send it to the gpu memory and for this we create a vertex buffer object for which we specify our vertex data now we have come to writing shaders then let's create a shaders folder and it will contain files for vertex and fragment shaders for them we use the name default but the extensions will be vert and frag respectively opengl uses its own glsl language to write shaders so let's go to the vertex shader here you need to specify the version used in the main function will become the entry point here you must specify the gl position variable as a four component vector since the vertices must initially be specified in homogeneous coordinates in the fragment shader we will also specify the version and leave the main function empty since we will return to shaders a little later now we can load our shaders using a separate method here we will use the context manager to load them safely but we loaded them in order to create a shader program object it is needed in order to compile our shaders using cpu and they will be ready for use on the gpu side let's access the vertex buffer and the shader program object through our class constructor [Music] and now we have come to another fundamental concept in opengl this is the vertex array object in which we just associate our vertex buffer with the shader program please note that for the buffer we specify the data format and attribute names in our case this means that in the buffer each vertex is assigned three float numbers and this group of numbers corresponds to an input attribute called in position okay then create a vertex array in the class constructor and move on to the vertex shader so we now have the coordinates of the vertex as a three component vector in position as input and its attribute location number is equal to zero we pass it to the gl position variable for the further rasterization process in the fragment shader we define that the fragment color will be determined by the frag color variable and here we define the location number of our frame buffer and now let all the fragments be read to render our model we need to call the render method on the vertex array object and here we must not forget to write a method to remove all created resources since there is no garbage collector in opengl well now let's import the triangle model create an instance of its class and call the render method and when the application is closed we will release the resources with the destroy method and as you can see our model is displayed in red so we can say that we got the hello triangle achievement but the triangle is not a very interesting model and let's now create a cube model if we place the cube in the center of the local coordinate system then its vertices can be assigned coordinates and in addition these vertices can be numbered and then you can very simply describe all the triangles that make up the cube there will be 12 of them let's use this information and describe first the vertices of the cube and then using the numbering of the vertices we will describe all the triangles that make up the cube and by the way in opengl by default the order of describing vertices is counterclockwise and for convenience we will write a static method for generating vertex data based on vertices in their indices then we use this method to get the vertex data and use the cube model instead of the triangle and if we run the program we will see something strange our entire screen is filled with a red rectangle but in fact this is a cube we simply did not take into account that the model itself must go through its own transformation pipeline that is our model must first get from the model space to the world space then to the camera space and then to the clip space all these transformations are done using the model view and projection matrices well then i suggest creating a new file called camera and install the handy pi glm module which is designed to work with opengl math let's import this module and first set the parameters for vue frustum now let's write the camera class in the constructor of which we calculate the value of the aspect ratio of the screen and then we have all the data to form the perspective projection matrix using the glm module and to access this projection matrix let's import the camera class in the main file and create an instance of its class now we can return to the model file and for clarity we will expand the constructor with the on init method in this method we will pass the projection matrix from the camera instance to the shader so in the vertex shader this will be a uniform variable of a four by four matrix and the positions of the vertices must be multiplied by this matrix and if we run the program we will see the same picture but this is because our camera is inside the cube let's move our cube back a little to make sure that we are really working with the cube model but for now we only see one side but let's leave the cube in its place and move the camera itself for this we select the position for the camera and the so called up vector to form the view matrix we again use the glm module in the look at function here we need to specify the position of the camera the direction in which it looks in the up vector such a function is very useful and saves us from extra calculations to get the view matrix and now the view matrix can be passed to the input of the shader it will be the same uniform variable that we multiply by the projection matrix and if we look at the intermediate result we can say that we really have the outlines of the cube since the camera looks at its center and is located higher on the right but our transformations are still incomplete since we need to get the model matrix we will get it using a separate method and now let it be a four by four identity matrix then we pass it to the vertex shader in the same way and it is multiplied by the view matrix since this is an identity matrix the result in rendering the cube has not changed in any way yet but let's still use the model matrix as an example for this we create an attribute for tracking time in the main file create a method for getting the time in seconds and call it in the main application loop next i suggest calling the update method before rendering the model and in this method using the rotate function of the glm module based on the time value we will rotate our cube around the y axis but then at each iteration we need to update the model matrix in the shader let's look at the result and as you can see our cube model began to rotate but since the cube is filled with one color this does not allow us to feel the full depth of the object and in that case i suggest moving on to texturing our cube let's go back to the method of getting vertex data let's take a look at the test texture so the texture coordinates are in the range from zero to one along the x and y axes and for our case for the convenience of working with them they can be numbered and if we look at the front face of the cube it is easy to understand how to apply texture coordinates to it that is with the help of texture coordinate numbers we can describe all the triangles in the cube model and then you can do the same as we did to describe the geometry of the cube that is first we define a list of texture coordinates and then using their indices we describe all 12 triangles from which the cube model is formed and at the end we use the numpy module to combine the geometry data and texture coordinates data into one array and since our vertex buffer has changed we need to change the buffer format in the vertex array object specify that the first two numbers are texture coordinates and then define the name of the attribute for these coordinates so in the vertex shader we indicate that we have a new attribute in tech scored we also define an output variable already for the rasterized texture coordinates for the fragment shader and let them be called uv well the fragment shader now has access to uv coordinates and since they are in the range from zero to one let's display them in red and green color components and now we are seeing a rather strange picture the faces in the cube are not ordered in any way since we need to activate depth testing to do this let's go to the main file and in the application class constructor for the opengl context set the depth test flag and as a result we can see the correct rendering of our cube along with the depth test we can activate the call face flag which does not render invisible faces the rendering of the faces is affected by how we set the geometry of the vertices for triangles by default the vertices are set counterclockwise but if we change the mode to clockwise we will see the internal faces of the cube so let's get back to our texturing and we're almost done and of course we now need to load the texture itself here we will use the pygame module that is we will write a method that loads the texture at the specified path and then we create a texture object already on the gpu side for this we need to specify the texture size the number of color components and convert the texture to a string so to move on create a textures folder and place the textures you need there then we use our method to load the texture at the specified path and then go to the on init method and here we need to specify the name of the texture variable and the number of the texture unit and call the use method for our texture so in the fragment shader we get access to the texture in glsl this is the sampler 2d data type and using the texture function we get the desired color for the fragment using uv coordinates and if we run the program we will see our textured cube but upon closer inspection we see that the texture mapping is reversed along the y-axis and we did not get the desired result this is due to the fact that in pi game the y-axis is downward so we need to use the flip method and flip the texture along the y-axis and so we have a correctly textured cube and so let's change the texture to something more interesting by the way let's reduce the rotation speed of the cube and i will load the texture of the old wooden crate well i guess our cube looks much better but at this stage i suggest giving us freedom of movement so let's move on to our camera class so opengl uses a right-handed coordinate system and the movement of the camera can be set using the corresponding vectors up right and forward that is we will simply move it along these vectors then we set these vectors as unit and orthogonal let's define a variable for the speed of movement of the camera and we will change the position of the camera using the move method and we will track the keys pressed using the pi game module and in order for our movement to be independent of the frame rate we need a variable for delta time the delta time value itself is easy to get using the clock instance in the main application loop and then we can start writing the move method here taking into account delta time we will get the velocity value and using the pi game module we will get information about the keys pressed and then we will change the position of the camera by moving it along the forward writing up vectors in accordance with the selected keys for moving the camera and we will call the move method through the update method and since the camera position changes we need to recalculate the view matrix so we will call the update method in the main loop before rendering and since the view matrix has also changed we also need to pass a new view matrix to the shader but if we run the program we will see that we have a rather strange control this is due to the fact that the orientation of our camera is always directed to the center of the cube then when calculating the view matrix let's set the direction of the camera from its position taking into account the forward vector and in this way we get the opportunity to move in space but as you can see it is impossible to change the camera orientation now so let's solve this issue so there is such a thing as euler angles with which you can describe any rotation in space as you can see from the slide this is pitch roll and yaw and for our engine we will use pitch and yaw let's then for convenience the values of position yaw and pitch come to the input of the constructor take them into attributes and define a variable for mouse sensitivity since we will change the camera orientation with the mouse now you can write the rotate method here we use the data on the relative movement of the mouse for the previous frame and based on them in the sensitivity we calculate new values for yaw and pitch while limiting the pitch within so that there are no unnatural turns up and down and most importantly we need to write a method for recalculating camera vectors taking into account yaw and pitch the forward vector is responsible for the orientation of the camera so using basic knowledge of geometry and trigonometry you can find the projections of this vector on all world axes and when we have calculated all the components of the forward vector then using linear algebra in the cross product of vectors we will first calculate the right vector using the fake up vector and then find the true up vector and do not forget to normalize each of the vectors and it remains for us to call our rotate methods in the vector recalculation method through the update method thus we got the classic implementation of the first person camera for our engine but we can make one more improvement so that the mouse pointer does not go beyond the window to do this using pygame you can use the event set grab method and also hide the mouse pointer itself well now we have a very convenient control and besides complete freedom of movement but let's go back to the moment when the object is painted in one color then you cannot use the shader and just fill the texture with the specified color and we are again observing our object without any sense of its depth and we have come to the point where we begin to implement lighting for our engine then let's create a file called light here we will write a class with the same name the attributes of which will be the position of the light source in its own color looking ahead i will be implementing a fong lighting model so we will set the light intensities for the ambient diffuse and specular components let's import this class in the main file and create an instance of it and now we can pass the intensity of the ambient component to the shader to begin with but we do this taking into account the fact that in the fragment shader for convenience we create a structure called light the elements of which will correspond to the attributes of our instance and then we can declare the uniform variable light we will calculate the lighting using the get light function in which for now we will multiply the color obtained from the texture by the value of the ambient component and let's apply this function and now we can see what the ambient lighting looks like but in order to move on to calculating the rest of the lighting components we need to return to the method for obtaining vertex data and talk about normals so in simple terms normals are vectors perpendicular to surfaces and in our case for six faces we have six normals and since our face consists of two triangles each triangle consists of three vertices then we make a list so that for every six vertices there is the same normal let's transform this list into a numpy array and again combine it all into a single array of vertex data and accordingly in the vertex array object we change the format for the vertex buffer and add a new attribute in normal let's make a change to the vertex shader that we have a new attribute and declare the output variable normal so the lighting calculation is performed in the world space and it would seem that it would be enough to multiply the normal vector by the model matrix without taking into account translation but not everything is so simple and in order for our lighting to be correct when the model is not uniformly scaled we use more complex mathematics first invert the model matrix and then transpose and besides for lighting we need to calculate the position of the fragment also in world space so in the fragment shader we have input variables for the fragments normal and positioned so let's go back to the get light function and normalize the components of the normal vector to determine the diffuse component we will use lambert's law according to which the illumination of a surface area is directly proportional to the cosine of the angle between the normal to the surface and the direction of light and in turn the cosine of the angle is equal to the dot product of these vectors then we will use this information calculate the direction to the light source and taking into account the normal we will find the value of the dot product of these vectors do not forget to use the diffuse light intensity value and calculate the final lighting color for the fragment and it remains for us to pass in the on init method to the shader the values of the position of the light source and the intensity of the diffuse component and if we look at the result then we can say that our cube finally looks like a three-dimensional object and as you can see lighting plays a very important role in the perception of objects so for the full lighting model we need to calculate the specular component in this case we need two vectors the first vector is the direction to the camera and the second is the vector reflected from the direction of the light source then the dot product of these vectors raised to some power will give us a specular highlight let's then calculate the vectors we need and find their dot product and let's say we raise it to the power of 32 take into account the specular intensity and calculate the final color of the fragment it remains for us to pass the specular intensity to the shader and we also need to pass information about the camera position but since the camera is not static we will pass the position value in the update method and also do not forget to declare a uniform variable for the camera position in the shader itself and as a result we can see the specular component on the upper surface of the cube the value of the number to which we raise the specular component determines the strength of shining the larger this value the stronger the light will be reflected so if we return the previous look of the texture then we can say the phong lighting model is ready and by the way we used a point light source without attenuation here and you can easily find information on the implementation of other light sources on the internet and now i would like to talk about the current structure of the project as you can see the model class is overflowing with various methods perhaps this approach is suitable if you work with a single instance but when creating several instances of this class it will lead to an unreasonable consumption of computing resources in memory since for each model will load the same shaders textures the same vertex buffers will also be created and so on and in the simplest case for this we need to reorganize the project approximately to the following class architecture where we separately have access to shader program objects and various vertex buffers based on them we can create the vertex array objects we need and in a separate class there is access to textures and for convenience all this is aggregated in the mesh class with the help of which we will already more efficiently create object instances so now our project is presented with a new file and class structure according to the slides let's take a quick look at it a class has been created in the shader program file where we will use the previously used method to load the shaders we need create an object of the shader program and we will access such objects through a dictionary called programs there will also be a separate method for releasing resources next comes the file for vertex buffer objects here for the convenience of further work a base class with three methods familiar to us is created and already inheriting from this class the cube vbo class is created in the constructor of which we define the data format for the attributes and the names of the attributes themselves and here we already have our method for getting our cube's vertex data and there is also a main class in which through a dictionary we will access the vbo we need so in the val file we import and create instances of the shader program in vbo classes and using the get val method we create the vertex array object we need which can also be accessed through a dictionary named valves next comes the texture file here the class of the same name is arranged according to the same principle we load textures using the appropriate method and access them through a dictionary called textures here for example three different textures are loaded so the file with the mesh class at the moment it is very simple we import the vowel and texture classes and create their instances with the destroy method and we import the mesh class itself in the main file and create an instance of it in the graphics engine class constructor and as for the model file here i suggest using the base class for the model which specifies the name of the vowel and the texture number and then accordingly through the mesh instance we get access to the desired vowel and through it to the shader program besides here we leave three basic method and then to create a cube class it is enough to inherit from the base model class specify the vowel name and texture number and then we leave everything as it was for the update and on init methods and if we run the program then we can still see our cube we reorganized our project in order to be able to conveniently and efficiently create a large number of objects for our scene and if we are talking about the scene then let's create a file with this name here we will import the model and write the scene class with access to the application in the constructor we will declare a list of scene objects in a loading method we will write a method for adding an object to the list of objects then in the load method we will place our cube in the scene and in the render method iterating over the objects also called the render method let's check the work of this class import it in the main file and instead of a cube create an instance of the scene as you can see everything is working correctly and we have come to how we can add more objects to the scene and for this we need to talk about transformations as you probably understand in order to place an object in space it needs to be given the appropriate coordinates and for this we will calculate the model matrix taking into account this position we use the translate function of the glm module and in the class for our cube we will also specify the input coordinates for its position then let's place two more cubes in our scene on opposite sides of our cube but only with different textures and as you can see we got a way to place our objects in space but in addition to this the object can be given its initial rotation around the axis so let's set the rotation around each of the axes in euler angles and for convenience in degrees and then first convert them to radians and then we use the rotate function three times separately for each of the x y z axes and also provide the cube class with the ability to set the initial rotation let's give the cubes a rotation of 45 degrees around the x and z axis that is the pitch and roll values so now we are able to set the initial rotation of all objects but there remains one more fundamental transformation scaling you can also scale an object along three axes we will similarly use the scale function of the glm module and it is important to note that the correct order of transformations is scaling rotation and translation and it is important to know that matrices are multiplied from right to left then for example let's stretch one cube along the y axis and the other along the z axis and thus we have the opportunity in our engine to apply basic transformations to objects okay but to talk about some more nuances let's add even more objects to our scene suppose we place a certain number of cubes in the xz plane so that we are in the center so we got a more interesting scene and i made it to talk about gamma correction let's assume that the x-axis is our calculated color values and the y-axis is the color value displayed on our monitor and everything is fine when we have a linear relationship that is what we calculated is what we got on the screen but due to the peculiarities of color perception by our eyes and besides historically it turned out that monitors use a power law when displaying colors approximately the degree is 2.2 and then it turns out that a color value of 0.5 is displayed with a lower brightness a value of 0.218 and in order to make the resulting colors linear again we need to multiply the displayed color values in linear space by the inverse gamma curve that is make them brighter so let's go back to the fragment shader and declare the gamma value and then we can apply the gamma correction to the final color value but if we run the program we will see a completely unexpected result everything has become too overexposed this happened due to the fact that we take the initial color from the texture but the colors of the texture as a rule are already in nonlinear space and therefore it turns out that the gamma correction actually applied twice to texture colors and then it is enough for us to return the texture colors to linear space and look at the result and thus we get the colors of objects that are correct in terms of gamma and eye perception [Music] so if we make a solid floor of cubes in the scene we will see ugly ripples and artifacts going into the distance this is due to the use of one high resolution texture for small objects to solve this problem opengl uses a technology called meep maps which provides a set of texture images where each subsequent texture is half the size of the previous one to activate this technology we go to the get textures method and first we specify the minification and magnification filter for the texture and then we use the build meep maps method so now we do not see that ripple but already strongly smooth textures go into the distance and here we can additionally apply one more technique and to improve the quality of the textures we will use an isotropic filtering such filtering allows you to eliminate aliasing on various surfaces but at the same time introduces less blur and therefore allows you to save image detail so in the next step let's figure out how to use custom 3d models in our engine for example in the classic obj format and for this we will find some free model and upload it let's put the downloaded files in a folder named objects and start with the vbo file for the obj format you can write your own parser but for convenience i will use the module pi wavefront install it using the pip install command and import it accordingly so for our object we will write a class called cat vbo here we inherit from the base vbo class and we need to define a method for getting vertex data then we use the installed pi wavefront module with cache and parse parameters after that our model may consist of several objects stored in the dictionary but in this case there is only one object then we get access to the vertex data and convert it to a numpy array and if we look at the data format of this model we will see that they completely coincide with the previously defined format and no additional changes are required and in order to have access to this vertex buffer then we will place an instance of our cat vbo class in the vbo's dictionary now we can create a vertex array object for our object and put it in the vowel's dictionary and since the shader program will be the same we simply use our created vbo also in the texture dictionary we must have access to the texture of this model load it with the appropriate path and now we can make a cat model class for now we will make it based on the cube model and here we need to specify the name of the vowel and texture let's go back to the scene class and add an instance of our model and place it away from the camera and as a result we can observe our model but we see that its initial orientation does not fit our coordinate system then you can easily change the initial orientation of the model let's rotate it 90 degrees and in addition let's add our cubes to the scene well in this case we can observe our model which is made in obj format you can use different formats of 3d models and for this there are various modules for their convenient loading so we have covered only the most fundamental and basic concepts for creating applications with python and opengl but i also do not exclude the release of further series on learning opengl [Music]

Channel: Coder Space

Views: 103,610

Keywords: opengl, python, opengl tutorial, 3d engine python, 3d engine from scratch, python 3d engine, python pygame, pygame 3d, python 3d, pygame opengl, python opengl, python opengl tutorial, python opengl 3d, python opengl cube, learn opengl python, learn opengl, python 3d cube, 3d engine in pygame, 3d engine in python, moderngl, 3d engine from scratch python, 3d engine pygame, opengl 3d engine, python opengl 3d engine, 3d engine tutorial, python 3d graphics, python 3d rendering

Id: eJDIsFJN4OQ

Length: 33min 31sec (2011 seconds)

Published: Sat Sep 03 2022