Unite 2016 - The Power of Procedural Meshes

am I on I think I am yeah all right hello everyone my name is Alexander Burke and I'm gonna talk about procedural measures in unity this is actually the second time I'm giving this talk at United I gave it at unite Europe about half year ago so what you guys are getting is actually the super-sized US Edition with more gifts and a picture of John Travolta as well so buckle up a little bit about me I am originally from Denmark but have lived three years in the UK so if you have a hard time placing my accident that's why and I've been working in games for about oh yeah I know five six years and I have done a lot of different things a recent thing I've done is to create my own studio called out of bounds games and this year I had my first title released on PlayStation Vita steam and soon hopefully on PlayStation 4 and sort of like my secret sauce in my game is to do procedural measures so that's why I know it's really powerful technique and I guess it's because I've always been a very like visually driven developer I guess we've all learned to program this way at least I did and it was always sort of like well I could get to understand the basic concepts but it was not something that motivated me a lot it was a lot better when I started to actually like do game development but even that has its limitations because you asked like limited by the pre-made assets you already have like the audio files and the textures and so on and what Mike's my personal experience when it comes to using procedural measures is that like what you're doing you can it basically becomes like clay that you can like either sculpt more freely at like pre runtime or while you're running the game and it keeps you a lot more like freedom to do what you want to do the like drawback is that it's not clay it's it's made out of like numbers in math so you need to have a fairly good understanding of it but when you grasp that you gain a lot more power to do exactly what you want to do so this is the agenda for the talk today I'm gonna start out by first like giving some example of like why do we want to like do procedural measures then I'm going to cover basic the basics of how to do it in unit C then it's sort of like it's a segment where I'm gonna break a bit of some of the stuff I've used him for and I am gonna end up with some like hidden gems on optimization and debugging that I had to take really deep on on the internet to find or head to this grub on my own as I said earlier you really need to understand maths in order to like get the fool out of this talks talk so if you don't I can recommend this book it's one of the few like books I bought while I was at university which wasn't like wasting my time it's super condensed and it covers a huge area of mathematics like vector math rotation like representation random number generation splines it's got it all it's a really good good read furthermore most of the stuff I'll show you in the basic section is actually from series of examples I've written units you already have their own procedural examples but they're from unit C free so kind olds and they also they suffer from the problem that they're trying to be shiny and surely the potential but also trying to teach you how they work at the same time whereas my examples are really basic but very easy to understand and there's also a port of something called triangle dotnet which is something I've used in my recent title myself so why use procedural meshes well I see three different use cases for it the first one is like player made content like sports a really good example like you wouldn't be able to get that like Lobby like putting different limbs together look that that game has without actually generating the the mesh from scratch another case is if you want to do like unique mechanics which is something I used in for a lot and I think Gish which is sort of like pre Indy Indy like I think it's from 2007 or eight and you control this like blob of goo that can either be sticky or heavy or slippery or like a combination of them and you wouldn't like be able to get that gameplay of trying to like control this blob without actually like generating the shape of it dynamically Bert which has been really popular recently is procedural generation actually like generating like a level for the player to walk around in and a really cool example which I don't think a lot of people know about is sir you're being haunted which is it's it's a survival game where you're being constant by robots that treat you like they it's foxhunting and they have a English countryside generator so every time you play the game it's like a new countryside it's very quirky I I recommend checking it out of course there's also tilt brush which is I think it's amazing like you wouldn't be like without procedural measures you wouldn't be able to generate something like this and provide this like kind of in section for the player yeah I think it's probably one of the coolest things I've seen made with unity in a long while all right so let's go over the basics and I'm starting really bad like really basic here a mesh is made out of points called vertices that are then combined into triangles that can be rendered by the GPU so that means that the most basic thing you need is a list of vertices a list of the triangles like how those dots are connected depending on how you want your surface to look you might need more information per vertex but I'm going to cover that as well so in unity the most basic things you need in order to like show a mesh and also to generate it is you need a mesh filter which is what stores your mesh and then you also need a mesh renderer or like a skinned mesh renderer or something equivalent in order to render it and of course you also need air script it doesn't have to be attached to the like the game object where you're doing like where the mesh is actually being displayed but in the examples I provide it is because it's the easiest so this is probably the simplest measure can make the way you do it is first you create an object of the mesh class then you give it a vertex array and as you can see it's just vector trees and after that you tell it like we're in the vertex array like a given triangle list so for this example it's it's it's very basic it can get complicated really quickly so if you want your surface to actually like like react to lighting it needs to know like in what like like what like direction it's facing and you use normals for that and it's basically a another vector that is pointing in the normal direction of the surface so as you can see here we again have the same triangle from before on the left but both because it doesn't have normals and because I'm using a colored material with that doesn't take take Lydian information it doesn't actually react to the directional lights zooming around while the triangle on the right does something that is not used that often in like the way that you would like you used to like call our models back in the day is that you would put color directly on their vertices and then like you would interpret from that nowadays should do that more with textures but if you're doing stuff procedurally I find it's really handy to be able to put in extra information in the color Channel and if you're doing something like a particle system it's still useful to actually have color come from the vertices as well in this example I am basically every update changing the colors based on HS HS v color space so that's why you can see it's basically like rotating around and normally it wouldn't like have that banding effect it's just because it's it's in a gif it's also worth noting that you can define colors in two different ways you can do it as floats and you can also do it as bytes and in general it's better to use bytes because you you don't incur a conversion but for completeness I'm showing you both but this is the preferred way to do it UVC is basically a set of 2d coordinates for each vertex and what they're most commonly used for is to like look up into a texture and like wrap this texture around the 3-dimensional object and they can also be used for a bunch of other things if you need to pass in like more information and as you can see it's again it's quite simple you create an a vector to array fill it with the information and you then assigned it to the Tuda mess mesh object in this example I'm actually also creating like an animated texture but you can see that in the example project so I'm given like triangle like setting triangle indices its own segments because it is what I have found to be like the like the toughest and not to crack in a lot of cases and it's predominantly down to the fact that your code executes in linear manner but you're trying to connect things in on a surface so at least on on it like a 2d surface but maybe you might even have something where it's it's a bit more involved than that you can just see like that's this is the code for just a simple plane and what you have to do is like go along like one line in the plane and actually reach over to the next row in order to get the indice for that so you add in the the width of the plane and then it needs to get the one in front of that so that's plus one to that and then you also have to do it for the other triangle that's in that like sub quad so prank rectangle and this is just a simple example it can get quite a lot more complicated such as this this is a fog of war system for like four weeks game that was supposed to be on mobile so we had to make this system super optimized and like my solution to that was to use a procedural mesh but I had to like create the triangulation for this and it's basically like hexes within hexes how the heck do you like triangulate that well what I did was I took some some paper and basically like drew it out and I started to like find like common features like how it does the topology how does the mesh like the kinetic connections actually repeat and I found out that these like lines going through it would like be of like would be similar and then I basically created like functions that would be like triangulate from A to B and and and so on and if you are like I think just actually draw drawing it out on paper and like like looking at it is in general way easier than trying to like first just generate the vertices and then like print out where they are and so and just draw it it works so this is some of the stuff I have used procedural meshes for in the past this is what's an entry for global game jam a couple of years ago where I think the theme was we don't perceive the world as it is we perceive it as we are so we got this idea of basically having a game where you change your perception of the world from being like flapped around and you use that like solve puzzles and that's actually done with like a procedural mesh so changing a mesh Collider yeah I guess this is actually like pre 2d physics it's kind of an old old game here it's it's it's cheating a lot like as soon as it's doing the bending animation it's actually not doing any physics it's basically just like disabling it and then translating it manually around and then I basically for the level being flat I stole like how the collider should look like and when it's completely round how that should be as well and then I enable and disable them had to start an end of transition the way that it's it's actually doing the transition is that I have these two Bezier curves which acts as like the unit vectors of the space but I'm curving them and I'm then using the like X X Y position to basically get like a Radian like a radial coordinate like in degrees and in like distance away from the sensor in order to find out where it should end up when it's done the wrapping you can't actually represent a circle with through a bish yes it's it's going to be a little bit out of it's not perfectly round so what I do for the last like five percent of the transition is I basically like slurp directly like directly to like the mathematically correct position but I still get like this nice effect of of the world bending so this is the current game I'm working on and it's called laser disco defenders it is a disco themed self-inflicted bullet hell shooter and what that basically means is that the core mechanic is that every like shot fired by the player stays in the level and keeps bouncing around so the more trigger-happy you are the heart of the game gets and for me it was like it was important that it was like a spirit as possible so in order to get the laces to to render that's actually one giant mesh as well and a given beam is consisting of the middle end and the end caps that are like bounded if the beam is too short to have like a middle segment it's just like the to start points that are like generated and in general it's like instead of I could have like every time it hits it hits a wall trying to calculate exactly like how it should then like where the vertices meet meet up but instead I actually just draw like a big flare and hide that I'm being lazy and it also happens to be good because then I can do like a circle check a collision check and see if I'm hitting something which makes the games a little bit easier to play when you have to hit enemies so yeah sometimes it's just the best to actually be a bit lazy and we have Johnny so like I did a lot of research into like disco and the visuals and so on and something that became quite apparent is that like like you need good lighting and you need a lot of it in order to create that like dancefloor disco feel as you can see here like each laser beam actually like gives off light into the environment I did have the problem though that I had promised my publisher that we were gonna do a Vita release so I had to optimize this quite a lot so I couldn't rely on units is deferred lighting because it's just too slow for like like yeah I say low-end mobile I'm not surely we can categorize the Vita as that but it doesn't have enough resources to do the third lighting and also have like bloom and some color correction post-processing in and that's something I wanted as well because it gave like the game that right like disco Luke so what I actually ended up doing is that the light is also a mesh and it's pretty much the same mesh as this for the laces it's just been like drawn out expanded and it's using a much more like diffused texture and like you can also have like individual light sources so it's basically just a component I add on to something and it will also like tell the lighting system like hey you need to match like draw a quad for me and this is the colors of the of this light and the radius that is then rendered into a texture by a camera that is like always taking the same position and like size from the the main rendering camera but this camera renders first and it's actually because I'm not concerned about like specular lighting I only wanted like diffuse light I actually found out I could reduce the size of this texture to only be 10 percent of the size of of like the actual like screen size which meant that I had a lot of like it was it didn't use a lot of like fill rate which meant that I had like most available for the post-processing effects I wanted to run it's probably also why you can see it's a bit like low resolution and if you're like kneel here and you're like ah that that what like where can I learn more about this I have actually put a blog post up where I go more into detail on like how I made the system and how it works and you can find that in on my own personal way sign I'm also gonna like tweet out the slides where there's like a link to it directly it's also how I usually react to box by the way so procedural generation is something that I also think is really interesting to work with so when I got the chance of like working on my own game full-time that's something I wanted to explore and I wanted to make something that was also not just like tile based because that's been done before and as you can see these are like the three zones in later disco defenders there's like Crystal Cave which has this like really nice faceted look there's the dance then which is like more of the traditional block block you look and then there's the Guru grotto which is like inspired by lava lamps where everything is like curvy and and blobby and it's actually all run through the same system it's just like how I generate the outline and like the contour texture or like the come to a mesh that that differs but it does prove a bit of a challenge to again get through the triangles as I mentioned earlier so the way that the system works is that I basically have a set of rules of like pre-made layouts and for given difficulty setting or like if say you have survived and silk cave ten then I take like how far you've gotten and I used that to grab like rules that are proving that that meets up with the like the current difficulty you need to face and then it basically just starts to like place them out and make sure they don't overlap and then it generates like an outline for each of them and in the next States they actually all combine together into one like continuous outline and then the last step is to triangulate it and how do you do that like this is like it changes so much from each like level like how it's actually they laid out I do have some ideas for how I could but I ended up being lazy and used something called dallona triangulation it's a method that was originally created for taking like a point cloud of data and like create triangles like connecting them with triangles so you could run like certain kinds of physics simulations on it in like in academia turns out it also works really well for this and it has fairly good like performance characteristics if you use the correct type of algorithm I like I can maybe generate 50 levels a second on on divita with the current situation so like it's basically as soon as you complete a level there's a new level for you to play so it's fast enough for my purposes and I basically taken and like ported an existing like c-sharp implementation to unity like the port was to basically replace all the like double values with floats but hey I did a port and debugging an optimization like again it's super important to like work visually I find like you couldn't you can't try and like debug your mesh by like just printing out the values it's tough I wouldn't recommend it a much better option is to use something like the gizmos and debug class to draw draw out stuff as you're creating it and you also have like the like the thing you're gonna like have the most problems with it's gonna be like like you're generating your triangles and you might actually have triangulated your mesh correctly but they're turning the wrong way so quite often just turning your camera around and see if the triangle appears is a really good option and you can also turn on wireframe rendering to see if the triangles turn up so yeah so you have a bunch of like way of optimizing meshes that doesn't actually move while the game is running ironically enough mesh dot optimized and does absolutely nothing and I have this confirmed from Unity themselves yeah it used to do stuff now it doesn't so and cold as that as much as you want so much better options is true either like it when you mark a mesh s like being static inside unity what it actually does in the background is that it takes all the static meshes and combine them into one giant mesh and then based on where your camera is looking it's it's basically like generating like in a triangle index array to like figure out what it should actually rent out you can do the same at runtime so say you are like creating a lot of prefabs or like spawning a lot of prefabs then you can call static batching utility and actually like ask unity to do the same like process that it would normally do just as you're like saving the scene or like editing the scene I do find like I have this world this is some of the stuff I got a lot of questions about back when I did the talk at unite Europe and I didn't actually I hadn't had to need to research it myself I've done a little bit of like up created some examples things for you to look at and I did find that static matching utility will bring you like your draw call countdown but it is not as good as if you actually like combine your message to combine the message yourself which is the second option it is a lot more work to do that though and you risk breaking stuff especially if you have like if you create like an automated process like what unity is basically trying to do with the static batching is like it doesn't want to like send an object for rendering if it's behind you because then you have instead of like doing frost from calling you end up doing like like calling the triangles on the GPU which is not efficient and you can easily like make that take yourself when you're like combining the meshes so dynamic meshes the like the only option you have from unity science is to say that the mesh is is marked as dynamic and it basically tells the underlying graphics API that it's it's gonna I need to change this every frame so try and like give me a memory buffer where like it's it's more easily accessible for that and this is one of those things that is not documented and even the guys at unity doesn't like some of them they don't like I couldn't get an answer for like why this actually works but I saw like like five FPS frame rate in increased when I like started to double buffer my my meshes my best guess is that it's basically lot not really releasing the like the active memory buffers in the background quickly enough for like every time like you you like keep using the same mesh object here every frame yeah I don't know I the only thing I know is that it works yes so if you can the best like data structures to store your mesh data in is in flat erase because that is what it's actually reading itself but of course if you don't know exactly like as you're creating that like before you start creating the mesh like how much how many vertices and UVs and so on you actually need I find it's a lot easier to just use the list and unity actually provides like a set of functions where you can pass in a list directly instead of having to do a conversion to an array so that can also give you a quite nice performance boost of course like if you're doing like the less work you're actually doing the better and I like I started with a very like naive implementation of my laces where I would actually like render all the laces in the whole level and I found out like I can actually just like do a really simple like frustum column myself both for like if the little ASA should be drawn but also if the light for it should be drawn as well and as you can see that gives me like that doubles my performance in this example which is what's very nice right here yeah this is sort of like a beautiful hack and I haven't done it myself but again in the slide notes there's a link to an article that described it you can actually create something like create add a skinned mesh yourself and then move the bones for it and unity will take care of moving the meshes for you of course that it's that's only good for something which is like actually something that is like writable but it can it's it seems like it can get like it you are really like nice performance increases as well and this is something I I did experiment with and failed horribly at actually getting to work units is API isn't thread safe like you can't just like access to the mesh object from like a threat you create yourself but if you're creating like like you are creating like they are like the dates are going into it beforehand so you might not actually have to worry about that and I think like they just announced the job queue system here at June I delay I haven't had a chance to read more into it but I think that sounds like a really interesting thing that we're now getting at getting access to and just to give you an example of like how much like performance you can probably get out of it like this is profiling examples from ladies discord offenders and what you see to start with is basically all the laces being like calculated where they actually are and then the next thing is where they're being being rented out and like like action that's not exactly true what happens is that its first doing the the like physics calculations and then it's also drawing delay some like the mesh for the laces itself and then it ends up doing the lighting calculations but the thing is as soon as like a laser is done being calculate like it it has like moved or maybe it has like been reflected on a wall or it's hit something it doesn't interact with any of the other laces so at that point I could actually pass it on to like some worker frets and have them generate like the the vertex data for me and like even if they are not put into like the exact same order in the array it shouldn't it still doesn't actually have a like it shouldn't be a problem like the all the laces are completely independent from each other another option it's this compute raters voice which is basically a way of generating data on the GPU problem is it's not available on an old platforms like it has to be something like fairly modern if you want to do some of these techniques in mobile I for example I wouldn't recommend relying on computer traders because it's still there's still a lot of devices out there that doesn't support it if you're only doing like console and PC it might be worth looking into and a good thing is that unity does like help you you don't have to like ride a computer for each like platform you want to talk and you can like write it in one language and unity will compile down to the rest so that's really nice alright I have a lot of time left I hope you have a lot of questions alright so in summary I hope I have given you a good idea of like the power of what you can actually achieve with procedural measures like it's something that you can use to solve like existing problems in another way and it's also something that will allow you to create like gameplay and like rendering solutions that you wouldn't otherwise be able to achieve and it's something that has really like helped me create like unique solutions and unique a unique look to my game so I hope that's something that you can use yourself and I'm looking forward to see what you will use it for thank you so I have really poor eyesight so I hope there will be someone from unity who can like help me figure out who has like hands raised yeah maybe sorry oh all right so do we have any questions so about the skinning one have you looked at how much more efficient the unity skinning is versus just computing the mesh from scratch like how much penalty do you pay for procedural measures versus skinning it's a good question and I must admit I don't actually know I do think that article mentions that he ended up doing it this way because it was quicker so if like and it does kind of like makes sense because your power I think you're passing way less data to the to the GPU like that's that's another problem with like dynamic created meshes is that you will probably hit like like a bandwidth barrier like how many how much data you can actually send over to the GPU that's why that like frustum calling trick I showed earlier works so well because it both saves you in like the calculations that you need to do but also the amount of data you have to send over so I think that's one of the reasons why it's it's more efficient thank you for the lesson I'm interested in baking ambient occlusion and curvature maps into vertex colors so how could I get the vertices in crevices and in the sharp edges would it be possible so I could just try and repeat your question so I I would like to bake ambient occlusion and curvature maps into vertex colors okay so how would I get the word this is of the mesh which I've been which I have modeled it earlier earlier oh okay that's super easy to do I thought you were like going to ask me how to like get that data into the mesh yeah okay I've never heard about a curvature met before what you would do is basically say you would get like the mesh filter component and then you would get the mesh from that so it's also like it's the same way that you also access existing meshes and and change them okay and I meant about like when there's a bird is in in a corner so how would I recognize with in the code that the word is is in okay so you mean like yeah I guess that's for them institution isn't it but I guess that's for the ambient occlusion like you need that information yeah do you have to like is this something you want to like have run really fast or is it okay if it's slow like they can is it like a baking process you want to do yourself or yeah yeah the baking I would like to do it in unity and really fast with the code right I don't want to paint paint separate vertices in right it's really modeling but it's something that would be like your workflow not something that would like run while the game is like played by the playwright yeah my workflow yeah okay well in that case you could use like like the in the indices array and actually try and do a lookup to like like connected like basically find out like what triangles are this vertex part of and use there to calculate the angles then see like if it's if it's narrow enough to like meet ambient occlusion okay I will check that out could I maybe contact you later we out your blog or something to maybe get the proper answer through that alright sure yeah thanks hey thanks for the talk it's for you thanks I'm kind of curious you talked a lot about rendering but how does how does the physics system deal with like changing collision meshes potentially if you're generating something like that yeah so I guess the best example is like this so like I am I'm cheating I'm cheating so much like I basically have to date like versions of the collision mesh and when I'm doing the translation I just disable physics completely and just do the interpolation changing a mesh Collider at runtime is really expensive I did like in like another even earlier game where I actually created like a destructible environment and that was all done with mesh colliders and I actually found out like a little dirty trick I'm not sure it still works it probably does that instead of like actually sending in like new mesh States and I would just change like the the indices array and then where something was destroyed I would create like a degenerate triangle and it was way quicker than like trying to actually like rebuild the rest of the mesh so if you want something where you're like taking stuff away or adding stuff in that can be quite quick especially if you actually start to use like shop like speeding up you're like this was like a worms game on on a planet so I basically had like a curve for the whole planet and then I would like generate the mesh around it but it would actually be a lot of like small separate mesh colliders which for like runtime performance wasn't very good but for changing the individual like for changing the actual like kalila that turned out to be a lot quicker so in jail like I think there is like it's it's not like a linear like calculation in order to actually generate like in the new coalition mesh when you sit in send in like new data it seems to be like to the power of two or three or something so if you can keep like the collided the collision mesh small you can actually do those chances really quickly and it's and if you still didn't need like a large coalition miss mesh try and like split it up into smaller ones and then do the change did that answer your question do we have anymore yeah I know you carried this to some extent but what are the best learning resources there isn't well I guess like units his own procedural examples and description reference was got me started on it for me it was just a lot of like trial and error and like like try and it's like see what I could do and what would like break like that thing would like for example like the planet and like like splitting it up into smaller chunks was just something I found out myself by by experimenting with it and sort of seeing like oh the bigger the mesh is even like there's like a nonlinear correlation there that's something like I basically found out on my own I'm like I have learned a bit of like OpenGL in the past I guess that also helps and that's actually another thing why that like math book I talked about earlier is so good because it actually has a section on shader programming and it does a better job of explaining shader programming than my OpenGL book from from University did so like yeah get that book that's that's my my advice I guess one thing I would add is that the the sir you are being hunted game that you mentioned earlier on from big robot games they actually did a talk about their approach to the entire thing which I believe is available on the unity youtube channel as well so if you type in big robot or sir you're being hunted in the youtube channel as a the whole thing about their their approach to it and I guess slightly different slightly different approach slightly different type of game as well so yeah yes I think like my experience has been with a lot of like 2d constants as you can see if you need something like like so you're being haunted it's a 3d game so if you're more interested in that maybe that saugus is a good place to start yeah definitely all right thanks for time

Channel: Unity

Views: 51,435

Keywords: Unite LA, Graphics, Unite 2016, Unite 2016 Los Angeles

Id: 8LTDFwWMlqQ

Length: 42min 33sec (2553 seconds)

Published: Thu Dec 01 2016