Unite Berlin 2018 - Harold Halibut and Making a Stop Motion Game

yes hi guys welcome and thanks for coming my name is Renata Kimo glue I'm the game designer one of the writers and the composer of Harald halibut this is Gabriel I am our lead programmer and also co-founder of anime and materials getting company and we want to tell you something about our content creation pipeline for Harald helluva today and also have an interactive part towards the end where we go into the editor and you can ask us all kinds of questions because it's been about like a little more than a month at that we moved over to the HD pipeline so it's also new for it for us but we had some things that we already experienced some really nice things and yeah that we want to show you so in case you don't know the game yet here is a trailer agent Harrelson you got me leads yet well that might be a minute Bob but you know you've got the best man on the job I'll have this figured out before dinner me it's always me who has to give us the ensberg headquarters said to prepare for a messy crime scene but this is textbook work out of a swish sometimes I do wonder what life would have been like on earth [Music] thank you thank you so everything you have seen and the trailer is actually handmade in a real world workshop part of our team is actually sitting there so if you have questions later about the analog creation part there they are there was a little bit shy but you can talk to them so yeah we built sets and puppets in the real world using classic model making techniques and go through a digitalization process where we want to give you some insight let's start with the analog creation yeah I'm making a video to give you an inside [Music] okay that was a little overview I also wanted to highlight a few specific things besides of building all the sets and puppets even if there are two the assets for example even if you have paintings in the game we try to aim we try to do these in the analog world too so this is an example of no this this looks strange well it looks normal here actually okay yeah here you can see how all our our character is painting one of the science of of the stores and because the whole story starts in 70s we also wanted to aim to recreate these classical sign paintings for example and we keep that analog creation process in the sound design too you can see how we use the tiny shoes of our characters actually to record their footsteps and once we created all these assets in the real world we move on to the digitalization step yeah we want to preserve all the little details in our digital models that were built in the real world before so we use a technique called photogrammetry to do 3d scans by taking hundreds and hundreds of photos from many different camera angles all these photos taken are processed by capturing reality a special photogrametry software what that basically does is matching all the points on the object surface throughout all the photos taken for being able to triangulate and get a 3d point cloud and this 3d point cloud is then translated into a 3d mesh you can see all the camera angles the little dots around the object are all camera positions where we've taken photos this mesh additionally to this mesh we get textures as well because photogrametry is a image based technology but one problem with these textures often is that we have baked in light so you you have the the present light situation all in your textures to get rid of these highlights and shadows we use a turntable and a controlled light setup the light comes from the camera direction so we with no shadows but if you want to do some photogrammetry outside where you can't control your lighting situation there's also a very nice handy tool in the unity asset store called the lighting tool what that basically does is extracting all the shadows and you'll get a very nice and clean diffused albedo texture which you can use for your for your models in units and relight it yeah but back to the geometry the the photogrammetry scans are very high poly from 1 to 10 million polygons and to use these assets in a real-time environment for example in a game we need to do some reduction of these polygons we do that most for most of the objects by hand because it's more efficient to do that by hand when you look on the the poly count yeah and for characters it's very important to do it by hand to get a very clean mesh because if you have a very messy mesh the the animations doesn't look that good yeah we decide of scanning the geometry we also scan the material I'm involved in another company called a Lumiere and we are specialized in digitizing real surfaces so we scan PBR materials here's a little example what you can see here is only a plane everything every details only come from the PBR textures beside of very accurate colors we provide high detail normals and physically physically correct specular and roughness and also we can capture transparency you can see some painted Leafs on a projector film and with our scan we get automatically a Alpha mask and information about how transmissive the material is what's interesting here by the way is although we try to create everything in the you know real world there are of course objects that are impossible to scan which is for example grass or something like that or yeah plans so we use we kind of create the plans on flat surfaces as you can see here our artist draws them on transparent sheets and we scan them in and put them together in the digital world yeah in unity it looks like that we scanned the the bark of the tree and the Leafs separately and then we brought it together on unity and if you want to do some material scanning by yourself there's a very nice tutorial on the block of algorithmic yeah they tell you how to use your smartphone as a material scanning device okay from that point on we have like our 3d 3d models and depending on what we are scanning so if you have the material scans we already have the PBR textures but for all the three-dimensional objects all the things that we 3d scan we of course only have the diffuse albedo so um what we then have to do is alter the materials which you do in a substance painter and try to achieve the same settings or settings for the materials like metal and so on that looks similar to our real-world objects what really helps is being able to look at the objects to shine a flashlight on them and so on so you can really test how the material materials work in the real life and even if you for example don't like build all your game assets in the real world it's still useful to use real world objects or materials to see their properties even if you want to recreate material properties in the digital world yeah and especially in the beginning we'll used a lot of mmm these shader collaborations charts actually [Music] because you have specific values for specifically reference and values depending on what kind of material you have for example and it really helps in in in that you know during the time where you get used to or get an idea of how material materials really work it it really helps you have specific numbers that you can kind of put in to see okay this metal should be having a roughness of this or another material should have a reference of that now we don't need that anymore because we are kind of used to it and we have like our predefined like material settings that we can reuse in our models since we have 3d models in the end so it's basically from there on the whole pipeline is a very classical game pipeline so we can normally Rick the characters and then are able to animate them also using motion capturing what you can see here is a live preview for from the motion capture system perception urine that we are using in the unity editor it's it looks cool but it's actually not so useful in the production environment but we use it to kind of get a feel for the characters and to stress tester rigging this is a nice little script from Katie wrote Akashi he has a great github repo you should definitely look into that all kinds of useful stuff and everything is free we also this actually no one has seen this this is a really quick test we did while ago so you are among the first ones to see our facial animation that we are working on currently yeah because we have 3d models we can create blend shapes and yeah and the eye is a little bit of as you can see yeah because it's 3d models we can create blend shapes as you would do with a normal model and use lip synching oh and wise is Lauren Hill by the way so I guess she won't be speaking the character in the end to mastership you know don't be afraid that we don't have anything to we don't have anything to let's not be mediocre in our greatness you know what I mean like think big think big and thinking doses thinking experiences and don't be afraid of experiences that teach you yeah we hope she doesn't sue us because of that okay and from that point on we move over to unity so everything comes together there I'll just switch this over there are so many things to talk about when it comes to the HDR and the pipeline and also we heavily use cinema she and the timeline and we were doing screen captures in the last days and thinking about what we should show and we just thought we'll let you guys ask what specific things you want to know and we will just look a little bit into the project maybe as a little introduction about our experience of experiences of moving into the HTT render pipeline we use a lot of lights because we are aiming for it is very cinematic lighting and that was that has always been a performance problem in the legacy pipeline because of the light culling that is present in the HDR and a pipeline it's much faster actually and since all the models we have are actually super low poly like most most of the walls you see are basically planes and they look so detailed because of the normal Maps we have because of the material scans and yeah a real properties of those materials I can actually that might might be interesting because the mouth sometimes disappears for example this and if I click on oh I think I have to activate the selection where okay IIIi thought it doesn't show but it shows and you can see there is only it is this edge so this whole ceiling actually consists of two polygons and we still have all these details because of the scans which of course saves a lot of performance yeah and because of the light coming in the HD render pipeline we got a huge performance boost again other things that we in our early tests immediately noticed were transparency for example that was always a problem because you know our artists like these transparent transparent objects behind other transparent objects in a transparent room with transparent things floating in front in the back and so on and you can imagine it's really difficult to get that right especially in the legacy pipeline we always had sorting issues and now it just works as it should we have another example here where you can see like these multiple layers of glass with chamfered edge and ya know transparently sorting problems anymore another cool thing that we use a lot is volumetric lights until recently we had to rely on third-party plugins from the asset store which are also nice but most of them aren't compatible with the HC pipeline yet and let's move to this room ok what you can see here is around the lights there is just like slide glowy thing which is because of the volumetric lights and the whole watery thing in the background is actually just lighting so we don't have water we were thinking about if we which is actually one of the main things we always do even if we add digital things to the game how would we approach that if we would do it in the analog world and in this case for example for the for the water thing in the background we would probably fill a room with haze and light it appropriately so we have this effect and that's basically what we also recreate in the engine yeah these are like the and the transition was easier than we thought we thought we would have to like rerender all like export all the textures again but there's a handy tool which I by the way discovered after exporting a lot of textures again so that's why I'm telling you there is a handy tool and I can actually show you here which says upgrade project materials to high definition materials or upgrade selected materials to high definition materials what this does as the new lit shader needs a different type of texture which is the mask map it's actually really nice because it combines several textures in one texture so it saves memory it's this one here not so interesting it should look at it because all the different color channels are actually different kinds of information so what we have in here is basically the ambient occlusion the roughness the metallic map and the detail mask which we actually don't use about so you can just leave it out if you don't use it and this too I've showed you and in that yeah what that actually makes is if you have a material which is said to the standard a standard shader it takes your roughness and it takes your metallic map and creates this mask map out of it it's really useful yes and from this point on you can ask us everything we yeah and we'll show you whatever you want any questions already anything that thank you [Applause] yes yeah so you scan everything you recreated everything yes almost everything besides of glass surfaces because we can't scan glass and hey don't even know if it would make sense what we actually do that's actually interesting because as you can see some of our older glass surfaces which we want to rework actually like this one it's just plain glass and what we tend to do in like more recent approaches is that our artists kind of paint a white surface with a little bit of pain in the corners to have some kind of dirt simulation and which we can scan again and just make it transparent extract the whites out of it so that's that's the approach with which we are going with now yeah but besides of that I don't know if there is something else which is not made in the real world of course we often do adjustments sometimes the things don't really fit for example because when our artists built the sets in the real world they of course built rooms and they built force aids and they don't like build a huge this this set for example would be about I think eight meters long so we split it in two parts and sometimes these parts just don't come together so we have to scale things or it just a little bit but yeah yeah I know we were actually talking about that in the last days I can't promise anything yet but we are thinking about it at least thank you thank you very much you're welcome I have a question about one thing you said when you are showing the real-time performance capture you said you it wasn't practical to use in production and then I was curious why that was oh it's only this live previewer thing and the problem is the perception neuron motion capture system has a very specific kind of rake and this rig is it's very strange actually I don't know why they use this kind of rig and so the finalized animation which is which is very targeted on our characters doesn't match this preview at all so it it feels the same of course you know the movement itself is similar and we have to adjust some things it's really useful to get a feel but it's if you want to have because that this was our initial thought actually it would be nice to have a direct feedback where I have to grab something for example or where I have to move with the character's proportions this unfortunately doesn't really work that well with that laughs preview so it's more that gimmicky thing but the motion capturing itself of course is in production news it's just only this preview okay thank you you're welcome I'm really interested in your pipeline of doing the crazy mega scan thing and having this super high complex mesh and then you mentioned that you by hand simplified those down I imagine that must have been a huge like time sing for you guys so I'm kind of interested like is is that the most effective workflow could you automate it and second question could I see your wireframes to see what you enter the course yeah so well they are on as you can see so yeah these are a little bit heavier sometimes you know a day it actually changed from from so2 acid sometimes our 3d guy is like super focus on details so he made these planks all stuff right sorry did you build that sort of stuff from scratch again over the top using the scanners reference or did you literally my hand simplify it because it seemed we may be building it from scratch yes so once you scan the assets we have high poly models worth three to ten million polygons and we tried out different techniques of reach apologizing also auto auto retopo tools which we actually use at one place which is these background buildings so these are automatically reach apologized and because they are in the background you don't see how crappy they look actually and it's a program so a program can never know what you want you know it's it's good enough for in cases like this but for all the things that we really use in the game we do it by hand because of both the efficiency so we can an automatic original village' tool even adds polygons to planes which don't need one friend in need any for example and also to have a good two apology which is really important if you look at the characters now so oh that looks a little bit creepy so yeah the crazy eyes yeah it's really important to be able to animate the faces in a nice way here's also the body okay yeah sorry did I answer your question yeah actually yeah I mean we are still working on that game and we yeah we tried out different things but yeah we would still do it the same way because it's never more efficient than it if you do it on your own and especially if you want to correct things later it really helps to have like a good topology otherwise you were like hacking your way around and yes but you're totally right the analog creation for example of a puppet takes about a little more than a week and then we have all actually the same amount of time also going into the return policy and texturing thing and so on a while the texturing is actually super fast because we have the texture it's basically retouching so we don't really paint textures it's more like you know setting up these material definitions and doing some what you usually would do in Photoshop you know cloning a little little pieces that didn't came and get over and during the scan process yeah hi I have a question about animations which is you were going for specifically for that handmade stop-motion style so I want to ask is there anything specific you try to do to stylize the animations yes that's a really interesting question and we again we had a huge a long time of experimenting and we also experimented a lot with that and we for example try to reduce frames of the animation to 25 first and we also tried 12 frames and song and the problem is it doesn't work as it would in film in film you have 25 frames per second but it's always these 25 frames in a game environment the frame rate change and you can never make sure that it's like they accept maybe someone can we can we could actually but you can never make sure like how which frame you are showing so that actually led to a crazy jitter that really looked bad the other thing is the gameplay itself and we wanted to have a you know fluid movement in terms of it that feels responsive that feels modern while still trying to keep that analog feel to it and so all of our attempts to reduce the frames and so on none of them worked but what really work is actually to not clean our animations so we have this motion capture jitter in there and we were just actually we wanted to try out some and just slap the animations on the characters and we were looking at them and it was like nice that saves us a little bit of work and it looks more like stop-motion and of course there's still a lot of adjusting going on so especially with the fingers for example there are not always super accurate in an immersion captured data so we have to tweak these bits and bytes here and there and yeah and also we sometimes exaggerate some of the movements because it's not real humans right so yeah thank you can you tell us a bit about the camera animation since it gives a cinematic feel yes um we are actually let me show you this we are using cinema sheen for all of our cameras and the what we basically do in game is to activate and deactivate these cameras where we set priorities to so this is the current camera we are on right now and I will just you know when we move to other rooms we have triggers there that basically activate these cameras if I activate this one for example now you can see it automatically blends over to the bar that's the bar camera which would normally be activated when we enter that door in the cinema scene brain we're able to set up blending time so we have a default value for further blending but we are able to specifically say it takes in this amount of time to blend from one camera to another and what we also use is for example in the when you go over when we want to combine more things than just camera animation we use timeline so this is the this is the timeline clip basically that's launched once you entered yeah this whole area and we blend in that first shot this this the first shot is actually this one and because we blend it in that's something really useful and it took us a while actually to discover that we can use it from anywhere so what this does is actually it really blends from any camera camera you are using now to that camera that you specified so it works everywhere which is really cool we also use that and it's really useful to use timeline for these kinds of things because in this case for example we also change the transparency of see the transparency of this front wall you will see it fades out when we zoom in yeah and it's really nice to be able to control these with keyframes both the materials the the camera and Gabriel for example wrote there there is actually the the playable the default playable in the asset store which is really useful to get an insight into what else you can do with the timeline and there is for example a light fader and so you can play it live with that we the problem is you have to kind of drag every single light into the timeline that you want to fade so Gabriel for example programs a custom playable for us with which I'm able to fade everything that's the child of a game object which is really cool for these things so it saves us a time again yeah okay thanks for coming [Applause]

Channel: Unity

Views: 7,702

Keywords: Unity3d, Unity, Unity Technologies, Games, Game Development, Game Dev, Game Engine, Harold Halibut, Stop Motion

Id: 9usssSQc0wQ

Length: 34min 29sec (2069 seconds)

Published: Mon Aug 06 2018