Creating a 3D digital double in Blender & Meshroom | Blender Part 1

today we're handing over to Calvin Serrano a VFX artist from Germany who helped us on rise of the dark side this is part one of a two-part tutorial today Calvin is gonna show you how he scanned Ollie into a 3d model then he's blender to simulate throwing him into a tree first off let's start up with the equipment you can use any camera that can be set to manual use for photos scanning the general rule is the more pixels the better and you should use a lens with around 24 millimetres or lower that doesn't distort the image you will also want to take some C stands with you for stabilizing the actor and of course you need a computer and in this case an Nvidia graphics card that can support CUDA let's get to the second part the preparation in the case you want to photo scan an actor or person you want to have as much detail as possible in the clothing since photo scanning is basically tracking thousands of points on an object you should also avoid all glossy transparent or reflective materials on your clothing or the actor next up is the location find a soft wet environment with unchanging light and background a inner courtyard conference room or Street on a cloudy day should work just fine you can also set up your own lights if it's needed but your actor or the object you want to scan in the middle of the space since you need a lot of space to move around have the actor do the t-post with the bit wider spread legs this will work best for the most cases since we see all surfaces clearly like the armpits and the space between the thighs the less one object includes the other the better but it should still look natural we need this little movement as possible to have a good result the actor should also focus his eyes on something in the distance and not talk during the photo process now we will set up the camera first off put all the camera settings on manual so it doesn't adjust anything automatically the shutter should be around 100 to avoid motion blur set the white balance to the current lighting in my case I used the cloudy preset the ISO should be set up so that we are one step away from clipping the highlights on the actor if the background Clips a little bit that's okay the f-stop should be four or higher to have the actor and the background as much and focus as possible let's get into taking the pictures make sure that when you move around the actor you don't cast any shadow onto him this would confuse the scanning program light on for the first round framed the whole actor in shoulder height so that there's still some space between him and the edges of the frame start from the front and walk around the actor once keeping the same distance and taking a picture every step for the second round framed the actor the same way but this time from knee height for the third round do the same framing but this time race the camera as high as you can now we will take a few additional close-ups of parts from the actor that are important for the shot like head shoulders hands thighs and feet all right back on the PC the first thing you want to do is copy over your files the next thing you want to do is download mesh room mesh room is very straightforward and easy to use program so all we got to do is just drag and drop all the images we have into this window right here once the image is imported go through them one by one and check if something is off like in this one for example his eyes are closed you don't want that so just go in and remove this one and generally you don't want like closed eyes or blurry pictures or movements so if you can find any of those things in a picture just remove it from this list okay so I deleted the ones that don't fit and now you only have to press Start and it will ask you to save and yeah you should always save so press safe and save it somewhere where your project files are anyway now that it's saved that press Start again and you will see it will start going through this list of notes you don't really have to change any of the settings here you can go into the notes and like change settings but the settings that are in there by default are already like pretty good this will take some time depending on the amount of images you have you can take also more than one hour now it's finished and we can see a point cloud on the right side here just go ahead and press this load model button when you see the 3d model just hide the structure from motion and you will only see the model you can rotate by left-clicking and just panning around and you can move around with the middle mouse button you can also zoom in and out with the mouse wheel so the result looks pretty good for the most part the feet up very good and also the arms are pretty okay but the face is something weird going on almost looks to me like some pictures are confusing the program like we have a double nose here and the eyes a bit low on this side then over here so yeah I think I think I would go ahead to like go through the pictures again and see if I can find something weird and just delete them and recalculate welcome to the second attempt this looks much better now I just deleted the closer pictures we took later on like this right here since I think he moved a bit during the photo progress so when we took these pictures later on his face and head on was already like in a different position or something that can happen sometimes so you really want to make sure that your actor doesn't move during the photo process if you started to wonder why I didn't show you the process with Olly who we actually used in the short film it's because we did a few mistakes while photos scanning him that's what I wanted to show you the proper way of doing it from scratch with a new model but yeah this is a 3d scan dolly and it didn't turn out that well you can see like a lot of weird edges and yeah just like overall it looks a bit weird a bit blurry a bit spiky especially like his head over here the lens flares moving across Ollie from the bright background that's definitely something you want to avoid when taking pictures another thing is that the light isn't very soft in this environment it's very directional another problem here is that all this outfit doesn't have a lot of detail as you can see his jacket and t-shirt are very plain and the shoes are also pretty reflective if you look at them closely welcome to the blender and simulation part of the tutorial you will see on the left corner right here the buttons that I'm pressing so if you're wondering which button does what you can always look right here first off let's delete everything by pressing a a that selects everything in the viewport and then press X to delete and yeah just click on this delete again and everything should be gone now we're going to import our 3d model from the mesh room you will see in the folder where you saved your mesh room project file also a mushroom cage folder and if you open it up you see every folder for each of the node steps and in the last one texturing you will find the text should mesh it's in one of these it's this one yeah alright so just copy this file path and in blender let's go into file import obj and then just paste the file path right here and yeah select the textured mesh and import obj now that it's imported I will introduce you to the basic navigation features so for orbiting around just hold down the middle mouse button and you will all be the round for zooming in and out just use the mouse wheel and to pan left and right hold down shift and the middle mouse button you also have all of these functionalities on the right side here with this gizmo right here you can also orbit around you can also click on any of these axes and you will snap into that position and you can pan around with this one and zoom in and out with this one now activate x-ray mode right here this will give us a bit more understanding about our chaotic mesh at this point I also like to activate quad view hold down ctrl alt & Q and your view will split into these four different views you still have your normal perspective view up here but the other ones are flat from only one direction this one example is X this one is y and this one is Z what I also like to do for this is adding a reference mesh for that just going to edit preferences add-ons and search for rigify this one right here just activate it and close the settings again now we can go into add amateur basic basic human now we have a basic human bone structure right here this bone structure is also properly scaled so we can use it as a reference we can open up our site menu right here and look into dimensions here we see it's 1.98 meters high so if you want to have your character smaller or bigger make sure you have your bones selected with left click you can select objects with left click all right here in the outliner so ya select the metric and with pressing s you can scale it up or down and doing that you will see the dimensions changing too so yeah I'll just cancel this by right-clicking and leave it at the default value you can also open close this menu with M something that I also like to do at this point that will be more handy later on it's changing the shading settings you have your four different shading menus appear wireframe solid material preview and rendered let's stay in solid for now and open up the settings right here select random in color that way every object will have a random color and it will be easier to understand the scene now we are going to move the mesh around so it fits the reference bones first make sure the cursor this little white and red circle right here is in the center if it's not press shift s and cursor to world origin that will reset it to the center now we have to change the transform pivot point to cursor we can do this right here select 3d cursor and now all the transformations you do to an object will start where the cursor is so now select the mesh and rotate it and move it so our character lines up with the bones through rotate you can just press R in any viewport it will rotate it in that direction to move something press G you can always accept change with left mouse button or cancel it with the right mouse button our mesh is also still much smaller than the reference so press s for scale so you can scale it up the top of the head should end up with the top of the last bone I will also save the project panel just go into file and save now that our character is roughly in place we can go into edit mode for that make sure that textured mesh is selected and change from object mode to edit mode we can also deactivate the quad View again now with ctrl alt Q now press alt a to deselect everything all right what we want to do is cut off the mesh around the feet for that press C and you will see a little circle you can make it small and bigger with the mouse wheel and with left-click you can select so just draw around the feet you'll have to press again if you want to move the camera around get into a better position and then press C again and keep on selecting now that we have it painted out press X and delete the vertices now we want to select the island of polygons that we separated for that hover your mouse over this island and press L this will select all the polygons that are connected to each other now press ctrl I to invert the selection and again X and vertices to delete it you can go back into object mode now and look at the result we can also deactivate x-ray mode again alright now we want to add when you mesh a cube alright move this cube down by either pressing g4 grab and Z to just move it on the z axis or select the move tool right here you want the cube to cover all of the floor so around this height now we want to activate another add-on for that go into preferences again and activate the bool tool all right select the cube first and then we shift the character you can also do this in the outliner by pressing on the cube and then here with control on the textured mesh now go to objects ball tool and under outer boolean choose slice now we have two separate objects the one on top and the one on the bottom we can delete the one on the bottom and you will see we have the feet very cleanly cut off for the next steps we want to have the RIC a bit more visible for that select the rig go into object settings and viewport display display as wire and activate in front with the rig selected we can switch back into edit mode and now we will delete bones that are not necessary for the physics simulation for example the breast bones press X and bones and these hip bones right here and also the heel and toe bones now changed from edit mode to post mode and also change the transform pivot point to median point activate the quad view again and also activate the transform tool you also want to change the transformation orientation to local now we will adjust with the rotation of every bone to fit the original mesh when you're done with that press ctrl-a and apply pose as rest pose you can also change the view settings to texture now until now we completely worked in the default collection but we want to add a new collection so press right click on scene collections and click on new you can double click on the collection to rename it and just call it old this way we can save old models in case we want to revert something go back into object mode and select the mesh we will duplicate it with shift D and press right-click to have it snap back and then just drag and drop the new mesh into old you can also go ahead and hide the old collection right here now we will clean up the photoscan mesh a little bit select it and go on to modify us add a smooth modifier and set the repeat to something like 30 the mesh should be smoother but it shouldn't be too smooth so I think this is like a good middle point now also add the decimate modifier if we extend the panel a bit by click and dragging on the edges here you can see that the current phase count is 300 mm and that's way too much for physics simulations we will try to get it down to around 15,000 by changing the ratio value I'll start with 0.1 all right that's better but we can still go lower let's try point 0 5 you will start seeing now that we have less detail but we can change this by changing the appearance and the object and shade smooth so now it doesn't look that much worse than before apply the modifiers from top to bottom now by just clicking apply on the first and then apply on the second now we also want to apply the transformations we did to the model by pressing N and you'll be able to see the location and rotations all messed up so press ctrl-a and apply all transforms now everything is reset and if we have to reset the mesh it will be in this position again select the mesh first and then the rig and press control P for parenting and select with automatic weights you will notice the mesh disappeared from this list right here but it's actually just in the metric now so if you open it up you can find it right here when you select the metric and go into post mode you can change the position and rotation of the individual bones and you will see how the mesh will follow but make sure to not change the pulse at this point back in object mode go into add mesh and plane you can scale this up a little bit with s and just type in the number 10 this will make it ten times bigger make sure everything you edit until now is in the default collection so just drag and drop the plane into the collection with the plane selected go to object rigidbody and add passive passive rigid bodies are objects that don't move but interact with other bodies now we will start with the actual rector of simulation for that I will be using the recto tools which costs something you can get yourself two since it's making the progress a lot easier but I will also provide you with a link to do it without this tool so yeah when you download it go back into blender and preferences and now when you're in add-ons go to install up here and search in downloads for the rectal tools select it and click install add-on then you will find it in the list right here just activate it here again and you should be ready to go there's one more add-on that I want you to install it's very useful it's called visual limits and you can download by clicking this button here and download zip and then when it's downloads just extract the dot PI file and install it the same way as you did with a rectal tool it's called visual limits just activate it and let's go so what you want to do now select the metric and you want to press f3 this will bring up the search menu so if you searching for any functionality inside of blender you can probably find it in here so we are going to look for a rectal and here this rectal to spy menu and select generate hitboxes this little window down here opened up so extended and in mesh select bones you also want to select use bone envelope that way the bones already have a bit better structure you will see the add-on created a new collection that's called direct oil hitboxes you first want to select one of the objects in this collection and then right-click on the collection and select objects now go into the object properties and viewport display and while clicking alt click on display s and wire when you hold down alt and change something it will change it for all the objects that you have selected at this moment so do this again holding down alt and pressing in front and now we will always see the hitboxes now if you press play you will notice that the rector was just falling apart that's because we don't have any constraints between the hitboxes and the bones yet so while you still have all the hitboxes selected open the rect all tools again and choose generate rigid body constraints now what you want to do here is changing the generic type to generic spring and activate all the spring angle properties but just click dragging down here if we play the animation now you will see it's working already there are a few things that are still want to change and I'll show you what those are first of all we can hype the collection for the rector hitboxes so we only see the constraints and the free selected constraint you can see with these anger visualizations how much the bone can actually rotate so for example for the elbow you don't want it to be able to rotate more to the back so selected let's select both elbows so we can change them at the same time go into physics properties and while holding alt change the X values change the X values something more realistic now it can't move all the way back but it can move more towards the chest now basically repeat this with every bone that needs adjustment we can also hide the metalwork now and hide the constraints and show the hitboxes again now we're going to adjust the hitboxes a little bit to fit better with the shape of the mesh for example for the head I'll activate quad view again with ctrl alt Q and then I'll go into edit mode activate x-ray again and now I will select the top vertices with C paint over them and now I can move and scale them around with the transform tool we can use this to make it fit the head shape a bit better you can also deselect vertices with middle mouse button in the selection node just do this for some of the bones that are not quite right yet you can also select and edit multiple objects at the same time right here we have a little problem because when we play this because when we play the physics simulation you can see them pushing apart and we don't want that so you need to select one of them and only select the top part here and then just move the selection a bit to the right do the same with the other side too and now it works you when you're happy with the adjustments select the whole rectal hitbox this group again and go to object rigidbody calculate mass and select beans soy this will calculate the mass of each individual bone so that the body parts have a more realistic weight I think I want to adjust the stiffness of the simulation a bit for that I'll activate the constraints again and select the whole collection and then while pressing alt click into the stiffness and change 10224 all the axes maybe even one yeah I think this looks more natural the head is also moving in a weird way so I will select the head constraints these ones here and just lower all of these values to like minus 10 and 10 let's see now that's a bit better now shall the hitboxes again and select the center one go into the physics properties and activate animated now if you extend the timeline up a bit and open the side menu to the left here we can extend this and we will see all the keyframes were making so on the side of animated click this little button here and you will add a keyframe for the first frame of the animation if you play it now you will see this bone won't move go into the object properties and activate the keyframes for the location 2 now move forward to frame 5 and move the hitbox to a position where you want to where you want your actor to fly towards and then click on the society again to add new keyframes now if you play it from the start you see the hitbox is offset for some reason that's because we have to have the transformation animation at least one frame after the rigidbody animation so select the transformation keyframes and move them with g1 frame to the right now if you play it it's staying where it's supposed to be now go back into the physics properties go to the frame where your animation stops and deactivate animated and add a keyframe so now if you play it you see it's just flying off you can also now easily adjust the force by just picking the last keyframe of the transformation and moving it to the right if you don't want to see any of these helpers you can also press this button here and it will deactivate all the overlays you might still see some glitches like here with the foot is starting to space out for that you want to go into the scene properties into rigid body world and change the steps per second to a higher number like 120 you see it's much more stable now all right now we're going to actually set this up for our shot first off let's open hitfilm and import the footage let's create a composite shot out of this but change the resolution to HD so 1280 by 720 now we just need to adjust the size for that right click on the footage and transform it to frame width now go to the export settings right here and choose the default preset PNG sequence and export now in and out area we are doing this to get a low res version into blender to have it play back in real time so now open up blender again and under add add a camera you can look through the camera with this button right here there's a special navigation method that you can use to move the camera like a first-person shooter to use it open the search menu with f3 and type in view navigation and now you can look around with the mouse and with WASD you can fly around as left-click you accept and with right-click you cancel again make sure that the camera is part of the default collection if you cannot find in the outliner you can also just left click on it and press m to choose to in which collection it should be now go into camera settings and activate background images you can expand this and click on add image now click open right here and navigate to the exported image sequence select the first image and you should be able to see it in this menu here change depth to front and also change source to image sequence here in frames you have to put in the number of frames the image sequence has for that you can also go back into hit film and change the time display to frames go to the last frame and you'll see you have 60 frames so type in 60 frames and when you play it back you see it moving now you want to use this offset value here to have the video start right in that moment rather recto should be activated so let's go with this frame right here yeah that should fit now we have to roughly line up the footage with the 3d scene for that open the side menu again and first off check what height the camera is in since the camera is in about eye level of the actress you will fight the focal length right here in the camera properties we used 24 millimeter lens so you can just type in the value that you used I will use the view navigation for that again with Q and E you can also go higher or lower I always start up with trying to line up one of the feet and getting and getting the scale right now let's get him into the starting post show the hitboxes and the constraints click on this little filter button right here in the outliner and activate this selection button so now we can choose which objects can be selected and which can't so deactivate selection for the metric and the mesh and the plane so that we can position the rest freely I will activate x-ray to be able to select everything and now with Si I will just paint over all the objects now go back a frame to frame 0 and press I while the objects are selected and select location and rotation now we have a keyframe for all the objects just in case we mess up now we can go back to the first frame and adjust the post if you're planning to animate a lot press this Auto keyframe button this way when you change something it will automatically add a keyframe so now we will move the hips into hip position and you will see we've got a new keyframe here at the transform pivot point select active element and now select all the constraints and hitboxes from the right leg at last you want to select the first constraint of Lac and then with Si and left mouse button D selected again that way you get the gizmo at that position and when we change to rotation mode go back to the camera view we can rotate the whole leg from that position repeat this for every joint you the positioning doesn't have to be perfect in since he's flying away pretty fast but when you're finished deactivate the auto-keyframe button and give it a play I will reset the cursor by pressing shift s and cursor to world origin and now I will select the floor oh I first have to make it selectable again select the floor and scale it up four times so it covers the whole floor now under ad I will add the mesh and cylinder we can use this cylinder as a physics interaction object with the trees scaled up as high as a tree but also make it a bit thinner with the move tool again we can move it to the position of the tree move this object into the main collection again now with the cylinder selected go into objects rigid body and add passive now we're going to adjust the direction he's flying into for that make the hitboxes visible again and select the one that you animate it first of all I will delete the last two keyframes I did before for the animation selected press X and delete keyframes now go to frame two and add with I another location rotation keyframe and now move to one frame before the physics start again so this is frame five fast I will change to the top view so we have a better view of the scene and rather Trias so now I will just move the object to the tree and also moved a little bit upwards with G and C like this so it flies up to and now I will press I and add a keyframe for location and rotation let's see how this looks okay this is way too fast so we will grab the last keyframe for the object and just move it back a little bit this way we basically just reduce the force we can deactivate the overlays again to have a clearer view now I think I'm going to make it a bit faster still yeah that looks kind of funny I like that you can also set the last frame for the animation now you see the footage cuts out at this point anyway so we're gonna set 41 as our last keyframe you can just ctrl-c and ctrl-v from the current frame to the last frame all right let's hide the hitboxes and make everything selectable again so now we are getting into rendering going to the render settings step right here and change the render engine from evey two cycles eva is a pretty good almost real-time render engine but cycles has the better motion blur which is pretty important for shot like this so activate motion blur let's look into the settings yeah half the shutter is fine and under film activate transparent that way we will have an alpha background if you notice your recto breaking when scrolling through the timeline that's because the animation is not baked yet let's fix that right now so go into the scene properties and on the rigidbody world open cage and bake so now it doesn't matter where you scroll the animation is baked in also for the cycads render engine you can either use your CPU or your GPU if your GPU doesn't show up I'll select mine right now but the fuse doesn't show up go into edit preferences system and up here you can activate your GPU by pressing on the code that's using select all the objects that you have just for interaction like the floor and the tree and go into object settings visibility and deactivate great visibility camera don't forget to hold down alt to change the setting for every selected object we will set up the lighting now to be able to see that we have to activate the render preview right here and rule out the texture and everything to render while it's doing that select the camera and then the camera properties change the depth of a background image to back and also make it fully opaque since this is a supersoft lit environment we can just go into the world properties and change the background color here to a brighter color we also don't need to see the skeleton anymore so just hide it but all these looking very contrast right now especially in contrast to the other footage so go into the render settings and the color management in look choose very low contrast so this will give us much more to work with later on now we can go to a frame that we like into the test render I will use this impact frame right here go to render and select render image so yeah this looks good to me I think we're ready to render that you can close this render pop up again and go into the output settings and right here we can select what file format we need PNG with RGB alpha is fine and for the output just select the folder where you want to have that image sequence now here's a little trick to save some render time you can just scrub through the timeline and see where Ali is actually flying through the space so for this shot it's just from here to here press control B while in camera view and draw a box around this area so this way lenda knows that it only needs to render this area of the image now we're ready to render go to render and render animation it will go through every image now and render every image individually that's it for the photo scanning and director of physics simulation tutorial I want to thank cloudy Oliver Leo for being my photo scanning model see you in the next tutorial where I will show you how to composite this render this is part one of two tutorials so be sure to subscribe and ring the bell to be notified when we next upload

Channel: FXhome

Views: 149,997

Keywords: HitFilm, HitFilm Pro, HitFilm Express, filmmaking, VFX, visual effects, how to, rise of skywalker, star wars, rise of the dark side, star wars fan film, fan film, star wars trailer, star wars fan made trailer, star wars hitfilm, star wars 2020, blender soft body simulation, soft body physics, how to create a digital double, how to create a digital double for free, free body double, how to create a simulation, blender simulation tutorial, meshroom tutorial, how to body scan

Id: bX_1q0N-EHE

Length: 38min 25sec (2305 seconds)

Published: Thu Jan 02 2020