How to Create a Metahuman with CUSTOM 3D Scan Store Textures

hey I'm Frank and in this video I'm going to show you how I use scanned assets with metahumans so it's a bit of a process that goes into starting with a scanned asset wrapping it to a custom model and then putting it onto a meta human but the way that we're going to do it is we're going to keep the existing metahumans the way that they are so that we can continue to animate it and use the meta-human assets like the hair and the eyes as well at the end of it so let's get into it [Applause] so in this video we're going to go through starting with a 3D scan store asset a scanned asset and then we're going to wrap that to our own custom model that we're then going to create a texture that we can then apply to a meta human that we've generated from our original mesh it sounds complicated but I promise it's not let's check it out so what we're going to do in this video is we're going to project the 3D scan store scans onto our mesh I think that's probably even the way I said it is a little bit difficult to understand we're not actually projecting the textures we're going to be deforming the 3D scan store mesh to match ours so that we can throw our one in the bin and then this is our new base mesh because it's already fit with the UVS and the vertices points from the scan data so the UVS will just be able to throw textures on and it will match ours perfectly so you'll see that in a sec the way you need to do that is you need to wrap this model to look like this and then we can get rid of this one and use that so the way that you set that up is your custom model needs to be on the bottom and the scan data needs to be on the top just so that you know the wrapping tool knows which one the two project two and they need to be on the same world space so just make sure they're in the same world space here doesn't need to be the same scale obviously the scan data and the model can be different but just as long as they're generally in the same spot so that when you run zrap they are the syncing the screen syncing works well which you'll see in a sec so if you don't have zrap you can just go to zrap online and download the trial for it it's just a quick install plugin I do recommend you buying it though it's a great tools for any type of projecting from scans so we're going to click this okay so now what we've got here is we've got our scanned head in here and this is that benefit that we were talking about before of putting them in the same world space so that week when we move them around they're going to be moving together and if it's not you can just click sync up there so what we want to do here is we want to place points on each model to the landmarks that correspond to each other so that sounds like a whole bunch of nothing so let's just do it and point there is zero so that point is zero I usually like to go through and do a bunch and then just corresponded over here so as long as the numbers what you're doing are lining up on each model you can put on as many as you want in a row all that matters is that these numbers so we go four five six needs to be four five six so I'm just going to go through and do that all through this model um go through and do the landmarks areas so the the eyes the nose the mouth the ears and then some of just the registration points like the chin maybe the sides of the jaw and then maybe up here so I'm just going to go through and do that and speed through this and I'll see you on the other side foreign foreign you're done with setting all the points what you can do is you go down to wrapping and it will recognize those two meshes and all the points that you've done you can just hit start wrapping and what that's going to do is that's going to now deform the 3D scan store mesh to match your model so then we can throw that one in the bin and then week our new base mesh is the 3D scan store so perfect you can already see that's works pretty well got it done okay so what we've got here is we can get rid of our model and go to the scan data now you've got your model with the 3D scan store assets on it and that looks pretty good so because I've done only done it with the low poly in this one you can do it with the high res the zrap plugin might run a little bit slower but if you want to have all the uh you know juicy details that come with the 3D scan store model you can do it with the high res but I'm just for this demonstration I've just used the low so if you did want to divide it back up you could do that as well but if you had the original mesh all those subdivisions would still be there as well as the textures so the point of doing this is now that we have our model with the textures we're going to bring that over into unreal so the meta human ID solver can use this texture information as well as the mesh to create our new meta human so let's just export this what you want to do is just delete hire for now export obj and obj or fbx and then jump over into unreal so I've just jumped over into Unreal Engine 5 and I've just started a new scene all I've done here is through a couple of wreck lights in here for our scene and a HDI backdrop it's none it's not necessary I've just kind of thrown it in if you don't have that available and you want to know how to do it you can just go over to edit plugins just type in hdri and then hdri backdrop will be there you can just click that it's pretty good you can drop it in and it kind of maps to the world for you so that's also not a bad little tip but what we want to do now is we want to import our model and our new 3D scan saw textures so you can just create a folder in your content in here this is just for the video I'm doing it's just scan to metahuman if you want to import into here so our custom model which is the one that I exported I usually find it zbrush and this 3D scan store size is a little bit different if you have trouble importing where it says that there's redundant uh polygons or the degenerate meshes I think is the error that comes up which you just go straight in it's because this import uniform scale is probably set to one so I usually set this to 100 uh and that you that would come in to the world size that I would think would be pretty accurate in here so if you click the import all there's your model boom instead of him in here and then also the we're just moving up into here to where the lights are you can see there perfect and the size is pretty good so what we want to do now is throw our textures uh onto this model which are going to fit perfectly because we've already wrapped it with 3D scan store so I'm going to import those textures face to fuse these are all the textures that come with the models and we can obviously use normal and displacement and all that in the future but for this we're just going to be using the color information as that's what the ID solver works best with and as you can see that just drops in perfect and creates its own material so this obviously isn't the best way to be doing this as um creating material instances and is the best way to be doing it um from a master material but for now we're just going to set this one and left click also makes a parameter here and we'll just set the roughness down we'll just make it 0.5 so she's not as shiny cool you can see that change there Beauty so this is going to be fine for right now so now what we want to do is we want to create fast we want to create a meta human ID for this so the way that we do that is you right click and then go down to metahuman and then metahuman identity scan ID let's just call it that so this is obviously your character name or whatever it is because this is going to be the information that is going to be stored for your character model open and then this is where we go into how to create the identity for your meta-human so before we add our meta-human into here just jump out of that so we've got him there what we want to be doing is setting our textures to our static mesh before we then put it into there so we can open our static mesh here and then we want to change the material to is that material that was generated for us safe cool so now our static mesh has the texture associated with it without us having to do it just insane here so if we go into our scan ID now what we want to be doing is setting in our new mesh so our mesh is called custom model 2. so let's look for that custom model too beautiful in there it now has our texture information as well so what we want to be doing here is setting our camera to set the markers just like we did with the scan of the other NZ brush so we want to set this focal length the same way that we would do it if we were doing a portrait because right now it gets you know feel to crank this up it'll look a bit like a frog if we were to come in you know how these viewports look so we want to crank that down to get the most accurate representation of what our character looks like so I usually like to do you know sub 20. gives you a pretty good cool so what we want to do is what the way that it works is you need to promote a frame that you set here to then put your tracking markers on so let's say there looks pretty good we want to promote this Frame so what that's going to do now is we have an active frame that we have our face set on in a neutral pose we want to then track our active frame it's going to load the trackers it's going to recognize where the face is so what this process is doing is it's going to recognize our mesh and say hey we want to make a meta human from this mesh so when that's done you can then jump into here and move the trackers around to fit a bit better it obviously does the best it can but this is just a 2d plane so this is the frame that we took the frame that we promoted and we're just going to move these tracking points a bit more to where they fit and then we're going to generate our meta human off this which is pretty crazy that it can do it just based off these uh these markers you also want to make sure that the mouth is closed as well um I've found that that is very helpful when doing facial animation as well so we'll just do a couple little tweaks on this just to get the ball rolling and uh get a mesh generated so we're just going to go ahead and hit metahuman identity solve and now it's going to create our mesh for us so what we need to do to make this submittable is we need to set a body template that matches your mesh so what we're going to do is we'll just do this one for now and then click mesh to metahuman so what this is going to do is now take our face create a new mesh and with this you know temporary body that we can use and then we're going to have a meta-human based on this so now what this has done is it has created a mesh a meta-human mesh based off our template there which is great so now we can use all the animations all the preset hair eyes everything that's going to fit perfectly onto this so before we can do that what we need to do is because this is going to be available in bridge now which you would sign in if you haven't you wouldn't be able to get this far if you haven't already created an account it's free to make one so just make a quicksil bridge account and in ue5 Bridge is inside the user interface so if it's not here you can just go up to window go down to quicksil bridge and then it will open up I usually just dock it over here so what it does is it creates a meta-human identity for you so these are some of the past ones that I've already done with this character but this is our ID which has no information in it yet so what it does the way that unreal works with the metahumans is that it has there's an online servery interface that you go to to create your meta human to give it a little bit of character and all that kind of stuff so we're just going to jump over and have a look at that now um by clicking this button so we want to launch this all right so now we are in The Meta human Creator and this is the mesh that it's generated from the identity solve that we've done so let's start with this guy edit selected cool so what we want to be doing go through ART direct this whole thing the way that you want it's not going to really matter as far as the skin and all that kind of stuff but as like the eyes that's going to matter because we're actually going to be using these eyes we're going to be all these assets that we're using whether it's the eyes head okay let's yes I think it was this is the one that I'm doing with the character that I'm working on uh so let's just you know change this a bit let's make it okay so you can make it whatever you want it to look like that's cool so when you're done editing in the metahuman Creator uh the live link to bridge that will be in your unreal it will automatically update your metahumans for you in here so what we can do is then download them and then add them to the scene so this was one that I edited before which is the character that I'm currently working with is already just say yes okay so when that has downloaded an imported into the scene you will have your blueprint class this is your this folder will be generated depending on whatever you've named it then you have your meta human ready to be brought into unreal so what we want to do is now make these textures fit onto this face see I think that the meta humans are great and they have a lot of cool assets with them the only issue is that the they do lack that Fidelity of you know real skin textures even custom textures even if you wanted to have paint or face paint or any of this other stuff it does have this kind of waxy look but we want to keep the great hair and eyebrows and eyelashes and all these things so now that we have our meta human mesh and our identity and all of our information uh so if we wanted to drop an animation rig or anything onto This Blink animations that would all work but the issue is is that it has its own UVS and its own mesh that doesn't correspond to this so what we need to do is we need to project the similar way that we did at the start of the video we need to project our information onto this mesh so the way you go about doing that is using this mesh so that we can stay inside this ecosystem so we never need to rip anything out we never need to start retargeting we can actually just export this mesh assign the textures to the UVS and then all we need to do is just drag that new texture in and apply it so the way that we go about doing that is that in our new metahuman identity folder which is when you download and then add to scene it will create a folder into here so you've got all of the parts of the metahuman plus your identity which is this character so if you just to drag this into the scene that is it um but for this we're going to be working on the face so let's have a look at the mesh that we want and it is this mesh here and a sign we're going to project our scan data onto that so we're going to go into asset actions and then over and Export so export this as a fbx to a location that you want and then what we're going to do is we're going to open that up in Maya so after an internity of waiting uh exporting the face mesh from unreal which was just sport there and inputting into Maya this is what you get so this is all of the levels of detail that come with the medium in plus all the eggs so the beauty of how complicated this all looks is we don't need any of it all we need is the one mesh so get rid of the skeleton foreign type history type cool so what I want to do also in here because we've just got our one mesh now take this separate so the reason I separate that is because it's got a whole bunch of other stuff we don't need so inside here we've got the eyes the eyelashes all that kind of stuff that's already in unreal so all we're going to be doing is continuing to use this but we just need to use the information from this mesh so you can delete this and also just in case you didn't know cool these things up it's got all that mouth detail in there with the teeth and stuff as well is great for when we're going to be using it inside but now what we're looking at is so we can get rid of this too lose our sure I'm just line up for now now we have a meta-human friendly mesh that looks exactly like ours now the last piece of the puzzle is that it doesn't have our textures on it because it has its own UVS which is great because this is the the universal metahuman UVS but we have now have our mesh with the metahuman UVS which is perfect so what we're going to do now is export this and import it into zbrush and we're going to project we're going to create a texture from our scan textures so I'm just going to go ahead and file export selection I export as an obj metahuman base mesh export that then we're going to jump over into zbrush and import it back into our scene so we're back into zbrush in our original scene so what we need to do now is import our meta human friendly mesh so we can project our textures onto it so let's just go into the mesh that we were on before our custom one that we don't need anymore and bring that in so we've got our scan data our friendly scan data that is fit the mesh we had before I've just brought in the meta human now this is going to happen so when I hit F to focus up on the full scene oh sorry on the thing that I've selected it sees the meta human mesh that's great but the problem is that's our guy down there before so what we can do is have our meta human base mesh selected go down to deformation and just hit unify and what that's going to do is do its best job at trying to fit it to the scene scale which is great so this is where it gets a little bit tricky in we want to redo the process that we did before so this is a pretty good view of what we can look at where what meta human has done in an attempt to create our mesh so when we did that identity solve over in in unreal we said hey this is our face try to make a mesh off it and this is the result which is pretty good it's not bad it's pretty close so it's not perfect and we want to make it perfect because the way that we're going to do this is we're going to need this to be Pixel Perfect to this and we're going to project this texture onto this and then we're just going to grab that and throw it back into on the entire unreal meta human mesh so it does seem a little bit back and forthy but for the moment this is the best way that I've found to project scan textures onto on on an unreal mesh so what we want to do is we want to wrap this guy to this so let's go ahead and set that back in World space like we did with the last one and we're going to do another Z wrap on this one so the difference between the first time and now is that our scan data is now our base mesh which is the face that we want to look like so we're going to need to pull that to the bottom and we're going to say hey adjust this meta-human base mesh to look like our scan data so we're just going to swap that order around a little bit and then we're going to go to Z plugging the same way start zero so we're back into here so now what we want to do is we say hey metahuman we want to look like this so we're going to go ahead and place all the points again like we did last time I'm going to speed through that and then we're going to wrap we're going to adjust our meta-human just tweak it slightly to look a little bit more like we what we want so then we projected our information it's going to be Pixel Perfect onto our meta human mesh foreign okay so now we've placed all the points for our mesh and we're going to do a similar wrap that we did before so that now our meta human is going to be matching perfectly to our scan data head and then we're going to project we're going to create a new texture for our meta-human mesh perfect so our meta human base mesh let's get rid of our scan data one don't worry about up there that's fine all when all we're using this for is for a texture projection so we're gonna this whole process is purely to make a texture so we can divide him up cool so what we want to do is go back to our scanned out ahead we're going to hide the metahuman for now what we want to do is take this texture information [Music] and take this texture information and project it via poly paint and then we're going to project that onto our Pixel Perfect mesh over here the texture information is only as good as the density of this so if we were to project let's say the textures off cool if we said the poly paint and say hey we want to grab that texture information and throw it onto our vertex information here there's only so much vertex information in there so if we were to hit texture from poly paint it's it's going to do its best job at trying to project that texture onto here so if we up the resolution of our texture map and then poly paint try that again with a lot more information poly paint from texture now we have a text jar that's fitting it's not using any texture at all it's purely poly paint and it's basically the same thing so now we have a mesh with vertex color from scan data that fits exactly to our mesh and we have a meta-human mesh that fits exactly to that so what we want to do now is we want to project our scan data onto the meta human so all we need to do is Project all hey this has poly paint information would you like to project the poly paint which is exactly the process that we want to be doing always yes perfect so let's get rid of our scan data and now we have a meta-human mesh with scan data on it poly painted projected now that we have our scan data we can turn that off all that matters is our meta-human mesh so our meta human mesh we've imported here we've projected our poly paint scan data onto it which is great um you can see that there's a couple of little Parts here which honestly doesn't matter because we're not even using this mesh but that's because I've just done a projector with one hit it's actually pretty good from uh not going through the stages like you usually would but what we want to do now is we just want to export this as a texture so if you go down to uh texture map it's you're going to see that we've got nothing in there but we want to go new from poly paint so now we have a meta human texture a texture map from the scan data that matches the exact metahuman UV so all we need to do is clone this texture and it will go to here and then we need to export that and now we can just drop that on inside of unreal and it will map perfectly to our meta human so what we want to do now is import our textures that we've just exported from zbrush our polypate projected textures import from wherever you imported them to now I've got them in here so what we want to do now is go into the material of the metahuman and swap out the texture of the diffuse texture for the skin so we're going to go through like we can go through and do the same process for the specular and the normal and all that for the maps we're just going to do with the diffuse here so you can see how it works so if we go into the face of our meta human what we're doing here is clicking your blueprint identity going down to the face and then just like any other object in unreal you've got all your different material attributes here like you would on a multi-sub so we're going to click on the face go into the material that's associated with it and then this is our meta human skin so what we want to do is we want to reassociate the textures with those new ones that we just did our base color so we're just going to go ahead and so it's meta human texture find that in here it is and just like that you can see how much better so we're going to copy this for all the different levels of detail because it's going to be changing as we move away from the camera so we're going to hit save and just like that you have scanned data textures inside your metahuman mesh so now that we've proved the pipeline for working with metahumans and changing the scan textures we're just going to have do a quick little demo on how they can be beneficial in using the rigs and using the way that it is set up so that we haven't actually changed any of the structure of how it's built so we're just going to delete this for a second and we're going to go into create level sequence let's just call it scan test uh segments cool so we've got an empty scene and what we want to do is just drop our meta human into the sequencer cool so now because we've continued to use our existing setup and the model is exactly the same as when it was generated from metahuman all we've done is change our textures so we can still use if you turn off snapping up here if you've got snapping on now our meta human works perfectly with our scanned textures so we'll showcase one last thing um this is obviously applicable to just any meta human but it's cool to test it when you've got new Textures in it especially if you've got new clothes or anything like that is to be able to continue to use the unreal ecosystem and the rigs that they have so the beauty of doing this means that we can download animation packets and new assets from the marketplace and just drop them straight in and they will work perfectly with your meta human because we haven't we haven't messed with any of the structure of the rig and we haven't broken any of the connections we've just purely changed cosmetic information so I'm going to be using a user that has been supplied to me by Gabriella who goes by feeding wolves I just wanted to say a huge shout out to her I'll put her link to her channel in the description I'm sure you've all heard of her it's just made the testing process for me very easy and it's really cool to see how U.S has come in after using a motion capture so thanks again for that so the way that you would do that is you go into the area that you want to animate whether it's the face or the body in our case it's the face so we're going to go to track and then animation I've loaded in a u asset of an animation sequence of a Face animation and it's purely just more of a test of the animation working and as you can see the scan textures are working pretty well so that's the end of this video uh thanks a lot for watching and I really hope that you got something out of this if you have any questions please hit me up in the comment section or on Art station um but uh yeah again thanks for watching and see in the next one

Channel: Frankie E

Views: 8,747

Keywords: metahuman, zbrush, metahuman creator

Id: mIFPvom1bas

Length: 34min 20sec (2060 seconds)

Published: Sun Apr 23 2023