AI to 3D Characters: Next Level For Blender, Unreal Engine and more!

today my friends we are trying out prompt to 3D as well as images to 3D using AI have I always looked this good something more like this and for this we're going to be using chat Avatar which will give you a fully rigged 3D character with 52 blend shape which is compatible with the AR face kit created all just from one prompt or an image a photograph or an AI image chat Avatar is still in beta mode it's looking to fully launch in April being beta it's not a perfect product just yet I have spoken to the developers and they want to include hair assets as well as clothes so what you're seeing when you go to the website and produce models is the beta version I'm not just going to show you an overview of this product in this video I'm going to show you how you can use chat Avatar in its current format to create 3D characters in blender with animation to the body and facial M cap for the face using our phone so we're going to just jump right in now here you can drag and drop an image to create from all or you can write a text prompt now I'm going to use Robert DeNiro here and I'm going to click on generate you're then taken to this window where you have the option to chat the chat assistant here on the right hand screen what this will do is ask you more questions about your character and create prompts for you on the left hand side you don't have to use that at all you can simply just type in your prompts over here if you come down to model you can pick what kind of model you're using dream face will give you a realistic 3D character hero star will give you a more stylized character tunify St will give you more Pixar Disney style and dream face version one is the predecessor and then down here you can see different pumpt helpers that you can select if you want once you're happy just click on generate remember this is not going to cost you any credits at this point as you can see it's created our 3D preview model here it's not going to be a perfect science it does look a little bit like Robert dairo and it's a good start as I said before this is not a perfect science just yet but remember it's giving us a base mesh with textures uvw unwrapped blend shapes as well as a rig I'm going to now download this character and to do this just go up to this folder up here and then click on 4K texture that's what I'm going with and then default body because I'm going to be bringing into blender not Unreal Engine I will show you that in another video and then I'm going to go rigged body and this is going to cost us one credit uh per item and I'm going to select mail eyes and teeth expression blung back head textures at this point just be aware the credit amount will be deducted from your account when you hit pay so that's going to take about 10 minutes to download if you come off the screen here and click on mine this is where all your Generations are created as you can see I've got quite a few once your character is ready to download just click on this purple folder which will appear when it's ready and then download that to your downloads folder and your computer keep it zipped now while we're waiting for that we can just come up here to the plugin options and you can see from here all the plugins that they have developed for chat Avatar to import into whatever tool you're using and you can see here in the grade out ones these are currently in development as we're using blender I'm just going to click on blender and that will take us to their Gum Road where you can just download the latest version of the chat Avatar tool for free so click that and then click on download and that will download directly to your downloads folder now keep that zipped up in blender we're just going to install that chat AER tool so delete everything in the scene and then go to edit down to preferences and then from here click on add-ons from the left hand panel and then click on install just navigate to your downloads folder and you should be able to see the chat Avatar import tool click on it and click on install add-on in the search bar here just type in chat and you should be able to see your chat Avatar import tool you need to activate it by checking this check box here go to this hamburger and click save preferences now click on close if you go to file import and then you can see the chat Avatar package click on that and then from this window click on import now import your character it's going to be a long number dot it and from here I'm going to select 4K texture because that's what we packed at the beginning I'm going to select default topology rigged body eyes and teeth blend shape and back head texture and click on confirm so it will give you a warning message asking you if you want to change your scene to Cycles because the shaders for the characters texture were created for Cycles you can automatically switch your renderer over to Cycles by just clicking yes so now just close this import window and we can preview our character this is pretty sweet now looking at this the topology is quite heavy you'll see that the eyes look really freaky so we're just going to fix that now that takes a couple of seconds with your character selected just come to material properties down here and then in the materials we're looking for a material called occlusion click on that and then expand viewport display and then expand settings and from the blend mode just switch it to Alpha hash and there it's fixed and then you have your your shape Keys down here now this is what we're going to be using to animate our 3D character and as you can see they're all working nicely I'm actually now going to bring in a female character so you can just see the differences in characters so you can see she's actually all uvw unwrapped her head as well as her body so let's take a look at this rig now this looks like a kind of mix and mo rig I'm actually going to replace it with a better rig I'm going to be using the acur rig by real illusion this is absolutely free and it's really quick to rig your characters so we're going to get some animation on this chick Pronto before we export this character just ensure that you go to the modifier tabs here and where subdivision is just press the cross here to remove that modifier all we need to do now is export this characters mesh as fbx so select the characters mesh only go to file export and then fbx we just want to make sure selected objects is only checked as well as mesh and just uncheck bake animation and just click on export okay so just head over to actor core the link is in the description and download for free on your PC once installed load up acurate and on this screen click on choose file select the fbx file that we just exported from blender and click on open from this window just go down to rig body from this screen we just need to designate the point so check on Symmetry and we're just going to move the shoulder to the shoulder point now if you're a little bit confused you can use the screen on the top right hand corner and it gives you an example of where that joint should be on your character so I'm now going to click on the rig right hand button I've got five fingers so I'm going to select five fingers and then repeat the process on this screen as well just move the joints to where it's indicated on the right hand example once you finished that just go to rig left hand and repeat so it's going to give you a preview of your character animating and you can actually change the animation as well just to check that your joints and your animation is working now I would inspect closely your hands as well just to make sure that the joints are all working in those if you find there's any problems or if you want to offset the shoulders you can do that with this panel here which is really handy if there's any errors just go back and just move those joints it's a very easy process once you're happy with the rig click on export and then export fbx from Target application select blender we're not going to change any of the material settings and we're just going click on export and just give your saved file a name and click on Save now we're going to go back into blender and import that rig so we need to go to file import fbx and import that characters rig just make sure you uncheck absolutely everything from this panel you don't want anything checked in any of these boxes and then click on import fbx okay so just select the bone in acuri and just move her across on the left hand side you've got your acuri character and in the middle you have your original character here now we want to transfer the blend shapes from the original to the Acura character so we can do moap to do this we're going to use the mesh transfer tool go to the GitHub page in my description and thank you so much to Hansel B if we go to releases on the GitHub page and go to the most recent release and go to Source code. zip and just download that now don't unzip that we're going to go straight back into blender and we're going to go to edit preference add-ons install and then click on the mesh data transfer folder and then click on install now if you don't see it straight away all you need to do is just type in mesh and it should come up there so you got the mesh transfer tool and just check that to enable that and click file save preferences so what you need to do is click the Acura mesh as well as shift select on the original mesh and press crl a and click on all transforms so that's going to just clear any transforms on the those characters sometimes their location or rotation might be slightly different from each other as I have spent days on this I know absolutely every error that you're going to get so if you do get stuck at this stage you can go to my patreon which is PR Muse and I'll be sure to help you out if you select the mesh of the new character that we just imported in and then go to data properties which is this triangle here you can see the shake Keys we've only got two we want to clear those by this downward Arrow and then clicking delete all shape Keys cool so when we go down this panel here you should have a mesh data transfer dialogue you want to ensure the shape Keys button is selected and with the source picker here just select the original mesh from here and then select vertex ID now once those are selected click on transfer mesh data and now you will see in the new mesh that we now have all those shape Keys transferred FD over to our new character which is great which means we can use those shap keys for a facial M cap now we need to transfer this material from the original to our Acura character and that's pretty simple so don't worry about that so we're just going to expand this window here and click on the new mesh which was from acur and go to your materials tab Okie doie and then we're going to shift select the original mesh here we're going to go within our material tab going to go to this downward arrow and click on copy materials to select it now you can see we have a slight issue with the eyelashes here left click select this character so only this character is selected hit tab on the keyboard to go into edit mode and I'm going to go into face mode here we're just going to select one face of the eyelash and go to select and go select similar and materials and that's going to then select all the eyelashes and then back in your materials properties down here we're going to go to eyelashes anyone will do and I'm going to click assign we're going to repeat that process for clusion so you're going to click on one face go to select select Simer then material and then click on occlusion from your material properties and click on assign you can see she has like frog eyes and to fix that you just come down to viewport display and under settings under blend mode from the dropdown click on Alpha hash we can actually now delete the original rig mesh so if you come up to your outliner right click on the original mesh and click on delete hierarchy so now we're going to jump into doing the hair and I'm going to be doing this using hair cards essentially hair cards are just flatten ples with material on them they actually less taxing or less resource demanding than hair particles or Geon noes however people might prefer the look of Geon noes and hair particles on their character lucky for you I have just done a tutorial on how to create hair using G nodes in blender so you can go and check that out you might have already heard of hair tools it's been around for a while and the developer has continuously throughout the years kept it up to date this product is a lot more substantial than I'm showing you in this video so I'm going to hit the end key and find the hair tool tab I've already installed this so when I go through tutorial I will show you absolutely everything in detail uh the tutorial will be coming do not worry we're going to be going to the library and we're going to just be using the preset hair from the library here I'm just going to use long wavy and click aend and that just brings the hair into the scene just going to grab the hair and just plop it onto our character like so what I can do now once that's in position just go to the modifier and it already has a shrink wrap on so I can just Target that to the scalp do not worry if I'm going really fast because like I said I'm going to be doing a tutorial on this and that's the hair all complete that only took a few moments I was using the preset so nothing nothing too fancy going on here we're now going to be adding facial map via our mobile phone in blender yep you heard that right thanks to the developer Nick Fisher he's kind of modified the Unreal Engine live link face to be able to allow us to use the plug-in within blender that is pretty cool not only can we do map we can also stream directly into blender as well you can download the live link files from his gumroad here you can download it for absolutely free zero don't unzip the file once you have done that if you jump into blender go to edit preferences add-on install and select the blender live link dotzip and then click on install add-on and then just make sure you've got the check next to the box so it's actually installed if not you can use search panel to search so Hit N on the keyboard and you should now see on your tabs you've got the live link tab in there okay with your live link enabled here you should have this dialogue box if you hop onto your iPhone and then go to the App Store and search for live link you should see it there once live link has downloaded simply open it up on your iPhone and select capture mode as live link AR kit and hi there I am so I'm just going to click on the settings button here and you want to go to live link under streaming and then here this is where you want to put your IP address the quickest way to find your IP address is just to hit the Windows key on your PC type in CMD and then open up your command prompt here and all you want to type is IP config all in one word hit return and there you have your IP address okay you're now ready to connect your character just click on your character's mesh and click on the plus icon here leave these as default and click on connect and you should have your character animating now so that is pretty sweet so that will give you really really nice easy way to do elim as you can see pretty lit ah another thing that you can do is load from a video so if I press this record button and do some really terrible acting hi this is a Blender character that was really bad but that has now recorded something called a slate so if you go to this bottom movie clip icon on your phone you can see that slate here has saved and then save that to files and that's going to save it to your iCloud and then back on my PC if I open up my iCloud I can just go to download and download a copy of that to my PC you can use an alternative method to get it to your PC whichever you prefer once it's downloaded into your downloads folder simply right click and extract that before we load in that slate what we want to do is click the mesh of the character come down to data properties and just clear all the shape and what this will do is reset all your facial shapes or BL blend shapes to zero once you've done that just go to load from CSV and just locate that file in your downloads folder and so it will be your slate. CSV and load that in and as you can see now with the characters mesh selected you can see on the playback timeline here you can see those key frames have been added you can actually go in and check that your key frame is auto keying and you can go in and adjust those key frames as well we are now at the last stage of creating our character we're just going to be adding some free map from the real illusion ACAC cor Library so we're going to head over to that website now the actore library is a fantastic place to pick up some mocha you can get it for as little as $2 so if you just navigate to the left hand side you should see a button that says free if you give that a click you'll see a library of free map these are pretty decent Maps I'm just going to click on the change pose character here and just click on download just click on the female robot and then click on next and then from the max texture size just keep that all as default on a Target application we're going to switch to blender move in place means your character will animate in place okay we're now going to click on download and that's going to download to our downloads folder so you just need to unzip that in your downloads folder and then head back into blender okay we're going to import that animation so go to file import fbx and go to your actore blender file motions and your fbx will be in there just click on import fbx and you'll see the rig will import into the middle of your scene so also if you bring up the player back timeline down here you can scrub through the animation and just see it there now what we want to do is switch to the nonlinear animation timeline which is here and you'll see your Armature which is the rig that we brought in and you can just press this button which is the push down button which means we can use it as an animation strip so if we come to this Armature here which which is your characters just click on where it says no action and on the right hand side from this drop down window just switch from the drop- down window to your animation which mine will stand and talk now we can just select the first Armature that we bought in with the animation and hit delete okay so if we just head back to the timeline you can see our key frames are there in the timeline and if I just scrub through you can see the animation from the mo cap coming through as well as the facial Mo cap that we recorded on phone it all works very nicely together I hope you have enjoyed this video and hopefully giv you an overview and a demonstration of how you can integrate this product into your workflow I just want to say a massive shout out to my patrons I see you for my patrons I have a couple of free credits to give you guys this is limited and this is first come first serve so head over to patreon to hopefully grab yourself some credits to use with chat Avatar I will pin any retrospective errors on the comment section and that will do it bye-bye

Channel: Prompt Muse

Views: 45,273

Keywords: aicharacter, #aiactors, #aito3d, aito3dcharacters, #charactercreator, 3danaimation

Id: 2s0TNVVBmqQ

Length: 18min 59sec (1139 seconds)

Published: Wed Feb 28 2024