Tutorial: Lipsync animations in Blender using Audio2face. Example on CC3 character

welcome to a new tutorial today we are going to see how to use audio two face from nvidia to make our character from character creator to talk first let's load the fbx exported from character creator duplicate the mesh in edit mode leave only the head select the head invert the selection and delete the rest leave the head selected and go to mocap import important export the model choose a place to save it now open audio2face open the exported file attention you will see two files select the one with fixed in the name choose open go to tab character in tools click on mail template move the head that showed up next to the head imported the best is to put the gray one on one side and the green one on the other side to see better add a light source on the drop down menu now select mark select your imported mesh and select the mark open mouth click on add points select the points around the mouth the tip of the nose the corner of the eyes the ear and forehead click on done if you like you can unselect the correspondent visibility option to hide the created points click on begin mesh fitting click on begin post wrap click on audio to face tab select the mark mesh click a 2f pipeline click yes to attach to the mark mesh and pressing play you can see the animation on the mark mesh and the on the imported mesh select the mesh to export select the one with result on the name choose a folder select the fps and click export as maya cash don't worry mocap import can deal with this created file go back in blender and with the head selected click on import as nla strip grab the maya cache exported from audio to face press play and you will see that it's working fine now we have to transfer the animation to the full body if your mesh has beard or brow on a separated mesh you will have to join to the body mesh do it selecting the beard brow and the body mesh as the last one press control plus j this will make all the selected mesh belong to the body mesh put the head used on audio 2 face back on its origin it should be on top of the full body mesh select the full body mesh go to the mesh data transfer addon select the origin the head click on world and then transfer shape keys now you can hide the head object open a window with the dope sheet go to shape key editor and select the shape key that is available and the full body mesh now will have the head animation the problem now is that the jaw is not moving but we will fix this using bones hide the mesh it will be easier to see the bones select the armature go to edit mode select the jawbone and duplicate it leave it at the same place increase the size of the bone to be easier to select it weight paint now we are going to create the weight paint to drive the movement of the lower part of the mouth select the armature and shift select the teeth change to weight paint mode select the jawbone that we have duplicated you can see that there is no vertex group with the jawbone name so to create one we click anywhere on the lower part of the mouth go to edit mode face mode and select at least one of the faces of the lower part of the mouth the teeth and the gum press control plus l to select the full mesh of the selected faces on the vertex group created be sure that the weight is set to 1 and click assign now the bone will drive the movement for the lower teeth and gum constraint let's add a rotation constraint to avoid the jaw to do weird movements set it to local space and select limit x y and z on limit z we will limit it to a custom value click the eye on the constraint to mute it and on the jawbone move it forward to beyond the max rotation you want it to have enable the rotation constraint clicking again on the eye icon and change the minimum limit until the teeth is in the place you want for the mouth close disable the constraint again and rotate the bone to the max position you want the mouth open enable it again and tweak the max value until it reaches the desired place test it to see if it's the way you want driving the movement on the jaw based on the lip sync go back to object mode and unhide the meshes go to the beginning of the animation select the full body mesh and go to edit mode change to vertex mode select a point in the middle of the lower lip or chin make the 3d cursor to move to that point using shift plus s and select the option cursor to selected go back to object mode and add an empty you can resize the empty to be easier to see it with that empty selected shift select the full mesh and change to edit mode we already have a selected vertex so we should press control plus p to parent the empty to that vertex now go to object mode the empty will follow the movement of that vertex now let's hide the body mesh to work on the bone select the armature and go to edit mode select the tip of the jawbone we've duplicated and make the 3d cursor to that point using shift plus s and select the option cursor to selected go back to object mode and create another empty select the created empty and shift select the first created empty and press control plus p to parent the new empty to the older one now when the first empty moves the second will also move select the armature again and go to pose mode select the duplicated jawbone and add a damp track constraint select the target to be the last created empty and we will have our jaw moving when the mouth moves that's it we hope you liked it and if you enjoy our videos and would like to support us please be a patreon the link is in the description below this will help us to keep creating tutorials tools and animations to help the blender community thanks a lot see you in the next time you

Channel: CEB Studios

Views: 8,056

Keywords:

Id: 6XjCi5LdNco

Length: 9min 0sec (540 seconds)

Published: Sun Jul 25 2021