Adjusting animation using the MetaHuman face board

hello and welcome in this video I will be sharing best practices for adjusting animations created with metahuman animator using the metahuman faceboard in order to identify what areas might need adjusting a good place to start is by comparing the performance footage with the animation on the metahuman identity asset in the metahuman performance asset in this example I am paying attention to the lower lips intersecting with the bottom teeth spotting the in the metahuman performance asset can be helpful in having a starting point in what might need to be adjusted when viewing the same animation on different metahumans there might be some variation for certain poses in this example with these two metahumans there is some variation in the lip contact another reason for wanting to adjust an animation is perhaps you want to transfer head movement from a facial animation captured using an iPhone on a static Mount to an existing body animation I will use these examples to go over a few techniques that can be used to make these adjustments using the metahuman faceboard and body control rig if you are new to using the metahuman faceboard in an empty level sequence add the metahuman into sequencer the body and face control rig will appear in sequencer by default for now I will not be using the body control rig so I will delete it once the control rig is added to sequencer a few new panels appear one of those is the animation panel in the upper left corner it has a variety of tools and settings for example by checking on hide control shapes I can hide the control rig or over here I can change the Gizmo scale another panel is the animation outliner in the upper right corner this lists all of the controls and has a search bar the third panel is the animation details which displays the values for each control this can also be used to create keys for the purpose of this video I'm going to create a second viewport by going to viewport options then layouts and selecting a two- pane layout I'm going to dock the animation panel in the upper right corner in the main viewport I have a camera set up with the entire face control rig visible and we'll use this to select and adjust controls in the second viewport I will change the cameras I have set up to show the part of the face I am modifying in closer detail feel free to leave the viewport at default or adjust it according to your preference before I begin working with the face control rig I'm going to make sure that the translation Gizmo is set to local space and that snapping is disabled to select a control click on any control and that control will be highlighted it will show up in the animation Det details panel and will also be highlighted in sequencer to select more than one control hold down shift and select another control to select multiple controls at the same time hold down control alt and left Mouse drag over those to create a key you can press the letter s or you can create a key in the animation details tab or in sequencer I'm going to move a few frames in sequencer and using the slider I will move the controls that I have selected I'm going to press the letter s to create a key for the right and left inner I squint then move a few frames over and reset them I can either use the slider to reset these or change their values to zero in the animation details panel or in sequencer I can use the sequencer filter located here and check on selected control R controls to display only the controls I have selected now I am going to animate the cheek raise controls by selecting them and pressing the letter s to set a neutral key for them I'm going to create a marker by pressing the letter M to label what controls I am key framing here by WR clicking on the marker I can see a few options I can label this marker by typing in the control name I can put comments in here and assign it a color I'm going to finish animating the cheek raise controls by selecting them moving the slider and creating keys by pressing the letter s moving a few frames over and resetting them I'm going to reset the sequencer filter by unchecking selected control R controls if I play the sequence and notice that the face board disappears I can go to the playback option and uncheck game view clean playback mode now when I play the sequence again the control rig will remain visible in a new level sequence I have key framed and labeled all of the controls that metahuman animator generates animation 4 as a best practice start by getting comfortable using the metahuman faceboard by going through the process of selecting and adjusting each individual control this will give you a better idea of what area of the face each control adjusts and influences when using the metahuman faceboard to make adjustments to a facial animation for certain areas of the face it will be easier to locate what controls you may want to adjust by being familiar with it if you are already familiar with the metahuman faceboard you will notice that some new controls have been added such as the inner lip thickness controls and some new tongue control controls you can find all of the controls being driven by metahuman animator in the metahuman documentation in the Epic developer Community to use the metahuman faceboard to adjust a facial animation one approach is to use the curve editor I'm going to begin with a metahuman already added into a level sequence since I am not creating an animation but adding an existing one I'm going to delete the body and face control rig with the face component selected I'm going to add an animation by going to this plus sign to add an animation track I am going to go to animation at the top and locate the facial animation this animation was captured with a stereo couple device and was exported from the metahuman performance asset at 60 frames per second I'm going to change the sequencer frame rate to 60 frames as well if I hover over the animation track I can see how many frames this animation has I can also rightclick on it and go to properties and view the section range end which is 173 frames I'm going to enter this number to change the playback range and use the end bracket to change the length of the sequence and now I'm going to go back to the beginning of the sequence to access the metahuman face control rig I'm going to select the face component and right click on it I'm going to select bake to control rig and select the face control rig I'm going to leave everything at default and select create this might take a few moments and once this is complete the animation has been baked to the control rig when I play the sequence we can see on the faceboard what controls are being activated for certain poses if I Collapse this control rig track and sequencer we can access every individual ual control to inspect control curves I'm going to select the right eye control and the left eye control in sequencer I'm going to go to the track filter and check on selected control R controls so that only these are displayed to view the curves for these selected controls I will open up the curve editor which is located here if I Collapse the right and the left eye control we can see they have a curve for the X X AIS and Y AIS I want to view the xais curves for the eyes I can pin the xais for both and view the Curves in this area I can use the middle Mouse wheel to zoom out or in by holding down the right Mouse button I can pan around this area if I hold down the left Mouse button and drag the cursor around an area I can select the keys in that area to view these selected Keys closer I can go over here to the left and select the zoom to fit option or press the letter F this will bring all of the selected Keys into closer view over here is the access movement tool this first option allows you to move a key on horizontal and vertical increments adjusting its time and value the X only option allows you to move a key only horizontally adjusting the time and the Y only allows you to move a key along a vertical axis adjusting the value I'm going to select the Y AIS snapping only with this enabled I'm going to select these Keys then using the middle Mouse wheel I will zoom out to make some room in order to move them by moving the keys of the x-axis curve up they move the eyes to the left and by moving them down they move the eyes to the right I'm going to unpin the curves for the xais and pin pin the curves for the Y AIS I'm going to select these keys and bring them closer to view and I'm going to select these two keys I'm going to move back now to make some room to move them if I move these Keys down they move the eyes down and if I move them up they move the eyes up not all controls have an X and Y AIS if I select the jaw forward and back back control the keys in this curve have values that range from 0 to positive or - 1 with the Y AIS snapping still enabled I'm going to select the key on this Frame by moving this key to a positive value it moves the jaw back and if I move it to a negative value it moves the jaw forward the curve editor has a variety of tools that are helpful when working with facial animations I'm going to select the mouse M Corner control and in sequencer I'm going to select the Y AIS curve for this control I'm going to drag my cursor over these keys and bring them closer to view when viewing the keys in this selection if I wish to reduce some of this noise and make the curve look smoother I can use a filter I can locate the filter options over here or by right clicking in this area and selecting filter I can use use the furri a transform filter which Smooths key frames with a lot of Divergence or noise between them I will select Butterworth which will remove high frequency noise along a curve when clicking apply if I press apply a few times we can see the selection of the curve begins to look smoother if I continue to press apply we notice this affects the boundary keys of this selection these boundary keys can be readjusted manually with the the Yon AIS snapping enabled another useful tool in the curve editor is the transform tool I'm going to select the left blink control and view the curve for this control I'm going to select this section of keys over here and enable the transform tool which is located over here I'm going to bring this area closer to view the transform tool allows you to adjust the key frame times and values I can use the various points on on the Lati by dragging the corners edges and Central areas to scale the selection relative to the center point if I move the center point to the upper boundary when I select the upper boundary and drag my mouse cursor down it moves the selected Keys up towards the center point increasing their values if I move the center point to the lower boundary and bring the upper boundary down it will move the keys down toward the center point decreasing their values now I'm going to go back to sequencer I'm going to locate a part of the animation where the lower lips are intersecting with the bottom teeth I'm going to use the transform tool to modify this area when scrubbing through this section I can see which controls are activated by looking at the face board since I have become familiar with the faceboard I know that I want to look at the push pull down controls and a lower lip depress I'm going to start by selecting the push pull down controls and view the curves for these controls in the curve editor when viewing the curves for these selected controls I can see that the lips are intersecting with the bottom teeth when the push pull down controls are close to a value of -1 I want to create some markers to indicate the start of the area I want to adjust for these controls I'm going to press the letter M to place a marker here this Frame is where where I want to indicate the start of the adjustment I'm going to locate the frame where I want the area of adjustment to end which is going to be at this Frame so I will press the letter M again I'm going to go to the middle of the section now I'm going to enable the selection tool with the Y AIS snapping still enabled I'm going to select this section and bring it closer to view I'm going to bring the selection a bit closer and by dragging my mouse cursor while holding the left Mouse button down I'm going to select the keys on this Frame if I move these Keys towards a positive value I can see in the second viewport it is adjusting the intersection the lips are making with the teeth I'm going to leave these two keys here as this is the value I want the rest to be at I'm going to select the keys in this area where I have created markers I can hold down the control button to deselect these two reference Keys again as they were reselected I'm going to enable the transform tool by selecting it I'm going to move the center point to the upper boundary and move all of these Keys up toward the center point by having the upper boundary selected and dragging my mouse cursor down I'm going to move these two keys I have left by changing to the selection tool and moving them back in line with the rest I want to check the keys on the edges of the area I adjusted to make sure that they are all lined up otherwise I can move those individually with the selection tool enabled by making this adjustment the lower lips are no longer intersecting with the bottom teeth now I'm going to modify the curves for the lower lip depress controls I'm going to select them and view their Curves in the curve editor when looking at this section I can see that the markers I created for the push pull down controls are in a fairly similar location of where I would set them for these controls as well I can see that these keys are close to a value of one I'm going to select the keys in this section and bring them closer to view I'm going to enable the transform tool and instead of leaving the center point where the upper boundary is I'm going to move it to the bottom boundary I'm going to bring the upper boundary down to decrease the values of these keys I can see that the keys on the corners of the selection are all lined up and do not need to be adjusted I'm going to scrub through this section and make sure I am happy with this adjustment now that I finished making adjustments to these curves I can go back to sequencer I'm going to reset the sequencer filter by unchecking selected control Rec controls I'm going to go to the start of the sequence what I want to do is create a new animation sequence with all of the changes that I've made to do this I'm going to rightclick on the face component and select bake to animation sequence I'm going to give this new animation a name and a folder I want it directed to and press okay and then I'm going to leave everything at default and select export to animation by baking this to an animation sequence I now have a new animation with all of the changes that I made if I compare the frame where I made the adjustment with the same frame from the animation that was not modified I can see that my change has been added to this new animation there may be cases where I want to adjust an animation without modifying the existing Curves in this case an additive layer can be used for more information on the animation curve editor refer to the Unreal Engine 5.2 documentation when using the metahuman face board to adjust a facial animation one of approach that does not involve modifying the existing curves is to use an additive layer in a new level sequence with a metahuman added in here I have already deleted the body and face control rig I'm going to add a Face animation by going to the plus sign on the face component to add an animation track I going to go to animation at the top and locate the facial animation this animation was captured using an iPhone device and was exported from the metahuman performance asset at 30 frames per second so I'm going to leave the sequencer frame rate at 30 frames as well by hovering my cursor over the animation I can see that it is 601 frames so I will enter that number and use the end bracket to change the playback range for this sequence I can now go to the start of the sequence I'm going to bake the facial animation to the control rig by right clicking on the face component and selecting bake to control rig and then selecting the face control rig I'm going to leave everything at default and select create with the animation baked to the control rig instead of collapsing the control rig track I'm going to add a section by selecting the plus sign and selecting additive once I do this an additive layer track has been added if I Collapse this we can see that this layer contains all of the controls I can use this layer to create animations on top of the existing one for example I am going to select the right and left inner I squint I want to create an offset for these controls with these controls selected I'm going to move the slider to increase their value and set a key for these controls this control will remain at this value throughout the entire animation now I'm going to go back to the start of sequencer so that I can delete these keys and reset them I'm going to use the sequencer filter and select selected control R controls to view only these I'm going to drag my mouse cursor over these controls to select them and then delete them by pressing delete I can collapse these controls and see that they are still at a value of one I'm going to enter zero to reset these I can reset the sequence or filter Now by unchecking selected control R controls this additive layer can be used to adjust certain controls in specific areas of the sequence before I begin making any adjustments I'm going to create a neutral key for the entire faceboard additive track at the beginning and then create a neutral key at the end this ensures that all of the controls in the additive layer are set to a value of zero I'm going to go back to the start of the sequence and if I Collapse this we can see that all of the controls have a neutral key at the start and end of the animation if I play the sequence and press pause at this Frame I can see that I would like to adjust this section where the lips are not quite making contact for this pose I'm going to create a marker here by pressing the letter M to indicate this is the frame where I want to make an adjustment by looking at the face board I can see which controls are being used for this pose I can see that the lips together controls are activated these controls can be used to bring the lips closer together when the Jaws open I could choose to modify these I'm going to move one frame back to reset this rather than adjusting these controls I can use a control that mimics the lips together without altering these controls controls in this case the lips toward teeth down controls would be an option if I adjust these controls I can see they are a good option to use for this adjustment I will press contrl Z to reset these I want to go back to sequencer and use the sequencer filter and check on selected controller controls so only these are displayed I'm going to go back a few frames and create a marker here to indicate the beginning of the area I'm going to adjust this is right before the lips are making contact so I'm going to press the letter M to create a marker here I'm going to move a few frames forward and create a marker here to indicate the end of the area I want to adjust which is right after the lips separate before I make my adjustment I will create neutral keys for these controls so that the change I make for these controls only occurs between the section I have placed the markers at otherwise this adjustment will remain throughout the entire animation Now by selecting my start marker I want to create neutral keys at the start of the adjustment I can expand this area by holding down control and using the middle Mouse wheel to zoom in now I can go to the frame where I'm going to make my adjustment I can use the slider to move these controls or in sequencer I can collapse these two controls and view their values in the section where I want to make an adjustment I can drag my cursor over this number increasing the values of the right lips toward teeth down control by doing so a key is automatically created I'm going to do the same for the left lips toward teeth down control until I see the lips making contact now if I go through this section frame by frame I can see that the lips are making contact and separ separating when they should be I'm finished making adjustments for this control so I'm going to reset the sequencer filter and now go to the start of the sequence and play this I happy with this minor adjustment I have made and I'm finished making changes to this animation I can go back to the start of the sequence and bake this to an animation sequence in order to create a new animation with these changes to do this I going to rightclick on the face component and select bake animation sequence I'm going to give it a name a folder I want it directed to and select export to animation now I have a new animation with the changes that I made and if I compare the frame where I made the adjustment with the same frame from the animation that was not modified I can see that my change has been added to this new animation now that we are familiar with using the curve editor and an additive layer in the next section I will be sharing different ways ways you can approach working with a facial animation that has head movement in this section we will be combining all of the tools and techniques that have been covered to work with a facial animation that has head movement I'm going to be using an animation that was created from a performance I captured with an iPhone device on a static mount it was exported from the metahuman performance asset with head movement at 30 frames per second with a metahuman added to a new level sequence I'm going to assign the facial animation with head movement by going to the plus sign on the face component to add an animation track and going to animation and locating it I'm going to leave the sequencer frame rate at 30 as this is the same frame rate of the animation if I hover my mouse cursor over the animation I can see that the length of this animation is 830 frames I'm going to enter that number here in sequencer and use the end bracket to change the sequencer playback range I'm going to go back to the start of the sequence and press play once I do this we can see that the head component separates from the body component this is due to the facial animation having transform and rotation values on the head component instead of the body component if I'm not using a body animation and wish to keep the facial animation with head movement I can approach this a few different ways one approach is to assign the facial animation with head movement to the body component to do this I'm going to select the body component to add an animation track and going to animation and locating it in order to assign the facial animation to the body component I will need to go to this Cog wheel and check on allow incompatible skeletons with this enabled I can now locate the facial animation and assign it to the body if I play this sequence now the face component is no longer separating from the body component and everything is working fine a different way to approach this is to retarget the facial animation with head movement to the metahuman body skeleton to do this I will want to locate the facial animation sequence to locate it I can right click on the animation and sequencer and go to properties I can now look locate the animation sequence and browse to its location by selecting the folder icon with the animation sequence selected I'm going to duplicate it by right clicking on it and selecting duplicate I'm going to rename it and we'll add retarget to body at the end of this name for this example to retarget it to the body I'm going to rightclick on it and select replace skeleton I'm going to to select the metahuman base skeleton and press okay and save everything if I open this up we can see that I now have the facial animation with head movement working on the body component if I go to the skeleton hierarchy and select the neck one bone the neck two bone and the head bone we can see that these bones have rotational values now in sequencer I'm going to delete this facial animation and I can search for the body animation and assign it here now if I play the sequence we can see that this works as well now that we have covered how to work with a facial animation that has head movement in the next section I go over how we can transfer this head movement onto an existing body animation in this section I will be sharing one approach that can be used to transfer head movement from a facial animation onto an ex existing body animation using the metahuman body control rig and an additive layer I have two metahumans already added into a level sequence this first meta human is from the previous example where I added a Face animation with head movement to the face component and then duplicated that animation and assigned it to the metahuman base skeleton and then assigned it to the body component this second metahuman has the same facial animation but has been exported from the metahuman performance asset without head movement and it also has an existing Idol standing body animation assigned to the body component as we can see this animation already has head movement in order to transfer the head rotation from the metahuman with the facial animation that has head movement to this metahuman with an existing body animation I'm going to bake both of the body animations to the metahuman body control rig I will begin with the first metahuman I'm going to rightclick on the body component go to bake to control rig and select the metahuman control rig we can see that the face control rig also shows up because we have enabled use incompatible skeletons I'm going to make sure I select the control rig labeled metahuman control rig now that this has been baked to the control rig I'm going to bake the other metahuman body anation to the body control rig we now have both of the metahuman body animations baked to the control rig for the metahuman with the existing body animation I am going to create an additive layer by selecting the body component and going to the plus sign labeled section and selecting additive I now have an additive layer for this body control rig to transfer the head rotation from this metahuman to the one with the existing body animation I'm going to turn on the sequencer filter and check on selected control Rec controls in the viewport I'm going to select the neck one control holding down shift I'm going to select the neck two control and holding down shift again I'm going to select the head control in sequencer we can see the controls are visible I'm going to collapse the head control the neck one control and the neck 2 control if I play this sequence we can see that there are values for the roll pitch and yaw what I'm going to do is copy all of these rotation values in order to paste them onto the additive layer I have created for the metahuman with the existing body animation to do this I'm going to go to the start of the sequence I'm going to take my curve cursor and hover over the area of the Roll pitch and yaw tracks of the head control I want to make sure I do not select the scale tracks I'm going to drag my mouse cursor over these to the end of the sequencer track now I'm going to scroll down to the neck one control I'm going to hold down shift and drag my mouse cursor over the roll pitch and yaw tracks of this control as well now I'm am going to scroll down to the neck two control and hold down shift again and drag my mouse cursor over the roll pitch and yaw of this control as well to copy these I will press contrl and the letter c I'm going to turn off the sequen or filter for a moment and now I'm going to select the other metahuman in the viewport with the existing body animation and in sequencer I'm going to select the additive layer track making sure I am at the start of the sequence and with the additive layer track selected I'm going to press contrl and the letter V to paste these Keys now if I enable the sequencer filter again I'm going to select the neck one control holding down shift I'm going to select the neck two control and holding down shift again I'm going to select the head control control we can see that the keys have been transferred for these controls to this metahuman now when I play the sequence I can see that the head rotation from the one metahuman has been transferred over to the metahuman with the existing body animation if I wish to adjust these controls perhaps I want to offset the roll of the head and tilt it up a bit I'm going to select the roll of the head control and then open up the curve Editor to make an offset to this curve I'm going to go to the access movement tool and enable the Y AIS only snapping I can select all of these keys and can move these to a lower value to tilt the head up I can now close the curve editor and reset the sequence or filter it is worth mentioning this is one way you can transfer head movement from a facial animation onto an existing body animation by transferring the rotation tracks to an additive layer I can disable the additive layer by right clicking on it and selecting mute and revert back to the original head movement from the body animation I'm going to unmute the additive layer now that I finished making adjustments I can bake this body animation to an animation sequence and create a new animation with all of the changes that I've made I'm going to right click on the body component select bake to animation sequence give it a name direct it to a folder and select okay I'm going to leave everything at default and select export to animation now if I open this up I can see I have a new body animation with the head movement I transferred from the facial animation I hope you found the information in these videos helpful and I look forward to seeing what you create with metahuman animator thank you for watching

Channel: Unreal Engine

Views: 30,004

Keywords: #MetaHuman #MetaHumanbestpractices, #digitalhumananimation, #realtimeanimation, #NPC, #videogamecharacters, #videogamecinematictools, #UE5, #UnrealEngineanimation

Id: Nkb4DEoZ_NY

Length: 35min 40sec (2140 seconds)

Published: Sat Sep 30 2023