SPEAKER 1: Hello and welcome. In this video, I will be sharing best practices for capturing performance footage to be used with MetaHuman Animator for iPhone and stereo coupled devices. For performance footage captured using an iPhone device, indirect lighting is ideal as it provides even lighting on the face, softening shadows. A light pointed at a white wall, a ring light, or a soft box light, are examples of light sources that provide even lighting on the face. Lights can be added to a head mounted iPhone to help keep the face evenly lit when there is movement. For framing, start by enabling the Depth preview in the app settings in order to gauge the correct distance the face should be from the iPhone. Moving too far leads to reduced frame coverage, lowering the resolution. By moving too close, black artifacts indicate the depth camera is failing to register part of the face. The ideal distance is where the face occupies as much of the frame as possible, while adjusting the distance once clipping is seen on the tip of the nose. It is fine if there is a little bit of black on the sides of the nose. For iPhones on a static mount, turn the head slightly to each side to ensure that all parts of the face will remain in the frame if head movement will be included in the performance. With the face framed in the center, position the iPhone at a slightly lower angle with an upper facing tilt to reveal more of the lip seam and upper eyelids. When capturing a performance, avoid arm or hand movement from occluding the view or interacting with the face. Handheld movement for capturing performances is not recommended, as this will affect framing and can impact the quality of the animation. For iPhones on a static mount, it might feel natural to include body movement when emphasizing emotions. However, the body should remain still. Head movement can be included in a performance as long as the face remains in the frame and the facial features can still be tracked. Avoid excessive angles where the facial features can be occluded by one side of the face. When viewing the processed footage in the MetaHuman Performance asset, in this example where part of my facial features are occluded on one side, we can see the quality of the animation is impacted. A front-facing frame showing a neutral head pose can be assigned manually to register the position of the upper neck before processing the performance footage. Otherwise, the system automatically picks a frame with the best reference of the front-facing head frame. For head mounted iPhone devices, head movement will need to come from a body animation. A well-fitted helmet can help prevent the iPhone device from moving around during a performance. Larger models can be fairly heavy, and require relatively long arms to hold them, generating a lot of force. Make sure the helmet is fastened securely, and check regularly for wobble if a single arm is used instead of a double brace. If an audio track is present, tongue animation can be produced, and it is solved solely on the audio track and not the footage. Audio is captured in the performance take using the iPhone's built-in microphone. External microphones can be connected to the iPhone device to help improve the audio quality. A directional microphone connected to the iPhone is ideal as it provides cleaner audio with minimal background noise. If capturing audio from separate devices to be used in the final content or added into a level sequence for playback, timecode can help keep the footage and audio aligned. In the app settings, the timecode is set to System Timer. There is an option to select an NTP Server or use a Tentacle Sync via Bluetooth to connect with a master clock, as in the previous release of the Live Link Face app. Timecode can also be used for full body and facial capture performances, making it easier to align the face and body performance with audio. For performance footage captured using a stereo coupled device, accurate calibration is fundamental to obtain good results. It is recommended to bookend a helmet session by taking calibration footage before and after a performer is wearing the helmet, and to timestamp it accurately. If the camera or helmet moved during a performance enough to invalidate a calibration, make a note of it and consider taking new calibration footage. For more information on Stereo Camera Calibration and tools, refer to the MetaHuman documentation in the Epic Developer Community. Once the helmet is mounted on the performer's head, ensure it is securely fastened to prevent the camera from moving out of position during a performance. Additional padding can help keep the helmet in place. Diffuse lighting can help soften shadows on the face when moving around a volume. The intensity of onboard lights can be adjusted to keep the face evenly lit while avoiding hotspots, which can challenge the trackers. A slight underexposure is preferable in this case. Ensure there is plenty of room at the bottom of the frame for the jaw to open fully so that all facial movements are captured correctly. For framing, the performer's face should be in the center of the image. The center of the image should align with the upper part of the base of the nostrils. The nasal labial and ocular area of the face is always worth making the focal point, especially for cameras that have a shallow depth of field. When capturing a performance, timecode can be used to align body and facial performances with audio. The audio WAV file can be used in the stereo footage conversion process to be included with the performance footage when ingested into Unreal. This allows for tongue animation to be produced when processing the footage in the MetaHuman Performance asset. In the MetaHuman Performance asset, we can inspect the quality of the footage once it has been processed by using the different Viewport modes to compare the performance with the animation on the MetaHuman Identity asset. Using this iPhone performance footage as an example, by enabling Depth Mesh, we can see if any clipping was present when the footage was captured. By unchecking Undistort, the tracking markers can be viewed on the footage to see if they are correctly placed on the facial features. By selecting the Single Pane Wipe mode, the MetaHuman Identity and footage can be compared as an overlay. By selecting the Dual Pane mode, the footage and animation can be compared side by side. If I switch over to the B tab, I can inspect the animation on the MetaHuman Identity asset by itself. The MetaHuman face board is visible on the side panel, and it can also be made visible in the Viewport so that the controls being used can be reviewed. The performance footage and animation can be evaluated outside of the MetaHuman Performance asset by exporting the animation as a level sequence. The animation can also be exported as an animation sequence, and assigned to the target MetaHuman in Sequencer to be viewed with the grooms and skin textures. The quality of the animation can also be evaluated by baking the animation to the MetaHuman Face Board. All of the controls can be made accessible in Sequencer, and the curves for each control can be inspected in the Curve editor and also modified. Now that I have covered the process of capturing and assessing the quality of performance footage, in the next video, I will share best practices for adjusting animations created with MetaHuman Animator using the MetaHuman Face Board.
