Stylized Lip Sync Animation using iClone AccuLIPS and Face Puppet

so why do you think you should be squad leader uh I think technically I'm supposed to ask the questions hi everyone I wanted to make a quick video on my process for enhancing lip sync inside real illusion iclone and that prism of light that glorious incandescent ephemeral Little Wonder totally gone that made me a little sad it's a pretty incredible program and the acyp feature for providing lip sync that comes with it does a pretty impressive job straight out of the box but it can be improved and like any good piece of software iclone provides a bunch of different methods for doing that some of which come with the software some of which require additional plugins and packages let's take a look so here's my character inside cc4 that I want to export to iclone Philip sync the first thing I want to do is go to the modify tab choose the facial profile editor and make sure that the facial profile is set to cc4 Extended what this will do is give your character a whole bunch more options for expression or blend shapes allowing for better lip sync this may increase the file size of your character slightly so you only really need to do it for foreground actors with dialogue and not any character who's only going to appear in the background of your video you don't need to save a facial profile unless you've made a bunch of custom changes in these sliders that you'd like to save otherwise just click no save your character and then go to export and send to iclone once that's opened in iclone we're going to add some dialogue I always do this before adding any animation so it's much easier to watch the lip sync before the head starts moving around go to animation add script and acul lipse you can either type in dialogue using this button record on the fly if you have a microphone or use some pre-recorded audio I generally use pre-recorded find your clip and hit open the clip opens inside acyp and you can play it down to check it so why do you think you should be squad leader then hit generate text you may find that it doesn't recognize every word so you can go in and correct some text manually [Music] once you're happy click align then hit apply the dialogue will then appear on the timeline along with a sequence of facial blend shapes called visin playing it down you see that it does a reasonable job so why do you think you should be squad leader but there's not much expression to it this is where we can go and and fine tune using a few different methods either separately or in combination there's no real right or wrong way the first thing I generally like to do is to right click on the dialog track and choose lip options then I set smoothness to one and the clip strength to something like 1.2 I find that this gets rid of some of the jerkiness that can sometimes be present and provides a bunch more expressiveness to the facial movements so why do you think you should be squad leader why you also have the talking style editor where you can alter the strength of mouth movements like this ooh you should be you should be be you should be squad leader you should be squad leader you can also zoom into the timeline and alter the strength of the vises individually for example if we want to emphasize that particular U you can find that Vis double click it and turn the slider up to 150 or even 200 you should be you should be if some of the mouth movements are still a bit jerky you can grab and move the VIS slightly further apart to further smooth it out but not too far or you'll lose sink these methods are great for improving the basic lip syn but there's still not a lot of personality that's where the next two methods come in face puppet and face key to look at these we want to switch on our expression track which we can do [Music] here let's look at face puppet first which is the simplest and probably the most fun that brings up this control box where we can choose to adjust different areas of the face in real time you can have Auto blink on or off but I generally prefer to have it off as I like to add my blinks in manually you can also left click during the facial puppeteering to create blinks but I find these look pretty artificial so let's go ahead and add some movement to our face we have a whole bunch of puppet control sets which you can play around with but for now I'm going to go to mouth lips and snil [Music] cuz listening to the clip it just sounded like there should be a lip curl or sneer on the second U so that's what I want to try and add in preview the movement first to get a feel for it adjust the strength slider if you need to then when you're ready move the play bar to the head of the timeline click on record and hit space so why do you think you should be squad leader so why do you think you'll see that it's created a new puppet track on our Express timeline if you're happy with what you got great if not just contr Z to delete it and try again so why do you think you should be squad leader now I can go again with a different control set and maybe add a real sneer to that moment using the nose cheeks and ears control set exactly the same process again do a quick preview to check the motion and hit record make sure the mode is set to blend otherwise each pass will rewrite the animation you did before so why do you think you should be squad leader so why do you think you should be sad leer why do you think keep adding layers with different Expressions until you're happy with what you have sometimes the trick is knowing when to stop the final thing you might add is a little bit of head motion if I've move the head as part of face puppeting the mouse motion can make it a little jerky and that's where I like to right click on the puppet clip and choose smooth expression clip and scroll down to head this is pretty sensitive so a setting of between two and five should be enough depending on how dramatic your motions were so why do you think you should be squad leader I haven't used the solo feature selection part of the face puppet much but it's quite simple in the way that it works for example if I click on the Brows you see that those will be the only facial part affected by this particular control set I can also switch off one brow by setting it to gray or click it again to make it bright green which allows you to have that brow moving in a different way to the other one click the back arrow to revert back to both brows moving together all the other control sets have individual settings as well that you can experiment with let's go ahead and delete that puppet clip so we can look at face key this is where you can get really refined with your Expressions almost on a frame by frame basis there are three different methods for doing this muscle expression and modify first off we want to create a clip to work inside take your timeline marker to the head of the sequence and hit reset to zero then move to the end pick any muscle set and move it just a tiny amount and that will create a new clip now you can adjust different settings within that clip I always like to make sure that key all Expressions is activated to save time creating new key frames for every facial part that you might move let's say we want to emphasize the first sew a little bit more on the muscle panel we can do that by selecting the mouth moving to the relevant part of the clip and adjusting the shape until we have what we want then slide a few frames earlier and click reset to Z zero so why then a few frames after and do the same so why do you think you can move these key frames around to alter how quickly this expression changes so why do you think let's add that sneer back in that we had before in the same way just remember that any green selected area will move at the same time so make sure you double check your selections I feel like the should be open a little wider as well so let's grab the jaw on the same key frame and move it down you should be squad leader you can even go in and fine-tune the mouth for individual words or letters for example making it tighter on the B of B you should be Squad be should be squad leader think you should be squad leader there are so many individual muscle controls available it can be a little overwhelming once you're reasonably happy with what you have then you have the option to smooth again this time for mouth blend base and move be aware that this will delete all your key frames so make sure you're happy with what youve got so why do you think you should be squad leader next to muscle is the expression panel where you can key between a number of preset expressions and key the strength of the Expressions over time as [Music] well so why do you think you you should be squad leader you should should be so why do you think you should be squad leader the final option is to go into the modify sliders and add individual Expressions at Key moments there too let's create a new clip in a similar way to muscle move to the head of the timeline and reset to zero then move to the end and adjust any slider back and forth that will create a clip for you to work inside then I'm going to go ahead and add that sneer back in do you think you should be squad leader a way I like to work is to maybe do this first to find the moments that I want to emphasize and then go into face puppet and build on top of that as it's a little more organic so why do you think you should be squad leader so why do you think you should be squad leader another option which does require an additional purchase from the real illusion Marketplace is to add one of the many expressions from digital Soul digital soul is a pack which contains a whole bunch of pre-animated expressions of different uh different kinds uh they can be dropped straight onto the character and then key frame on top of that using face puppet or face key I found it to a pretty invaluable resource I hope this information helps to improve your animations please like or comment below to let me know what you're working on or subscribe for more tutorials in the future if you're Keen to try out iclone or character crator for yourself there are affiliate links below that would go a long way towards supporting this Channel or you could sign up to my patreon for early access to my films thanks again and see you soon

Channel: Reallusion

Views: 5,114

Keywords: iClone, character animation, motion capature, facial mocap, mocap animation, 3d animation, 3d facial animation, video mocap, mocap, motion capture, motion capture animation, face animation, animation creator, mocap editing, realtime editing, webcam mocap, reallusion, acculips, Motion LIVE, facial expression, facial animation, lipsync, lipsync animation, lip syncing, Face Puppet, character expression

Id: RxCYHsU5s2A

Length: 13min 3sec (783 seconds)

Published: Fri Feb 16 2024