The Secret Workflow for Animating Dialogue

hey welcome back today we're going to be talking about animating lip sync dialogue we have talked about this a little bit in the past we did a video on three tips and tricks kind of to keep in mind while working on a lip sync shot i don't want that but today i want to get a little bit deeper we're going to break this into five sections selecting your audio breaking down the dialogue listing specific workflows the four component of language and finally polish things to add to really seal it all together and the link to patreon is down below as well in case you like what i'm doing i want to support the channel or you want to participate in the weekly animation dailies where we review work together as a group super fun now let's talk about lip sync execute order 66. first let's lie down some gram rules if you are working on something for your demo reel that's one situation versus like an assignment and exercise something just as quick practice if it is a more serious shot that you're gonna spend a lot of time on i would highly recommend not picking anything from a movie tv show or video game or if it's mildly popular avoid it like the plague uh the reason for this is because if you pick a line from forrest gump as an example or from the office i've done that bros before home somebody has probably also seen that and so the moment that your shot comes on and they hear that audio they're gonna go oh that's from this thing they're now distracted from your work and really you know focus on who is that actor i know that voice like they're not looking at your work or what's even maybe worse than that is it kind of goes into these two different sections where there's the perception of the audience where they kind of already know what's behind they know what was happening in the scene the character's train of thought what was going on in the movie the plot like that might influence how they perceive your shot and therefore whether they think it's good or bad or whatever because you're being compared to the original the other perception is yours as you work on it maybe your choices have been tainted because you saw it one way and maybe that's the only way you can see it or maybe you're trying to avoid it even though it's a great acting choice either way your decisions are kind of compromised because you're no longer just purely being creative and coming up with your own stuff you have a little bit of a bias when it comes to that dialogue so there's nothing wrong with doing it this way it's just not it's often not the best solution for these reasons now for an exercise a quick test anything just for fun do it who cares like have fun with it this is something that i don't think a lot of people talk about there's kind of this divide that should exist between demo real shots and like assignment exercise stuff that most people don't really think about everything's all just blended you know if you work on the walk cycle for school and then the the weight assignment for school often it's like well that's my demo reel and maybe it is because that's all you have but you should be doing little practice exercises that are small and easy to get through and like make big progress and then apply it to bigger more creative pieces of work there should be kind of that separation so if you want to do a quick clip from the office because the dialogue's hilarious and you just want to take a stab at it go for it but if you're going to really put some time into a demo real thing then maybe find something a little bit more original and quick side note never pick anything animated like if you have dialogue from an animated something don't do it that's kind of why games are in there as well uh first of all you've already seen that thing animated so you're you're tainted and again you're comparing you're then gonna be compared against the original done by a professional so maybe not the best play so where do you find good audio well podcasts live streams there are so many audio books there are so many places that are really untapped gems if you are gonna go for something from a more famous actor or a character i would recommend interviews often you know people have seen the movie but may not have seen all the interviews for it so my old hiccup shot that i did known toothless for 12 years and but he and i haven't worked together in a while but it was like riding a bike as soon as we kind of we just we have a thing and we just kind of connected and all of us did i pulled i wanted hiccup's voice but i didn't want something from the movie for all the reasons i already told you so i went to the hatta trainer dragon interviews actually i take it back i took audio from uh this is the end an interview for a different movie that he's also in and then i took i listened to every time hiccup said the word toothless in both one and two at the time movies and i found the one that matched the cadence i was like ah that that instance of toothless and i spliced the audio to make it sound like he's talking about toothless so it feels like it could have been in the movie but it was from some interview and that's why there's some background noise there's like music in the back you know toothless for 12 years when you are selecting dialogue not only do you want to pick something unique and from a source that most people haven't heard for all those reasons you also want to find something clear and clean and nice to listen to i'm not a huge fan of old pieces from the 40s and 50s or you know old movies that sound like they're old movies just the audio technology wasn't as good and so it's distracting there's like a fuzziness you want the focus on your animation and not like wait i'm not really hearing the dialogue because then your lip sync really has to sell it so look for stuff that's been shot on a nice microphone with not a lot of background noise and you don't really have to think about it it just sounds good and the key to when you are searching for dialogue to avoid the entire first half of everything i just said when you are searching search by ear only don't look like if you're you know going on youtube looking for interviews you're going on live streams listening for stuff like don't look literally take your hand and cover the screen if it's on your phone or put the window off to the side on your computer just listen grab the url and copy it to an mp3 file like do whatever you have to do to not see it you want to go in completely blind you want to just have an audio file so that your imagination is allowed to run wild and you can do whatever you want to do with that shot whatever you think makes sense go for it and maybe you do want to go back at one point look for reference but come up with the original ideas yourself it's gonna be a huge help to you the last thing in this section and the most important thing in my opinion is having a moment in your dialogue something that stands out not every shot needs this not every shot has this but something that generally i look for is finding something in that dialogue in that file to have some fun now this might be some kind of like a lip smack some kind of you know like my grimmel shot that's exactly what i was looking for this is it but it's not only about that just a weird sound maybe a syllable a stutter maybe it's silence having just nothing having a pause that can be really powerful and really helpful having dialogue dialogue dialogue right that middle section is where you can have some fun that's where you get to go off script you get to do something with the characters perspective their thought process something that really makes your scene shine and memorable because maybe the dialogue is a little bit flat maybe you don't really have a much of a choice about it but if you have that pause you can do something that makes the audience think a little bit for more information on that you can watch my subtext video interview with jp that's a good one dives into this hand in hand perfect stuff but let's dive into breaking down the dialogue from a lip sync animation perspective not subtext just straight technical dialogue mouth animation the first thing i always recommend doing and i do this with the students that i mentor in my patreon mentorship stuff we do this in dailies i do this when i do reviews over on agora whenever we're talking dialogue i almost always end up just writing down the audio just writing it down in phrases and trying to break it down visually before trying to go into the computer this can be really helpful we're going to come back to this as well so as you might expect i broke all of my rules when making the preparations for this video and all the audio clips i picked are things you'd probably recognize well that's mostly just because i thought you'd have fun with it and because they are more of those exercise assignments they're not for demo reels or nothing i'm gonna be like look i did creative things it's a technical exercise and that's the point so sometimes it's okay today let's roll the dialogue war war never changes now if you've played the fallout games you know this dialogue and if you haven't then i'm sorry you're missing out on some great games by the way fallout yeah so in this technical exercise let's break down the dialogue so here's what i would do i would write down the whole thing now if this were a longer piece of dialogue i'd break it up into phrases i'd have war and then i have warner changes and if i had more i'd continue doing it in these different kind of blocks separate from each other if you've already watched jp's subtext video that we did that will show you how to take this dialogue break it down into a subtext into its emotional component and then act that out and show that in your animation but that's not what we're talking about today today we're looking at this dialogue and trying to find landmarks now i'm going to keep coming back to this this exercise but the idea here is sometimes there are emotional cues you know maybe you can hear that somebody's smiling in some dialogue you can hear there's a little bit of a laugh being stifled maybe there's someone's like trying i don't know not to cough or something you can hear things right and those are things you want to take note of you want to make little annotations to yourself but the big thing is to focus on the sounds and the flow now i'm going to come back to this when we get to the four components section but the main things i'm going to be caring about in this are any sounds in this that have hard rules because like a lot of things the dialogue here some of this is very flexible very fluid and a little bit subjective to like what are we going to do with these letters but certain things you cannot make these sounds any other way which makes parts of this pretty easy now to give you an example of that i'm actually just going to write a word i don't know why this popped into my head but i literally just thought mumbo jumbo that's the word i'm choosing so let's go with it this is going to be a significantly easier word to break down sound wise because of a few things m's are kind of easy in the sense that you know exactly what's going to happen you cannot make the m sound without your lips closing there's no way to do it you have to close your mouth and um right even for one frame it has to be there so we kind of know what's gonna happen on this m and this m and b does the same thing the b is created by the release of pressure built up from right it's just the air being pushed out of your lips that's been built up so same thing with b we kind of have rules for those we have another m i missed that one so like half of this word has already been accounted for in terms of like what we're gonna be doing with it and we kind of all know that oh sound is like oh like like we're gonna be going oh it's gonna be circle how big the circle how why different things like that's flexible but we kind of have a general idea and we're gonna come back to that again the u we have two of the same sound there so you can kind of copy paste and i don't know what to expect there the j is the wild card wild card the j is compression of the lips the right is this it's this whooshing of everything kind of coming together and pushing out the j if that makes sense you're gonna see a lot of these funny little examples today but this word gives a good explanation of how i would break down the sounds where you look for anything with a rule and you look for things that you have kind of an idea of what's happening and then you find the stuff that's like okay well what's that well that's kind of this and then you work it out together and then it becomes the puzzle of putting them all together but not in a way that feels really choppy but where they can sort of blend together in the right way let's keep that in mind and we'll come back to that for our dialogue here war war never changes because how we tackle this is gonna come down to your choice of workflow when it comes to lip sync now i've done a video the seven professional workflow or what is it the seven workflows of professional animators it's like a 40 minute video it's one of my favorite videos i've ever made i'm very proud of it and if you haven't seen it please go watch it a lot of that translates really well here to dialogue the ones that i want to focus on are sort of a straight ahead what i call looping phrases pose to pose landmark style lip sync animation or more kind of a layered passes also and before i even skip it technically there's auto generated at this point if you haven't seen nvidia omniverse there's a new tool audio to face which i've been playing with i'm going to do some testing where you can just feed the mesh an audio file and it will generate full facial data for it for the lip sync and you can put it on whatever rig you want that's super interesting i'm going to figure out how useful this is and how to you know use it in our workflow so i'll get back to you on that make sure you subscribe if you want to miss that oh and while you're at it you can hit the thumbs up if you're enjoying this please do that it helps a lot thank you starting off with the more straight ahead workflow you pretty much just kind of begin at the beginning and you just move forward and try to put stuff in as you go and i call it looping passes because you kind of just have to take loops through it you you you almost never get it right on the first time you have to keep watching a few times and adjusting and digesting and going oh i kind of forgot this i kind of missed that one and just kind of keep going through it it's sort of the most like unplanned workflow for this i guess if you're just like i'm just going to do some lip sync and i don't know you might have video reference that you're pulling from and so that can kind of give you a guideline as well of where to go but you'll probably go through and you know get what you get from your reference go back through see if you missed anything go back and push it there's a lot of repeat stuff i don't feel like there's a whole lot to say on that one now a more post-to-post workflow for lip sync could be looking for landmarks so again coming back here maybe you block in all the m's you block in all the o's and you're more focused on the specific moments of your dialogue and so starting with the biggest shapes or the smallest shapes or the most interesting sounds can also really help kind of chunk it up into phrases and now onto our last workflow by the way everything i'm about to show you works for all three of these workloads it really doesn't matter it's kind of what works best for you but the last workflow i want to talk about is layered passes and that's where you kind of do like a jaw pass just up and down of the jaw you you do you know just the the tongue by itself and you you kind of just go through and do you focus on one thing at a time as layered animators would do and you do it in layers and you do it it's not that you're using actual animation layers you're not like creating new animal layers in the bottom right corner it's just that you are kind of taking it one one piece one layer at a time and this is got it i'm good this is now a perfect opportunity to jump into the next section of the video the four components of language why are we still here just to suffer now everything you're about to hear i have made up not that i created this no i've pulled this from a lot of places this is just kind of what i've collected this is the section i'm like hey like here's the thing i want to share don't use this as la though so add to this with your own experience the four components of language in my opinion are corners in and out mouth up and down lip shape and tongue these four things together can more or less give you everything you need and if you take them one piece at a time make sure they're all working together and flowing properly in context of each one respectively i think you're gonna have some pretty solid lipstick animation so let me show you how this all works all right so returning back to our examples here let's talk about these with these four components let's start with mumbo jumbo the easier one now the jaw we made a lot of these rules already we already said that the m the m the b the other m's and b's those are jaw that's the jaw open and closed it's got to be closed so we know that if we were looking i'm just going to start to draw kind of graphs if the graph editor freaks you out i have a video that should hopefully help um you don't have to think of this like the graph editor i'm just trying to visualize it so and i'm a graph person so if this is zero and this is one and then let's just say that this is negative one it doesn't really matter what the numbers are it's just that this is our baseline that's neutral that's like t pose character comes in super bland neutral zero right with the jaw the m is gonna be closed and m that's that's m and b are down here m and b we know they're gonna be roughly here now every time there's another letter and a u an o potentially a j and a u right that's sometime that is going to be up so we know the jaw is going to open at some point this part is pretty easy you know the jaw's going to be opening and closing but now you kind of know when it's happening now i have a video on the worst animation advice that i have ever heard part one i have to do more um in that video one of the things i shared was my dislike of the advice to have the frame mismatch by two frames like oh like your dialogue and your animation need to be to frame off sync to properly work i don't like that advice because i don't think it's detailed enough um and you can watch that for the full context but one of the things that i hear a lot is like oh like things need to be mismatched i don't think it's really true there is one way where you can mismatch it and it works but if you mismatch it the other way it doesn't work it's completely off and it'll look awful there's stuff like that and i talk about that but the big thing i want to kind of look at is sounds blend we mumble we talk really fast we combine a lot of sounds together when you prepare a sentence in your head you're not preparing syllable by syllable you're preparing collections of words and therefore sounds and then when you present them they all kind of have to just mush together because you have to get it out so when people mumble you're blending that really aggressively but if you are speaking very eloquently and taking your time things get a little bit more separated it's a little bit easier to have each sound be represented by its own shape so those phenomes or phonemes or however you pronounce them that library of the a shape the b shape the c shape those sometimes work they're often a great starting point a great spring springboard to you know lead with block it out but you usually can't just stick with it you can't just go plug in all the sounds and call it a day it almost never works and it usually feels very like staggered and jarring because there's no blending you need to have it flow so back to the thing about that advice one of the things that a device i think does fairly well is it's trying to communicate that sometimes it's not that the you know you're you're mismatching your lip sync to your dialogue it's that a lot of sounds you have to prepare for some sounds happen like the moment the lip makes the sound that's what's making the sound other times it's when that when that shape is broken or when that shape is prepared that you get something and so therefore it it's off sync let me explain what i mean i can i can close my mouth and then make the m sound way later right like i can make the sound but it's not until i actually make the sound with my vocal cords and then having my mouth closed is what i get the um if i have my mouth open ah it's a different sound it's the same effort i'm still doing the same thing with my vocal cords i'm just going momma it's the same effort right it's just whether my my mouth is kind of muffling it or releasing it as just a burst of air ah that's one thing where like you could have the lip flows way back here and be prepared for the m like you don't have to hit that on this frame you don't have to go and make the sound you usually do want it there early if you can have it and then go and then i have the pressure do something to show that the b for example having a b sound we have that here having a b sound is not about having the mouth closed that's not as much of an m like the m it's the rule like it has to be closed the b is a little different you do have to have the mouth closed for the b but the sound doesn't come from the closed mouth the sound comes from the opening of the mouth it's if you've ever played a video game you kind of know how this works you can hold the trigger down and then when you release like both of those actions are registered to different things there's pressing and there's releasing and it's kind of the same thing here there's pressing the lips together but then there's the release b is the release b is the burst of air coming out terrible example you get the idea you can build it but it doesn't happen until it pops open p and b work the same way that's why they're called plosives well that's not why they're called poses but they're called plosives because explosives it's the explosion it's the plosive i don't know i don't know words that's the deal so the b this has to happen like the b has to land on when this leaves so if you had to line it up it's like yeah the b might be closed but then when you hear the b sound it's opening and so you have the opening happening on the b so you need to have it closed before this frame it needs to already have been closed otherwise your b is going to seem like a week to a week part of your animation so it works really well we've got m and b because m needs to be closed b needs to be opening easy that part's really simple so we kind of have m can be closed for as long as you want it's going to go up to somewhere maybe it's one and it's open all the way wow like that's a you know 100 you might be like mambo jumbo probably not kind of depends on your audio but it's probably somewhere down here mumbo jumbo something kind of comes up closes stays closed now something you can also do is it doesn't have to go mum bo it doesn't have to freeze you can go mum and you can compress you can push it so you can actually have this dip in here and go past zero you can actually have it go and squash squash and stretch you can do all that with the face so you can push this down below its neutral position you can post things with the lips you can do different things but just the jaw you can push that down mumbo jumbo and then you know do different things again we're just focusing on landmarks when i'm doing this right now the way that i pronounce the word there's a there's a focus mumbo jumbo i could go mumbo jumbo or you go mumbo jumbo right can you hear the difference to me this is bigger than this mumbo jumbo that's how i look at it so i'm thinking that this open of the mouth is going to be bigger than this open of the mouth so we kind of have a baseline of like that's going to be bigger than this that gives us contrast it gives us an opportunity to push things we know kind of we're not just going to use the excessive but we want to push it a little bit make it a little bit more interesting and so we can go bigger here you know we can we can probably take one of these compressions and make one of these lower than the other one take your pick but this is the fun part this is where you get to break it down and do some things this is kind of how i'd look at the jaw the open and closed that's going to give you the muppet version of your lip sync which is a lot this is a lot of what makes it work what is in the box is it cookie no it is not a cookie bye-bye how about wait a second if your lip sync if the jaw movement isn't working that's usually the biggest i mean it's one of the bigger they're all important but it's one of the most obvious ones i guess what you don't want is your jaw to do this you don't want the jaw flapping up and down we usually like to group sounds together with the jaw motion so wherever you can it's not that everything's gonna be super smooth and go mumbo jumbo like it's not just gonna go up and down and be super smooth sometimes what you'll have is it'll come up slow down go up again and then come down sometimes it'll go up kind of pause and then dive it's generally always moving in this direction until it has its inflection point and flips that helps with the flow because you have a general idea of which direction things are going even if it's going up kind of slows down but still goes up still goes up that can help but sometimes it does just straight up reverse but it's these quick changes in direction that can be attention grabbing so when you have that and you look at your graph and you're looking at your drop in and close sometimes that's how it is so don't like get rid of it just automatically it's a flag for you to take a look make sure that's how it would be try it a few different ways see if it works and just consider it so that's just something to look out for that's the jaw section of this word whoa the next thing is the corners in and out now up and down at the corners is often where you smile on your front i'm mostly just focusing on the in and out the width of the mouth is extremely important when it comes to flow this is probably the biggest one when it comes to making the dialogue make sense i think this is one of the things that makes you that could make your lip sync not just looking accurate but look like broken so what's next same thing applies you don't really want it to go back and forth we usually again it should be elastic and it should be you know quick but it shouldn't feel jittery the jaw is all about the rules of the sound of that letter and of of the word but i feel like the corners are more based on the adjacent sounds the connecting stuff the the the neighboring sounds than the one that you're on right now at least in terms of direction of like where you're going with the flow so this is the one that can be really really really misleading when you are recording reference one thing you don't want to do is record your reference really slowly if i go mumbo jumbo like if you're trying to figure it out war war never ch you're going to get these really exaggerated when i say this you don't want that they're going to feel too much overacted unrealistic authentic and that's usually because not just because like the jaw movements get really crazy but it's usually because you end up focusing on individual sounds with like if i just go a e i o u u u right you can see it's like wide narrow u but if i say hey you yeah it starts off wide hey you but it's because the h and the e hey u if i say the letter u i'm kind of saying you if i say it slowly but if i just go u like this if i just say you it might not you know it completely changes based on your pronunciation so this is where like accents and dialogues and like this is where things get a little bit more complex but the thing is to not fool yourself by sitting there in front of a mirror and going slowly don't do that you're going to get inaccurate results so you need to stay at speed stay at the right speed and you know try to memorize your line say it with confidence and just see like what are the what's going on here because if i say mumbo jumbo most of that is a pretty in um corner thing mumbo jumbo i don't think my mouth ever really goes wide for any of that if i wanted to push it you know maybe i'd go mumbo jumbo so you can you can slow it down to see where would i push like where would go big you could also do the same thing with the m you could start with like a wide like a smile like mumbo jumbo all right like i hope this is working so you could start wide here you could go a little wider here but it's probably gonna be widest here a little bit less white here um but the rest of this is is pretty pretty pushed in because of all the o's and and b's well the b can be big so this is maybe not the best example here let's look at the war never changes one for this so if i say war war never changes that's one where there are a few points where it does widen so we say war war never changes so this kind of n e is a widening point war like the r has no desire to widen your lips that's a really weird sentence to say but r has no desire to make it any wider right or like it's pretty it's like an o or u it's all pretty in and the a is flexible you can you can say a in different ways a heart a which i think what is that like a a because it ends in the sound of an e [Music] [Applause] [Music] aye e wide if it's ah it's kind of middle again wide so again it's all the sounds anyway war is like an oh oh sound war war so it's it's in so in in never we widen so we kind of know it has to go up here hey jumping in while editing because i start to just devolve at this point and i don't explain this very well what i'm trying to say through all this section and what i should have spent more time talking about is with the corners of the mouth in and out there are guidelines but you i don't want to tell anybody that you have to do it a certain way because yeah having an e sound with a wider shape usually makes it read better but you can also do it the other way and i don't spend enough time talking about that because you can say e but you can also say e you don't have to exaggerate it that far you can if you want to for effect if it helps it sell the shot but there are certain things that you can really go either way and it doesn't really have a preference like i start with war never changes the the changes you can say changes or you can say changes both work you can hear that it kind of changes the way i say it a little bit and so your dialogue might kind of dictate that if it sounds happy changes more of a smiley sound versus changes sounds more i don't know something else and usually when you're stuck between do i go why do i not go wide it comes to three things number one what just looks better it might just look better one way or another and then you go with that because it's animation it's meant to be appealing and entertaining so go there but it also might be based on the emotion with which it said again you can hear a smile you can hear a frown you can hear it a different way and the third thing is adjacent sounds if you had a sound right before like this war never changes and i had never and i'm saying okay let's do that wide well then you're starting wide never changes that means you could go from wide to wide keep it wide and then just kind of you know it doesn't have to change much and it could work um also never changes you can do that but you can also go the other way never changes you can totally go the other way and it more is based on like the attitude that you want to like embed into it so it's super dependent on how you want it to look how you want it to feel and how it's meant to come across and blend with the adjacent surrounding sounds so it is very subjective i'm just giving you an example here of kind of the thought process of one way to interpret all of this it's not the way to interpret it and it's not like these what i'm finding in this shot and the the answers i'm coming to in this example aren't something that you can copy and paste into your own work and be like this is what he said this will work every time this is just kind of the way to think about it and figure it out the way i do it anyway and so if something doesn't sit right with you and you're like you know i don't think that works for my shot try it the other way i just wanted to share my process but anyway back to the video s is often a very wide shape it doesn't have to be you can say so you know so but that's what i meant by the the corners are kind of directed by what's around them that's what i mean these sounds don't act on their own if you slow it down then yeah but if you say it at speed it's these combinations that give you the vibe of the full word saying just the letter s going super wide there we go so so then it can be really narrow because the o is in charge the o is stronger than the s in this situation and so that's the thing where like at the end we end with an s with nothing on the side e like the sound of an e that's wider so changes it gives us the opportunity to say like change change the g is the most in part of the word e is flexible but the e kind of wants to be a little bit wider so it might be a little bit higher up and then the s also wants to be pretty white so that's one where you can kind of lean into it so all right let's let's let's ease into a so you might end up with this this nice flow where you can kind of go low to high and a little bit of playfulness where you know but what you don't want is like never changing like you don't want it going a lot of up and down within a word you do within a sentence try not to do it too much within a word unless you really need to for some reason it's probably not necessary so that will give you flow of corners that's x that's translate x for those for jaw we've already talked about it but what we're going to do for war never changes say let's hear it again war war never changes i feel like this war is bigger than this war so i put it higher war never changes changes to me is the big one that's the big sound so i look for those landmarks and i say all right well changes is the the high point that's my extreme up that's my biggest changes v is gonna be pretty low because v has a rule v and f have the role of pretty much teeth on bottom lip and it's a slightly different amount of kind of bottom lip curl the pose is a little bit different looking but it the f and the v is upper teeth bottom lip so there's a rule of how that's going to look which is hinting to the next step lip shape this is one where this can help you get a lot of the shapes that you're missing the big shapes the big sounds come from the jaw open and closed making that actually feel like the correct letter is being said largely comes down to corners but when you don't want to miss the things that are kind of blending in there the stuff that's kind of hiding and sneaking by that's where the lips come into play i mean there are some sounds that completely rely on it the v and the f are a great example where that's entirely lip shape but when we're not talking about those one with the rules well uh lip shape for s is just kind of getting out of the way the top teeth you want to see a little bit of those teeth for the s um that's another thing with like where's the lip come from a lip a lip is is is when you have the uh the tongue interrupting the teeth during the s shape i'm trying really hard to do a list actually used to have a lisp um a long time ago but that's i had to like figure out like why do i have a lisp i don't like the way i'm saying s's there's something wrong with it if you have a list but i just i just didn't like the way i was saying words and i was like why do i have this and i realized i was like no one taught me how to say s's properly my tongue isn't the way i'm saying f f i don't want to say f of one of the s so the tongue if you have a character who has a lisp then you need to involve a tongue afraid i can't help you it's gone you know it's it's it's nothing but code now and the lip shape it's kind of like the f and the v right you have the tongue underneath the teeth but yeah for an s you want to see the teeth and you don't want to see any tongue and you want to have usually wider lips and so that there's kind of a guideline for the s there's a rule for the v guideline for the s kind of what you're looking for when it comes to the g when it comes to the r when it comes to a lot of other letters here you can kind of go different directions with this there are some things they're going to sneak by without it and there are some things that it'll just kind of help you push it the uh the sneer controls a lot of brakes will have this right you can do all right i feel like this is a very fun video i'm getting a very good flavor of what i'm actually like in my day-to-day life sound effects and funny faces that's a good example of like lip shape i mean that's it's not the oh jaws on the corners it's not used in every sound but it can help to push things and so with an r you can just kind of go r and you're going to have like like the curl of like the top lip going up the bottom looking down right you can have that like right like the nose the mouth that's also it's often drawn like that i think the fairly oddparents it's drawn like i think the fairly valid prince is literally drawn like that it's like this kind of like shape but you can also push it further if you're going like really like if you're in like if you want like more of like a like a facial flex of like a growl or something you can you can push that and you can have those sneer controls kind of come up and go like i don't know if this is working for you but hopefully you're getting what i mean that you can use these to either accentuate a shape make sure it doesn't get missed or things like changes changes i was doing the war never changes and i actually did a version here i'll show you war war never changes war war never changes you can see how the ch feels a little bit more aggressive here on the right and a little bit more like a ch the ch the flex of the upper lip i feel has helped here k is another one where it's on release it's an open mouth it's like a bee but with your mouth open right it's it's set off by like air that you kind of trap with the back of your tongue and in the in kind of the back of your mouth and go and you send it over the top but often we say okay like it often happens on the side of the mouth okay and you don't have to go okay it doesn't often happen right in the middle that's another thing just in general we'll talk about with polish is like appeal and pushing poses but lip shape is like you can you can tweak things you can push things you can go more on one side to the other that's actually something with uh elastigirl in the incredibles but the actress behind elastigirl she has this kind of southern accent she kind of talks out the side of her mouth sometimes i'm not all dark and angsty i'm elastigirl i'm you know flexible no theme song or i'll turn this bus right around and you can hear that in her dialogue and you can see it in the animation there's a lot of stuff like that when you animate a character with a certain dialect or way of speaking you want to you want to push you want to put that in there somewhere and lip shape is a big way on how you can do that whether it's just where you put the lips or how you shape them things like that you don't want to just leave it at a default you want to you don't want to open the character's mouth and just have the football mouth you want to actually shape that pose the last one is the tongue and this one is extremely important people forget this all the time i often critique work where there's just no tongue animation and it makes like half the words not make any sense there's not a lot that the tongue does visually it's a very subtle thing but it's very very important we catch a lot we see a lot of things we're very used to talking to people and so we can pick out when something doesn't feel right there are certain things that have hard rules an l a d there are other sounds where the tongue just kind of helps out of the experience making it feel organic so an r for example let me break this down if we have an uh an l i feel like that's the easiest one an l is the tongue o is that a threat i smell tongues at the top of the mouth let go the l sound let go the t also does the same thing let go right the l is a pressurized tongue pressing against mouth happening when this flies down so again you can prep this early you can have the tongue you know be low and then you can have the tongue come up early and then when you go that's when you get that pressure the tongue really pushes let in the moment that you get a different sound is when the tongue flicks down from up here down to here and it's usually a big spacing that's what these lines represent bigger spacing on one frame smaller spacing on the next string meaning one frame is here next room is here and then maybe a little bit lower all this has to come back up the tongue is a very fast muscle it does not do a lot of easing you can't just set like pose here pose here and the top the tongue cannot float the tongue is a very very fast muscle here's some x-ray footage to show how much really the tongue does it's kind of gross it's kind of weird but the tongue is this fast but the tea coming off the tee let you can see it the t is also the top but it's not as pressurized let go it just kind of oh it just kind of hits the top just kind of kisses the towel let go well let go right the t has to come up here to block the air let well there's the way i say it is to kind of just stop pressurizing any dialogue and say let and i like stop like let but let go let right it's the burst of air from behind the tongue so if you go let's go let's go like there's a little bit of a tongue release there but if you just say let go it still goes up it still just goes up but it may not pressurize and push as hard it might go but and it just kind of seals it i hope this makes sense this is i mean and this is exactly why i break why i write it down because you can kind of make notes for yourself okay well this is how the l works i can build it up pressurize it and then when i hear this break between these letters that's when this happens but the tongue's got to come right back up which gives you an opportunity to maybe have like a like a quick like wave through the tongue is like the top comes down but the back is already moving up and then you have it kind of like you can have that you know that yeah whereas if you have like an r when i say r my tongue isn't doing much but it's not laying on the ground or if i don't allow my tongue to move at all it sounds very different are you okay to have my tongue there are you okay my tongue is actually kind of filtering some of the air in my mouth as it comes out right are you okay versus are you okay so the tongue is doing stuff it's not just dead in there so you can't just leave it alone when i do the r i like to have the tongue kind of in the middle of the mouth r and it's kind of floating in there so feel it out try it like look at what your character's doing look at where the tongue is and try to talk like that if your character's tongue is sitting at the bottom of your mouth it's going to sound very different it almost sounds like i'm doing an adam sandler bit because he always talks in voices like this but he does it in a really rude way and that's what i call high quality a tool but you can hear how how drastically different that is right if i didn't have a tongue it probably even more different because it's still probably doing stuff that i can't fully control you know some of the back muscles or whatever but that should get across the point that like you can't talk without your tongue in the way you'd expect so it is very important l t d same thing the d so if you take any dialogue you're working on you break it down you look for the landmarks and that's the thing each word each sound each letter has different landmarks depending on what filter you're looking through if you're looking for corners jaw lips or tongue there are different rules for different letters and by the time you get through all four layers of that you've got most of your dialogue pretty set and everything else that's left is just a matter of making sure those things flow looking for those like those valleys and those kind of reversals of directions and making sure you've got the contrast where you want it to be and shaping things to help kind of blend between stuff right so that is how you can look at an entire scene of dialogue figure out very quickly like where you need to be focused and keep all that great organicness in the because it's also very organic you should not feel robotic there's a lot going on here getting all that in there breaking it apart can help with that so finally we can now go to the last section of the video the four steps to polish it's treason then there's a lot to look at with the whole character's face and the head and the body performance and all the stuff i'm just talking about lip sync again so there's still more to talk about but just talking about the lip sync the dialogue you're going to be very connected to the face there's a lot of stuff that's related but the first thing that i would generally look at because it's usually the biggest thing to look at is breath you can hear in a lot of audio breathing and while sometimes that affects the mouth sometimes it doesn't but it's always something you should consider really some it's always something you should consider because you might have someone who goes and then starts talking that's going to give you an opportunity to do something with the nose which we're not really talking about but you know that's a nose thing it's also a mouth pose it's a lip shape it's it's a jaw push up i think i just spit it's a jaw push up it's a lip shape it might be a corner thing and then you go straight into all right the mouth's already open you don't have to go all right you don't have to like close it then open it that's that's one of those reversals that's like it's changing direction doesn't need to can you just flow into it that is also one thing with the jaw that i forgot to mention that like you don't always have to return to neutral if you have your mouth open it can stay open nothing says it has to go back to the t-pose version of your face also uh one thing i i forgot to say as well you can have these things kind of overlap with each other like you can use them to fight against each other what i mean by that is like take the m an m is usually it's usually slower when it opens than a b or p b and p plosive they pop open that's the whole point they pop gives you opportunity for cheek stuff but the m you might say mom mom the jaw can open without the lips opening you can you can kind of blend those two you can have if you just go jaw mom um they might just come together but if you manually animate the lips to go against what the jaw is doing mom you can get that peel you can get that organic fleshiness and that's where you can start to get some of this polished stuff i'm jumping ahead i'm supposed to talk about breath breath is important sometimes you can hear it sometimes it's here but it's often here i have a whole video on animating breath i should probably do a more updated one with more detail but breath is important with lip sync it's you have to consider it because all that stuff comes from somewhere so you want to show that in the body performance in the neck wherever you can the next thing i'm going to tell you is connectivity and fleshiness which is kind of what i was mentioning before the whole face is connected when you do stuff with your mouth it affects your cheeks which affects your eyes your ears like everything your neck like if you scream go ah you freak out you get these lines all of this is connected so you can't just do a mouth pass and call it done like you have to integrate all of this which is why facial animation is so closely tied to lip sync because they're the same thing just different parts of it flashiness is what i meant by like the m thing i'm just having like that's fleshiness it's having this squish and the squash and the stretch things like that next up is pushing graphic and appealing shapes shape i actually have that in the three tips video so that one i dive a little bit deeper i won't spend too much time here but the goal with that one is to not just have default 3d poses or just some generic shape you draw whatever it is to actually shape it with attitude to give it a motion and intention so that the sound makes sense in context of how the character's feeling when they say it the goal here is to have interesting shapes the volcano mouth things like that that i talked about in that video and finally for polish something that can really make it feel fun is mouth flow now we've talked a little bit about shape flow and having things kind of work together but actually moving the mouth around the face you can do it for a peel you can kind of cheat at three quarters for a more 2d thing like your character's looking this way and the mouth is very symmetrical just over here there's all this wasted space you can move the mouth go move it over yes but when you you know so you can have the mouth kind of moving like it's a drawing have it feel like an organic thing that can move around the face so if someone goes whoa you know like i can only just go whoa like it just kind of stays in place but you could have the character almost throw their mouth around their skull and go whoa and like it goes like the lips can come off the surface you can get fun silhouettes let me just show you an example of something that i've been i did this test on twitch a while back i think this is actually gonna be really fun it sucks because now your chances of getting killer are like super low the two things to look at here hopefully you laugh when he goes that's the thing i'm hoping you laughed at that was the focus remember in the very beginning of the video i talked about having a moment something to make it stand out and memorable for me that's the part that's the thing i was looking for and getting it to actually push off the side of the face as silhouette that was me pushing the poses for appeal to make it fun but at the end you might see the mouth kind of moves and goes it kind of feels flushy that's you know on my twitch demo that's what i was going for and so that's what i mean by mouth flow being able to actually move the mouth and have it move with the shapes feel like it's an organic part of the performance with arcs and all that kind of stuff i've named a lot of videos in this video that i've already made that are like related to these topics so i've dropped links to those below so if you want to catch up on some of that stuff there you go there's time code chapter markers if you need to come back to this video take some more notes like this is a lot of stuff that i've pulled from industry talks panels classes i don't know a bunch of different places and just some stuff i've come up with on my own i have a friend who's a speech and language pathologist we've kind of had conversations about this stuff so there's a lot of different sources i hope you found this helpful if you did please hit the thumbs up drop a comment all that stuff that helps with the youtube algorithm so that more people see this in the channel does well subscribe if you haven't already and if you have anything else to share please drop it below by all means share your information share your knowledge with discord server all the links below and if you want to get your stuff looked at get some reviews of this kind of thing talk about it more one-on-one link to my patreon is below as well where we do dailies every week and you can get your animation reviewed and we can talk about it as a group alright i think that's it thank you for watching this video i hope you enjoyed it i hope it helps you and i'll see you in the next one [Applause] you

Channel: Sir Wade Neistadt

Views: 126,559

Keywords: lip sync, lip sync animation, animating dialogue, animating audio, audio animation, animating lip sync, dialogue animation, animating acting, acting shot, animation demo reel, how to animate acting, lip sync tutorial, animate talking, lipsync, rigs for acting, maya rigs, blender rigs, 3d face rigs, mouth animation, character animation tutorial, advanced animation, 3d animation, animation mentor rigs, animation test, how to animate, animation, animator, maya, blender, c4d, anim

Id: 5cIxEZwZmS4

Length: 47min 49sec (2869 seconds)

Published: Tue Jun 08 2021