EASIEST Way to Add Blinking & Lip Sync + Final EXPORT Steps! Blender Anime

what's going on people two here and today I'll be talking about blend shapes in blender which allows your 3D models to Blink look around talk and have facial expressions this is a key step for VR chat models as well as V tubers as well as game characters that have dialogue this is one of the final steps in blender before actually exporting the Avatar over to outside software and I actually prefer using shape Keys over rigging the face for all of these things because VR chat and vtuber models actually have built-in integration with shape keys so it's pretty simple to implement so if you're new or you mainly create VR chat or vtubing content then I recommend shape keys for you anyways here's an overview of everything we'll be going over today everything will also be timestamp organized in the description so that it's easier for you later anyways as I mentioned this step happens fairly late into the process here we have a character that's already fully modeled but the meshes are still separate from each other and that's a good starting point for this video because I'll Now teach you step by step I'm going to merge everything down in this project and get ready for shakies and exports so I'll start the merge down process first I will merge an entire model into two different objects please pay attention to exactly what objects I will merge into these two objects this is the first object which I will call external or the outer layer all the external stuff so including the hair and the clothes and maybe some external body parts like the ears and the tail if you have them are going to be in this one object I'm going to apply any modifiers that they have on them in order from top to bottom and then I'm just going to join them all into one object the second object that I'll merge everything down into will include everything in the head and the body so as you can see here all these different parts of the eye were different objects at one point but we've merged them all into this one mesh containing everything on the face the head and the body this also includes the inner mouth Parts the teeth and the tongue because when we create the face shape piece everything needs to be in one object so that we can edit them all in one shape key now this is important if on your face and body there are subdivision and or solidify modifiers unlike the first object do not apply these we're going to actually apply these after we make the shape keys and here's a model after I've joined everything down to two objects in case you're confused I'm gonna be toggling these so that you can see exactly what's in each object and I've renamed both objects so that you know exactly what it contains in the name and again for object 1 modifiers are applied for object two modifiers are not applied now before we move on to the actual process we need two plugins I know there's a lot of prerequisites but these are actually necessary these two plugins are one SK keeper and two cats SK keeper allows us to apply modifiers even after shape keys are made that's something that you actually normally can't do and what the cats plugin does is it generates mouth visines for us which I'll explain more later I will leave links to these plugins in the description but they install the same way as any other plugin I assume you already know how to install plugins in blender before we move on it's also a good idea to apply all transforms on these two objects and hopefully when you do this nothing changes visually it's also a good idea to save your file as a new version since you're joining stuff and applying modifiers this is very hard to reverse so make a backup copy also side note I know I haven't made an instructional video on how to do the teeth and tongue yet but as you can see they're very simple shapes they're basically just curved rectangles with subdivision applied to them and the top and bottom teeth are duplicate these tongue and teeth meshes are modeled separately then of course you join them to the face object and you just stick them inside so let's move on to the shape Keys shape keys are a basic way to create deforms in our character and kind of a way to animate things like our eye and mouth movements without having to ring the face instead of rigging we create these deforms using edit mode and sculpt mode the shape key menu is found in this green tab I'll start adding shapes to this object by pressing the plus button on the upper right of shape Keys it will create a basis first which is the default position for your object these next three shape Keys I've named are viseme shapedes I'll explain those later I will also a pretty one for blinking and two more for different Expressions anyways I'll mainly focus on the core shapes here the AO Ocho the advising shaped as well as the blink starting with ah let's turn this value up to one and I will start editing what ah looks like over here in edit mode so basically when you make the sound ah what does your mouth look like and that's the same for every viseme in case you're lost just say the viseme's name out loud and look at your mouth shape when you do that that's the shape that you're going for anyways I've created a Vertex group for my top lip vertices this is not a necessary step but it will help and save a lot of time as I can just easily select the top and bottom vertices of the lip with just the click of a single button so I'm going to do this for the bottom lip vertices as well so that when I create the other visemes I can just click these vertex groups and easily pull them up or down I'll eventually want to go to sculpt mode to finish this but there are a few prerequisites over in your modifiers I'll turn on this on cage this triangle simple thing and this allows the shape keys to persist in sculpt mode so that I can freely edit them in sculpt mode opening the lips in edit mode is also prerequisite so that the sculpting can go a lot easier and also turning on the Symmetry mirror y so that I can edit both sides at the same time here I'm just using the default grab tool pushing this bottom lip to a line with the face and opening up the mouth a little and also pulling down the chin a little because naturally when you say ah your bottom blip moves but also the bottom part of your face of course then I select one vertices on the bottom teeth Ctrl L to select everything that's linked and that's how I easily move the teeth and tongue into place your teeth obviously are not together when you say ah and there you go the ah shape key is pretty much done I'm just going to fine tune the position of the upper teeth here now to move on I'll change the R value to zero and move on to the next shade key o by changing its value to one I'll use a top lip and bottom lip vertex droops to easily separate the bottom and top lip preparing it for further editing in sculpt mode we also do a little more pre-prep here by changing the angle of the teeth and the lips and the tongue now we head over to sculpt mode to try to get this mouth into an o sounding shape there's a small visual error here on the lip texture and we're quickly going to fix that and it messed up the inside of the mouth texture but I'm gonna go fix that later I actually didn't catch it when I did it here anyways when your mouth makes o sounds your lips turn into this circular shape and also becomes narrower so that's what we're going to try to go for here and we're going to make sure it looks good from the side as well the upper lip was a little too inward so we just push it out a bit and that is our ah and our o the teeth are also kind of not visible when you're doing an O Visine next we're going to move on to the true Visine so same process I turn all the other shape keys to zero and increase the shape piece value to one then like the other ones I'm going to quickly select the top and bottom lip using vertex groups and separate them I just quickly uh Google image search to advising to make sure that I'm getting it right and it's pretty simple your teeth have to come together to make the chin sound and your lips pucker out a bit so let's recreate that by having the teeth close together and positioning the lips as such later using the cats plugin we can take these three visemes and the plug-in will generate every other Visine for every other sound that your mouth makes here I fix the small texture error in the mouth box and then checked every Visine I went back and exaggerated the old viseme a little bit more by making it more narrow once these are done I will move on to the blink Visine so change shirt to zero change blink to one and there's a few things we must do in order to simulate a blink all the eye parts will pretty much be moved here and I'll start with the eyelashes where I'll take the sculpt mode move them down a bit and move them into this closed position making sure that symmetry is also moving the other one on the other side at the same time next I'll select all the vertices surrounding the eye socket so a quick alt plus left click on one of these edges on the eye socket will select the entire circle around the eye socket then I'll press s and Z to scale everything along the z-axis and I'm just going to scale it all the way down to close them shut the iris and the little eye shine need to be moved back so they're not clipping so I just move them back until they're out of view and next I'll just have to adjust these lines so there was one line at the bottom of the eye which now I have to align with the eye and also the ones on top I will scale them a little bit down move them down a little along with the eyebrows I'll also move them down just a little bit and after that I just work on a couple of different expression shapes but as I said I won't go over these too much it's at this point of the process where I use the cats plugin as well as the SK keep plug-in the first one will be SK keep which will allow me to apply subdivision and solidify so with the face object selected I go to object menu apply chosen modifiers and select subdivision and solidify but not Armature you will never apply the Armature modifier and yes this is necessary to have this plug-in because if you try to apply modifiers while you have shape Keys blender will just tell you oh you can't do that and you might be wondering why don't I just make the shade Keys as the last step this is why because when the subdivision and solidifier applied there are like Millions more vertices that you have to work with well not literally but it will be practically impossible to make shape keys with that many vertices moving on we'll use the cats plugin if properly installed you will see a cats tab along your sidebar so under the cat's plugin under the visines menu I'll select my face mesh and as you can see there's three slots for your viseme ah o and Chu in that order and once you press this magic button called create visemes it will generate literally every other vizi mouth shape that you need and they'll show up in your shape Keys menu as vrc.v underscore viseme name and that's pretty much it for the basic shape Keys now our shape keys are done and all the modifiers are applied on both of our objects now our entire character could be one object Once We join those two mesh objects now rigging I will go over in another video because that's a whole video on its own watch out for that later this week I'll even show you how to rig the clothes automatically after you've rigged the body as well as what to do for stuff like the hair as well as jiggle physics for the sake of this video Let's just pretend we already rigged everything but if you look in my scene hierarchy here I only have one mesh object which contains all the meshes of this Avatar and it's parented under one Armature and the mesh object only has one modifier Armature modifier every other modifier was already applied in our process if any one of these conditions are not met then once you export to let's say Unity you'll notice that there will be no blend shape functionality like you made all of those shape keys for nothing so make sure that you meet all of these requirements anyways this entire video would be pretty much pointless if I didn't export it into at least one thing so let's do file export as fbx and today I'll just show you a very condensed export to Unity as a vrm file which will make it usable for v-tubing this is going to be very quick and I will do different versions of exports in different videos later but for now let's just do a quick Unity One so let's just export that fbx and assuming you already have Unity you can Google uni vrm click their GitHub link scroll down under Unity package under installation click latest release scroll down to under 0.x and download that udvrm Unity package you can then drag and drop this into Unity into any 3D scene go ahead and press Imports and just like that the vrm0 drop down will be available now in your drop downs now let's drag and drop that fbx we exported into our scene and as you can see it's all great let's click on that base fbx file in our assets go over to rig change generic to humanoid and press apply then configure usually it auto fills in all the bones pretty well I don't have a left and right eye bone though it thinks the ears are eye bones so I'm just going to remove those but the body and hand bones are all correct once you've confirmed those you can press done at the bottom right and apply now to fix this gray model we can now head over to materials and get that sorted out as you can see it does have materials but they're all blank so let's press extract materials and all that will really do is take these six materials out so that we can configure them now let's convert all these into two shaders since this is an anime character the vrm comes with an m-tune Shader which is a pretty decent tune Shader so I change all these into N2 and change their lit color into white although I should probably do that for the shade color too this was just a quick example so I did bother but if your model already has baked Shadows you don't want additional Shadows on top of that so you don't want the shade color once everything is now a tune Shader I'm just gonna go grab my image textures they're already in this file so I'm just going to lazily copy and paste them into this folder and you'll just take these image textures and drag and drop them into the corresponding lit color Alpha squares as you can see here as for the material named outline they're just solid colors so a solid gray and a solid black down the road I'll teach you how to create different kinds of materials let's make our face the same way as this blue z-axis because that's needed for the vrm zero export now go to vrm0 drop down export to vrm 0.x fill in these red boxes with the literally anything and our character is already T pose so we don't have to do anything else let's save this brm to the desktop like the organized gentleman we are now we can hide or delete this first iteration we will take that vrm file we just exported and import it back in but this time when it's imported it will already have all of the textures on the materials takes a few seconds to load in but it's there it has the same name as the old one but this time it has colors so that's how you can differentiate it and you'll notice it has scripts attached the most relevant to this video is this blunt shaped proxy script which you can double click blend shape Avatar here and this is where we assign the shade Keys we made in blender to these blend shape Clips so here we clicked on a so we drag the a visime all the way to 100. for I I drag ih200 for you I drag ou200 arrows there are more shape keys and actual blood shapes you can assign them to if we were exporting to VR chat then we would need all of these but just go ahead and fill in aeiou and also blink I also have two mini Expressions here which are joy and shock Joy is actually a default expression but if you traded Expressions that don't exist you can just click create blend shape clip and once you're done with the blend shapes you can just click the root object which in this case it's f41 and then you do the vrm0 export to vrm zero point x thing again of course this will export a DOT vrm file which I'm going to open in a program called vcface which is a program made for v-tubing and it looks like the mouth and eye shapeys are working as well as the Expressions again I forgot to turn off the material Shadows so there's some weird double Shadows going on but whatever there are lots of other vrm features that I want to go over later too but this video has taken long enough already to edit so I'm gonna have to cut it short here hopefully you still learned something please support the video If it helped you in Subway and good news this channel has finally been partnered so thank you guys for your support and that does mean I'll be trying to upload more regularly and so I'll see you in the next one peace

Channel: 2AM

Views: 13,618

Keywords: blender, blender anime, 3d, blender 3d, 3d anime, blender npr, blinking, lipsync, lip sync, visemes, mouth, eyes

Id: Qyrz4JoMP28

Length: 16min 32sec (992 seconds)

Published: Wed Apr 26 2023