FaceBuilder for Blender — KeenTools LIVE (21/08/20)

hi everyone my name is artem and today we are gonna look at our facebuilder addon for blender i assume it was a long-awaited stream because we have many questions uh how to use it in blender how to install it in blender uh why it's like everything okay kind of questions my role in kin tools is uh quite difficult to explain i called myself humanizer so i'm like you in contools your site and can tools i often reply to your emails and try to push the user a user opinion of our users into the development process so uh yeah let's see uh we uh are going to i'm going to answer your questions of course you can ask them in the chat on youtube here you can also come to our discord channel and ask them ask your questions in there there will be all like show shown to me here in my chat uh and yeah i'll be happy to answer all questions today uh this is our streams plan uh we already covered the left side which was pin tool geotracker facebuilder in nuke and now we are coming to blender with facebuilder and [Music] after that we turn back to nuke with face tracker and some other difficult complicated cases that people are working with in nuke but today it will be face builder in blender and uh why did we came to blender with fate builder because as you know we uh started with nuke uh it was our first product was geotracker it allowed tracking geometry in nuke you can import any kind of 3d model into nuke and then track it and then use it in your vfx job but then we saw that people are usually using joe tracker to track faces because you know movies are about people and yes and they were tracking faces with jaw tracker and static models and that wasn't the right way so we created first the face builder tool to create uh models of actors uh in a in an easy way uh without 3d scanners and stuff like that and then we created face tracker that could morph the model to track facial expressions and then we noticed some demand from users of other software like blender and maya and 3dmax and whatever else people wanted our tools in here and there and we we are really yeah we we already have a question about geotracker from blender uh we think it it's possible and at some point uh i think we'll come with joe tracker to blender as well but yeah we started with facebuilder because it was like a 3d tool it wasn't like tracking tool or something that you would use only in bfx and now we see that people use it for uh like sculpting to kickstart their sculpting projects to for 3d printing and anything else for all kinds of silly videos on instagram uh we always are happy to watch them so please tag us if you share something in social networks uh so yeah we came to blender because it's free and uh it like with the version 2.79 it's it has got the revamped interface and it's it's really now a really promising tool that we want to see like the whole world of the whole 3d world coming like paying attention to blender now because it's now it's awesome so yeah um let's start uh with installation because it's often a problem for many people so we will uh see it in all details right now so what you need to do is first come to our website which skin tools dot io uh you can use this button you can go to products facebook for blender and here's the big black button that leads to the latest version of facebuilder for blender now it's been downloaded as a zip file here a node for our macos users often in macos you you get zip files automatically unpacked once you click here so don't click here go to straight to your downloads folder and adjust blender and find this zip file installed because to install the addon and blender you need to install a zip file maybe they will change it in future hopefully so for macos users we made a workaround so they download not a zip file but dmg file which is not being automatically unpacked but mounted as a disk drive virtual disk drive so you uh come to blender and find uh open this settings preferences go to this addon tab which is here and press this and stop button and go in my course users you will see a face builder driving here but here we in windows and we go to our downloads folder and here's fitbuilder for blender before we hit our uh kintool score which is we'll talk about it a little bit later with which what is it and how to install it uh many people downloaded uh the core library as well and they try often try to install the core library as an add-on you don't need to install core library as another you need to install facebuilder add-on which is please pay attention to this so you select this file which is called kin tools the version number face builder for blender that's the add-on install it uh it appears here then you need to tick this check box and now done is switched on and you can already see it here but it needs a car library because we we can't ship it uh with our addon uh because of blender licensing restrictions uh so yeah you need to install it separately you can press this button uh to come to this settings pane you have to accept this license agreement that's you can read it uh it's pretty standard it's an agreement and then you can install it the recommended option is install online uh push this button here you can select the ninth version later stable version which what's the difference uh the latest stable version is the it is what it is uh like uh it's the version that we released uh recently and we have tested it and there's nothing there should be no bugs no uh problems in there but if if you encounter some feel free to like do drop us an email uh if you uh want to test some new features and new stuff we are working on we have nightly builds they are being built almost every night once we push something in our card some test features but here's a one moment that you have to know uh if you install if you want to try a nightly version you have to install the nightly version of the addon now we install the stable version of the add-on so we can only install the latest table later i think we will remove this menu at all so you wouldn't need to think about it just press install online and it installs now it's been downloaded uh like that and yeah it's it's installed some users uh at this point some users are again being a bit frightened it asks for a license already it doesn't ask you for a license straight away if you installed it first time you will see uh the lines here that will tell you that you have a trial license for like 14 days or 10 days or else and if you have installed license uh which was installed here before we you can see that license is active it's a personal license and we have uh almost a year left uh those are like different ways of installing the license uh they're probably quite simple to understand and uh yeah we have a question do you know any simple way to generate 52 blind shapes needed for hierarchy tracking on an iphone found the blender plugin that is a some ai to do it but not sure if it will work uh at this time uh we know uh we've been conducted by the developer face it i don't uh it's called perhaps we can find it on the internet let's see yeah perhaps it does what you have been asking ask him to and at some point uh it will support or maybe it already supports our dome but not like in uh you need to do some manual adjustments after afterwards uh but yeah it's something that will work and we are planning to implement our own set of blend shapes that will be compatible with air kit i think as well maybe with some other libraries and and softwares but at this at this point in time i think you can try face it um let's get back to installation uh yeah that's that was the online way of installing the addon which is the simplest one and we recommend to use this way because it's usually this works well we can back to our settings and let's try to install it like in offline mode because sometimes you don't have access to internet on the machine where the uh blender is installed because you know maybe some studio restrictions or something like maybe you work on a movie or a project with strict ndas and whatever else and that happens so let's remove the addon and here uh one one thing about updating the addon at this point in time if you want to install the new fresh version of the addon you need to remove the old version close blender because otherwise uh the other the old version will be used again so you will launch blender go to preferences and install the add-on again the new version of course uh and then again you have to turn it on accept the license and now let's install the core library from the disk from offline mode so now we need this uh can tools core library zip file you can download it at our website as well let's see we need a browser so we go to go here go to download section this time and here you can find the core library uh and here it's a little bit more complicated because the addon itself is multi-platform it doesn't care what operating system you use but the core library is uh depends uh on the operating system so if you want to install the addon on macos you download the marcos file if you want to install it on windows and linux you download the proper core library files and here it's again zip file so be careful don't unpack packet here you can select the version the knife version is hidden here same for uh the addon uh if you want a nightly version you go to download section face builder for blender and then you select the version the nightly version is here so yeah let's pretend we download that file so we come here and we select this coral zip file it's installed now we have it on and yeah we you can even uh you can either press this button here to see this toolbar you can press n button on keyboard it will pop up and you can use shift a mesh and add fade builder head but you see uh we have some people that who emailed us and told us that it's what's happening of course you need to default this uh delete this default cube uh before so let's pretend we don't have a field builder head we have this initial uh scene so the delay select this uh box which and you can either right click and press delete or you can press shift a or just a just a sorry uh which will select everything the camera the default light source and the cube and then you can press x which is the shortcut for deleting and then you press delete now you have a cleansed scene i see we have some uh other questions uh uh okay yeah it's just compliments okay thank you very much yeah i really appreciate our community it's it's it's awesome to build things for community like this uh and yeah let's create the hat it's been created now we uh have it here and yeah let's run through the ui sections yeah here's the button that yeah i told you before it's uh the short card for the settings if you need to for example install the license or remove the i don't install fresh version you can press here not going to here here and looking for can tools are done in there just press here and it will just drop you in there uh then we have a camera section it will be uh available on only when you uh have photos here then this is the your main button add images it opens the file dialog then you can set up the model you can switch on and off different parts if you don't need them uh and uh you can like uh when you're modeling you can switch off some parts if you turn them on later the model will like uh will be correct nothing will break so you can like turn them on and off at your own wheel whenever you want uh here's the rigidity stating that uh we'll talk about it later but uh in briefly it's like um it tells facebuilder how uh how deformable the model is the lower the neighbor number here the more soft the mesh is so it's easier to shape uh the greater denial more number the less the less soft uh the model is and usually it's it works well with automatic settings so you don't need to think about it this reset button just to say it resets the shape you can yeah and these panels we can look at them with photos being loaded so let's start for example with um something oh yeah let's start with talk about how to take photos and how we are loading them you see this uh we have the no new panel exif uh it will show you exif information for the photograph use you've opened here uh if if this information is present exif is perhaps everyone knows what is it but uh just in case uh it's some meta information that camera for the camera stores in inside of uh photo file uh and we use it to there's a lot of different kind of information but we use uh focal length uh and the sensor size it's not always available but when it's available it helps uh to setup the virtual cameras for every photos like you see you can see once we loaded photographs uh now we have cameras created for every uh photo we have here and yeah as we jump here we can see again we can see the camera settings here uh previously you need to setup it manually but uh in the fresh 2.1 version we made it all automatic so you don't you usually don't need to think about it if you have exif in your photographs it will automatically detect everything and set up the cameras if you don't have exif it's better and you don't know the camera settings like of the focal length and the sensor size it's better to rely on focal length estimation you can turn it on and off here and you only if the in the case when you uh shot your photographs and the exif information isn't stored in there but you know the sensor size of your camera camera and the focal length that you used to shoot your photographs then you can open this uh panel here and type in the focal length in 35 millimeter equivalent so if you for example use a micro four thirds camera which which has a crop factor of two then you need to double your focal length so if say for example you use a lens with 25 millimeters like a fix of lens then you need to multiply it by two and so you have 50 millimeters here uh yeah and here you can find some more advanced settings but we don't recommend getting in there but you can if you are brave enough and if you want to control everything it's possible yeah let's move on and what's next um yeah sometimes you let's see these panels you can set up in size like let's talk about them later first start with wireframe sometimes you can see the wireframe clear enough so you can go to this panel here which is called wireframe and you can make it less remove like increase the opacity or lower the opacity so you can see it cleaner or like you can always press tap button to hide it temporarily and then it reappears you can see all hints here in in the lower part and then you can change colors like manually or you can use presets sometimes they help like let's keep the standard one and you can turn it off the highlighting the different parts uh the pin pins panel uh let's start pinning to see a pin you create a pin by clicking anywhere on the mesh if you click here pins are not being created of course uh then you create a pin and you can you just start dragging it um it's usually let's talk about pin size your pin size you can increase it like to ridiculous size you can keep it small can uh uh change the handle radius it's uh the radius like if i have really small pin size i still can use like take this pin like here it's perhaps it's pixels so yeah pay attention to them uh let's keep the settings and uh remove it here uh usually people start with the tip of the nose it's totally okay to start with then uh you better to another point yeah uh another important thing thing to know is that it's usually better to start with a three-quarters view which is this because especially if you have like if you can find uh only one photo in the internet of or of the person or if you have limited time to shoot someone shoot three quarters to you first then uh then switch to like frontal and side use and here's uh another important thing if you shoot uh someone if you shoot them yourself make sure the person is not turning their head sideways or upwards or downwards because when when a person uh turns their head in such way the muscles here are straight uh straightening and it slightly change the shape of the head and you don't want this so we'll say this the these photographs are photographed that we are taking wrongly so let's remove them from here one by one you can do this all the time you can open another file like here like replace the image you can replace this image we delete them all and we add images again and now we come to the proper photographs of the same person it sometimes takes some time to load photos because we are reading when photographs are loading we are reading exif data and setting up the cameras so we have the same person but now the pers the person stays still and the cameraman it was me is running around the person and shooting the person from different sides and uh here again yeah here again i asked her to ignite her head and it was something wrong you need to like just use the chair to be a little bit higher and then crouch to be a little bit lower to shoot the photographs from from top from the top and from the bottom but let's start pinning on this uh set of photos so we move the nose first and then um usually it's better to start with corner corners of the eyes and when you start pinning it's usually better to take the keep the maximum distance between pins because if for example [Music] you create another pin here and start rotating it's like it's really imprecise because the distance is small and you need to be really like it's difficult so if you create in here you can move it more smoothly uh we have questions i see how to deal with lenses first i will tell you how to deal with that distortion a little bit later because uh yeah we uh we presume that you have no lens distortion in a photograph that you allowed here usually that's not a problem because uh let's talk about this later just a little bit later um creation purchase takes too much time to uh there are many ways to do the same work and uh so you can use character creator free to create models you can use our addon there are uh other things uh to do the same kind of work but uh it depends depends on the project depends on the task that you need uh to do and uh yeah it's it's always different if you if character creator is good for you that's that's great yeah uh that's just another option um yeah what where have we stopped yeah here uh it's a perfect case again it's not perfect let's find the perfect case because i don't like it she's inclining her head let's see what we have here like yeah we have this person her name is lisa she's a great musician and helps us a lot with her photographs and yeah here you can see i was crouching i was taking photographs from the top and please just stay still not moving at all and that's the perfect case and in in the perfect in this perfect case you want to keep the focal length the same and you want to shoot the photographs with the same camera and another thing to note is that you want the even lighting so you can see that that there are not so much of the shadows we can see some like but when you when you want to create the texture it's really important to have even light and basically that's uh the easiest place where you can get even light is the shadow shadowed place somewhere uh on the street uh in the daytime if you have overcast that's perfect you can just be anywhere if you have a sun shining in there then just find a shadowed place and shoot the photographs in there it will be it will get get you pretty good results yeah the question uh about nuke two yeah of course we will go to nukes uh after the stream again we have already covered facebuilder nuke in our preview stream you can find the record recorder streams in our channel and we will go back to nuke in our next stream but that this time it will be face tracker so we'll be making faces and then tracking them right uh let's start pinning so i prefer to start with three quarters view because it gives you information about the frontal view and the side view and you have depth information it's uh from the start you want to roughly place the model and give the give facebuilder the like uh basic understanding of what's what size it it yeah it is and what's uh the proportions are so from the start it's better to just run through all the images spin the model uh like roughly everywhere and only then switch to uh modeling the parts here i you can see i chose to create pins and inner corners of her eyes that usually is not the best choice i like i said before it's usually better better to start with outer corners uh because you get like uh more distance between the pins and that should give you a bit better results at the start but sometimes it works like just like that uh if you don't see like this corner is you oh we can see it so we can put this model here and let's make this wireframe a little less visible so we can see the mod the her face and later when you uh yeah it's better it's better to start with corners uh that you that usually don't change when the expression changes and when the uh that you can clearly see like for example if i create pin here on the on her cheek i don't know where the spin on the 3d model is on her face right so it's impractical to create pins here so let's uh probably pin her ears and then her chin and then her ear again and here you can see this hole here in her ear so uh facebook doesn't know uh about such things so you don't need to make your ear like this uh you need to use your imagination thing like uh perhaps it will be somewhere here because if you do this it will change the whole model and perhaps it will give you not really good results so yeah in such cases use your imagination and place pins where you think that they should be this here here here on this neutral expression uh how i should know people with neutral expression uh just ask the person to close their eyes to imagine they're sleeping and then relax their face totally while their eyes are closed because they can be focused on their muscles and things and pretend ask them to pretend they are sleeping and then slowly open their eyes like relax that the person should be totally relaxed uh mouth closed uh eyes open uh straight pose uh sitting or standing that doesn't matter but the face should be relaxed as as relaxed as possible uh and that gives you the best results and yeah here in this ideal case when she's relaxed we can uh place everything quite precisely uh we don't know where the head box ends uh so we don't touch it uh we just let face builder decide where it is in in such cases if you how if you don't if the person doesn't have uh hair on their head it's perfect you can model everything uh then let's pin this view again three quarters we are we can see the corner of her right here we can use pin here we can move it here change it a little bit and place it here um perhaps it should be here and then the neck i think that's that's okay we don't need like the uh perfect precises right now again we are just uh placing after you pinned the three quarters you uh i think in a deal in an ideal case you better switch to views like this like from the bottom and three quarters from the bottom because it gives you even more information let's let's do this we have um here it's here can corner mouth um perhaps that's okay and let's not do this in this use let's then you can go to the front on side views the problem with those views are that you you don't have the information about death so it's better to have yeah i'm jumping here and there but i i hope that's okay when you change the model on uh if you paint model here and then you switched here uh and you tweaked it a little bit here and then you went to another photo and tweaked it a little bit here it's okay that on the play like on the first view uh the model is little bit chain uh is out of place it's not totally out of place it's just it it has changed and uh it has more information about the face and you can see it here that we need to uh move them again so it's it's an iterative process you need to like roughly place the model everywhere then you go for example to the frontal view after three quarters use place everything you can see it sits pretty well with minimal number of pins and kind of kind of that then we can be in the side views using the corners of her eye and yeah and in the case when you don't know how to like place like if the for example the model was here and you don't know how to turn the model just pin something like like you can take the ear and move them like now it's happened sometimes it needs like you to rotate head a little bit and then you'll find proper position let's find here and here corners chain and it's sitting pretty well yeah the um one of the complicated places is here under the nose uh we are working on the new deformation model and it will be better in the future but at this point in time you need to like do some maybe have more pins here and yeah it's not always possible to if the nose is like really edgy uh sometimes it doesn't you just can't uh make it too engine-like and you can use sculpting sculpting after you created this model in future versions i think we will solve this problem uh because we always are getting more data statistical data and uh our deformation model becomes better and better and yeah let's pin it here and you can see we don't have a lot of pins the model sits already pretty well it's probably not the most difficult phase in the head but right here sometimes you need more pins sometimes you need less uh a couple of words about the quantity of pins the more pins you have the more heavy it is on your computer so and it's when you have like many many many many many pins here and there then uh if you for example move this pin here and you want you you need to do a lot of small adjustments to make something some little differences and that could be cumbersome so try to keep number the number of pins as minimal as possible uh let's we have questions please consider improving your details and next questions yeah sure we we are aware of this problem as well we perhaps we're going to release a new uh topology in our soonest release uh maybe in a couple of months or so and we'll have low poly model we will have uh this this would be uh the model you have we have now is with will be a high poly model we will have mid poly model and low poly model so probably people from animation and gaming industry will be happy i hope yeah let's see we have been to a lot of use here and let's see where we move on yeah let's just run through all views to check everything is okay like you can see slightly off the place here so yeah we are doing this iterative refinement now because we have more information and we can we've we have been to a lot of views now like one two three four five six use and we can switch to details now but there's i can see there's not much to improve uh like for example you but that that's the possible problem if you have a pin here it's probably inadvisable to create another pin here uh sometimes it can help you can try but usually it's better to just use this pin here because it already uh it's already working on this area you know you don't need to create a lot of pins here and then fight with them try to keep them at minimum and we can probably fix the nose as well a little bit and here and here and here what else here yeah let's refine this um sits pretty well [Music] um yeah sits pretty well and this view oh yeah the nose is a little bit out of place here that the advantage of the bottom used that you can uh you can make the nose and the shape here like this this place and the nose you can do it really precise from the bottom because when there's just a side view or something like that it's difficult to understand what's going on here and maybe maybe for some reason i think you can use let's use this view yeah too as well it should the more views you pin uh the less time you need on the next use because oops thank you blender because the model is like already pretty pretty precise i'd say and it sits really well without too much of adjustment and your chain but the more views you have the more photos you have the more precises you can get i don't think that you need more than uh 11 photos uh usually even seven isn't like usually it's uh yeah oh we can see the problem here usually it's enough to have the frontal view of course three quarters from one side three quarters from another side and everything is okay uh side views from one side and another side and then some top and bottom shoot shots that will give you some extra information but the bottom and top shots are also they are they can give you really good results but they are not like uh essential i'd say they are like for people who need extra preciseness yeah let's let's stop pinning because it it's like you can you know it could be you can do this endlessly uh it's like some meditation yeah let's move on and uh uh yeah let's look at what happened in the scene as you can see uh when we pin the uh shots we the cameras associated with the photographs are being placed as well so you're not only changing the shape of the uh head you're you also are placing the cameras uh the cameras are like of the photographs that the like this is the frontal photograph this is the three quarters one three quarters two those are the side views uh so those are the uh bottom views and let's say you want to watch the result you have in 3d what you can do is go to 3d mode then you can press alt d uh nope it was no not it was window d and sorry you press alt d and then you have a clone of this model and then you switch back to some of the photographs and then you can see it here so if you for example move it here um we are changing the camera so it's probably work better here you can move here and you can see the the head this head moves first and back because the camera moves uh when we are changing the something less for example you can see how it changed so you can watch the result in 3d like that and you can delete this and yeah let's go to texturing and we have another question from wiki jane if you want to edit the model how would i do it assume i didn't urine you use a built-in blender uh functions like sculpting or something else like you can as you can see we have uh like if you go to the edit mode you can see polygons and you can see the mesh you can change it manually and you can even uh like for example we can for example do something like this and so uh let's switch on here here you can see the model is okay but if we can change it for example we can tweak her things it's like it's it's like a per sheet not it's yeah sometimes you uh perhaps now we set the model yeah yeah at this at this point in time you need to clone the model or if you want to make some changes that will persist uh and uh yeah let's let's let's just try i'm just i think it will work like that if we for example i don't know let's see that's interesting let's move this that's oh no it changed it's a clone yeah it's clown it changes the model okay okay so the model will be the topology will be reset when once we come here and go back yeah it's reset so you need to do a real clone which is shift d and then you go to the edit mode and do something like that let's let's present this she's a punk punk and that's her hair uh yeah like that and then you can use sculpting you can go here and change something like that and like that uh do whatever you want using standard blender capabilities you can go to the edit mode and select something and delete this part so the polygons are being deleted and yeah just use the whatever you have in blender uh you can do this uh with this head as well if you are like done with uh with pinning and you don't want to pin the model anymore so you can just do it with this model as well um yeah so let's go to texturing uh it's uh an interesting part uh first a couple of notes on the interface what it all means we have uh first we have different kind of uvs and let me show this perhaps maybe we can show it here oh yeah i have new here so let's see to nuke it will be easier to see in there um yeah we'll come to nuke later i will show you some interesting tricks uh i have texture builder here let's see the texture here texture is being created and created yeah let's see uh now we are using max phase uh uv we can use the butterfly which is default just in a moment we'll see this it looks like that and if we for example um let's try to see it more clearly let's try to like that does it work no it doesn't work we need to somehow okay it doesn't matter uh then we can use the legacy it was our first uv uh map that we used it's uh it's advantage uh one of the advantages it has it's that it has the minimal distortions so although you have a lot of parts and it's like not the whole texture distortions are here at minimal at minimum then we have spherical map and i'll show you one trick a little bit later and then we have max phase the max phase the point of the max phase map is that we have the most resolution here for the face part so you can change like it's really like has has a lot of features here uh there are on what we have we have questions the geometry tracking available in blender version not at the moment but we are i think the next hour dawn for blender will be face tracker i don't i can't tell you when it will happen uh but yeah we are moving that direction as well are you considering any face builder related facts to it would be yeah we are planning to add uh fox blend shapes into face builder pretty soon i think uh again no estimates you know because it's the development process is really unpredictable so we can uh tell you when uh something happens uh but yeah i think i think it will happen soon maybe in a couple of months maybe a little bit more but it's something that we want to implement as soon as possible uh yeah let's go back to blender and see how yeah here uh these two fields uh define how large is your texture in pixels uh usually it's square texture so we can see it here perhaps let's select it now we have let's go here um object mode please nope let's see uh what's what's wrong what's wrong um okay let's just select a yeah here's our texture uv map this is the default evmap it's uh it has the square dimensions and we can change it here for example if we want uh like for example if we look at this spherical uv map it takes a lot of place here and then square and it's distorted it's really distorted here because it's in square so we can for example keep this high and double the width so let's for example say it's four thousand by two thousand and um let's see um editing it's it didn't didn't update oh yeah we have to have to change something uh no we have to create a texture yeah before um [Music] yeah when you create uh to change the uh to make this uh changes effect if you need to create the texture and you change it uh what happens when you click this button create the texture uh you click it and you can select the images that you want to use for texture because uh we have it because sometimes you want some pictures uh to be just a model reference not uh not a reference for a no you don't want to take picture texture information from the photo because perhaps it has different lighting or it's a [Music] really low resolution or something like that so you can select photographs and the photographs that don't have pins on them they are automatically disabled so they won't be used uh yeah and we press the ok button and we wait a little bit because this is a time like resource it can it consumes our resource some resources and we uh it called clay's detector what it does does is uh it looks from for to the pictures using those cameras and projects the texture on this on this model uh using the views and the position of this uh model in the in the photo and yeah we can see it here and let's return to dimensions and now you can see the texture became horizontally uh like white not not square and let's let's see so yeah all this information is here you can see it and uh what it gives you it gives you a little distortion uh like horizontal distortions here so it's easier to uh make it better perhaps and let's let's see what what we've got in here in 3d mode uh pretty nice texture of course we have some shadows here because uh the light was coming from the top on the photographs you can see all the light is coming from the top and we have shadows here and for such cases we have some advanced features here there uh all experimental but let's try first is to equalize brightness it will what it does it tries to even the brightness on the different photographs and let's try wait for us to let's make the texture a little bit smaller so it takes less time like 2000 by 1000 create a texture okay now we wait a little bit sometimes it gives really weird results because it's it's still experimental uh and sometimes it gives pretty good results and we are working on completely new texture and algorithms that will create albedo textures but it will come sometime soon as we can see it has removed some shadows maybe not here um it's heroes the same uh uh works with the color if you for example have photographs which one of them one of them was taken taken like midnight with with a flashlight and another one was taken during the daytime and they have different color perhaps and so this might help again it's experimental feature so don't expect many things from them from it and now let's see what those uh things do and for that we will perhaps use just one photograph in the frontal one and let's see what happens here with uh the value of 10. uh it's uh the unit of this value is uh something mystical you don't need to think about it the minimum value is zero the maximum value is 100 when it's at zero uh it takes let's see what what what it does creating the texture and yeah it takes all the possible pixel information from the picture from this frontal picture like for example we have a really narrow angle here so we don't we can't be sure what color here uh what color of the skin we have here right and with the uh and go streak this off zero we still take this information because we we can see this model is like somewhere here so we can try to take this information if we go to 100 uh and create a texture the algorithm will only take uh the polygons the information of the polygons that the camera is looking at night looking at at 19 degrees so you see it's not a lot of information here that we can use and for example we can let's create something weird we can use this expand edges feature which basically expand edges like you see edges are here what it does it takes this color here and it just stretches it so let's see like one like five percent what are those let's see now we'll we will probably see something really bizarre oh no it's just even it you see it we are like covering the holes and uh enlarging the texture a little bit here and there uh sometimes it works sometimes and doesn't uh yeah again we are coming with a new a completely new texture and mechanic since in closet future yeah what's next what's next um yeah uh an important button here uh is export button you can export the texture into a file like for example let's just save usually it's better to use png files because these black areas will be transparent and it's easier to work in photoshop with them or in in gym if you prefer open source software so we can export the texture into a file and it will be just a png file let's see where it is yeah it looks like just like that uh and so you can of course open this file in uh let's say photoshop or fill the gaps [Music] retouch something and then you can uh apply this uh texture back to your model here and how here's how to do this and let's say let's say let's say let's say we will apply a texture of another human uh to her model because the model is the same the texture map is the same why not just add another texture we go to shading and we go to this node here and we will just remove this file here and open another file i will go to the um let's say let's say this texture uh it was it was looking like that let me show you um it looks like that it was uh on i'm sorry it was uh yeah it looks scary but uh it's how it works uh you know if you work with textures you probably are ready to see the most scary things in this world yeah so it looks like that it we have some color filling here we have the cloned hair here uh if you want to grow like hair in in blender for this model like you have seen in our demo video you can you probably want to remove everything except the skin color uh from the texture and then you probably want to remove the eyes polygons and then install some like bowls in here and then apply texture to them and whatnot i most modelers probably would want more detailed ears because like now it's let's see how they look um they look like that and it's sometimes it's okay sometimes it's not what artists want and uh yeah and here we have our texture again oh what next what next uh we that's that's yeah let's say we have many cameras uh let's let's create a new scene you know save anything uh because who needs saving where are brave people um yeah let's say we have photographs of someone uh taken with different cameras but the person was uh keeping the neutral facial expression that's like one of the cases that you can encounter uh so for example we can [Music] we can probably go here and over here take those photos uh here a couple of words of yeah we have expressions here so let's try to find the most neutral we don't have neutral faces here um okay okay let's see what we have in just a moment okay let's let's present it's a really rare case i think when you have many different cameras and but still neutral face so let's just jump in into uh into a case when you have many cameras many different cameras and facial expressions right so we go here and let's say you took some images from the internet there's a there's a lot of them but we don't need we didn't need any ever every photograph and those are taken from an internet it's our friend who kindly agreed to take part in our process of making demos and stuff so uh we have many different photographs uh some are neutral by the way uh but they are all different like different aspect ratios different zoom levels any anything like and here's a one important thing that if you shoot something with a zoom lens you basically shoot something with different cameras because uh let's let's see let's try to look at it how it works for example come on come on what's wrong sometimes programs don't work let's say let's see yeah now it works uh here's your sensor of your camera here are the lens and pretty rough but i think that will work and this is the image that you get for this lens and this is uh for example 50 millimeters and the person will be like standing here uh so here what you have at 50 millimeters is uh so let's project it here it works uh like usually not usually but all the time in into uh in the reversed way so you lens reverse the image but that doesn't matter right now so the person is here it stands like that and if you change the zoom level the person's uh perhaps this is another lens uh the person will probably appear like it's probably not a good explanation but i hope i hope you will understand i wasn't really practicing it before and okay the person will be uh this is like another lens let's see uh let's say this is a different zoom level like that was 50 millimeters now we have 70 millimeters and now we see uh like this part of person and uh this basically means that the sensor you get it will be if the sensor is larger but it stands here in the same plane i have this person here and here uh so again so yeah basically what it means is that if you have different zoom levels like this is 70 millimeters this is this is 70 millimeters this is perhaps 50 millimeters and yeah we basically have different cameras with different sensors let's return to from this awkward explanation to the blender and in this case uh we don't know what the camera is so blender our addon in blender automatically detects that there is no exif information we don't know anything about these photographs and any of them and then it turns on the focal lens estimation so you don't need to think about it if you don't know the camera settings if you know camera settings for example if we say that in this on this photograph we have i don't know 70 millimeters we can turn it on off and type it in manually uh or we can just use it there are cases uh where for example let's add another photo uh from here let's see what we have here i have exif information but the photograph is rotated and usually we take the information from exif to rotate the photo automatically so you don't you usually don't see such things but for some reason here the information about rotation was lost and we can see in uh in this image that first the camera tells us that the image was 3000 by 2000 pixels but the real size of the picture is a little bit different so then we decided it was cropped digitally and then we turned on the focal lens estimation but uh we still use the 35 millimeter equivalent focal length from the exif just as a start uh value we start from 70 millimeters and then we will uh estimate the real focal length let's let's um start here let's make it more visible maybe maybe use something like that because on dark shots like bright mesh works better and what we have here is uh yeah the camera will be estimated independently for each shot here because uh in this case we don't have uh let's see another case i will perhaps need a button to remove all the photos but let's just add some some more for example we take again um perhaps these ones yeah you see uh we have this one digits here what it means is that uh facebuilder included those photographs into group number one and what it means is that it works with these photographs uh we detected it using exif we see that uh those pictures are all different we don't know anything about them so they are ungrouped basically they are having their own groups uh but it's it's good to think that they are not grouped at all and yeah what we have here is our group and those images will be used kind of together to estimate uh if we have estimation focal length estimation in this group the focal length estimation will work only in this group uh like this will affect pictures of this group if we don't have focal length estimation and we know the actual parameters of the for the camera uh it will be used only for shaping the model let's remove them from here because those are real rare cases i think if you have different photographs on different socials they are all different so it's better to either manually setup them or use focal length estimation everywhere so what happens now is let's make it standard yeah and a little bit more visible maybe not that much and oh maybe maybe this will work here and without but yeah it's it works like that better um we can see that when we move our pins here uh first two pins they uh just rotate the model and third pill starts rotating the model in 3d the first two pins rotate the model in 2d the third pin rotates model in 3d and when we add fourth pin and still doesn't estimate anything yeah estimation starts you can see phase builder is with with the fourth fourth pin you start uh shaping the model actually shaping it uh with the fifth pin you see the estimation started focus length animation and what we can do here is uh perhaps we need to we need to make it like that we can see his ear ah thanks bender um you can barely see it here that's the wrong place you can perhaps sometimes move in pins without mesh as well because many people ask us if blender or if our add-on uses the pictures in some way at this point no we don't use the pictures we the pictures are here only for your reference so facebuilder known knows nothing about your photographs except uh the focal length that it's been estimated or typed in manually or and the pin position so yeah what what changes the model is only the position of the pins uh let's say it's something like this let's take another color scheme and move this here um it's barely visible and no parts some sometimes the parts help sometimes this highlighting make it more difficult to see what happens so you can always turn it on and off when you need it and this is a like pretty neutral phase so we don't need facial expressions here which we are coming too slowly and this is i would say again it's a neutral head neutral phase compressions and let's move here and again we are estimating the focal length here oops and like that like that like that sometimes you just want to move pins at random like putting them in random places but usually it's better to know what you're doing i'm pinning this person like maybe i've pinned him 100 times perhaps already so i'm just relying on my experience uh and yeah let's see what we've gotten in 3d like that so we already have someone similar to the person photographs let's try to create a texture here and try equalize those two features oh yeah we have those pictures selected we wait a little bit and what happens what happens is your computer is working yeah we have some texture here you can see even not too rough transitions between the color of photographs because if you look at the photographs we can see it's yellow here it's pinkish here but our latest version of algorithm already handled it pretty well let's try equally scar off color feature oh what it gives us is i hope something good something good nice uh yeah now it didn't work well uh let's try to make it worse because we're having fun aren't we and now we are using uh equality brightness as well so what i expect here is yeah grayish something uh yeah sometimes it works sometimes it doesn't and i think it will work better better with our new algorithms in future uh yeah let's see let's talk about facial expressions uh yeah uh yeah there's a i just remind you that if you want uh good results with your texture you want even lightening lighting you want uh minimal shadows uh you want the person staying still not turning anywhere their head anywhere and you're running around the person with the camera and taking photographs with the same exposure with the same white balance with the same um what else uh focal length of course because it's there's no point to take pictures with different focal lengths uh you only make the process a little bit converse more cumbersome so uh let's jump uh into the question about distortions as you probably know there are no perfect lenses they all have distortions and what you can and face builder uh expects you to having no distortion in your photo photographs at all and uh usually it looks like uh let's clear this mess first uh photographs are being like distorted like that so instead of being like even it's not too even but i think you you get the idea they are being like bulged and the lines in the shot are going like that uh and yeah and the good thing about it is that in the center of the frame there are usually minimum distortions uh that's that's distortion still the distortion and if you have a heavily distorted lens you can just place your purse uh your actor person whatever subject into the somewhere here if the if the face is somewhere here you can just don't care about less distortions though because usually on all modern lenses this area is pretty straight uh yeah but uh i i'd say that on modern licenses usually uh this area is safe and you only see distortion somewhere here somewhere here and let me show you the real example here hello scary texture um yeah we have a photograph like that let's say we want to see the frontal face and let's see what happens with lead insertion and you can see uh it was a lens uh nikon 50 millimeters 1.4 it was produced in early 2000 like some somewhere maybe even in 90s so it's not a perfect lens but even here we barely can see any distortions on his face distortions are here or here uh so it's like is the lens of 20 year 20 years old and uh pretty average lens nothing special and it still gives you like this kind of result so you usually don't need to care too much about distortions but if you want maximum preciseness it's better to undistort your photographs or uh as another hack you can use the focal length uh correction uh distortion collect correction from your camera because usually modern cameras have the lens or maybe lens for the modern cameras have uh built-in distortion profiles built in them so if you shoot raw uh you probably have distorted images if you shoot jpegs you have undistorted images already because the producer of the camera sorted that out for you yeah so what you can do about lens instructions and blender is use this phrase blender undistort on google or whatever search engine you like or you can use this qr code it will lead you to a blender community let's take it stackexchange.com there's a large large post on how to undistort pictures and blender using nodes different nodes like distort nodes and then you can like import pictures in blender and distort them export them as files and then import those files and distorted files into facebuilder for blender so yeah uh let's go to blender again and finally we are coming to facial expressions our newest feature uh what it gives you uh is the facial expressions from face tracker that were first introduced into face tracker for tracking phases and then we decided that we can add those expressions into facebuilder to let people create faces with photographs of people who were not uh standing like sometimes you just can't find proper shots and you perhaps just have shots from a movie or somewhere else and you cannot find a scene a frame where the person is having neutral facial expressions so yeah you can use all uh facial expressions here in blender uh let's say we will start yeah with this photograph and yeah there are two ways to work with facial expressions and then in face builder from blender at this point in some cases and there's no way to tell when you have one case when you have another case at least i don't know such way yet maybe we will find something else in future like how to do how to know how to start but then i think blender our don will make this automatically for you but at this point uh there are two approaches that you can uh two ways you can take uh one way is to first [Music] set uh take the most neutral shots or maybe not even neutral shots like uh do we have any no neutral shots here now they're all pretty neutral let's see let's let's let's take another set of pictures and we go to uh yeah we go to go to where we go to go here and we go here and i take those pictures and what we do we have is this is lab our programmer he by the way works on worked on tech new texture model that will be released soon and now he's kind of working on porting face tracker on unbinding face tracker from nuke so you can perhaps thank him thank him for bringing face tracker in other two other hosts like blender and anything else in the future um yeah so what do we have here uh we have two ways as i said before uh first we go through all photographs we without uh turning on the facial expression and here we have information about the camera it's been taken from exif data so we don't add the focal length estimation to the equation so our facial expression and the modal expressions and the model will be pretty precise i think um yeah so so what what happens here yeah we go through all photos and we try to ignore the expressions and use the parts of the face that are not being changed with changing x ex with with this expression and usually those are the corners of the eyes i say you can hardly move your the tip of your nose and you can move the corners of your mouth quite a lot but in this shot i'd say they're quite neutral here's a tricky part uh we cannot say if gleb is has his mouth really closed his lips are closed but his mouth might be open behind the lips right right so we but let's pretend that he has closed mouth and we yeah we said his chin here and perhaps you can hardly move your ears here but you can move them here quite a lot some people can do this quite to quite a huge degree and his nostrils i'd say they are looking like neutral so let's um yeah let's keep it here and move the ear here and move it here and fix the neck because it's kind of in a neutral position and then we go to the next shot and here we have perhaps not the dramatic change in expression so we again rely on corners of bite say again neutral mouth eyes we don't use eyebrows here because i don't know what's their neutral position in here it can be shifted let's pretend he's my oops he's closed yeah to move around like that uh what you need to do is to uh just in case you didn't know and you were struggling using something like this you can use this button here and you can use this to zoom in though out but if you want to move effectively uh you can use shift button and the middle button on your mouse like which is usually a wheel button and you hold the shift button press the wheel button and you move around you can use scroll wheel to zoom in them out and if you use only the scroll but uh scroll wheel button uh you will go into the 3d mode that's how blender works so we decided not to change it not to override this and yeah let's move here and here we have another silly expression and i'd say we can place the eyebrows here because they look neutral and like that and i wouldn't touch his chin because it's somewhere somewhere i don't know where and let's turn this and then we move on to this shot again the nose the corners of the eyes the ears is quite safe to use we don't touch his mouth here probably can't touch this uh we don't touch his eyebrow here and yeah like that i think his eyes eyes are partly closed but we don't need to close them here because if we close them here we may just make them give them the wrong shape and let's move don't don't move his nostrils and another one and oh yeah let's pin this one uh you can change this and you can hardly change this uh i'd say it's somewhere here even though i cannot see it nice i hope it's not too boring we'll move on shortly and you know shots like this these ones i'd recommend to switch over the neck so it doesn't give you them like it doesn't ask you to place it somewhere because you yeah you see the head is at some angle here you can you don't know what what angle and in our current implementation the neck is static so you can't uh incline the head in our model it will be as well change it in future but for now you have to know that it's static and you can't move it and what's what [Music] here we have we can as well go and uh fix uh the parts of the face that can move and we don't want them to move sometimes it's uh it's worth to do sometimes you don't need to do this but uh if you're not lazy you better fix them for example uh we see the mouth here is pretty neutral so we don't want to be open right so we fix it with two pins and move it here perhaps oops and yeah we don't move his eyebrows but perhaps we want to keep those points here and uh here again we don't want him to move his lips so we fix them the nostrils as well and what else what else everything's fine here oh yeah let's fix his hand here and move this a little bit and what we can fix oh yeah let's fix ice because i'd say here they are in the neutral position so let's fix them like that they we will like make them more precise in a moment and here nothing i'd say and here uh maybe just a little bit uh no his nose is perhaps not in the right position okay and now we can turn on facial expressions nothing changes from the start um but once i move at least one pin here the uh the huge uh equation will be solved again and we will get uh and facial expression will be taken into account this time so you will probably see some we can see some weird things things here like uh we will see so let's try yeah yeah he has opened his mouth let's move it now to the right position like here let's move this cheek here and let's move the chain oops wrong point let's put it here and perhaps move here uh open his mouth properly uh sometimes when you move mouth you can get like those polygons here uh a bit skewed it's better to uh keep them straight if you see that the mouth being opened in a straight way like the mouth is open just downwards not being skewed so we'll fix this and we don't even need to fix the eyes a lot because uh when the expression changes it takes it changes everywhere and facebuilder kind of knows uh it really it actually knows based on the statistical data that we had how faces work with expressions so when you for example close one eye a little bit closest another eye and etc uh yeah here we can see some weird things uh his mouth decided to be a little bit off so let's move it here move it here you see it getting getting there and we are moving now we are skewing his mouth so it's okay if we if you want to um set a pin uh in the in that corner but you don't know where the corner is you can just open the mouth set pin here remove this pin and get some wires uh yeah that's uh let's say it's a pretty difficult expression i say and i hope our deformation model and future will be capable of hand and clean it with more preciseness but at this point in time um we're having something like this and his ear and let's move it cheek here oh yeah a couple of words on where to put the like where the edges of edge of the model should be uh it really depends uh because as you can see here it's a little bit blurred so you don't really know like in pixel terms where the edge is because we have some hair here and it's blurred because the focus is somewhere here on his eyes and his eyes are too blurred as well oops let's come back where are we yeah here so in terms of texturing it's better to keep the model inside the contour so when the texture is being created you don't take uh the contour like outside you don't get outside of the of the real face so it's better to be a little bit inside or if you have like a pixel perfect focused image that you can you like you know this uh you know you can see the edge to the every pixel uh then you can of course place it precisely but you but usually it's just enough to keep the model inside the face like the edges of the model and as you can see here nothing's really changed and i created another pin here i don't want to move his ear i want to move his brows by the way and yeah that's fine i'd say no here and yeah that's fine uh and here what do we have here uh the mouth is being a little bit opened we'll fix that a little bit uh the mouth is a little bit tricky again but it's getting better and better with every new release so yeah it's uh about updating uh it's better to update on often because we in like in large production studios people are afraid that uh updated software will break something in their pipelines and that indeed happens sometimes but we have like some extensive testing routines been run automatically on before we release anything like uh when our developers push our code to the repository it's been automatically tested uh before being built and there are quality tested tests uh there are like tracking modeling tests and anything any kind of test so you can be pretty sure you wouldn't break anything and we we always keep the backward compatibility so if you for example open an old project with a new face builder or uh a new note and nuke uh everything everything should work you can probably lose the ability to change something because uh i don't know maybe we will introduce new deformation model and it will be incompatible uh in terms of changing it but it will still work you still have your results you still can use them in the old scripts and old projects because yeah we we are trying to keep the sound under our control and yeah let's say that's all done and what do we have here is oh yeah we're not done let's finish with this view [Music] and i can see many questions but it's okay to ask them so yeah if you have something to ask we will shortly come to the end of the stream i think there are not too many things left and so you can you can start uh spamming spamming us with your questions um i mean in the in a good way no of course not in a bad way we will a lot of questions we'll have feedback without our community and it's like it's really like i said it's really great to be to create products for community we have uh it's it's amazing it's truly amazing and let's close his eyes a little bit a little bit here a little bit there and and like that i say i say we we are done what we've got here is our 3d glap and let's see how rigidity works because i skipped it then but now i think it's a good time to show how it works like you can see uh there are some some rough edges probably here or there yeah that's better and yeah maybe we can maybe we want to the mo our model to be a little bit more smooth and as we can see the authority automatic algorithm will decide that rigidity should be uh 1 1.65 etc here uh it's being automatically changed with every shot you pin uh but let's say we want a smoother model so we what we do is we increase the rigidity let's say take five it doesn't change changes anything right way but if we move pin and look in 3d the model becomes a little bit more smooth maybe maybe we can ah right i forgot i didn't let's see five again yeah now you can see how everything moved the model became a little bit more smooth and now it's a little bit harder to change everything and let's see what happened in three yeah it's a lit it's now it's quite we removed all the ralph angles and stuff like that and let's say we want auto uh yeah and we if you if you feel that you move the pins but the model is really don't don't doesn't want to change you can lower the value like for example let's say we want 0.5 uh and yeah 0.5 and let's see what happens whoa everything moved and it's so soft you can you can do whatever you want it's like becomes a sculpting tool at this point in time i think um yeah let's see what happens in here it's quite ugly uh so yeah let's return the default value it jumps back to 1.6 again and yeah now we've got our model um we can try to create a texture uh here again let's see what happens what we can get off of this model and what are we having here is yeah it's quite yeah uh a couple of words on how do we call it uh persistent persistent pinning yeah uh you want uh not persistent what what what was the word uh anyway you want every little consistent yeah that's what's the word sorry i'm getting a little bit tired um yeah you want every part here to be exactly in the same place on all across all photographs you have uh because otherwise like texturing algorithm will take this uh we will decide that this is an i and it will take this as a as an eye and mix it with other textures from other views so to get a precise and accurate texture you need preciseness and consistent consistentness uh in your pinning so every part ever basically every polygon in any id in an iu case you want every polygon you have here like this one to be in the same place on all photographs then you'll get like superb results it's not always possible uh and it's not always important but the pres like it affects the texture let's make it a little bit a little bit smaller so we have less time to wait and create texture okay uh because you see the like views were not consistently pinned and we have some madness happening in his eyes like uh now it's maybe a little bit better but you like again you can turn off uh ice completely and install that's scary and install the eyeballs here yeah you can always do that and let's say yeah we again we can replace the texture here uh like export it enhance it and expand it and we can place hair here using hair tool and blender and all kinds of things and here perhaps some people think that i want this expression like i see the expression here and i want it in 3d and in 3d i have only neutral head we uh we decided to not release uh to not let people do this at this point in time because it works uh in a wrong way we want to bring a better technology like blend like fox blend shapes for example so you so you would be able to manipulate the model using facts blend shapes and you would change the expression in the right way because the blend shapes uh we use kind of blend shapes something internally to shape the model shape the expressions but here in this case you're not only changing the expression you you are also changing the shape and by the way those kind of blend shapes we use they are all statistically based they are not a human human parabol a parable so it wouldn't be like a good way to good thing to work with it's they are really uncontrollable by hand only heavy mouth can handle it so but so still you can actually you can do this there's kind of a hug if you really want here's how to do this you first create a clone with alt d here here will be our clone and go back here for example and you see the model in 3d has changed but it's a like linked clone so it wouldn't be it would be reset once you in out of pin mode and if you want to keep it selected in pin mode again that's a hack and i wouldn't recommend this as a good way to get the expressions but if you desperately want it you can do this and then you press shift d and perhaps you can press y to move it oh no maybe we can move it in x like here and here we have a unlinking clone and here's another another cool hug we can turn it into blend shapes there are those are not good plant shapes because uh the shape is changing a little bit as well uh not only does the expression but it works like that select the head next oops let's move it here let's take this hat in this head and we go here into the shape case and we join a shape use this menu here and select join us shapes and then we go here and then we change the value now you see you can change the this model it was used as as the base basis model and yeah you can clone it or you can like for example press shift d again here the neutral head and you can for example use this head again and this and remove all that and this is now a link clone so we just like this like this and create shapes and again we can open this model you see it it changes a little bit uh but we can animate it somehow and this model isn't changing because it's unlinked right let's move on uh and what we have now is um yeah i had some points on taking photographs from the internet if you ever happen to first thing to keep in mind is that people change over time their faces are changing constantly and you can see i saw difference like really significant difference in photographs of one person taken like uh across four or five months uh i don't know sometimes people get a little bit uh thinner sometimes they had they are getting more rounded let's say it's called like that and uh yeah so people change over time and some like especially young people they are becoming like they are not finalized if we can say it like that so they they change a lot uh if you know the camera settings like for example you sometimes can download photos from for example pixels.com and sometimes photographers uh do uh note the camera settings like the what camera was used and what focal length was but the this information is not stored into exif uh uh fields of the file but you still know it you can find uh the information about this camera like for example it scan and mark the canon 5d mark v you can find the sensor size and you can if you know that it's canon 5d it's it's roughly full frame 36 in millimeters millimeters 36 millimeters a sensor uh and uh you can just use the focal length that's what used if this uh was like something of micro four first family uh then this is a crop factor two you just uh you take the focal length and multiply multiply it by two and use this uh setup it here in camera manually it will significantly improve your results if you don't know camera settings forget it just use focal lens automation uh and when there's no exif information in photographs uh but the photograph is turned for example it's rotated like that and facebuilder didn't detect that the photograph is rotated you can rotate it mainly here counterclockwise clockwise you can reset the orientation just yeah you can do it here as i said before you can replace the file using this menu here and what else you can reset expression using this uh for example here uh i don't like the result i can just reset it it will be neutral and you start over again you can reset camera position so it's in the center you can remove all pins uh from this uh view like like that and then when you move it uh when you add fourth pin it this sole the solution will be reset uh like the all the information that was used from this photo will be forget forgotten uh and you start over again and you can always undo this but due to some blender specifics uh you need to press for every action you're doing you need to press ctrl z and command z on mac twice uh maybe we will somehow improve it in future but at this point you need to press it twice and what else what else yeah another thing that i wanted to show you is our really ex uh the most experimental tool ever we built it was it's not even supported it's it's uh released as an open source add-on for facebuilder you can uh go to github uh to the account of alexander milovsky is the developer the initial developer of facebuilder another blender and he uh we had a kind of a hackathon in our team uh when the isolation started and we decided to have fun a little bit uh so everyone was occupied with some funny thing for a week and alexander made a face recording and now i'll show you what it does it's again it's released on uh under a gpo license it's open sourced you can go there you can improve it you can clone it you can do whatever you want with it and we really are looking forward to the feedback you provide us because we can perhaps make it uh really usable for people who want to bring some animation in blender for now uh how it works uh you go to github you download the zip file and then you go to preferences and add-ons and you install this addon here let's search here it's called kt phase streak master master because it's not from from github and master is a master branch of the repository so it's like it's the latest version and yeah you install this addon you switch it on and what else let's remove all we don't need it we need gleb's head and ah that's yeah like that like that yeah what we have here is this w new tab here and here's a here's an instruction it's not really clear but i i think uh we'll make it more clear maybe some uh just just to make it a little bit more usable but now i will show you how it works how it really works what you need uh let's say for example that we have an animated uh tract face from face tracker now it works only with face track by the way here's our tracked face let's see how it works it strikes a nuke the model was created with facebuilder and the face was tracked with fake face tracker and now we nuke is really slow at playing videos for some reason well it's caching now you can see the guy's singing and here what happens in 3d uh let's see again this is the same model without ears without the neck and what we do is uh sorry for everyone who is not familiar with nuke there's nothing special at this moment don't be scared with this mess it's just this test project we will need this model and we create this right geonote it's basically nope it's basically a node that writes geometry files like 3d models and we specify the path here to uh i already exported but let's export them again like singing abc you want to use abc format because the animation is getting is being turned up transferred better with abc and some we tried it with fbx but uh at least in nuke it doesn't work uh and abc works and so let's say we save it as a singing abc file and we choose a java format hdf works a little bit worse as far as it can remember and we execute execute means that we just save and we save frames for from first to 100 now it's being exported and done now we have this head from facebuilder the neutral head let's see where it is all right it's not here it's here this is the neutral head we will need this too as a reference and let's create another red geo node connect it to face builder choose another abc format here's the format is not that important but it's better to keep the same format just to avoid random glitches again java we don't need cameras uh we don't need we only any geometries in there by the way i didn't switch it off here but that doesn't matter again execute with only need first frame because it's a static model and we come to blender and here we import those files let's import the first mmm where is it it's here first the neutral head it's here let's move it right now we have it here our neutral head and let's import the second one sinking model and it's been translated and it has a camera that we don't need so we remove it nope it wouldn't delete this camera and delete this camera because it has to head to cameras some reason uh yeah we this model is place it in there but we can reset it because academic format allows it here here in this object constraint properties you can turn fluence to zero then go to the item and reset everything everything everything rotation translation to zero then move it let's see here you see uh the sizes are a bit different unfortunately but it doesn't matter at this point in time and alexander said that at this point it's better to switch off eyes and the mouth parts of the model so let's remove them mouth by the way is uh that's a mesh in inside the mouth and where are the eyes we don't know in them yeah now our model is a little bit holy how george from harry potter would say um [Music] yeah we have the animated hat here and we have a static one uh what we do now we go to this top and we select face builder head and generate rig well it's here now we do skinning i don't know why those are two different buttons perhaps it should be one button and nice the skinning was done now we can go into the pose mode and perhaps choose this bone and yeah you can we can see this we can change everything but we don't want to do this so let's get back to the object mode and what we do next is we select neutral animated mesh neutral animated mesh which is probably not a correct wording because here you want to select neutral mesh first and then you need to select the animated mesh and then you press neutral mesh to tell the addon to kind of set up the link for that one which is neutral and which is animated now it's now they are linked in the right way and then you select the animated mesh and then you select the rig of the model and then you press this bake animation which is not exactly baking but or like transferring and what we can see now if we turn on the texture as well we can see where is it the clap is singing like this person was cool now you can export this head an animated head into a file import it into unreal engine anywhere you anywhere you would you like to import it including this uh the bones and uh here's the important button if you want to export it uh you want to press this export scene button but before that you probably want to press clear scene button to remove the uh models that you don't need in this case and then you export scene and it opens the predefined the export window with predefined options so here you just type in the name for example singing clap f wait x here you can use fbx because blender works but and you export the scene and you can then you can import this model into any 3d software you like okay i think that's all for today uh let's let's that's what uh let's see if we have any questions uh and andrew our marketing head is telling me that we have no questions in this time cool perhaps i was so clear that everyone understood everything i hope we are always happy to uh answer your questions since our in our discord channel and see your feedback uh it's really it you you just it's invaluable uh because it helps us to make everything better and you can subscribe to the stream newsletter we will tell you uh it's on the right we will tell you when this next stream happens the next stream will be about face tracker nuke and there are many other like i say there are at least three parts the basics the first replacement and you don't make up maybe we'll come with some real lighting some some advanced techniques yeah i see chris stargazer joined late and now he's been confused and yeah we uh in this in the very end we talked about uh face uh rig addon it's unofficial you can download it on github you can improve it yourself if you like improving things on that are open source it's it's open source uh you can download it from github you can install it in blender and then you can watch the stream again uh it will be like save it here in a recorded version uh in the last part i will tell you again my recorded version will tell you again how to move this uh how to uh operate this experimental done in blender and how to transfer the tracked emotion uh traction expressions uh to the model of in blender like how to animate one model with another model uh that's it uh i see some things and i i really want like thank you too because it's yeah you're the best we have and i think we are going to stop right now right so goodbye everyone

Channel: KeenTools

Views: 8,708

Keywords: facebuilder, keentools, blender, 3d, 3d sculpting, vfx, blender add-on, compositing, blender3d

Id: T41XQpKjY24

Length: 137min 5sec (8225 seconds)

Published: Fri Aug 21 2020