How I put 2D Animated Characters on 3D Sets with BLENDER

want to know how i did something like this where there are 2d characters on a 3d set watch the video to find out more [Music] hey guys it's tini kumantoa and today i'd like to talk about how i did a test of 2d characters composed right into the 3d set first of all no this isn't a thing where the background is pre-rendered and then i do the characters on top of that 3d footage these are actual image planes placed into the 3d environment casting shadows reacting to lighting and having shader and material presets that make 2d animation feel more than just a drawing one of the things i'm known for and why a lot of people did follow me on social media was because of these casual dancing dog animations i did where i placed them and motion track them with live action footage an effect you'd kind of see in a movie like space jam every time i did a new dancing dog loop i would do it with new live action footage but this time i wanted to use a project where i could learn and implement some new media experimentations instead of using live action footage why not use 3d scans i had an iphone 6 for years it was almost just a year ago that i upgraded to a 12 pro did you know that the iphone 12 pro has a lidar camera or scanner this is what allows it to do 3d scans and i guess some advanced facial motion capture i got this idea from the channel corridor crew one of my favorite youtube channels out there i highly recommend you guys to follow them if you're interested in experimentation vfx animation ai stunts film it's a really entertaining fun channel where it's just a bunch of guys experimenting and making stuff something that i just want to do so i visited a park during the day and using my iphone i used an app called scaniverse and this is what allows me to scan my overall environment and turn it into a 3d model the way it works is like you're recording a normal footage but you're gliding your camera over the surfaces of what you're scanning or you're recording and what you'll notice is that it's slowly starting to create a 3d mesh with the actual texture and it'll start filling in empty spaces for that mesh and then once you're done you can save it as a low res or a high-res model using low-res or high-res textures then once that's done and once you're ready you can then export that into a file format to your liking fbx is what i use a lot for production and storyboarding and it's something that blender can also take in too you can save it in your phone or export it into a cloud so you can export it into google drive and then just download it from google drive into your pc there are other 3d scan apps that you can use on the iphone like polycam but scaniverse was the first one i could find and that there wasn't a paywall for the export options scaniverse is also free okay we'll save that stuff for later now i'm going to talk about the 2d animation site which i use tv paint and it's also what i used to do the previous dancing dog loops i just watched a bunch of shuffle dance videos looked at them frame by frame and heavily referenced them for my new dancing dog lube then i exported it into an image sequence i usually use alphachannel png with no background just to character animation with color and cleanup i personally like to have animated image sequences in its own folder with nothing else in there now i know a bunch of you guys will be asking why tv paint and not grease pencil if i'm going to use blender why not use grease pencil the simple reason is that i'm much faster on other 2d animation programs besides grease pencil and i know i'll get a lot of comments saying why not use grease pencil or why i'm not taking advantage of it and i get it grease pencil is nice it's free and it can be powerful it's free like blender personally i'm just not a fan of grease pencil of how it feels and i'm not someone who draws strokes using the z-axis i don't draw things coming in and out of depth i tend to keep things in 2d and also i like the feel of other 2d animation programs besides grease pencil also there are some great advantages when importing your animation as an image sequence such as being able to utilize 3d shaders on your tv animation an example of this would be things like bump mapping or normal maps or specular maps or even material mapping where you assign color codes apply a different type of material to coat it with the color of your animation with grease pencil by default on its own you can't really do that unless you export it into an image sequence and then import it back into an image texture since grease pencil has its own unique shaders but that's only if you want to go extra with the image textures and shaders you could still fully animate and color a character in grease pencil and this effect will work just as fine at this point it's really just a matter of preference the reason why i choose to use tv paint is because it has some great functionality for td animation tvp 11 for example has a great feature called ctg layers and coloring which has changed my approach to coloring and tv paint what it is is that it uses a source for this case the line art of my animation then you can indicate strokes to tell the program where to fill the colors and you can even color code palettes and it does a lot of like ai and calculations so you don't need to do things like gap closer or range or anything like that manually it just calculates it there's tutorials of this on youtube if you're interested in that now it's time for me to put everything together using blender just letting you guys know this isn't a step-by-step tutorial it's just me talking about the process so now we can start importing our 3d scans once you download your 3d scans and it'll be in fbx files you can import them as an fbx and it'll import it as a mesh with its own textures too now this part is super cool you can arrange your 3d scans duplicate them rotate them scale them if you're really ambitious you can apply material shaders onto it but i like to keep things simple i'm only using three different scans a hallway an outside of the building and a pond but i'm duplicating multiplying and arranging them to make a completely new environment something like this can be really awesome for setting a storyboard or layout scene or maybe even prototype level design for video games now the reason i say that is because scans themselves are very chunky and blotchy they're not polished models but personally i find this stuff pretty appealing okay now it's time to add our 2d animation onto the scene you're going to want to have the blender add-on switch on called import and export image as planes you can find this in the add-ons menu just make sure it's switched on this is what allows you to add image planes as an object you'll notice a new option pop up when you add an object there's a thing called add the image as planes find the folder you exported your animation and import it as an animated image sequence so for this case i totally forgot to do that but no worries that can be fixed when you open the shaders window you know find the node with your texture and fix some parameters change it so it's recognizing it as an image sequence make sure you have the total frames match your actual animation file if it's too little the animation will end abruptly too much or maybe too far blender won't be able to find those frames because it doesn't exist it'll just replace it with some weird texture or mesh so if it's a loop make sure you put the exact frame number and turn on cyclic this allows it to loop alright so when you switch your viewport shading options to something that's close to the render now it's going to implement lighting now there's a few things that you're going to notice the character animation looks a bit too dark and the jarring thing is that the shadows don't match with the actual animation it just matches with the plane or the animation itself isn't transparent and actually renders a black background as a replacement and there's ways to fix that on the material settings window you're going to want to switch the shadow mode and the blend mode to alpha clip so now it has no background and the shadows actually match the transparent properties of the original image texture and animation and if you feel that the animation is too dark if you go to your node window you can actually drag the point of the color of the texture towards the emission and then use the emission strength values to crank it up or crank it down if it's too much it'll glow and it won't really react to things like shadows now from here on it's just me playing around with the lighting the world and some textures now you probably noticed that i was looking at the world with flat with you know no lighting or just default lighting but you can switch that with the viewport shading and set it to the rendering shading so you can actually see the world with lighting and some material information for my render engine i'm just using ev cycles is a bit too extra and evie can do the job really well and from the render engine settings i could adjust bloom effects screen space reflection maps motion blur if there's animation and camera moves and such i also changed the properties of the world shader and this allows you to add things like sky boxes sky colors or any other texture that you'd like or a color i even experimented with the texture of the animation itself and i gave it a bit of bump mapping and if you guys don't know what bump mapping is they're basically height maps that instruct the material and how to react to lighting i didn't make a new animated texture like i would do with the nighthound project i just used the same cleanup final color animation that i did for the dancing dog animation and just use that and it's good enough but that's the main advantage of importing 2d animation as image textures you can really take advantage of the 3d shaders that grease pencil by default doesn't have now it's time to make a camera this allows me to do things like depth of field or change the length of the lens and i can even animate them change its properties while it's moving around i even animated the sunlight in the environment special thanks to my friend alan he also has his own channel on blender tutorials that you guys can look up but he was helping me walk through this stuff and he was showing me some great tips when it came to blender showing me stuff like adding modifiers to the camera and all this gizmo stuff on planes and things like that it's just too much to cover in this video now i experimented a lot with different shots basically just duplicating my dog characters putting them in another place in the environment and i animated that texture maybe running down the corridor i have multiple cameras for different shots and they're all happening at the same time i just selected a different camera when i rendered the output for it there were a few cool tips i learned from alan regarding cameras so let's talk about depth of field you can actually set the depth of field so it focuses only on the character so everything behind the character and in front of the character will be blurred out or instead of setting the depth of field properties to the character you can set it to an object known as an empty and if you guys don't know what a null object is i talk about it in after effects it's something that won't appear in the final render but it's great if you want to parent characters on top of the null object so when you move the null object you're moving the character to without affecting the character it works the same way in blender so i can use this empty object to determine the depth of field which is highly useful for things like rack focus and that's just it there's a lot of possibilities that are quite endless like i was being shown on how to do a quick water effect using a few shaders that create that cheap water illusion and honestly i found something small like a character's reflection being shown on the water while the character is 2d animated mind-blowing to me while reflecting the rest of the environment and this is the type of automation that i like to see when it comes to marrying 2d animation and 3d graphics i experimented with a lot of random stuff maybe another world was falling into the scene while the camera kept rack focusing back and forth from the background falling into the scene with our dancing character maybe i could duplicate all my dancing dog characters on one plane of the world and i could duplicate all of that and have it flip upside down and now that world is falling into the main world and while this is happening i'm adjusting the focus and i'm adjusting the length of the lens so it looks like it's foreshortening and things are getting crazy and it's all super exciting and it just makes me realize that blender can also be a great compositing tool for 2d animation and 2d assets now like i said all this stuff is taking place at the same time all these different shots are happening at the same exact time i just happen to have multiple cameras for each of these shots but when i render them i'm basically rendering each camera one by one now for the render engine like i said before i use ev because i still think it looks great and i don't need anything more than that but if you really want to bust your computer's nut you can try and use cycles which is basically ray tracing at that point and if your computer is not strong enough it'll take forever to render you can render them as image sequences but for something like this that's pretty small and nothing too serious i just rendered them as videos you can even change the directory output the format the size the quality in the video codec you can even automatically do this by pressing shift and f12 or by clicking on render animation under the render tab but before you hit that render button make sure you have the camera selected you want to render as for the final color adjustment and compositing and adding things like noise vignettes or posterizing effects i just use after effects adjust the color curves the color settings the the color strength and apply other post effects so that's about it i had a lot of fun doing this and i'm glad i recorded and documented myself doing this since since there was a lot of things and tips and tricks that my friend showed me and things that i was experimenting with that i might soon forget that's why you should record and document every project that you do if you guys were inspired to do something like this because of this video give me a shout out when you do share experimentations and i'd love to see them and share them on social media if you guys want to see a step-by-step tutorial of this let me know in the comments down below anyways thanks so much for watching this bye interested in learning hand-drawn animation or learning how to finish an animated shot from beginning to end have a look at the store where you'll find the complete introduction to 2d animation video course tutorials and other resources learn classical animation approaches drawing lectures techniques and other process videos visit the store through the link in the description below

Channel: Toniko Pantoja

Views: 136,889

Keywords:

Id: _8H0rAHym0k

Length: 14min 45sec (885 seconds)

Published: Tue Sep 14 2021