Using A.I to Convert Real Footage To Animation | Computer Generated Cartoonization

[Music] hi everyone it's been a while since i've done any ai related videos and i'm back today with something which is quite fun to play with we're going to combine two powerful ai software to turn real live videos into anime or cartoon style and no you don't need to have any digital drawing skills or spend hours stylizing your footage frame by frame so let's get started the first piece of software you'll need is called eb synth which is entirely free and mainly developed to convert video footage into stylized mainstream paintings but it's also commonly used to turn real-life videos into other styles i've even used it for digital de-aging before and i have a whole different video on that as input eb synth requires the individual frames of the source video in standard image file formats and to apply the desired style eb synth requires at least one stylized keyframe as a reference the common practice is to digitally paint the style reference keyframe on top of the original one but soon in this video i'm gonna show you another ai tool that you can use to auto generate some really cool styles first of all let's convert our video to an image sequence i'm going to use adobe media encoder for this but you can also use pretty much any editing software change the output format to jpeg choose an export location to keep things organized i'll create a new project folder inside which let's create another folder i'm going to call this one original frames go inside and hit save to start exporting the individual frames and as you can see each frame is labeled with a number in my case starting with 1000 and ending with 1236 this is very important so ebsync can identify the images as a sequence later on next you need to choose one or more frames to stylize the rule of thumb here is to pick a frame where the main subjects are clear and visible and to select additional frames whenever there is an obvious change of perspective or new objects come into the frame there isn't much of that in my clip here the very first one features pretty much everything in my scene so let's go with that once you select the frames you're gonna go ahead and copy them over to your smartphone it's important to keep note of the file name as we're gonna need to use that later now i'm moving over to my smartphone because i want to take advantage of a really cool photo editing app called prequel that has all kinds of filters and styles that can be used to transform images and although i have tried other computer based solutions such as photoshop's neural filters i couldn't find any styles that are as good as the ones that prequel has so let's open the app tap on the plus button at the bottom here to import your image and then you'll see down below all these different filters to choose from but the only ones i'm interested in today are the ai filters there's this cartoon filter which i think looks pretty cool but the one i like even more and is closer to an anime like style is this filter called arcane i was really excited when i saw it and knew that this is the one i want to try on videos i don't really need the dust particles here so let's get rid of that i'll also make some quick exposure brightness and contrast adjustments once you're done hit on next and save photo bring the exported image back to your computer i'll create another folder right next to my original frames let's call this one keyframes and paste the newly stylized image in rename it back to the original frame name and now let's launch eb synth to begin let's click here to point to our reference keyframe and hit open next for the video input let's browse to our original frames folder and select the very first frame as you can see eb synth has identified the keyframe number and the last frame to stylize our keyframe happens to be the same as our first frame and that's why this stop field remains blank eb7 has also generated a new path where the new stylized frames will be saved so we don't have to worry about that now let's go to advanced and change the synthesis detail to high notice that you can also add more segments here this is useful for cases when you're using multiple keyframes we don't need that here so let's close it and hit run all if you start the process and this error window pops up no worries all you need to do is open the stylized keyframe in photoshop go to image image size and notice that the image height says 1079 while our original frame size is 1920x1080 this is clearly an issue with prequels export simply change this to the original height of your video and then hit ok finally save the image using maximum quality settings and now the process should work just fine without any errors the process of stylizing the individual frames should take a couple of minutes depending on your hardware and the video input as well once we have the new frames exported we can go ahead and stitch them back together to convert them into a video i'll be using adobe premiere for that right click here and choose import find the output folder select the very first frame make sure image sequence is enabled and click on open next right click on the imported sequence go to modify interpret footage and change the frame rate to match it back to the original video next drag and drop the video here to create a new sequence this is where you can also restore the audio from your original clip if you want otherwise go to export choose a file name adjust your export settings if needed and then finally click on export let's have a look at the video the outcome looks pretty cool already but what i like to do here this is completely optional is to run the animated video through video enhance ai not only to improve its quality but to also get rid of some artifacts i'm okay with default settings and up skating to 4k sounds pretty good to me so let's go with that and let video enhance do its magic i believe that after this step the animation looks much smoother and cartoon like it's a small thing but makes so much difference in my opinion now considering the amount of time and effort that this kind of transformation used to take before i truly believe that the result we get from this method is pretty impressive and i can see it being used in many other creative ways now i have to be clear that this method struggles when there's so much movement or new objects are introduced in the footage in those cases it might help to use additional keyframes and spend a bit more time merging the outputs but the great news is that this technology is continuously improving and will get faster and easier with time especially with future updates and the introduction of newer and more powerful ai resources i'd also like to mention that i came across a cloud-based solution that is developed for a similar purpose and meant to work on both images and video files right now it only provides one style and i still prefer the ones that i can get on prequel in many cases it ended up scaling my image down to half its original resolution for some reason and when it comes to video you're limited to 30 megabytes for the file size and when i tried it with different clips it didn't even work so let me know if you're familiar with any other tools that work well on videos and i'll happily test them out if you're interested in learning more about the power of ai in post-production check this video out other than that don't forget to subscribe so i can see you in the next video peace

Channel: MDMZ

Views: 842,009

Keywords: cartoonize yourself, turn video to animation, video to anime converter, automatic animation software, ebsynth tutorial, how to turn real video into cartoon, Live Action to Animation, video to painting, cartoon effect on, cartoon effect on video, animation effect for video, how to turn video into animation, ai animation video, ai generate animation, artificial intelligence animation

Id: n5DZFbQTOus

Length: 8min 16sec (496 seconds)

Published: Sat Jun 04 2022