Generate 3D short films from 2D AI images!

in this video we will travel to the stops using the power of artificial intelligence we will explore a never-before-seen world learn about a foreign culture and gain insight into their unique cuisine on the way I will show you the tools that I use to create this animated sightseeing tour to an AI generated world and how you can use them to create your own short films but first we need a script and of course that will also be AI generated I always dreamed about human space travel discovering new planets and imagining what cities on other planets might look like so I had this idea for a universe where this is already long possible and there's a travel agency offering trips to these foreign worlds so let's use chechi PT to create a little advertisement video for one of their most popular destinations so first I prompted jgbt by saying you are a copywriter for a travel company that offers trips to distant planets and galaxies and I got a swift response sure how can I help you with your copywriting needs write a text promoting the city on Mars as a travel destination include the sites and special attractions of the city and recommend the best restaurants and what I got was absolutely perfect I used it word by word for my final video no changes necessary I also asked chechi PT to write detailed descriptions about all the places it just mentioned to use as a prompt in an AI image generator called mid-journey I copied over the descriptions of the different locations and tried out different styles my goal was to create something that looks close to realistic or high quality 3D animation so I used keywords like cinematic artworks photography or octane rendering and the results were genuinely impressive and I had a lot of fun generating these it really felt like exploration like I was discovering these new worlds and actively shaping them by choosing the images I like most and developing them further the only problem was when I had two concrete ideas of what the city should look like and I wanted to bring too many visual elements together for example I had asked chechi BT water City on Mars would probably look like and I got the answer that it would be half underground and made out of stone and Martian soil to be protected against the radiation so I wanted to generate an image with a city that is underground on Mars in a lack of a volcanic cave or something with a lake in the middle and a nice Skylight and I gave up after a few attempts of course you could always use Photoshop to merge different images together Photoshop actually has some great AI assisted tools to help you with that but my goal was to create as much as possible in mid-journey so now that we have all our images we can start generating the 3D models out of them and last time we used 3dphoto.io a website where you can upload any image that you want and it will generate a depth map a depth map is basically just an image that tells where the pixels are in space so the brighter parts are closer to the camera and the darker parts are further away so in blender you can use this Dev map to drive a displacement modifier and that will push your image apart based on these values the only problem is that the camera is very limited where it can go because once you start moving it too much you can see all the stretched areas and see that there's nothing behind the objects there's just a stretched mess but this time we're using a tool that can not only generate high resolution depth maps it can also generate 3D in painted models and 3D videos the tool is an extension for the automatic 1111l stable diffusion web UI and that can be a bit tricky to install but I put in all the links in the video description plus an amazing tutorial by Olivia zaricus who will walk you through all the steps but once you get stable diffusion running on your computer the installation of the high resolution depth map extension is really easy you just need to come down here on their GitHub page to automatic installation copy this link go to stable diffusion hit extensions install from URL put it there and hit install and you're ready to go so once it's installed you'll find this depth button here in your stable diffusion and this page is already going to need it's super simple to use just upload the image that you want to use and by the way it doesn't have to be an AI generated image you can also use photos or your own artwork and just look at how much better it is compared to the last technique but now you're saying wait this is exactly the same like what you did last video it's just higher resolution and better no because this program lets you do much more than that we can for example generate a stereo side-by-side image to admire your creations and mind-blowing 3D using those old 3D glasses that I don't have or you can use the cross eye technique wow 3D okay that was not what I wanted to show you it's this button down here generate 3D in painted mesh and we can also generate four demo videos so when we activate those and hit generate again it will take a really long time okay I am back with a fresh cup of coffee and a finished 3D model and before we import that in blender let's check out one of the demo videos so this tool not only generated a 3D model it also impainted the parts that were hidden behind the model so now we are way more flexible with how we can move the camera and blender but maybe you don't want to use blender maybe this is enough for you you can just generate all the videos for your short film that you need inside of this program and just use them but it's even more amazing what you can do with this in painted model in blender so let me show you how to import it first thing we're going to do is delete everything hit import ply and then we need to go to the output folder of stable diffusion and import our 3D model and this could also take a few minutes depending on your machine because the models can be pretty big so once it's done you are greeted by this and you're probably thinking oh no it didn't work this looks like garbage I can do everything again but it actually did work what you need to do is just create a new camera and put everything at zero then we need to rotate the camera up because the scene is oriented upwards and then we can just select everything and rotate it around to just make navigating it a little bit easier so when we zoom in here we can actually see oh there's our house and if we look through the camera we can see the house and all the geometry in the right place we can change the focal length a little bit to get everything in frame but how do we get our color back just go to shading click on the 3D model click on new Shader and if you don't want to put new lights in the scene and have them interact with the 3D model you can use an emission Shader add a new color attribute and plug that into color and now we have our color bag and the scene is ready to animate so now that you know how to set up a shot like this let me just show you what you can do with it with some examples from the city on Mars short film so for this shot for example I did a few things I of course animated the camera to highlight the 3d effect I added volumetric lighting I also animated the depth of field to reveal the background as we're moving in and I also wanted the camera to come out of a cave but there was no cave in the original picture so what I just did is I cut out a piece of rock from this side of the 3D model and copied and pasted it over here so yeah that's the thing I did in a few scenes for example in this one I took the the rock structure that was originally further back in the image copied it and brought it to the front of the picture to increase this 3d effect also in this shot I wanted to be even more flexible with how I can move the camera so in Photoshop I painted out the tower and put the background on an image plane that I could then move way back into the distance so this is a shot of a dish shift in one of the restaurants on Mars and what I did here is create a really strong depth of field effect to highlight the scale and I also added a new light to the scene so we can see all the textures of the food as it as the light moves around this is a shot I didn't end up using but it really highlights what you can do with volumetric lighting and just a slight camera move to bring a single image to life so I didn't do too much to this shot I added this slide camera move then what I did is when I exported it I also rendered out the sequence as a z-depth part so every frame also has a depth map and I could then use this depth map in After Effects to mask out different parts of the image so I could easily add dust effects and just bring the image to life with some stock footage so the last thing I needed to tie everything together was the voice over and this time I wanted to try out play HT so I quickly browsed through all their voices and I decided to go with Larry I was really happy with Larry's voice but Adobe just released a new AI tool called Adobe podcast that lets you use AI to enhance audio so for example this is the audio coming straight out of my phone and this is the audio coming straight out of my phone after being enhanced by Adobe podcast so I thought what if I paste the AI generated audio from play HT into Adobe podcast will it actually make it better one of the top sites to see is the Martian Space Center where you can learn about the history of space exploration and see exhibits on the technology used for the first human Mission to Mars I think it did and I don't know how to really describe it but the AI generated voice had like kind of the effect as if someone is talking into a can or something and Adobe podcast pretty much removed that so that's a really cool workflow for natural sounding voices so the final thing left to do for me was just to put everything together in Premiere and render it out join me on a little vacation in the city on Mars [Music] are you ready for an out of this world Adventure look no further than the city on Mars this futuristic Metropolis offers Travelers the unique opportunity to experience life on another planet one of the top sites to see is the Martian Space Center where you can learn about the history of space exploration and see exhibits on the technology used for the first human Mission to Mars the city also boasts stunning architecture including the Towering Martian Tower a skyscraper made entirely of red martian sandstone for outdoor enthusiasts the Martian Landscapes National Park offers breathtaking views of the planet's rugged terrain and unique rock formations take a hike or take a ride on a martian Rover and immerse yourself in the natural beauty of this alien world when it comes to dining Mars offers a diverse selection of restaurants for a truly Martian experience try the Martian Delicacies restaurant where you can taste the unique flavors of Martian grown fruits and vegetables for a more Familiar Taste head to the Earthling Eatery which serves up dishes inspired by Earth's cuisine [Music] don't miss out on the opportunity to visit one of the most exotic and exciting travel destinations in the universe book your trip to the city on Mars today [Music] I hope you enjoyed this little trip to the city on Mars if you did please leave a like And subscribe and also if I was able to inspire you to use some of these techniques for a film for example please send me a link I would love to see it see you next time

Channel: Mickmumpitz

Views: 219,504

Keywords:

Id: PFybvdKH8hc

Length: 11min 29sec (689 seconds)

Published: Thu Feb 02 2023