How to create an AI music video (FULL WALKTHROUGH)

hello my friends Nico here from newer frames I've been building this platform to create videos from text is particularly suited to create music videos and today I would like to show you how to create a video like this one [Music] we will go through all the settings in under 10 minutes let's go the first decision we have to make when creating a new project is which AI model we want to choose neural frames has a total number of six so-called standard models those are popular AI models trained on specific use cases for instance we have three all-rounder models and can depict anything you would ever want and then we have three Specialists for instance for realistic vision for analog photography and for comics and mangas new frames also has the ability to train custom moddeds here you can train an AI model on yourself or on any other object this can look like this [Music] but today we would like to pick a standard model for instance I really like the dream shaper model at the moment so let's select this one next up we have options to either upload an image which serves as a starting frame for the video or we create the first image ourselves in this case this is what I want to do this brings us to the first frame editor here we can type in some text prompt which describes what we want to see my motivation for this music video is to show the evolution of humankind so we could show prehistoric cave with fire something like that we have a powerful button called Pimp My prompt that uses AI techniques to enhance the prompt to describe it a bit better for the AI model then we can choose the format of the image in this case I would like to use 16 to 9 and we click on render which brings us here we had four images to choose from which is the starting frame of the video and let's say we like this one now we pick that and we are in the video editor here the video editor consists of three elements the the bottom element is the timeline timeline consists of three parts one is for the prompt inputs one is for modulation if we wanted and one is from music if we want it then we have the preview window here and on the top left we see just the settings in January now we would like to add a song actually so I will do that double click on the audio timeline click agree and add a song here and what newframes does now is extracting the stems of the song so you get the individual elements of the song for instance the snare or the kick drum and the Hoy song by the way sounds like this [Music] what we could do now is is just render The Prompt that we already have with some settings we we have certain type of um trippiness settings that you can see here also we have movement settings that we can select and we also have a pro mode where we we can influence the individual settings we could just go ahead and render like this and what we get then is this year [Music] but since we want to make a music video we can go one step further and add some modulation based on some element of the song in this case I like to add a modulation of the snare which is always a cool effect I would recommend picking either the kick drum the snare or the hi-hats for modulation but of course you can also pick the other elements so we can either double click on the modulation timeline or we can click here and and get the snare element here right away we can also make this larger right it says now that for this audio Source we recommend to use a lower smooth value to understand this comment let's talk about the two important parameters that are here strength and smooth so the way these videos are generated is each image is fed into a newer Network called stable diffusion to generate a new image and the the magnitude for how much the new image will differ from the OED image is called the strength so a high strength we form a very different image out of the old one while the low strength will stick very much to the oils image this can be seen for instance here in this video where you can see the strength being passed through and see that the images become more and more flickery sometimes you actually want it very flickery for instance when you change the prompt you change something or you don't like the image that you actually have then you want to introduce a high strength to get the new network off the old image that you didn't like and generate a new image from that now the other important parameter is the smooth between two neuro Network outputs we actually interpolate the images and and the smooth is a magnitude for how much we interpolate between them I have an example image here for instance a smooth value of 30 we introduce 30 images between two neural network outputs making it much smoother but also the image quality suffers a little bit from it and you can see a lot smooth actually makes the images much more flickery see it again so the the comment that we saw here for the modulation that it recommends us to use a low smooth value means if we want to modulate the strength based on the snare and the snares just the clap right it's a short amount of time if you have a high smooth they might actually be not a newer Network output on the hit of the snare because it might be a smoothing frame and then you won't actually get a modulation of the strength so my recommendation rule one if you use modulation use a low smooth value for instance two or one okay if you're not in the Pro settings just use trippy that's also fine it's the same okay this is very important if you use modulation based on one of the Rhythm elements use the low smooth value otherwise you'll be very disappointed and I really don't like disappointment of my users by the way the one last thing I would also not recommend you to change the smooth value um Mid video this usually looks a bit weird and also can lead to some weird behaviors choose a smooth value for the video and just keep it the way it is cool so I would like to start without camera movement so I will enter zeros here what I will do now is I I will keep a relatively low strength something like 0.3 and because I have the modulation of the strength on the snare the modulation of the strength will actually reach higher values right you can see it here it will reach at the peak something like 0.6 of the strength battery which is already a reasonably high value we can also go a little bit higher but for now I would like to try it like this one so the strength will be low most of the time and then the snare will come in this the strength will go up Suddenly Okay cool so we have the first prompt I will add another one maybe um prehistoric caveman sitting by a fire Click put my prompt we get something out nice add another prompt something like prehistoric cavemen fighting neanderthals cool add something else early humans from early agriculture cool right so maybe we can also start with some movement here he already slide zoom in maybe keep it here also all right and then we click on render and see if we like the results or not now newer frames at any point in time we can always watch what we created [Music] and if we didn't like something we could also change something here and then re-render from there but actually I think what we created here is already fantastic so we don't need to do that what I'm going to do now is add more prompts until the end of the song right so here we haven't told something like 80 seconds so I will do that and then I show you the end of the video the final video foreign [Music] [Music] foreign [Music] foreign [Music]

Channel: neural frames

Views: 30,570

Keywords:

Id: gY9B9x4Ku7Y

Length: 10min 34sec (634 seconds)

Published: Sun Jul 23 2023