At the very end of my last video I showed a frog
cyborg music video and I ended up getting a lot of comments about it so I decided to make a
tutorial on the whole process of creating it It was done using four completely
free AI tools but there's other tools that could be incorporated into
the process so I'm also going to make a second music video with a different
song and a different set of tools and then a third one with another tool So this should be a good comparison of the
different tools and demonstrate all the options to choose from then you can decide what will
work best for whatever it is you want to create The main tool in common across
these is Suno to create the songs It generates the music and vocals which is the
most mind-blowing part of this whole process to me I generated everything in Discord in
the last video but they released a web version recently which makes it much more
user friendly so I'll use that this time It's pretty straightforward once
you're in here in the prompt box just write the genre and topic you
want it to generate that will have ChatGPT write the lyrics for you or you can
switch it to custom to use your own lyrics I'll use the same Frog Rock concept then it
generates two options in just a few seconds These come back up to 80 Seconds long
now so I'll just play parts of these [Music] I'm into it but it's a little more experimental
than I'm looking for I wasn't very specific with my prompt you can guide it a lot more
than I did but let's try number two [Music] That's what I'm looking for and over
here you can either re-roll it with the same prompt or if you like the song
you can continue it that will pick up right where it left off then you can add
more lyrics like a second verse and chorus but I want a guitar solo so I'll put that in the
lyric box and the style let's check that one out [Music] That doesn't really sound like a guitar but the
drums with the double bass pedal are awesome So that's the process then
you just re-roll and generate different variations and continuations
until you have a full song you like For the music video we need
to start with some images The free method is with playground
AI if you watched my last video I did cover this you can jump to the
next chapter if you'd like I use a completely different combination
of tools for the next music videos Here's the generation box I'll just
drag the corners for a 16x9 aspect ratio frog playing a guitar for the prompt and
I'll generate four options then over on the left I'll select a style neon mecha
gives the futuristic look I'm going for it generates pretty quick and that looks great I also need a singer and a drummer that made it seem really easy but in reality I generate a ton of these
and then pick the best ones and that neon Mecca style looks
awesome for this there's a ton of other really cool styles in here
depending on what you're going for Next up we can animate these
for free with Pika Labs Pika 1.0 is the web version that just came out
it has a big jump in quality and user experience it is still on a wait list you can sign up
for it at pika.art they are letting people in a lot more regularly now if you don't have
access you can use it in Discord until you do So I'll upload the image here then
add a basic prompt of frog playing guitar down here there's options for
aspect ratio and frames per second then we have camera controls then negative prompting which is really
helpful to get better generations I'll do just a standard copy paste
here and then run it with that right that looks solid but there is
some weirdness to it so just like with the images I re-roll it a bunch
of times and pick the best options with the web version the organization
is so much better all your re-rolls pop up right here underneath and
once there's a good generation you can extend it beyond 3 seconds if
you'd like it's another big upgrade you also have the option to inpaint
where you can just change specific areas of your video and a bunch of other
features it's a big step up from Discord and I know they will be coming
out with a paid tier for this but I don't know any of the details about that yet The most difficult part is
editing this all together well this one's actually the most complicated to edit
out of all three of these the final one doesn't even require any editing at all you'd think this
cartoony frog one would have been the easy one since this will vary based on what you're
creating and what software you're using I'm not going to do a step by step I'll just cover
the main issues I ran into and how I solved them This is the timeline in Premiere Pro the first issue was lips syncing,
the common lip syncing tools don't work for characters like these.
I'll go over them for the second song because I use people in that video,
but for this one I had to get creative I took the best generations of the singer's
mouth moving and played it forward and in reverse at varying speeds to sync it up the
best I could and that was pretty tedious Let me know if you have a better
way to do that I would love to know The other issue is I could not get any
generations to go well that involved the full band. I'd only get one
of them looking decent at a time so to solve that I took an image of the
full band then cropped a version of each character and animated them separately,
then I layered them on top of each other with masks you can notice where they
interact in some areas if you look close those are the two main issues I ran into which
may or may not apply to whatever you're making other than that it was fairly basic
editing just some zooms and picking the right clips and syncing it up to the song then I exported the full video
which was in 720p then CapCut has a free 2x video upscaler under magic tools so I'll drop this in there and
run it which ups the quality a bit I'll play this one again before we jump into
the next song this is the same video from last time so all the clips were generated
in Discord before Pika 1.0 came out first one more thing we need is an an
album cover let's see what Chat GPT can do That is perfect. It didn't get the
text quite right that's an easy fix but more importantly it's missing
the umlaut. I can't let that slide There we go now we can listen [Music] Honestly I love that song it
gets stuck in my head every time Let's jump into the next one. In the last
video I used the lyrics from Purple Haze with a random genre in Suno and the
song it came back with was amazing it's crazy the amount of emotion
that came out in the vocals [Music] it's weird to call it emotion in an AI
voice but that's what it sounds like I just extended the song a few times picked
the best options and spliced it together For the images I need a good female vocalist and I want it to be more realistic this time so
Midjourney is going to be the best option I have a channel in my Midjourney server
where I enter random cool prompts I come across this is one I saw Heather Cooper
share on X. She's a great follow, she shares awesome Midjourney stuff all the time but purple is one of the main colors and it's
a little psychedelic, I think that will fit the vibe of the song perfectly so I will copy
that and then run it in a different channel and that looks solid so I'm going
to run that a bunch of times and test out some different variations on the prompt alright, instead of having you sit through
me generating images for an hour basically what I did was run the prompt a lot and then
upscale options I thought would work well from there I'd used subtle and strong
variations to fine-tune the results I also used zoom out on some of them so I just did that a bunch until I ended up with some images I really like then I
did a 2X upscale on my favorites I also used the same base prompt but
for someone playing guitar and also for a man because there's a male voice
that comes in later on in the song this is what all that looks like on the Midjourney site with all the images laid
out right next to each other This is one of my favorites of all these
so I'm going to animate the singing I'll be using HeyGen, it's a paid
tool that's really easy to use but there are free tools if you want to learn them wav2lip and sad talker are the best I've seen here's a comparison of those hello I am Eve the lead actress of fictions dot ai hello I am Eve the lead actress of fictions dot ai hello I am Eve the lead actress of fictions dot ai there is a learning curve for those
and it runs fairly slow unless you have a capable computer and can run it locally so I'm doing this the easy way with HeyGen I just click create video select Avatar and go to Photo Avatar then upload the image here
then upload the audio and click submit Easy as that and it will
finish in a couple minutes here's a glimpse of what that looks like l have another image I'm going to
switch to throughout the video and also a male vocalist so I'll do
that exact same process for those now I want to animate the guitar images I had, also the zoomed out vocalist was a little boring
so I want to animate the background of that Since I showed Pika Labs earlier
I'm going to use RunwayML this time they have a bunch of tools but I'll be
using Gen 2 for this which is right up top I'll start with an image, I have a
few I want to test out to see what the animations look like but this
was my favorite so I'll start there and I want to add a prompt,
I'll just keep it simple and this motion brush feature is really
cool where you can select which part of your image you want to animate and choose how
it will move. I don't think that will work well for this one so I'll just adjust the motion
slider and test out a few different numbers the camera motion tools here are great as well.
The visual for how it will move is really helpful but I'm just going to leave it as is and add
a little motion later in Premiere if I need it That's the process. I'll generate
a bunch of stuff and then come back The hand was the biggest struggle for sure, but
the waves looked amazing on a couple of these this other one had a couple good
generations too. I extended one and it did this sort of dissolving
effect I thought was pretty cool Some of the other guitar images I
just couldn't get to work out well I also animated the that same image
of the woman and I'm planning to layer it with the one that's singing and
I'll just add a mask to combine them and that should be everything I need The editing for this one is pretty
straightforward, the main part is I wanted to merge some of the singing clips from
HeyGen with the animated versions from Runway for that I'll just layer them on top of each other and then add a mask around the
face with a lot of feathering and then I'll just do that same thing
with a couple of the other clips so nothing too difficult there, much
easier than editing the last one and the last thing that I already
did was if you can see where these audio tracks are stacked here, I added a harmony to do that I isolated the vocals from
one of the generations I ended up not using. I used a software called
Ultimate Vocal Remover. it's completely free and open source and
you can download it to your computer I'll go over this part quick you
just add the song for the input select where to Output then switch process method to
MDX-net if it's not already then select the model the best one for this is Kim
Vocal 2. it won't be here your first time you open it, so you'll click download more models, select
MDX-net -then select Kim Vocal 2 from the dropdown I only need the vocals for this one but leave
this unchecked if you also want the instrumental everything else can stay the
same and start processing this is a short clip so it's done pretty fast and now here's the vocals [Music] then this is the before [Music] here's the after [Music] I did that in a couple parts and that was it This is the song, the vocals in particular,
that I've been most blown away by from Suno. And this is the worst it will ever be imagine
what these tools will be like in another year [Music] I'm going to do one more, so back to Suno I've been messing around with some different
genres, this one had some cool results groovy neo-soul song then we'll say about the singularity all right and I ran that a few more times
this Echoes of Infinity was my favorite [Music] to animate this I'm going to use Kaiber Kaiber is great for a lot of things,
especially with music. some big artists like Linkin Park and Kid Cudi have
used it in some of their music videos I use it most for things that I want
to look like dreamlike or psychedelic I have an image of this cybernetic woman I'm going
to use as an input and I'll be using flip book which gives it that kind of flickering effect
because it generates each frame independently I like that look they have an option for
smooth animation as well if you use motion here's a quick example of that
from the image I'll be using then the transform option is for adding
a style to an existing video you have But I'm going with flip book here I'll add the image and the
audio, then move on to the prompt cybernetic woman transforming into cityscape then they have a bunch of
curated styles to choose from, I'll go with photorealistic then it
adds a bunch stuff to the prompt here now on to the video settings for camera movement I'm going
to have it zoom in the entire time then I'll change the directions periodically and I always uncheck the show uploaded
image in the first frame otherwise it's usually a weird transition right at the beginning we'll use evolve at five and I'll
switch the audio reactivity to high then Kaiber will add little zooms or
speed changes and sync it up to the beat so I'll max that out and then generate previews I'll go with this one on the
top right that looks great now I want to transform this into different scenes
and in different directions throughout the song to do that I can add a scene for the next one we'll say
robots merging into cityscape then video settings and I already found
points in the song where it will be good to switch at basically just when a line ends or
the music changed the first spot was at 8 seconds and I'll add down to the camera movement then click generate previews, it
won't give options this time then I'll just repeat that a few more
times, so add scene, I'll leave robots merging into cityscape but I'll add the left
to the camera movement and start it at 16 then just keep it going, robots transforming into cybernetic woman switch it
to right and at 23 seconds now merging with the singularity, this time I'll take off down and right and
then we'll do rotate clockwise, that's at 31 then the last one we'll say
swirling cybernetic singularity I'll bump up the evolve and
that starts at 40 seconds and that's it now create video, this is a longer one
so it'll probably take 10 to 20 minutes and easy as that ,it's all done they have the option to upscale to
4K right on the site if you want to I'll just play this exactly how it came
out no additional editing or anything [Music] I had a ton of fun making these, I hope that helps you create something awesome If you want to stay up to date on the latest and greatest ai tool, check out futurepedia.io You'll find exclusive deals all over the site, and it's designed to be easy to navigate. You can narrow things down by category, then subcategory You can easily find the exact tool for your needs, and create a profile to save your favorites
