[Drum roll] Greetings, children!
Captain Disillusion here. Visual effects are everywhere. Even in things that are
the opposite of visual, like… mu-u-sic. According to research, over a dozen people on Earth are practitioners of
this strange art form, emitting rhythmic noises
from their mouths… and other things,
to create pleasing soundscapes. Let's listen to one now! [song playing] -[electric crackle]
-[thunderous blast] Gah! Sorry! Ah, sometimes the
companies responsible for spreading awareness of
the music don't like it when it's… played. Now, because a song involves
the lesser sense, it often needs to be accompanied by
a video to be detectable at all. And these "musical videos"
have historically been a breeding ground
for VFX innovation. Once in a while, you guys ask me
how'd they do "Star Guitar" with all the stuff in
the window passing by matching to the beat? Well, that's actually
a great one. See, if you look closely, a smokestack-- -[electric crackle]
-[thunderous blast] What a chilling effect! [clears throat] Some of the most iconic
music videos experiment with old
effects techniques or use the latest technology in unexpected ways. Think of the artsy rotoscoping
in Ah-ha's "Take on Me" -[electric crackle]
-[thunderous blast] or the grungy projections in "Dead Leaves and
the Dirty Ground" by the White Stripes. -[electric crackle]
-[thunderous blast] Or the incredible
stretching sets in "HomePod - Welcome Home" directed by Spike Jonze
for Apple, featuring FKA Twigs, set to the song "Til It's Over"
written by Michael Uzowuru and Jeff Kleinman, performed
by Anderson .Paak. -[electric crackle]
-[3 thunderous blasts] Dammit! Isn't anyone in the music
industry chill anymore? Hmm… At-a-ra-shii Gak-ko? -Ahhh!
-[Captain] What the... [speaking Japanese] -Hah!
-[Captain] Argh. Okay. I'm just gonna adjust
my translator and try again. [various beeps] [clears throat] I'm Captain Disillusion. I teach people about
visual effects and stuff. The viewers out there. -[all] Oooh? [speaking Japanese] And…you are? [speaking Japanese] Students, huh? With your own songs
and dance routines? Mmm. Hmm. How would you guys like
to earn some extra credit on a special assignment? -Mmmmm! [speaking Japanese] ["Toumei Boy" playing] To tackle a topic this big,
we gotta have a system. There are countless music videos
but they tend to follow trends. If someone experiments
with datamoshing, someone else is gonna do it too. So, lots of videos can
be grouped by a few tried-and-true
visual conventions, like… Skipping time through
messy little morphs Attaching a panoramic camera
to the artist Abstract randomness Capturing everything in
one long take, or making it look like you did Shooting things in reverse Cloning and layering people
and objects using a motion control
camera rig Using long takes
AND reverse motion Using long takes
and motion control Using all three together! Hmm, I can't help noticing that
this combo approach tends to yield better,
more memorable videos. Maybe… observing what works
in other people's stuff and then combining
those elements in new ways, with some personal flare,
is the key to creativity. Okay, now this is the studio and -I want you to be real careful--
-Whaaa! Don't...don't-- ["Koi Geba" playing] Atarashii Gakko are… special and have been making fans
geek out on their unique brand of J-pop
for a bit longer than it takes to graduate
high school. Suzuka is the leader. Also Rin's a leader. So is Kanon. And Mizyu is too. They're all class leaders!
Don't you get it? Now, personally I can't relate
to being a costumed character, stuck in time, entertaining
people with a niche skill. But I'll use the power
of imagination to direct AG in real spec VFX shots
for three songs from their recent EP, SNACKTIME. If I could just come up
with some ideas ["Free Your Mind" playing] "Free Your Mind" is a classic
rock & roll-inspired dance tune clearly designed to coax zoomers
out of depression. It makes me think of...
jump-kicking a kite, taking over the streets,
being in charge, and constructing
your own reality. The kinetic mayhem
of OK Go stunts somehow captures that energy. At least, I like a DIY vibe
in a warehouse. And there was that cool
'90s video for the song "Virtual Insanity"
by Jamiroquai, where the floor slides around
in trippy ways. I wanna try something like that. Now, we can't build
an entire movable room on casters like they did. All we have at our disposal
is the help of a few interns as well as a modest 70-foot long
green screen cyclorama. And I'm sure these green
uwabaki are gonna be... fine. But just like in that video,
I'll move the camera instead of the subjects
to create the illusion, using a handheld gimbal and
a super high-tech dolly. We'll tweak
the usual choreography and shoot it
in three separate passes. First, a static master
with just Suzuka and Mizyu. What's great is,
some of the girls are free during every take. In Japan, students must clean
their own classroom. On my set, they'll have
to be dolly grips. ["Free Your Mind"
playing on set] For Rin's pass,
keeping the camera at the same exact height,
we truck around in an L-shape while she stands
still, then resumes the dance. [music playback stops] -[Captain] So strong. -[Susuka] Yeah
-[Mizyu laughing] So strong -[Susuka] Very strong girls!
-[All laugh] Stronger than Joey,
Joey was struggling. -[Joey] Right?
-[All laughing] For Kanon's pass we do the same
thing in the opposite direction. Once combined, we should have
two sad girls sliding into place to be invited into
the happy gang. You might be wondering why it
was necessary to physically move instead of just sliding their
images wherever we want in post. The answer is because the human
body is actually 3-dimensional and distorts with perspective. We don't want them to feel like
flat cutouts. Especially because we really are setting them up
as flat cutouts in our 3D scene. Using a brilliant technique from
a Blender sensei, we'll reconstruct not only
the camera's motion but the proper depth placement
for the girls' keyed footage. With a bit of driver code,
generously contributed by an anime avatar on Twitter, we'll convert the sliding camera
motion into object motion, and reframe the whole scene
through a new animated camera. Now it's a simple matter of
creating the entire digital environment from scratch I like the artificial vibe of
the Jamiroquai video and want to combine it
with city imagery -roads, traffic, graffiti… Let's model simple chunks
of stage scenery that look like they were built inside
a warehouse, accented by a giant LED screen We'll make it feel like
a display by actually dicing up the surface into red, green
and blue components. And on it we'll put…
this stock footage of an intersection in Ginza.
Why not? We'll have the girls slide on
moving walkways that intersect. Try building that in real life. It's all a bit brutalist
and bare, so let's mix in some whimsy. Something like the
cartoonishness in chelmico's "Easy Breezy" video, which,
by the way, was shot with a really cool volumetric capture
technology called HOLOSYS. Ho... HO-lo-sys... Ho-LO-sys?...
HO-los-ys. Let's draw a bunch of
inoffensive graffiti and texture-map it
along the walls. There's an interesting video for
the song "TU" by Tulipa Ruiz that uses cutout animation. It inspires me to take
a huge shortcut by adding some flat cardboard
elements like it's a puppet show or a school play.
That's… something! Once lighting is in place, the camera projection mapping
method automatically helps us with realistic shadows
and reflections. But Suzuka and Mizyu can't
always touch the floor at the same time, so one of
their shadows sometimes doesn't connect. What's the solution? You and I are probably thinking
the same thing. Yep, break up the entire render into its component optical
passes, mathematically separate the shadows cast by the girls
and subtly warp them as needed. Then reassemble everything back
in the final composite. This should look epic! But first, we gotta shoot
some more stuff. The chorus needs
an even bigger spectacle. Something with orbital motion
to contrast the angular moves of the first part. Definitely not this overused
trope though. We get it, life is a flip
of a coin. Enough already! I've been fascinated by
this technique in the video for "Nee" by Perfume, where
they cloned the artists in a radial pattern
by using an array of three synchronized cameras. This was done even more artfully
in "Fear & Delight" by the Correspondents, with
a whole bunch of cameras inside a green screen cylinder. I'll set up four GoPros
aimed at a center point as precisely as possible, while an intern filming
the filming of this is filmed. After working out the logistics,
each AG member performs inside the volume individually
and as a group. ["Free Your Mind" playing on set Now if you're following along
at home on your giant green screen stage I recommend the doing
the opposite of what I did. using the deep side of the cyc
for closeups and the front for background so you don't end up rotoscoping
countless dancing silhouettes against a bare soundstage. [Playback stops] I think that will work.
I think it's gonna look cool. Oh, Captain. After syncing and matting
28 layers of 5K footage, we project them into
our 3D scene, this time on curved planes, which will give a sense
of parallax when we orbit
the camera slightly. I'll stick with the abstract,
theatrical set vibe and build a sort of dance floor, inspired by legendary
Gondry videos like "Let Forever Be"
and "Around the World". To weasel out of any
floor shadow issues, it'll have light-up walkways, driven by a simple texture
animated to the music. And after a few final touches,
it's time for a world premiere. One, two, one two three four!
["Free Your Mind" playing] ♪ Walking and walking around
Walking and walking around ♪ ♪ Sad face, you got a sad face ♪ ♪ Talking and talking around
Talking and talking around ♪ ♪ Bad place,
you're in a bad place ♪ ♪ Gotta dance, gotta sing ♪ ♪ Yeah, one more time ♪ ♪ Gotta dance, gotta sing ♪ ♪ Yeah, one more time ♪ ♪ Gotta dance, gotta sing ♪ ♪ Yeah, one more time ♪ ♪ Gotta dance, gotta sing ♪ ♪ Gotta move, gotta scream ♪ ♪ Gotta free your mind
YEAHHHHH ♪ ["CANDY" playing] The second song, "CANDY"
is about… eating candy. (Really?) and features hilariously
cartoony choreography. ["CANDY" playing in room] It definitely puts me in
a 2D animation mood. The zany work of Cyriak
springs to mind. Or the White Stripes'
"Fell In Love With a Girl" Lego brick video. Also, the many
delightfully cheesy Moving collage
offerings of the '80s. But mainly, I'm reminded of
the genre standard defining music video "PONPONPON"
by Kyary Pyamu Pyamu. Probably not an accident since she's lik a
big sister to AG. Let's combine the "PONPONPON"
aesthetic with 2D animation and human stop-motion videos like "Easy Way Out"
or "Sledgehammer". To do this efficiently,
I've devised a technique that shifts the burden of effort from the director
onto the artist. Instead of posing the girls
frame-by-frame for hours, I'll have them perform
the routine with extreme precision, at
half-speed, four times in a row. [Suzuka hollers] That way we'll have alternate
versions of every frame for simulating a messy
stop-motion look. ["CANDY" playing on set
at half-speed] [Girls exclaiming in Japanese] Of course, jumping, stepping
and Rin-ing in slow motion requires incredible muscle tone
and cardiovascular endurance… -Argh!
-[laughing] of which I have plenty. -[Captain] Okay, that's one.
We gotta do three more. -Okay.
-[laughing] We capture these takes in
a full wide shot at high res so that all the positioning
can be done in post. If I had to do it over, I'd probably film
each member separately, to avoid all kinds of issues
when they intersect. I sync up the four takes and conform them
to 12 frames per second. An expression then
randomly toggles between the different takes
on each frame, making things wiggly. And another
simple piece of code-- contributed by members of the Official Captain Disillusion
Patreon Discord, allows me to limit and override
the frame choices when I don't like them. Now time to make
everything else. I model a library
of delicious 3D suites. I also draw a bunch
of low-effort 2D ones. You can't have a song about
candy without lots of candy. Candy! Then I channel
my inner surrealist and spend some time gradually
turning this whole thing into a sugar coma fever dream. And the result goes
a little something like this. ["CANDY" playing] ♪ Where, where is my candy? ♪ ♪ Over there, over there's
your candy ♪ ♪ Pick it up, pick it up,
pick it up ♪ ♪ Pick it up, pick it up,
pick it up ♪ ♪ Don't, don't take my candy ♪ ♪ Give me, give me your candy ♪ ♪ Pick it up, pick it up,
pick it up ♪ ♪ Pick it up, pick it up,
pick it up ♪ ♪ Sugar high ♪ ♪ Sugar low ♪ ♪ Sugar high ♪ ♪ Sugar low ♪ [comical chewing] ["Pineapple Kryptonite" playing] For the final track,
"Pineapple Kryptonite", we're gonna knock things
up a notch. Bam! It's a song about defeating
an invasion with a secret weapon, so obviously we're in epic
Hollywood blockbuster territory. But we're not just gonna go
for tongue-in-cheek like a Beastie Boys video. Heeey, wait a minute!
What's going on here? I want to steep our shots in
the sometimes overbearing CGI look of modern
blockbuster movies. There's a long history of CG
environments and characters in music videos, from the boxy guys
in "Money for Nothing" to a skinless Robbie Williams
traumatizing everyone with "Rock DJ" to this cool video for
the song "Animation" by Young Juvenile Youth. Aaah! I'm also mesmerized by
this video for the Chemical Brothers/Beck
collaboration "Wide Open", in which Sonoya Mizuno
gradually transforms into a shopping mall sculpture via an astounding amount of motion capture, manual tracking
and 3D reconstruction done by artists at The Mill. Yeah, let's do all of this! For the first of two shots, I film a piece of the dance
on the gimbal, simulating a crane move. ["Pineapple Kryptonite"
playing on set] Again we can extend and
exaggerate that move in 3D, and toward the end,
when I dropped down, we'll instead keep
the virtual camera in place, making it look like the girls
levitate up. Then we build a realistic desert
environment with the help of some stock 3D elements
and a Blender feature I will not master in
my lifetime: geometry nodes. They'll also help us
inflate the projections for a realistic lighting pass. Speaking of lighting, we'll add
some UFO abduction spotlights and emphasize the power of
the tractor beam with procedurally
animated debris and texture sprites
of dust elements. Oh, and uh, once the girls
begin to levitate, I want them to transform into
alien beings of unknown origin. For that I'll use abstract
3D human figures and make them look melty with
moving texture displacement. To skip the arduous task
of proper body motion capture, we can use a free AI tool
to reconstruct their movements directly
from the footage. Hey, it works pretty good! Unfortunately it doesn't account
for the camera motion so I have to manually
un-animate each of them into the proper place
throughout the shot. But now we have a CG alien
version of each member aligned with the real one. I'll transform them
from one to the other by trailing this procedural
texture with an old-fashioned, genuine, Michael Jackson-powered
2D morph! Hashtag bring-back-morphs! But we're not done yet. For the second shot, we'll
pull out all the stops. In this avant garde portion
of the dance, the girls are just…
punching each other. I want to raise the stakes by
putting them atop a tall cliff and turning two of them
into gigantic floating heads that require a bit more than
a punch to take down. First I capture Suzuka and Rin
in a dynamic orbiting shot by doing what almost no other
full-time YouTuber can do: running. With my physical body. Several times! -Okay, let me see what--
-[laughing] While an intern controls
the camera's rotation remotely. Sure, it might look silly now,
but once we 3D-track the camera, reproject the footage,
stabilize and reframe the shot, and put some temporary heads
into the scene it starts to come to life!
["Pineapple Kriptonite" playing] I'd like to fix these obvious
fist-face gaps, so I'll do some
compositing hacks on the footage in those moments. Also spend several lifetimes rotoscoping
the reverse portions. [growling] [screaming] The environment is a post-battle
space-scape full of geonode-driven
asteroids. With light sources on every
side, the shadows from the camera projection
don't really hold up, so I have to build and
hand-animate dummy AG legs to occlude light more
realistically. I shall call them… Tokyo Nightcrawlers. As for the floating heads,
that's easy. We simply need
to create accurate, fully articulated 3D replicas
of the girls' faces and hair, then animate them singing…
with motion capture. First, as a backup we'll use
an iPhone's TrueDepth camera to sample the contour and
low-res textures for each face. But we'll also snap detailed
survey photos as reference for building the
high-resolution models. These images projected onto
the resulting models can then be stitched
into textures, cleaned up, de-lit
and reapplied. The meshes will have a set
of standard Facial Action Coding System
deformations. It's the same tech that makes
Memojis possible, but used for good
instead of evil. Once it's set all up, a real face can drive
the digital avatar in realtime. Mom, Dad… [Captain/Mizyu in unison]
I'm a V-tuber now! But let's have Mizyu and Kanon
provide their own performances. ["Pineapple Kryptonite"
playing on set] ♪ -[Kanon] Dream with me how
the world can be better and ♪ ♪ -[Mizyu] Sing with me
We'll be always together ♪ ♪ -[Both] Before you die ♪ ♪ Sing a lullaby ♪ ♪ Close… ♪
[payback stops] -[Captain] And that's... cut. -One more. Smile.
-[Captain] You want to... Okay. With all the motion data
applied and cleaned up, I have to perfect the hair
by growing, cutting and styling thousands of simulated strands
for days, to the point that the sight
of pigtails makes me curl up into
a ball and hiss-cry. [wheezing] But after all the struggles, the
render passes, the tweaks and the JJ Abrams amounts
of lens flare, we finally get… this. ["Pineapple Kryptonite" playing]
♪ [Suzuka singing in Japanese] ♪ ♪ Higher, higher, higher,
higher, higher ♪ ♪ Higher, higher, higher,
higher, higher ♪ ♪ -[Kanon] Dream with me how the
world can be better and ♪ ♪ -[Mizyu] Sing with me
We'll be always together ♪ ♪ -[Both] Before you die ♪ ♪ Sing a lullaby ♪ -[all cheering]
-[Suzuka] Yeah! Nice! -That was fun!
-[AG] Yeah! -You guys did a great job.
-[AG] Mmm. But now I'm afraid
it's time for us to go. Class dismissed. [speaking Japanese] I'm sorry? -[Suzuka] Hoo! ["KOIBUMI" playing] ♪ [singing in Japanese] ♪ ♪ I wanna koi, koi, koi (love) ♪ ♪ You wanna koi♪ ♪ -[Suzuka] This is the koi ♪ ♪ I wanna koi, koi, koi ♪ ♪ -[Singing in Japanese] ♪ ♪ I wanna koi, koi, koi ♪ ♪ You wanna koi ♪ ♪ -[Suzuka] Oh no! ♪ ♪ [Singing in Japanese] ♪ [Song ends] ["Quesera sera" playing] ♪ [singing in Japanese] ♪
Made a playlist in Youtube for all the videos that he has referenced for those that want to check it out.
Includes the videos that come up as references for styles.
That last one was especially really fucking cool
Seeing The Correspondents given a shout-out for their fantastic music video Fear & Delight was quite an emotional surprise; Tim Cole, the guy playing piano in the middle of the room, was a friend of mine who passed away from a sudden pulmonary embolism back in 2020. Love seeing the duo get some recognition from the Cap, their music is fascinatingly unique and incredibly good.
Youtube doesn't deserve Captain Disillusion
Being a Captain D fan is like getting a big bowl of candy and living off the sugar rush for 8 months or so. Not asking for more because you know that once the next bowl comes, it's gonna be equally delicious and with a tonne of new complex flavours.
Amazing as always, can see the camaraderie forming between the captain the group. Listened to whole bunch of atarashii gakko's biography and they do have an unique sound that works great. It really does fit cpt D, which explains why they got together to make this. It just works.
Wow this really gets the creative juices flowing again, something I've been lacking lately, the possibilities are endless!
I know the pain of hours of having to manually rotoscope a subject from a video. To put it perspective, for just a 10 second video that's 300 frames to work with.
I'm glad AI video editing tools have started popping up to help with some monotonous tasks, though they aren't 100% accurate at times and so doing things manually is still needed.
I knew this had to be a banger because of how long it was taking. I don't think anyone could have expected this much effort. Brilliant video, although I was hoping we'd see Captain D in a Seifuku.
This may definitively be the most work ever put into a video for Youtube only. Holy shit.
This is incredible. There's no possible way that Youtube rewards this amount of work with videos and monetization, but I love that people like Captain Disillusion are still willing to cram this amount of time and effort into it.
I was excited when I saw he had a new video... then more excited when I saw the topic... and so on and so forth and that didn't stop the whole way through.
Awesome.