Virtual Production for the Classroom | Unreal Educator Livestream

>>Luis: In an effort to make virtual production accessible to everyone, Epic Games released The Virtual Production Field Guide, which is downloadable as a PDF to everyone, including students and educators. The virtual production field guide was released in the summer of 2019 with an associated blog on unrealengine.com called The Virtual Production Field Guide, a new resource for filmmakers. This guide can be very helpful for those interested in workflows, roles, hardware and software related to virtual production, and in general for students and educators hoping to integrate virtual production into the classroom. If you're looking for resources about virtual production, make sure you go to unrealengine.com and seek out the blog-- The Virtual Production Field Guide, a new resource for filmmakers, and download The Virtual Production Field Guide as usual. This amazing learning resource is free and can be very helpful as we learn about the amazing field of virtual production. Students and educators should be aware that every month, Epic Games gives away free Marketplace content. If you go to the Epic Games launcher, the Unreal Engine section, there's typically news. And at the beginning of the month, we announce the free Marketplace content there. Also if you go to the Marketplace tile, the free-for-the-month content packs are viewable on the start page. Additionally, in the Marketplace tile, there's a free drop down menu with a free-for-the-month selection. There you can dig into each of the content packs and see what's available. This month, you can see that there's a modern city downtown with interior content pack that retailed for $149. If you look at this content pack, you can see what's available and how valuable this content really is. You can see that it's a really highly rated pack that comes with a great many textures, materials, static meshes, decals, and even blueprints that are incredibly valuable for learning. I strongly recommend that you instruct students-- and even as instructors-- to go every month and download every one of these content packs, even if you don't need them. Someday down in the future, they can be valuable as a learning resource, or even as a teaching resource. Also in the free drop down menu is the permanently free collection. The permanently free collection, as it states, it's permanently free. Here you will find a variety of packs including vegetation packs, some content packs including broadcast studios, static meshes, skeletal meshes, effects packs, and some of my favorites including the super grid and the advanced locomotion system. These are amazing projects that, once again, can be really valuable as a learning resource, but also can be super valuable as students work to stub out levels, do gray boxing of different types. Or even just need maybe a water plane or some wood materials, or things to help build out environments for their capstone projects. Also in the free menu is the Epic Games content. These are additional material packs, full projects-- including all of the Infinity Blade content-- additional plugins that can be helpful for media and broadcast. But really importantly is the entire suite of Paragon characters and environments. This is also an incredibly valuable learning resource, because each character inside of Paragon is beautifully created with amazing materials, stunning animations, effects, and audio work. And all of this content can be used even to ship games inside of Unreal Engine. Make sure you check out this content, learn from it, and use it in classrooms to teach and to learn. >>Luis: Hello and welcome to this week's stream. We're really excited to have our special guest today, Matt Workman. And today, we're going to be talking about virtual production in the classroom. We've been looking forward to this stream. And as usual, we have Tom Shannon with us this week. And we also have Mark Flanagan. And up in the corner, shrouded in green, is our friend Matt Workman. >>Matt: How's it going? >>Luis: So we're really excited about the stream. I think that virtual production has been an exciting topic that a lot of people have been asking us about at Epic, specifically in context about building curriculum around virtual production, what it is, and basically how to understand what it is. And I think one of the things that we've been excited about is going to the work that Matt's been doing on YouTube, going to the work Matt's been doing on the Facebook groups, and really sort of following the journey. A lot of people have been doing a virtual production. Hopefully many of you are aware that we have a virtual production page on unrealengine.com that shares a lot of really valuable resources that can be really helpful. But I think that one of the things that's really helpful, and one of the things that we've been excited about, is that we've been working with Matt. And you presented at SIGGRAPH. And you've presented at a variety of different events. And you did something really cool things with us at previous events where you presented material that you've been building from the ground up. Because there's many big companies that are making amazing films like The Mandalorian. And they approach it, I think, from the top down. But one of the cool things is that you've been sharing your experience, in many ways, from the bottom up. Which I think helps educators and students understand that if they, of course, don't have and millions of dollars, and if they're not Netflix and HBO and a big film company, that they can also get that grassroots learning. And help them to understand how they can teach students. And even though there are many schools that have huge budgets to buy lots of equipment, you still have to access the learning. And in many ways, we're still pioneering a lot. I love the analogy of building the car as you're driving it. Virtual production is very much building the car as we're driving it. But before we get into all that, tell us a little bit about yourself and introduce yourself. I presume there's plenty of people on the stream that maybe don't know a whole lot about virtual production and maybe haven't met you. >>Matt: Right, yeah. Thanks for the intro. Thanks for having me on the stream. My name is Matt Workman. And for about 10 years, I was a live action cinematographer in New York City. So filming with real cameras, real lights. I shot a lot of commercials and music videos. But I ended up working on very big VFX projects. So a lot of previs and 3D was happening. And as a DP, I wanted to be able to communicate in that space. So I started to learn Maya and build my own previs tools, kind of based on what the third floor had been showing and different previs companies. And after that, I decided to make those 3D tools that were designed for filmmakers available to other people. I started a company around it called Cinematography Database. And within the two years, I have brought all my experience of being on set and then in software to Unreal Engine. And that's turned into Cine Tracer. And it's turned, recently, into an effort to make indie virtual production known to the world, and show how that works and what it looks like. SPEAKER: So you've been at this for a while, right? Because to go back to what you mentioned, Cine Tracers is a tool. For those of you who are not familiar, Matt makes a tool called Cine Tracer, which is not dissimilar to Fortnite Creative. Basically it's a runtime tool that you can get currently on Steam. It is an amazing tool. If you are not familiar with it, go get it. Because it's really amazing. And you jump into it and you have full access to camera control. It's very easy to use. You have all the tools. You have dolly tools and truck tools. And you can tell us more about it as well. But you get to basically do previsualization inside of this real-time tool, where you get full access to sets and lights and all of it is tuned to real world cinematography standards and real world production standards. But you kind of also built this tool a long time ago. But you built it in, what, Cinema 4D, right? It was five, 10 years ago when you were doing this work. You realized that there was a need for a tool like this, or you even got into Unreal Engine. Give us a little bit of background about that. Because you saw and you filled the need. >>Matt: Yeah. So it started off as a need for just me as a cinematographer and working with The Mill specifically. And I wanted to do motion control, which is where you have a robot move the camera. It's like, so what's the move going to look like? So that's primarily done in Maya. So at first, I just learned Maya and Python and Mel and dependency graph programming, and 3D modeling and rigging in Maya, so that I could work with big VFX companies. That was where it started. That got a lot of attention, just that alone. And that was kind of private, too. That was not like-- I wasn't blasting that super hard back then. But it got around. And it seemed like, hey. A lot of people want to do this. It would be great to be able to bring this to other DPs specifically at the time. And so I moved that into Cinema 4D. Because I thought Cinema 4D at the time, like five years ago, was the easiest to teach. Because I knew I'd have to teach filmmakers how to use x program, right? I'm making plugins for programs. So I thought Cinema 40 would be easy to teach. And I think it is. I think it's still one of the easiest to learn, and best UI, and very simple. I like it for 3D stuff. So I made Cine Designer, which was essentially porting everything from Maya into Cinema 4D. And Cinema 4D has Espresso, which is a visual programming language just like Blueprints. Which is why it was so easy for me to go from Espresso to Blueprints was like, super easy. For filmmakers, it's very similar to working with like Nodes and RESOLVE. If you're used to putting together power windows, doing color connections with Nodes and RESOLVE, Unreal Engine is just that except you're making real-time graphics decisions or logic and whatnot. My coding is mostly visual scripting. So for a filmmaker looking to get into it, if you're already kicking ass with RESOLVE, I think Blueprints and Unreal Engine-- and not everyone knows this, you don't have to type. A lot of it is visual. So that was the first step was Maya to Cinema 4D. And that became a pretty big product. A lot of DPs are using that still. And then we made the switch to Unreal Engine, which was the most fun. That's been the most fun so far. SPEAKER: So you went from working inside of Cinema 4D. And then you discovered Unreal Engine, I guess. And then you were like, well, why not just rebuild it and tool? And I guess part of that is because Unreal Engine was much more real-time. So there was a value in bringing all that stuff to a much more real-time environment where you could move things around much faster. >>Matt: Yeah. It gets rid of the rendering paradigm. In a 3D application, you're typically like looking at a simple, not so good looking preview. You hit render, and x amount of minutes to hours later, you see what's happening. And the hope with Unreal Engine was that we're just rendering the whole time. And you just see it. Because it's a video game. And I became aware because of the GDC Star Wars demo. Where I was like, oh, real-time ray tracing is coming. So that was a huge motivator for me to get Cine Tracer up and running for when that would become available. I'm happy to say that I was ready for it. We had a pretty decent sized community in alpha and beta. And when Unreal Engine 4.22-- I believe was the first RTX release, because I updated immediately onto 28 ATI and we were ray tracing. So I kind of was rushing towards this like real time ray tracing revolution that was happening. And for the gaming community, I think it's still slow and steady to get there, where people can afford that hardware. But for what we call the provis-- the pro visualization community, architects and previs for film-- we'll jump right into the highest end hardware if it's going to get us better results. So we can shoot the jobs better and plan better. So really the film industry was a great candidate for being able to jump into real-time ray tracing. And that's one of the main reasons I did Unreal Engine. I was like, I really want to see what that looks like and let people try it. SPEAKER: Yeah. So you had to learn Blueprint and build this tool in Blueprint because you have a programming background. I mean, is your background as a computer scientist? Or, your background is a filmmaker. So how did that go? What was that like? >>Matt: I have a mixed background. I did actually start in computer science. So I knew Java and C++ a little bit. But it was not my strength. And I ultimately left it. I don't really enjoy "coding" coding I don't have to. So I very much relied on Blueprint scripting and just YouTube. If I had to describe how I learned Unreal Engine, it was by going on YouTube, following Blueprints tutorials until I could do everything I wanted. And I think fortunately for me, I had a goal. And I think that makes learning something very structured. I was like, I need to make this camera move. How do I do that? I need to make this light get brighter when I move a slider. How do you make a slider? So I had some very specific goals of basically recreating the Cine Designer, my Cinema 4D product in Unreal Engine. So it was a very structured, pretty straightforward march of just learning it. And I learned all of it from YouTube. And Docs, Unreal Engine documentation. And then the community around it. >>Tom: You already had a base in Cinema 4D structure for everything. You just had to take that, bring it into the Blueprint. And so all of Cine Tracer is made in Blueprints? >>Matt: Yeah. There is one Blueprint function library that's C++ that I regret terribly. >>Tom: [LAUGHS] >>Matt: I wish it didn't exist. Just from switching between engine versions, if you're all Blueprints, it's nothing. It's really not difficult. It's when you start modding the engine in C that you can run into-- just things changed. And so now you have to do them. It's not impossible, but it's a friction point. And it's kind of like, do I really want to? So I'm actually hoping-- I think we were talking about it-- is that my C++ is really to do file IO. If I need to move a file around, that's not typically what games need to do. And if they do, you're going to custom do that. So as soon as-- if it might exist in 4.25, I haven't looked into it. But once I can just move files around and make a new folder in Blueprints, I'll just completely get rid of C++. And every time I upgrade, even to UE5 potentially, it's pretty straightforward. You do C++, you're going to be coding for a little bit. >>Tom: That's pretty incredible. Because the application runs really well. It looks beautiful. It's fairly complex. There's UIs [AUDIO OUT], Fortnite, [AUDIO OUT], Cinema 4D. Or I usually call it something like Cinematography, the game. Or filmmaking, the game. Because it's really-- when you're doing it, it feels like a video game. [AUDIO OUT] It feels very game-ish [AUDIO OUT] So it's wild that's something that complex with so money interdependencies [AUDIO OUT] Pretty cool. Very impressed. >>Matt: Thank you. I think the C++ mega performance heavy stuff is really important when you're doing Fortnite. This is running on a server, and there's 100 people. So every millisecond of performance you can gain back is extremely important. And you're looking at clicking on things. And it has to register very quickly and precisely. I'm still a single player game running on a single computer. So even if I was to fully optimize it, I'm not sure if we would really notice if anything would really change. I don't think I don't think I'm breaking the Blueprints bottleneck yet. I think I'm running at full capacity. Switching to C would be a nice long term goal. But currently, I think it wouldn't perform all that much better for the user. I don't think they would notice. So yeah, Blueprints can definitely handle the Cine Tracer, which is somewhat complex. I would love to do a series just explaining the structure of it. Because it's basically like how do you recreate the Unreal Engine editor is what I think. I'm basically trying to take an Unreal Engine editor interface, or a Maya interface, and how do I simplify this? And I think projects like Dreams on the PlayStation do a really good job of that to looking at the scripting level. And so I think that having an abstraction of simplicity over even Unreal Engine editor, for less technical people, is a fun exercise. Definitely all Blueprints. I have everything I need. Except for file IO. That's about it. SPEAKER: Yes. So I think one of the really cool things is that you actually document all your experiences on your YouTube channel. So as you were working through Cine Tracer and building a lot of the things and adding a lot of the features to Cine Tracer, you put those up on your YouTube channel. And so we can actually track that journey. And then you started working on a whole separate section of your YouTube channel, which is the V-log stuff, when you started really digging into virtual production. That's a whole separate part of your YouTube channel. And so for those of you who are educators and tune in because you are learning different things about an Unreal Engine and/or just tuned in because you're interested in virtual production specifically and are not aware of the Matt's work, he's got a whole section on his YouTube channel. That really started maybe about in January, February? Maybe a little earlier. That tracks-- you were like, I'm digging really deep into virtual production. And one of the things that I really enjoyed is that you started in like, I got a mocap suit. And I'm going to put on the mocap suit. And then I've got my DSLR camera. And I'm on hook up by DSLR camera. And I'm going to start finding a way to track my camera. And then you go to the next step, which is I'm going to get a green screen. Tell us a little bit about transitioning from, OK, I've got Cine Tracer. Now I want to get to the next step. What were you sort of thinking about that led you into the journey of going from Cine Tracer, which was already pretty successful and doing really cool stuff to virtual production? Because I know you were already working with us. And you had gotten involved in the project we did last summer at SIGGRAPH, where you came in and did this amazing project that we worked on with the motorcycle. And Epic contacted you and said, can you get involved? You were the Director of Photography on that one. Is that correct? >>Matt: Mm-hmm. Right, yeah. So really that project, the LED Wall that we can say is like the mini Mandalorian set essentially, that was really what pushed me to keep going further. I was always looking-- and still am-- to take the full power of Unreal Engine, make it a package thing, give that to people. And that's Cine Tracer. And I'm still doing that. But when I saw the LED Wall Volume and saw virtual production LEDs, I was like, I would like to keep doing that. I would like to keep shooting in it. I would like to develop tools. There were things when I was on set that I was like, I wish I had this tool. That tool now sits behind me on that very nice camera cart. That's what I've been building essentially, is the tool that I want when I'm shooting those types of projects, either green screen or LED Wall. I have my take on it and that's what I've been developing. So the YouTube-- I've done YouTube for a very long time. And when I did Cine Tracer, the first year of building Cine Tracer, I streamed eight hours a day on Twitch for a year. The entire thing. And I knew nothing in the beginning. It was literally opening up the third person Blueprint and being like, so. And I streamed that entire process. The vlogs are all gone, because it's Twitch. So this time, you can't really live stream virtual production. It is very difficult. Because there's a lot of hardware and things turning off and on. I was like, I would like to document this now for YouTube. That would last for a lot longer. Those videos can last for a very long time. And I just started the road back to the LED Walls. That's essentially what I'm doing. What do you do from a little camera and your own gaming computer all the way up to Mandalorian. What is that path look like? I still don't know. I have a good insight, I think, on it. But I'm still making my way there. So I decided to document it on YouTube. And it's been fun. And what I really need is the community around it to help me. Because I have a lot of blind spots in virtual production. Virtual production is the most cross-industry collaborative industry there is. You're talking real-time graphics and then live lighting concert tech on top of precision camera tracking tech from broadcast to everything. Like, real cameras, lenses. It's so many disciplines put together. So I don't stand a chance by myself to learn it. So I put the YouTube videos out. I built the community. And I honestly just bug them all the time. I'm like so, why all this DMX, Unreal Engine, sky panels? How? Right? And people come in there and they know. They might know one part. They know really well grandMA platform and sky panels. And someone else is literally the person writing the Unreal Engine plugin. Geodesic, shout out. And they're like, oh. These two people get in a thread, and it's like, this is how we do it. And I'm like, cool. So it's very much cutting edge stuff that like not really-- I don't think any one person has the full knowledge. It's too big. It's like programming Fortnite. No one person knows Fortnite. There's lots of people coming in and building it. So that's a virtual production is right now. And it's super cross-department. I run the Facebook group. And the question is where did you come from? How did you find us? It's like, oh, from a lighting design forum. Like, real life lighting. Then there is like, virtual DJs told me about it. The disguise people told me. Houdini people told me. Like, filmmaking forums. It's like this crazy mix of people. So I try to have the YouTube channel be a beacon to find people. And then the Facebook group is where we have to all jam. Because we all have to get along. And it's from a hardware level, too. I talk to like lots of hardware manufacturers and software people. And they're like, what's virtual production? We heard of that and we'd like to do it. And so just connecting the dots as fast as possible. And Facebook just currently happens to be kind of the best one out there. Because you can just post your videos, your what not. And it's private enough for us. It's not like we're on Twitter doing it. It's just a really great place. So that's kind of what the journey has been. SPEAKER: Right. So in a little bit, Matt is going to share a presentation and is going to talk a lot more depth about his journey and what he's discovered. But I think that with regards to those people that are discovering it for themselves and figuring this out and learning, it's really valuable to go to the vlog portion of his YouTube and sort of track the trajectory. Because if you start from the first video and see where you've got your mocap suit. And then you were like, well, I need to get a camera talking to Unreal Engine. You started with our virtual camera that we provided. And then you started going in and wanting to track that more effectively. So you got a VIVE puck. And then you connected a VIVE puck. And then you connected the Blueprint Nodes to start making that easier to track your own camera into Unreal. And then you started discovering other challenges. How do you actually get that to sync correctly? And then you were like, well, I want to now be able to put a green screen behind. So there's the next part of the puzzle. How do I actually put a green screen and put my own virtual background in there? And to watch that journey is really interesting. And I think from an education perspective, what I need to learn, what I need to teach, and how I need to formalize this education is really fascinating to track through this learning. And I think some of the other stuff is that you're, like, OK. Now I have a light in front of my practical element. And I've got a virtual light. And there is a section in one of your videos where you're like, OK. I can move my virtual light. But now I physically have to move my practical light. But then a couple videos later, you're like, haha. I figured out how to connect my virtual light and my physical light. Now I can move one and they both move together. And I think that that's what we all-- and even the professionals-- have been doing. I think as you talk to the companies that are now providing the service of virtual production, they've just built more tools where the hardware is talking to the software. And they've created tools to do this. And they sell them as a service. But what I think is kind of interesting is that you break it down in a way that like, hey, this is how you can do it with just some Blueprint Nodes as well. And that's what was super valuable to the education communities. Because you're like, hey, you can go and spend $40,000, $50,000, $60,000, $100,000 to buy this thing. But you can also just go into Unreal Engine and connect the hardware and the software yourself. Can you tell us a little bit about that learning process? And the difficulty or ease that you discovered in that? And how other people might be able to benefit from that on the streams? >>Matt: Yeah. So I wanted to be able to do-- virtual production is a lot of stuff. But being able to do it with off-the-shelf hardware and stock Unreal Engine-- like no plugins, really nothing that you'd have to purchase. Mostly what I've shown, Unreal Engine just does on its own. People don't know that. You can chroma key in Unreal Engine. You can bring in the video. The plugin is free. You need the hardware. You can track cameras. All of that stuff-- I think up until now, basically-- is all stock Unreal Engine. So just for myself, because I didn't want to necessarily spend all that money just yet, I was like, what can I get done with just a live from Amazon, my computer, and like a Black Magic deck link card, which you just buy on Amazon. Anyone that wants to do it. And I just wanted to push that as far as I possibly could and document it. And I think for a large portion of the community, that resonates with them. It's like, there's the high end people. And I talk with them as well. But really, what can one person get done with somewhat modest equipment? And I was just personally wanted to see what it was. And this is not a canned, scripted progression. I didn't know if I could do it. And it's been rewarding to be able to just go through those steps. And I think that for people to see that it is possible is all people really need. And I kind of knew that about the series as well. It's like, oh, he's doing it. I must be able to eventually be able to figure it out. Versus if you only watch the Disney behind the scenes of Mandalorian, you're like, I don't know if I can do that. But watching it in an accessible YouTube format where it's in my basement-- I have a nice basement, but it's my basement. It is possible. I think that I wanted to see what I could do and then share that with other people. >>Tom: Yeah. People see that Mandalorian stuff and they want to do that. Like, I want to do that. Like, I want to build an LED Wall and do this stuff. But where do I get started? How do I even get there? So having a resource, seeing someone go through that process, I think is really, really helpful for people to really understand how difficult the process is. And is it easy, is it achievable? If you just see someone do a tutorial or a tech demo, it always looks easy. And it always looks polished and great. So seeing someone actually go through the motions and possibly fail at what they're doing is so valuable for people. And to document that is unusual. Usually I think studios go through this, and schools go through this, and individuals go through this. And all that information stays inside there. So for you to be doing it openly is such a cool resource for everyone to see and to start to understand what it really takes. Because at first, I was like, anyone can do it. You can just buy a couple things off Amazon, plug them together, and you've got your basement Mandalorian. Shockingly, it's not true. >>Mark: I think that's even more impressive in this case. Because the whole thing about virtual production is it is still being discovered. We still have a lot to actually learn about how things work. I think that's a lot of what is great to see you doing. You're actually trying things. You're trying things not knowing when it's going to work. It's not just trying to replicate something on a budget. You're just saying, will this work? And then trying something else. Will this work? And that is really fascinating. That's great fun. SPEAKER: Yeah, where it really started come together in one of your videos-- for me-- is where you are actually starting to put characters in the environment. And now we see you in the virtual production set with a virtual hand-held camera. And you're filming inside the environment. So you're in a green screen. There are environments there. There's a virtual character. Now you've got two cameras that are capturing the entire scene. And in some of your more recent videos, you're actually invested in even more hardware. So now you have your cart. And you started, of course, with the game controller. And you're like, well, I don't have the kind of control I really want. Can you tell us a little bit about how it became more sophisticated? And you were like, well, I really want to create a simulation. And then tell the story of that simulation through virtual production. Because I think that for those of you who haven't gone and watched some of these videos yet, there's a level of sophistication that evolves to where in Matt's more recent videos, you're starting to see where there's a movie that is being made now in his basement with just Marketplace content. And the equipment has become more sophisticated because he wants to do more sophisticated stuff. So tell us a little bit about what you're doing now. Because it's amazing. >>Matt: Thank you. And for people watching, we are going to-- I will show you pictures and videos. And we'll talk about it shortly. So yeah. It started with a VIVE controller. And I put it on a handheld camera rig that you would normally put a camera on. And I move that virtual camera with a VIVE controller the same way. So that's where it began. And then you're like, well, I want to be able to move around that space. And you can't physically walk very far with a VIVE control. I can walk five or 10 feet, but I want to move across the room. So I needed an Xbox controller. And I mapped the Xbox controller to move what we call the camera stage. And so I put that controller on a tripod. And so I had my handheld camera on a tripod. And then I was like, you know, I'd like to be able to do a gearhead move. So then I got inertia wheels, which are amazing. They are like the virtual production camera wheels, go ahead and check them out, made by Nodo. I was like, I need someone to put them. And I had in the back of my mind that I wanted to make this look like being on a film set. So that camera cart back there, that is the camera cart you see on film sets for DITs, for assistance. If you're doing-- those wheels are normally used to control heads on a techno crane, or big cameras. It's the same thing. So I just started building out the accessories I needed. I have the farm simulator wheels, which is pretty hilarious. Like, the farm simulator is this big steering wheel and a tractor controller with like 100 buttons. And you can map any of that stuff to do whatever you want. And I have the feet pedals-- one of the demos I have, we'll show it, we have the feet pedals. So I'm like, driving a car. And each one of those is a discovery of how do you get that into the engine, how do you work with it, how do you map real world things like follow focuses and film tools. Those can all be mapped into Unreal Engine. And I'm making what I call the most expensive, difficult, filmmaking simulator in the world. Because it's like, everything. We're trying to map all the real world film equipment to the virtual side of it. And so it's been this progression. And I'm not even close to done. That cart is going to keep-- we're going to more carts by the time I'm done with this. But it's not to show off. It's pretty much for me to discover how you do that. And when I have all the things that I believe I need, I'm going to release that framework so anyone can do it. And I've been working with the hardware partners to make it smoother and better. And I'll share the designs of the carts. And hopefully help move hardware that needs to be-- those companies need to make money for them to make accessories that we need. And try to provide the education, show how it's used, and then help bridge it all together so that you don't have to do what I'm doing and one by one figure this all out. I can release the framework that does it in an open source way, possibly, or source accessible anyway. And then the hardware will be bespoke made for this as well. Whereas I am very much MacGyvering it, we're going to have bespoke specific tools meant for this workflow. And my design is to make it so that you can control a camera virtually with that cart, but then you can also control a real world camera on a green screen with the same cart. And then a real world camera on an LED Wall in cart. I control the lights from the different hybrid boards I like. It could be all virtual. It could be green screen sets or LED Wall sets. It's all the same exact hardware, same exact platform. So that's what I've been building in Unreal Engine. So it's hardware plus software plus framework. To make it so that-- your average DP is not going to build that themselves. So I'm trying to build this bridge for the bigger stages. Cine Tracer is very much like the-- and all this stuff works in Cine Tracer too. But it's like, the consumer version. And it's like the previs version. But now I'm building it for like indie TV studios, indie movie studios, and make it more fluid. So you don't have to switch your mindset as much. I'm working all CG. I'm working on a green screen. I'm working LED. Well, I'm trying to make that all the same thing. So that's been the progression. SPEAKER: But do you want to show us and share with us the material that you prepared? Because I've taken a sneak peek at it it's pretty exciting. Tom and Mark, did you have anything before he jumps into that? >>Mark: No, just looking forward to seeing this. SPEAKER: Yeah. Pretty exciting. >>Matt: Can you see my screen? Is this working? SPEAKER: We see it, yes. >>Matt: OK, excellent. OK, cool. So virtual production for Unreal Engine educators. So we're teaching how to teach Unreal Engine. And this is basically how I think about and understand virtual production. Again, I have my blindspots. I also, as a disclaimer, come from the production side of the world. So if you're a VFX supervisor, you may see things a bit differently. This is very much the point of view from someone who is a director and DP, kind of how I think it needs to be structured for them to understand. OK. So this is what we're going to be going through. Can you see my mouse by any chance? Is that visible or no? SPEAKER: I don't see it. >>Mark: I can see it. >>Matt: OK, cool. So I might be moving it. SPEAKER: Oh yeah, I see it. >>Matt: Right, yeah. It's probably not the best. But I'll move it around. So we're going to go through four steps here. The first one is, what are the three types of virtual production? This is a simplification. There's much more. But this is a good way to segment it. Number two, hardware and software. What do you need? What's recommended? What works? What doesn't? Again, from my understanding of things. Number three is a step-by-step what's a good teaching progression, or learning? Like, if you want to learn this, where do I start? Where do I go? And number four is resources. So that's what we're going to be going through here. So the three types of virtual production. Which one do you want to teach? Which one do you want to learn yourself? I basically break it down, at the moment, as all CG or in engine, mixed reality, or LED volume. So those are kind of the main three that I'm interested in, that I hear a lot about, and most people fit somewhere into those categories broadly. Of course, there's much nuance to all of this. But if we want to simplify, that's how we do it. All in engine you'll see as a computer. It's very much an animation workflow. 3D animation with a little bit of spice in it, a little bit of handheld. A little bit of mocap. A little bit of real world stuff going on, but it's primarily 3D. Mixed reality LED volume, that's a film set. This is a film production. While there is visual effects, there is real-time happening. This is a film set. So if you're a teacher, or you're a student, I imagine you tend to fall to one of these more than the other. Like, do you have cameras in your classroom? Are you interested in working with a real camera? That puts you over here much faster than it will in the engine side of it. Maybe you don't really care about real cameras or real lights. Probably are not-- you may not be pulled into mixed reality and LED volume. I just want to make that big distinction upfront. They are all virtual production. That's something we want to keep in mind. But the first one is all CG in engine. So we can call this VR filmmaking, that's a very common way to think about it at a consumer level. It's basically 3D animation but with some real world input, like a handheld camera, which I'll be showing, and mocap, which is real humans moving. And so a high end example of this is the Lion King remake, primarily enabled by Magnopus. They did a lot of the hardware interfacing to allow real world filmmaker Caleb Deschanel, the DP, and Rob Legato to shoot Lion King like it was a real live action film, but it's all CG. So this is an all CG, all in engine execution VR filmmaking. And this is a not safe for work YouTube channel, but I just have to shout him out. JoshDub, who uses VR chat and consumer VR games to make machinima content. His is the most interesting I've seen in a long time. Not safe for work, don't watch this in your classroom. But it's I think an excellent example of all CG, all VR filmmaking. So this is a big category, more on the 3D side of it. Recent to the virtual production squad is Corridor Digital. They recently tweeted virtual production has blown our minds and hopefully yours with the limitless possibilities it offers. Here's how we made our first project happen from start to finish in Unreal Engine. So that's pretty awesome. Very recent. So Sam from Corridor Digital, we've spoken a lot. This is virtual Sam here, if you can see my mouse. We've been talking a lot about this process and continue to evolve our own personal takes on virtual production. Everyone does it differently. But seeing what's in common, how we can help each other, he's a filmmaker, a YouTuber, VFX artist, now working in Unreal Engine using Xsens mocap suits. I'll play just a little bit of this. There should not be audio, hopefully. And so this is them remaking an episode of their very well-known VFX artist react to XCG. So huge YouTube series. I had people from Weta Digital and ILN have come on to watch it. So they scanned themselves. They used Xsens mocap suits. They're using the iPhone, I believe, for the face mocap. And so, these are YouTubers executing at this level. This is all made an Unreal Engine completely. And so go check this out if you haven't seen it. Pretty awesome. And there's a behind the scenes of it as well. Next, we move into mixed reality. This is a picture of my basement a month ago. It's a lot different now. This is my wife Diana, shout out. So we have a Blackmagic camera. We've got a VIVE tracker, a green screen. And I can see it my little monitor here the live composite. And if I move my camera, the CG world reacts accordingly. And you really have this illusion of being in that space from a camera operator point of view, which is lovely. From a lighting point of view, I can then light backgrounds in CG, and I can light the foreground in real life and match them, and see them. And it becomes a much more live process than separating out, I just shoot green, and I see this bottom screen and I have no idea what's going on. I see what's happening. And I can move the camera freely however I want to. And this is just a pretty affordable Blackmagic camera, which we'll talk more about, and a VIVE. That's it. That's the only hardware. Stock Unreal Engine, nothing special as far as extra expensive plugins. This is just stock Unreal. And so I cover this on YouTube. Not how to do it yet. I'll be doing that. But showing that it is possible. So this is very commonly referred to as a virtual set. They've been doing it in broadcast for a long time. It's a lot more advanced now, though. You can move the cameras. A whole bunch of augmented reality stuff is happening. And you'll also see this in virtual concerts, which you're going to see a lot of virtual concerts in 2020, let me tell you. I've been talking to a lot of them who do it. So shout out. Zone VR did a great one. And The Famous Group, they're doing a bunch. And Fortnite too. [LAUGHS] They're doing a lot of those as well. So shout out. So I got into indie virtual production because of Richard right here. So he has a YouTube channel. And he was one of the first people to make this known on the internet that you could do it. So he is actually the person that got me to do it. So I wanted to shout him out. He has a studio called Virtual Star Studios. This is in Sweden. And this is his studio, work in progress. While this is probably unattainable to a single filmmaker, this is a pretty indie soundstage right here. This is an insert size stage. Like, you're not shooting a main movie here, but you can shoot insert. Just so that we know, this is an insert-sized movie stage. You could still get a lot done, trust me. But it's not a big, big green screen that you'd have on a movie. A pretty indie jib, Blackmagic camera, VIVE tracker, that's it. Unreal Engine. So that's a shout out to Richard. So this is a video he posted recently of the results from his system of green screen, jib, viewing it, moving the camera around. And this lovely gym here is made by Dekogon and is available on Unreal Engine Marketplace for $50 or something. And it's great. It's a very, very high quality gym. If you shot that cleverly, it would fool everyone. So that's mixed reality and a good example of it at an indie level. Next is LED Volumes. This is Mandalorian style. If you haven't watched the ILN behind the scenes of Mandalorian, good. This is again live action production, real actors, LED Wall now, not green screen. And we capture the actual LED while in camera. It's all in camera. If you want to see a more broadcast execution of that, if you go look for Katy Perry Daisies. The American idol performance is an incredible LED execution with a floor. And they did an incredible job with AR extensions and what not. This is happening right now. That shoot happened. Like, this is not the future. We are doing LED right now, even in mainstream, even in indie. Indie LED virtual production. It does exist. This is a stage in Australia. And shout out to Dave McDonnell, who is a visual effects supervisor experimenting with this. So these are two LED Walls, much smaller than Mandalorian. A similar size to what I did at SIGGRAPH. Not even curved, just right angled. And they're testing with it right now. This is indie right here. You can also use your computer screen, but the effect is not as convincing. Another behind the scenes from that same shoot. And if you look up Dave on YouTube, on Vimeo, and other places, you can see this stuff. This is test footage. It's looking Mandalorian-esque I would say. This is using a VIVE tracker and an LED Wall in Unreal Engine and nDisplay. Nothing else. So primarily the Unreal Engine and nDisplay part are free and what really enabled this. So VIVE tracker not all that expensive compared to the upper echelon of tracking, which we'll cover as well. This is the footage. They shot this in like, five minutes or something. They did not spend a lot of time with this. This is Alexa Mini LF. This is indie virtual production that's going down. If you want to hear more about that shoot specifically, you can check out the Wandering DP podcast. Patrick was the DP who runs this podcast and shot this demo. Go check out the Wandering DP podcast, listen to this episode. You'll hear a lot of nitty gritty about everyone's first time on an LED Wall. And shooting this, what they learned, what's going on, what do we want to do better. Hear it firsthand from these guys. I still don't have an LED Wall. So they are ahead of me. So those are the three types. Those are some indie examples of how it's going down right now. I think we're all familiar with the high end examples. But those are indie ones. Number two, hardware and software. What do you need? This is a preview of that on the right. This is pretty much it and how we're going to get started. So all in engine-- so it's 3D animation, essentially. So you're going to use Unreal Engine. You're talking about doing previs, you're talking about level design. That's all happening in the engine. You can use something called Sequencer for animation. It's very similar to After Effects, the key frames and timelines. Same thing for Maya. Same deal. And you'll use something called Take Recorder if you want to record live data. Like if I'm handheld moving my camera, or I'm doing live input with a gamepad, or a full human with mocap data Take Recorder. It's all built into the engine. You don't have to go outside of it necessarily. Other supporting 3D software. You've got to make 3D things, so we're talking DCCs like Maya. Just let it be known that Maya is still the tightest integration to Unreal Engine. So if you had to just pick one, pick Maya. Houdini is the best DCC in my opinion, love it. Sorry I couldn't presented at GDC this year with side effects. I was going to. I've been doing a lot of Houdini to Unreal Engine. Did it for six months. Also live streamed that. Not available anymore, but I did. And Blender, if you're a student, you're a filmmaker, you're like, I don't know what 3D is. You want to learn it, don't want to spend any money? Blender is free. Open source, you could fork it if you wanted to. Blender and Unreal Engine, the pipelines between the two get better every single day, even to rival some of the higher end implementations. So high end, Maya, Houdini. Beginners, Blender. That's all you're going to need. We're looking at mocap stuff. That requires its own software that comes with the hardware. I personally work with Rokoko. Love it. It's very fluid, very easy to get the data into Unreal Engine. Neuron Perception, those are both very affordable. XSENs, a little bit more expensive, but still probably much less than an optical tracking solution like Vicon and OptiTrack. If you're new to this stuff, I don't expect you to follow most of it. But I want to just put these names out there associated with the right categories so you can do your Googling. You can do your searching. This is very much a high level, let's run through it, not a deep dive. Any one of these categories, I could talk to you for a month about any one of these things if you wanted to. Another great piece of software made by Reallusion is iClone and Character Creator. And that's what I currently use for Cine Tracer. This makes it a very fluid way to make procedural 3D humans, dress them. They're pre-rigged to animate the face with an iPhone or other high end solutions. And iClone is kind of like this simpler way than say, Maya. To clean up mocap or bring in mocap from multiple places-- it's currently what I'm using, is iClone to bring in my face mocap data if I want to then edit it. Whereas into the engine, primarily you're getting raw data, very fast stuff. I believe there will be workflows to edit the mocap data, especially given that character rig is out in 4.25. But currently for me, it's much faster to work in iClone, so Reallusion iClone. And from an education point of view, if you are not aware of this and you're a filmmaking teacher or a filmmaking student and you're, like, I have never heard of any of these before. This is truly what Cine Tracer was designed for. You're a filmmaker. I'm used to cameras and lights. But I want to start doing previs. I want to start learning virtual production. That's where I see Cine Tracer fitting in, from an education point of view or a student point of view. You can actually track live cameras in Cine Tracer. Now you can do V Cam, which we'll be showing in Cine Tracer. It's a good place to start. Much more affordable than getting deep, deep into it. So it's a good way to get started is Cine Tracer. Hardware. This is a bit of a mess of a diagram, but starting with hardware. EC tower is always preferred, always preferred. You can get a laptop. Get a good one. Get a brand new one, a gaming one. Or if you have it, an older one, you can get started with it. But just know that we're pushing computers pretty hard. In this case, you can still use Mac. Mac OS is still fine. An iMac Pro, a Mac Pro, anything with a dedicated AMD GPU. All animation workflows, you're still fine. Rokoko and some of the hardware will work on Mac, but you can still get started with it. But you will probably end up, the further you go into PC tower world. Just letting everyone know that at the front. This is also a game engine. So we're talking game input, gamepads, Xbox controllers, PS4 controllers. Any controller you have, if you've ever played a game with a flight Sim sticks, midi, OSC, wireless DMX, SAIN, any of those things now work on Unreal Engine 4.25. Plug them in. You've got to learn a little bit of scripting, but you can make those all talk to Unreal Engine. That's what I do for a living now. It's super fun. I'll show you a demo very shortly. VR hardware is the breakthrough hardware. Steam VR, Oculus Rift. Tracking. The tracking capability that comes from VR headsets is amazing. Really good. I use it every day. iPhone, AR kit, iPad. Also, there's a virtual camera, which we'll look at what a virtual camera is just in this next slide-- for free from Unreal Engine. You can just download it. So if you have an iPhone or an iPad, you can do this stuff out of the box for free with that hardware as well. But VR stuff is a little bit nicer. Mocap hardware again. Rokoko, Neuron Perception, XSENs. Google those names if you want to learn about it. Join my mocap Facebook group as well if you want to know even more. iPhone X, you've probably all seen demos of this. You can do facial mocap with an iPhone. Lots of different ways. On the more mid to high end facial mocap, Faceware and I never know how to pronounce this other one, Dynamicxyz. I will be testing the Faceware system end of this month on YouTube if you want to see what that looks like for a more high end facial mocap workflow. So here's a quick video that has a lot of info. What does this look like when you do it? This is something I filmed in Unreal Engine, completely stock. And we'll watch through just this bit of it. This is like an all-- this was supposed to look like a high end game cinematic, borderline maybe a Love, Death, and Robots type film. And I'm filming it by myself with my hardware. And the interesting part-- we'll skip right here-- is that I'm filming it live like a video game. This is not rendered in frame by frame. I'm playing the game virtual production. And I'm recording it using the Atomos Sumo monitor. So this is live recorded, live captured. And it's supposed to feel like you're actually shooting it. So all these movements are being moved by me. I can move the sun. There, the animation for the characters are pre setup. But the capture of it feels like a live action set. And that's what I'm always after. And that's what it looks like. You can watch this better quality on YouTube. That's all in camera. And so we're now moving on to mixed reality. I'm going to throw a lot of names that you if this is your first time hearing about this stuff. But again, go Google it. This is where you want to start the search. So mixed reality, again, we're talking green screens and real cameras. You just bumped up a big notch in complexity. But the things you're looking for in Unreal Engine that makes this possible are Composure. You can live chroma key quite well now in Unreal Engine. You can bring the video into your computer using the Blackmagic or AJA plugins and capture cards. If you need to get video into Unreal Engine, that's Blackmagic. If you want to composite it, key it live, that's Composure. That used to cost a lot of money from a software point of view. That's now Unreal Engine for free. Things you're going to need to Google around with, media framework, virtual production plugins, and Unreal Engine. Again, included and free, you just need to turn them on. SPEAKER: Hey Matt? >>Matt: You want to have all these things here. Yeah? SPEAKER: There are a couple questions about the capture cards. Can you talk a little bit about the capture cards? And what your experience is with them and what is sort of required? >>Matt: Right. So the card that definitely works is the Blackmagic DeckLink 8K Pro. It's the one I have. It works perfectly with my Blackmagic URSA Mini G2. It is an SDI card with four in or out ports. That is the card if you can afford it. Just get it. Save yourself the headache. It works. As you come down in price, there is listed on the Unreal Engine website which cards are technically supported. Generally speaking, the DeckLink line from Blackmagic is supported. I'm not as familiar with the AJA cards. I do not have them. I haven't personally tested them. But I know studios who prefer them for different reasons. It's technical. It's a video engineering technical thing. If you really want to do this at a serious level, you're going to want to get an SDI camera that can output timecode and genlock. So look into that. The cheapest, most affordable-- which I bought-- again, URSA Mini G2 is $6,000 for the body. Hard pressed to find a cheaper, better camera for this than that. Still my opinion. Of course, the high end cameras work. Broadcast cameras generally work as well. Those are more expensive. You would know if you're going to get a broadcast camera. And broadcast lenses they're still $100,000 to $200,000 for such a camera. So you would know if you were going to do that. On the indie level, people are going to ask immediately does my a7S work? Does my DSLR work if I only have HDMI out? You can technically convert them using Elgato hardware. And it'll show up as a webcam. You will not have time code or genlock. And you can technically convert HDMI to SDI. You will also not have time code or genlock. So you will be able to pull off indie mixed reality, will not be able to get perfect sync with those cameras. So you can start, for sure. Like a a7SN, I don't have them readily available. I can show you demos of people that are doing pretty good work. a7S5 tracker, getting it into Unreal Engine, looks good. It's never going to be perfectly tight, but you could get started for sure, feel good about it, then upgrade your camera or rent a camera. If you want to just get into it and scale, URSA Mini G2 is your best start. Get a Blackmagic deck link card, 4K or 8K. Those are the get started quick kit. If you want to make it work with whatever you have, you can start with an HDMI camera. SPEAKER: So there's now some follow up questions about genlock and synching. You had a great video where you very specifically talk about the issues related with moving a camera and not having your practical elements sync up with your CG elements in the background. This is a real issue in virtual production. You definitely want whatever is real, whether it's humans or non-humans in the world synching with your background, especially with moving cameras. And that was a particular learning curve, right? And so hardware plays into that very specifically. Can you just spend a few minutes on that? >>Matt: Yeah. I forget the actual vlog, but you definitely want a camera that can output timecode and genlock. And there's nuances within just saying that alone could be an hour long talk. But if you're serious, skip to the genlock cameras. You need it to play ball. That's how Live Link is going to be created. Everything syncs to that. There's a lot of settings. If you look at the pro video section of the Unreal Engine documentation-- I don't have the link. It's linked in my YouTube video. There is lots to read there. The scope of just teaching video engineering on what is timecode, what's the nuance of genlock, that could be a very, very, very long talk. SPEAKER: This is hours of savings, right? The people who are going to bang their heads against the wall until this is solved. >>Matt: If you think you're going to try to make money with this, or really get serious about it, dedicate time. Don't mess around with HDMI cameras. Just jump right to genlock cameras. Like, how much is your time worth? SPEAKER: Super important. >>Matt: Yeah, in general. You can make it work. It's not going to be super fun, though, long term. SPEAKER: If you want to teach it, teach it right. Teach it with genlock. >>Matt: Yeah. Blackmagic G2 is $6,000 for the body. That's pretty good. That's a disruption to the cinema space, let me tell you, Blackmagic. SPEAKER: Yeah. Just put it on the line item and make it happen. >>Matt: Yeah. Skip right to the end. That's not even the endgame. Skip to that part, trust me. To mess around with non genlock and what not, it's just going to be annoying, bad, and hacky. And also not the way it's done professionally. SPEAKER: Right, exactly. >>Matt: You can hack around it. I could show you. But I would not recommend it at all. SPEAKER: That's not the point. The point is to do it professionally and right. Yeah. >>Matt: From an education point of view. If you're one person, you have an a7S and a VIVE and you want to make it work? Join the Facebook group. We'll show you. It's doable. But you're always going to have subpar results compared to a genlocked camera. >>Tom: Yeah, and it's one of those things where you end up spending so much time-- you want to spend so much time trying to fix those errors and work around. And then as a teacher, are you teaching workarounds? Or are you teaching a way to actually deal with this? And genlock and time code is something that in the professional world is a common thing. And it's one of those things that's divides the prosumer with the pro and the consumer. It's one of those things. So if you're teaching this, it's really important to make sure that you're [AUDIO OUT] that aspect of it. It's key to this all working together, especially once you start to get three or four different actors and cameras and everything all together. >>Matt: Yeah. From a teaching from a teaching point of view I definitely would teach the SDI. If the students then can't afford that and they're using HDMI cameras and DSLRs, the image quality is going to be great. And they can do that. But as they graduate into the real world, at onset, if they don't know how to work with a basic genlock or know the concepts of it, you're at a pretty big disadvantage. Especially when it comes to mixed reality virtual production, it's all-- there's a tight knit video engineering broadcast aspect to this type of work. Because a lot of this, at the high level, is live. Live to air virtual production. So this is not like, oh, we messed it up. We'll roto it. Like, no. This is live. Katy Perry, American Idol. Live. So this is precision. And now you're talking genlock. You're talking very high performance level of this. This is not messing around. So if you want to teach for you know the broadcast application for this and go to work at those studios, you need to at least teach it. If you end up executing at a lower level to just get it done, that's fine. But it definitely needs to be covered. SPEAKER: Yeah. It's not called real-time because you want to fix it in post. >>Matt: No. It's live. This all threatens to be live streamed to the world. That's the fun part, too. But it does require genlock and time code. SPEAKER: Well, thank you. >>Matt: Yeah, for sure. Any of these categories I could spend 10 hours just going into it. But I'm trying to give the overview so people know where they're interested, where they want to keep Googling, and not actually getting into the nitty of it. SPEAKER: That was an important one. So I definitely wanted to pause for a second. >>Matt: Excellent, yeah. Moving on, we're still talking software in mixed reality. We went over the stuff that's in Unreal Engine. More technologies to do all of this are in the engine itself. That's been a huge disruption to get that. You can look around. The software that used to do this, you can't afford it. [LAUGHS] Unless you know you can. It's very expensive. This is free. You can get started with the stuff for free. Other software that supports this, again, Steam VR. If you're going to track your camera with a VIVE tracker or controller, or the headset, even. Still people do that. It's fine. It works. High end tracking solutions other than the VIVE, we're talking MoSys, Ncam, Stype. Google them if you've never heard of them. Start looking around. Those are them. There's others, but those are the ones I'm working with this year. Other software that sort of sits on top of or between Unreal Engine. Aximmetry, Vanishing Point, who I'm working with currently. And we'll be demoing their Vanishing Point system very soon with Unreal Engine. The basket-- oh no, never mind. Disguise is another software suite that does something similar. I'm also working with them a lot recently. A lot. And ZeroDensity. Another name. Pixotope, another name. If you've never heard this before, it's like, what are these magical worlds you're talking about? Those are the names out there, just so that you can get a head start. And if you hear them out there, it's related to virtual production. Hardware, OK. PC tower. If you're doing mixed reality, PC tower. No laptops, no Mac. Just leave. That's it. That's just what this is. You need two PCI slots for the capture cards. You need two PCI slots for any GPU that could even have the hope of handling such a thing. My latest computer, the HPZA back there has six PCI slots. I might need more. We'll find out. Blackmagic capture card, AJA capture card, graphics card you're talking-- at this point, I heavily lean towards NVIDIA Quadro. NVIDIA 2080 ti will get you there. Don't use anything less than that. It's not quite worth it. If it's what you have, go for it. If you're talking new purchases, investments 2080 ti is like at the bottom of the barrel. It's incredible, but this stuff pushes really hard. Quadro is really preferred. I haven't personally tested the latest AMD GPUs. I don't see them out there in the field, anecdotally, all that much. They probably do work for mixed. But it's a whole lot of NVIDIA. A whole lot of team green. Camera tracking, again, VIVE. Controller, Oculus Rift works as well. VIVE is a little better. It's just a little bit better. If you have Oculus Rift in your school, use it. If you're purchasing, VIVE. 1000% VIVE. Again, Vanishing Point Vector, I'm working with that system this month. MoSys, I'll work with that system later this year. And Canon Stripe. Just names of hardware tracking companies. Throw them in the Google tabs right now. Just looking up what they are, those are the names you want to know. Cameras in mixed reality, we kind of covered it. My base recommendation, don't go lower than a Blackmagic designer URSA Mini G2. Just don't go below it if you're purchasing now. That's what Richard was using on his basketball demo. It's what I use. A lot of studios-- there's a lot of things special about the Blackmagic camera. I made a 40 minute video talking about it if you want to watch it. Just go for it. Don't use the other ones. On the high end, Alexas work great. Reds are to be tested still. Alexa works 100%. And broadcast cameras-- again, you know if you're going to buy like a $100,000 Ikegami or that type of broadcast camera. Lenses, EL lenses. Or, at least, cinema lenses. They could be EF or E mount. But they need to have a big focus barrel and iris, ideally, if you're going to do this seriously. Now at the indie level, I'm still using SLR lenses. I've been able to do everything I do. So it can be done. If you're looking to build an indie virtual production studio where you think you're going to charge clients or teach this at a level good enough for students to graduate into a working studio, cinema lenses. For a cinema workflow, broadcast zoom lenses Canon, Fugi on with serial ports ideally. And something I'm experimenting with, like, today is the new extended cook. I did an XD data by ZEISS, ZEISS extended data. This stuff is crucial. Currently, this is one of the main bottlenecks in the industry is getting lens data back into the engine. But I'm working very hard, and so are a lot of people, on making it easier. ZEISS extended data is going to play a big role moving forward. So if you buy lenses like the CP 3s with XD, Fuji Premista. Higher end lenses have this data. It will make your life a lot better, and your end results much better as well. And I could talk about that for an hour. Studio stuff. This is a film shoot. You need a green screen. You can have a cheap, muslin one. I personally recommend the Lastolight backgrounds back there. That's two of them stuck together. They're pretty stretched, out perfect for keying. Really easy to do, and very lightweight. They're portable. I really recommend it. You can also just get muslin hanging off a backdrop, paint a wall green on the indie level. But it's studio. So if you're a school that has a cyc wall that's white, paint it green and get started. And it's film production for the teachers and the students. Do you want to make a short film or a music video filming real people with lights and all that? We haven't even touched lights, that's beyond the scope. You have to light all this as well. I could go on for a long time. But let's look at some quick mixed reality here. This is a YouTube video you can find on my channel. So the mixed reality part is me walking around. So I'm in there. And then I'm also doing VCAM at the same time. I'm doing one more thing as well. And there's the basic AR, which it may be hard to see it, but there's actually a 3D camera moving around in there. And if I foreground it, that would be AR. So this is a mixed reality AR and VCAM all at once. And I might need to skip through it. Oh, man. It makes it real tough to click through here because of the old Google slides UI. But I'm going to get it. There it is. Am I going to move it? It's coming up real soon. I moved it, I missed it. OK, here we go. As an additional thing-- Lewis referred to this before-- I'm going to move a real light and then also move a CG light that's tracked to it, just using a VIVE controller. Nothing crazy. And the demo here is that I'm lighting the real world and the CG world in the same manner. Not perfectly, but it's a concept piece. But if I go stand in front of that light, I get the light. And so does Shinbi here, a free Paragon asset. I hate to refer to her as a Paragon asset, but that's what she is. She's a great actor, awesome. Really good to check out really high end 3D humans as well, Paragon assets. Definitely go get them. So yeah, this is that demo. This is mixed reality. This is what I've been messing with, and AR, and light matching. And now that we have wireless DMX and SACN, I'm going to be matching sky panels virtually for the real world, for mixed reality-- whoops, I hit Escape. There's a lot more to cover with that. Let me hit present instead. Loading. So that was mixed reality. Islands, that's a big one. If you can get through that one, you're ready, perhaps, for this one. This is LED. LED walls, Mandalorian. How do we Mandalorian? It looks simpler, because just says Unreal Engine End Display, but End display is the thing you're going to want to Google. You're going to want to search. You will end up on my Facebook group, however. It's very new technology. It's different every day how this works. Literally every day. Cutting edge stuff. We do it differently than how men-- it's different now. Things move very quick in this space. I'll show you-- I already showed you. While it's pretty challenging, and maybe a little expensive to get the walls-- which we can address as well-- people are doing it. These studios are up in mass already. Way more than you would ever expect, this is being done right now. Unreal Engine End Display, Google it. That's the core tech. There's a lot more to go into, however. But that's where we should start. To go deeper, you'd have to be already doing it, I think, to get much from it. Other software, Steam VR. If you want to track at a consumer level, it can be done. Demos are not bad. A company I'm working with right now very closely is Disguise. Something you can Google. It's beyond the scope of this to discuss how Disguise and Unreal Engine interface, but something to look at. Hardware, PC tower. Everything else, just get out of here. I work with HP Z8s, very much recommend it. Zero Density, a company that works in this space for, again, live execution broadcast, high precision, high tolerance, they standardize on the HPZ4. The one even below it is what they're running for all these systems. NVIDIA Quadro GPUs, don't use anything else. It's those. NVIDIA sync cards, you literally cannot do this without them. I just simplified your purchase order. That's what it is. SPEAKER: [LAUGHS] >>Matt: Camera tracking, VIVE, MoSys, Ncam. And in many cases, you're going to have to go to something optical, Vicon, QualSys, OptiTrack. Depending on the size of the volume, again, a lot to cover about tracking back in volumes depending on the size and the shape. Cameras, bare minimum, URSA Mini G2. I've seen it done with others, but just don't. It's so much harder. This is very technical. Make it easy on yourself, get a G2. Pretty affordable. You need to time code. As a reason why, the time code and the frame-- like the frame coming out of the G2-- the LED Wall syncs to that. And so does the render engine, Unreal Engine. The heartbeat of the entire system is coming out of the camera. So your camera heartbeat has to be pretty strong, genlocked, time coded, etc. It's not really going to work out so well with any other type of camera. High end Alexa, high end broadcast cameras, they all have that built in. It is a professional feature to have genlock, SEI, and time code. We need that. So if you're looking at LED walls, I mean, you should be looking at these cameras already. Lenses, broadcast lenses with serial ports, EL lenses, really any, because you're going to be doing external encoding possibly, or extended ZEISS data, Cook extended data for certain systems. I'm still working on that. This year you'll see me talking about that more on YouTube as I'm personally testing at the moment. LED walls, how much is an LED Wall? It costs money. They do. What you want to start to look into as a start are ROE. Saves you a lot of headache. Look at the company ROE, shoutout ROE. We're talking a little bit already. A model, the ROE Black Pearl 2 is what we used at SIGGRAPH. Possibly what was used at Mandalorian, I think we can say that at this point. The higher end are the ROE Sapphires, with a lower pitch density. Those are pretty cool. And those are better than what they used on Mandalorian. This space is moving very fast-- very, very fast. And with ROE, you're going to very commonly see Brompton processors. So if you want to start to price these out, look for people that sell ROE Black Pearls and Brompton processors and the needed networking hardware that goes with it. You'll be able to price them out to find that. I heavily recommend consulting with companies like Lux Machina, who did the design of the walls for Star Wars and those type of projects. Also, on set facilities with Asa Bailey. These are great consultants in this space. Or, how do you virtual production LED Wall? There's a lot. You want to work with some consultants if you're looking to invest into a stage like that if you know very little about it. Projectors work. You can do this with a projector. If you come on the Facebook group, someone posted a short film trailer that they did with it. Looks great. You have some different lighting requirements. You have to have the footprint of the stage to use a projector instead of an LED Wall. It can be done. It looks great. It's pretty awesome. There's some commercial projects that are not public yet that I've seen and talked to the people doing them at high level executions for big brands that you know. Unreal Engine projectors, they look very cool. Much different execution on set, though, however. And then LED lights. You need to have your lights talking to Unreal Engine. You can do it. That's a whole category of learning, the ETC and the grandMA. The pipelines right now, Unreal Engine can actually speak to the LED lights through sACN, Art-Net. I'm starting to just get into that. You can actually do it directly in Unreal Engine now, too. On the bigger sets, you're going through a grandMA, you're going through [INAUDIBLE] pipelines, all that stuff. I work with a lot of lighting designers right now. And then, regular old film production. That's just the tech. That gets you an LED Wall and a camera. But you need real people, you have to light them. You need to have audio, you need to do all that. And LED walls look better when there's a really good practical set. So the art department is still here. That's a whole lot to get done when you start the LED walls. That's an overview, though. Companies to look into, things to search. There's a quick video. This is the LED Wall shoot that I was the director DP on. That's an LED Wall back there. Most people maybe have seen this already. Alexa LF, ZEISS 28 to 80 full frame lens, no XD data, running Unreal Engine-- I forget. Some special version, probably. But this is the stage. It's relatively small compared to Mandalorian. Everything is pretty small compared to Mandalorian. But still, this was the LED shot I did. And again, kind of what fuels what I do is I would like one of these walls. I would like to keep filming on these and designing software. So this is the LED demo that I was on. And I showed an indie one earlier on too. OK, so that's a lot of tech. If you're brand new, that's a lot of names to Google, but good start. That will get you quite far into the process. So step by step, where to start, where to go. This is kind of the way that I see it currently. You start with previs all in Unreal Engine and you're doing in engine stuff. So you need to learn how to make a simple set in Unreal Engine. Use Quixel assets, they are free. Mega scans, use Paragon characters. Also free. Photo real humans and photo real sets for free. That's very new. That's did not exist when I was doing this before. And use Sequencer. Google it, Unreal Engine Sequencer. How do you make a little movie in Unreal Engine? Previs, start there. Next you're going to go into VCAM, which is where I have a tract camera, using VIVEs and Oculus Rift. If you have an expensive camera tracking system, you're an animation school with a mocap studio, you can obviously use a VICON TRO or something, very easy. Now you're going to use Take Recorder and setup that little sequence you made in Unreal Engine. Film it live now using Take Recorder with a handheld camera. So now the camera person is a live horse into the engine. I do a lot of that. If you can do that, next step is mocap. Now do that with a human. Put on your mocap suit, learn that workflow. Live record the camera and the human in Unreal Engine. Make a movie. I'm still working on that one, still tricky. Mixed reality is next. So now you've done this stuff that's basically all at your computer, essentially, with a little bit of human interaction. Now you have to bring a real camera into the mix. Now it's film production. Learn how to get the video footage into the computer. Blackmagic, look at the pro video section of the Unreal Engine documentation. You might as well just print it out and put it on the wall. Very, very important to get you through that step. Composure for compositing, and learn how to live record it. A lot of nuance with that too. Next would be LED Wall or projectors. I think projectors are probably a lot more accessible for some people. So you could just throw a projector against the wall and learn how to do nDisplay. That's the key word here. If you want to do LED walls and projectors, nDisplay. That's what you're going to be working on. That's the progression I suggest for teaching and learning. And you're also looking at the road map from my YouTube channel. Once I start the tutorial series, this will be what I am going to be doing. I have to acquire all the hardware and knowledge to do it first, but I'm well on my way to do that. SPEAKER: Hey Matt, real quick. Explain what nDisplay does for folks who just aren't familiar with it? >>Matt: I don't know if I can. It's a doozy. But essentially, you have to, let's say, calibrate the 3D world to the actual camera you're shooting to the actual LED Wall that you're filming. All those things need to be perfectly aligned to make that effect work. nDisplay also as a core technology in taking, say, multiple LED walls and stitching multiple Unreal Engine renders together. So say you have a gigantic LED Wall and you don't want to spread 4K resolution across 50 feet. You would actually take four 4K renders and stitch them together while keeping all off-axis projection and all the tracking together. nDisplay does that for free. A little tricky to do, you can imagine. But that's what it's doing. So if you want to sync a real camera, Unreal Engine, and an LED Wall to make this trick work, the evolution of rear projection, essentially-- that's like rear projection, the video game. nDisplay is the core technology that does that. There are others, but they cost a lot of money. nDisplay is free in Unreal Engine currently. And you don't need a big LED Wall. I wish I could show the demo. I don't want to do it because they're from the Facebook group and I didn't ask permission yet. But there's people that do it quite convincingly with a little toy on their LED monitor, the computer monitor. Same setup process to do it for a little TV as it is for a gigantic LED Wall. So that's nDisplay. Syncing real camera, Unreal Engine, and an LED Wall, or your little TV and a stuffed animal, maybe a Fortnite llama. That's the core technology that's needed in Unreal Engine. SPEAKER: We like free. >>Matt: Yeah. Yes, for sure. The hardware is not free. That costs money. So it's great that the software is pretty accessible. Last is just resources, so we kind of covered this before. But I guess there is specifically a virtual production part of the Unreal Engine website, check that out. Unreal Online Learning, a lot of free courses here. Not too many specifically on virtual production, but enough to get you into how you make a landscape, how you do certain basic blueprint stuff. If you watched all of those, you'd have a lot of knowledge to get in there. Might switch to game development all of a sudden like I did. Like, hey, I can make a game. Maybe I should. Unreal Engine YouTube channel. The canonical UE4 tutorials. I've watched all of them twice. And so that's how I know Unreal Engine. If you want to know Unreal Engine, that's a good place to start. Watch all of them. Watch them all. Some of them are not up to date, but the concepts are still pretty much universal for any real-time rendering. If you want to-- I'll be making these tutorials eventually. But how do you make an Xbox controller move a camera like you see in my demos? I'll show that specifically. But I learned how to do that by how do you make an Xbox controller move a character walking around in the world? It's exactly the same, except at the end of that tutorial, you're moving a character around. Those are the sources. And then this, the documentation from Unreal Engine. That's going to get you a lot into there. But watching videos is very helpful versus reading. Not everyone can read and learn. A lot of people like to watch things. I can't stress this enough. If you're thinking you're going to do Unreal Engine, virtual production, especially with an LED Wall, you need to join this Facebook group, Unreal Engine virtual production. Hop in there. I probably have, like, 1,000 pending requests right now. That's the place. That's where all this source material from the indie stuff is happening. That's where I go to learn Unreal Engine, and it's my group. When I need to learn something new about it, I speak directly to Epic Games employees. I speak to the developers of the hardware solutions, HDC is in there, not to blow them up. But this is where it's happening. There's other places as well, for sure. But this is the one that I can tell you-- and I manage. It's happening right there. And then my YouTube channel is Cinematography DB. If you've ever looked up Unreal Engine virtual production, you found it. It was that. And so right now I'm doing the virtual production vlog, which is documenting, getting me up to speed. Next step once I'm up to speed is the Unreal Engine VP tutorial series on how you should do it, and how to do it. I need to get to speed, though. It's probably going to take me to the end of the year to get all the stuff I need up and running, all the partners lined up to do it. But once I can do it, I pretty much want to make that tutorial series just to get more people into the ecosystem. So that will be that. And that wraps up-- oh, I went the wrong way. That wraps up this lovely Google slides. I'm supposed to shout out this free slide theme here, Slides Go. Lovely. I like Google Slides. And feel free to reach out to me casually, Twitter, Instagram that's fine. LinkedIn is specifically if we're working together. That's primarily how that works. Nothing personal, it's for if we're literally doing jobs together. And then email is less preferred. And if it's about contract DP work, I have an agency which is Gersh. If you're working at that level, you probably already know Gersh. So we go through those contacts for freelance DP work or consulting in this space. That be the presentation. SPEAKER: That was pretty awesome. >>Tom: Amazing stuff. SPEAKER: Yeah. And thank you, by the way, to all the folks in the chat that are contributing. It seems to me like we've got a pretty decent group of folks, professional virtual production people in the chat that are helping contribute. So thank you all. There definitely been quite a few questions that have come through as you were presenting. >>Matt: Really? [LAUGHS] >>Mark: Hundreds of them. SPEAKER: Especially talking as Matt Workman tends to talk, at a pretty good clip. So I think there were questions. I think you answered questions about the capture cards, and that was really good. And I think that some of the questions that were asked you address later on in the presentation. There's people that we're not 100% sure about how Cine Tracer fits into the big picture, what it is exactly. Once again, Cine Tracer is an application that Matt makes that sits on top of Unreal Engine that sort of simplifies a lot of the visualization process. It doesn't have IO, in other words, you can't put things into it. And what is the output from Cine Tracer? So everyone's 100% clear? >>Matt: My ideal output is an 8 and 1/2" x 11" storyboard that you print out and bring on the set. I try to keep it very practical. And we're not even outputting movies. You can record it with OBS. But I very much built it for people to in 30 minutes have an idea of how things should kind of be set up, throw a couple of cameras in there, mess around. Storyboard them right there, you just capture it, print it out, bring it on set. That is the workflow of Cine Tracer. It may be evolving into a more virtual production workflow, but we will see. But if you are serious about virtual production, especially at the professional level, just jump right into the Unreal Engine. That's why I've been really pushing that. It's slightly confusing at the moment because I am building something currently. All this stuff you see me shooting is my framework that I'm writing. But that's not available yet. And doesn't have a name. But I do see people confusing that with Cine Tracer. It is not, though. It is nuanced. If you're brand new, you wouldn't know what from what. But that is actually something that's not released yet. But I kind of alluded to it. It's a framework that should hopefully allow Unreal Engine to be used for all in engine, green screen, or LED. Just trying to unify the controls for that. So that's not out yet. SPEAKER: Right, right. >>Tom: And that will be an amazing asset to have for folks trying to do that. SPEAKER: There's also lots of folks that probably are experimenting with DSLR cameras and HDMI. And I think that as you start watching your very first videos, you are working with those. And you are experimenting with those. But if you go and watch, you had pain points. And you moved through them because you wanted to get to the fun stuff, right? And so I encourage everyone to go and watch-- I think there's what? 14, 15 videos in the vlog? There's a bunch in there. So you can go and watch and experience. They are at the most 15 minutes long. Most of them are about 10 minutes long. So it's not like you're watching an hour and a half per video. You can go and see the progression and why. And you can go and watch the genlock video. And it's very clear why you want to go and do certain things. So just go check it out. Because it answers the questions very clearly on why you want to get past these pain points. And there was one particular comment that I found really telling, which is you want to get to the art. You want to go and do filming. You want to go and do virtual production. And you don't want to get mired and caught up in, oh, I've got to get this camera set up. And why is my frame slipping from my practical? And why is my green screen not working well? And why am I getting weird edges on my characters? You want to get past that as quickly as possible, because you want to tell the story. You want to do whatever production you want to do. And you want the lighting to work. And all these things. And so I think a lot of what you're presenting here-- and a lot of what we want to be able to teach to students, or if you're a student you want to be able to put into your portfolio, or if you need to go get a job and be able to present on set-- is that I know how to solve these problems. And the analogy that I used at first, driving the car as it is being built, we want to be able to show up some place and say, I can drive the car and fix it while we're driving it. And so I'm invaluable to this virtual production workflow. Hire me today because I already know what I'm doing. I think that's really important because we are driving the car as it's being built right now. And so what you just presented was super valuable for those of us who haven't gotten in the car yet. >>Mark: I've got a question which relates somewhat to that and the art of things. One of the things I think a lot of universities are interested in using virtual production is because it is an incredible tool for learning. Would you say you have learned a lot? Not just about the technical aspects, but about being a DP and the way in which shots work? And have you changed the way in which your shots work now because of this? And is it actually helping to drive a whole new range of creative visions within your work? >>Matt: Definitely. Yeah. It's like the Matrix. It's like a video game. Specifically on one of the last videos I did, it was a cyberpunk looking film. And I had a Lamborghini car, the doors opened up. And I drove-- I played the video game of it coming in, like spinning around. And then getting out. And he talks, gets back in, and he peels off. And does a drift around the corner. I've never filmed that in real life. Ever. I can't imagine. I've shot over 200 music videos and I've never filmed that. But I filmed it in Unreal Engine. And I approached filming it the way that I would have filmed it in the real world. So have-- >>Tom: And you did your own stunts. >>Mark: [LAUGHS] >>Matt: I did. Yeah, please don't watch any of my dancing videos. [ALL LAUGH] I'm learning mocap. And that's part of it. Yeah. I'm filming and doing things that I can't do in real life. And that's why I like doing it. The helicopter? I've not filmed in a helicopter. I don't ever really want to, to be honest. They're quite dangerous. But I filmed the war movie in a helicopter, a short one. And then I filmed the cyberpunk thing with a car. And I approach it the same way I'm filming real stuff. And I build the tools and make that possible, and feel like it as well. So, yeah. I am learning. Because I don't get to film those things in reality very often. >>Mark: I think the first time I saw any kind of British production system was actually in the special features for the Tintin movie. And Steven Spielberg actually with WETA. And I thought, oh, I'll never be able to get my hands on these tools. And he was like a kid in a candy store. Because he was able to do shots that he's never been able to do before. And he was absolutely loving it. Now, everybody can do it. >>Matt: And I think there's something about one person alone being able to iterate without the whole machine around them. And I get contacted by directors at pretty high levels that are interested in the same exact thing. Working on a big, big end pull project, there's a lot of people around you. And you don't just get to do whatever. You very much have to make things-- you know, decision by committee. And what I've really found is interesting, while I love working on set with a film crew, there's always time. You're on a set, everything costs time. And everything is money. So to be able to just sit in a virtual world and just take my time as fast as I want to explore it, that's really valuable. And I think a lot could be learned there. And so in an all virtual execution, all you need is that cart. You don't-- that cart is very much my dream cart. You could do it with a laptop to be honest. Even on a green screen or LED Wall, it will require some people. But it will require less. That's not necessarily getting rid of jobs. But you can execute being on Mars with less people than actually being on Mars. And so I think that there is a lot from an educational space-- like if you threw three students into that space, it was all set up, and you just let them do whatever, then a lot can be learned very quickly because there's not as much in the way. It doesn't move as slowly. It's very fast. It's real-time. The real-time part allows them to experiment and learn. And even as a professional, and the big, big directors and the players I've been speaking with, it's like, they want that, too. They want freedom. They want to be able to make maybe edgier content, things that are more personal. The content that wouldn't necessarily happen on a mainstream level. Virtual production can enable them as well. So it's good for students and it's good for high end executions as well. It's just more control, maybe more time with it not costing as much money just experimenting. Being alone in a room with a piano is kind of how I look at it. And for me, a drum set. Like, that's my drum set back there. Just leave me alone. Let me just figure it out. I'm not beholden to all these people around me because it's so expensive. I think we're very close to being able to enable a new generation. And a lot more people could have that freedom to just mess around and have the results be as good as, you know, what top professional results would have been like 10 years ago. >>Mark: It's so cool. >>Tom: It is. It's really wild. It's almost like having a laptop. And suddenly now you can take your computer outside the office. And do things outside the office. As a DP, I could set up in my garage. And I can-- oh, I've got an idea. I can go out to my garage. I don't have to wait until shoot day and have everyone around and then maybe squeeze in a little bit of an idea in there into the shot schedule. Just not going to happen. That freedom to do that at home. I mean, can you imagine today going into the shots and people are like, go check out the shot I did last night. Let's see if that works. Yeah, at home. >>Mark: I have another question for you in terms of where the people who are using virtual production are coming from. Are they coming from on set into virtual production? Or are they coming from the effects into virtual production? Because I'm hearing both directions. >>Matt: They're coming from everywhere. One of the main drivers that I hear, because I have a pretty deep connection to the music industry, is the music industry. En masse. Extremely interested. Every concert for 2020 is canceled. That's a big moneymaker for the music industry. But guess what? A lot of virtual concerts are spinning up right now. And they need to be good, really good. So the music industry is pushing that hard. I'm working on a couple, kind of publicly. But like that cyberpunk demo is a pitch to some record labels to do all CG music videos where we don't even have the artist necessarily. He or she could do the mocap for the face if they want to. That's really a specific kind of process. But pitching that and being like, is this good enough? You know to do this? That costs way less than you would think. I filmed that in my basement quote unquote "for free". I'm not working with anyone right now. I have the hardware and the setup. But like, let's do a couple. The green screen virtual mixed reality stuff? That's going to be every concert, every product reveal for the year, everything is going to be that, just so that you know. All the virtual production people here probably can't even read all the emails they're getting at the moment. The bid is out. So that's where it's coming from. So yes, VFX and film clearly. But everything. If it ever was a production or an event, it is moving into this space. So DJs, music, sales teams, everything is moving into virtual production. SPEAKER: And probably once it goes that way, not all of it will go back to practical. I mean, why bother? >>Matt: That's a sad story. I don't own want to talk about. Yeah. [ALL LAUGH] Yeah. There's going to be-- I mean, the majority of my friends are film production people who haven't worked for four months. And that's not a good thing living in LA and New York City. And I have a lot of friends who will probably never return to the events industry from the lighting side of things. I built lighting plugins, I know that industry pretty well. They may never return. So there's definitely a practical need for it right now in working remotely and getting things done. Clearly this is a massive priority. It's what my inbox is every single day. SPEAKER: But it's a good time to skill up on these particular skills. And it's a really good time to realize that Unreal Engine is free. A lot of the stuff that you talked about is free with regards to the software. And a lot of the learning material from Epic Games is free, which you pointed out. >>Matt: I think it's the time to look at what you do-- any industry. Film in it. Film and VFX is the most straightforward move into real-time. Do I even need to pitch? It just makes so much sense. But basically, any industry, it's time to look at what do I do, and how can I do it in Unreal Engine? And let me tell you, I talk to people that are not film, that are not gaming, and they want to build sales configurations. They want to build virtual walkthroughs, virtual sales presentations. That's every company and every product in the entire world needs that now, and needs it in Unreal Engine real-time. Pixel stream it to the web. Pixel stream it to your phone. There's all sorts of ways about it. And film and VFX? It's super-- watch The Mandalorian. It's like, what more do you need? But I get it from every angle, specifically music and now sales. Because I worked in commercials and advertising. Advertising agencies are going to start deploying in Unreal Engine. SPEAKER: You know, I think this is really important. So if you're a traditional university, and you've got an audio department, you've got a film department, you've got a television department, and you've got all these departments. In some cases, they're not even in the same schools. One is in the School of Communication, one is in some art school. And that means that they don't collaborate very well. It's pretty valuable, if possible, for them to kind of find a way to get together. Because there's so much demand for these unicorns coming out of schools that know enough about lighting, audio, how to run a camera, and how to do these things in real-time. Because if they're going into school and they're going to come out three to four years from now, that landscape is going to be pretty different. And the demand for real-time is going to be pretty different. And a school that can tune their curriculum and change and start to realize that the pencil has changed, right? The tool has changed for telling stories and for doing commercials and all the things you just said. And now is the time, especially if we get any time to breathe, like being at home, to go how can I take what is available and this information and in the fall or winter or whenever, start to think about it differently and teach a little differently. And prepare these kids a little bit to address what's going to be a year or two from now or six months from now. >>Tom: Right. What's going on now has pushed the timeline of all of this so far forward so much faster. We've been-- Luis, you and I have been talking virtual production in schools for years now. And trying to make sure that every school that's teaching, like Matt said, any product visualization or anything to jump on real-time. And especially the film and TV schools. And we're having to shake them by the shoulders and be like, you've got to teach this. Because by the time these kids graduate, the way you're teaching them is already outdated. And in the last two or three months, I think that's just really pushed it even further. People who were dabbling and experimenting are like, let's do it. Let's go all the way. So the floodgates have been opened. And the students that are graduating in a year or two years? This is where to start now. At least that's my opinion on it from where I sit. And I see the industry moving. It's such a huge opportunity, too. If you can have that under your belt, when you hit that job market it's going to be a much better place. If you don't, you'll see a lot of job postings. And be like, man, I wish I learned the Unreals. >>Mark: It's something I'm certainly hearing a lot of from universities that they really, really know that this is happening. It's great to hear. Because sometimes universities take a while to find if this thing is actually an important thing. And as Tom and Lewis can attest, they've been saying it for a while. But it has become very focused, even before the current situation. Just the start of this year, and I think after SIGGRAPH last year, it became obvious to everybody this is going to be revolutionary in a lot of industries. SPEAKER: Which makes us really appreciate you coming on, Matt. And a lot of this information. Once again, the fact that you took your journey and put it out there for everyone to see, and then can come up here on the stream and speak with authority about hey, listen. I bled in front of you all so that you don't have to bleed as much. We really appreciate it. [LAUGHS] >>Matt: I'm still bleeding. I'm still bleeding out over here. [ALL LAUGH] You just don't see it. SPEAKER: We're coming into having been on here for almost two hours. So if there's any final questions, maybe we can take one question here. And really, again, thank you very much for all your insights today. And thank you all for participating, for those of you who attended and asked some really helpful questions. Anything, Tom and Mark? And Matt? >>Mark: It doesn't feel like two hours. It feels like 10 minutes. I could sit here for the rest of the week. SPEAKER: Well for those of you who don't know, Mark is actually in the UK. So he's probably ready for dinner. >>Mark: [LAUGHS] Always ready for dinner. SPEAKER: But again, thank you Matt. And we'd love to have you on again, maybe in a little while after your journey has continued on and you've figured out some more stuff. You're always welcome. >>Matt: Thanks for having me. I'll trade you an LED Wall. I'll come right back in. I'll show you how that LED Wall looks. How about that? SPEAKER: That sounds like a great plan. >>Tom: That's a deal. >>Mark: I'll see if I've got one. SPEAKER: I've got an LED monitor, will that work? >>Tom: [LAUGHS] SPEAKER: Only if you're doing a really small movie. >>Tom: We'll pull all of our phones and iPads. And stitch them all together with nDisplay. It'll work. We can make it work. SPEAKER: All right. Well, again, thank you all for showing up today. I will remind you that these typically get turned into YouTube videos that show up on unrealengine.com's YouTube channel. So if you missed anything today, come check with us by Sunday or Monday and you'll be able to watch it again. And next week-- next Friday-- we'll have another stream for you. It'll be focused with Linda Sellheim again on secondary education. And we've got some other streams coming up that we're really excited about. So thank you again for joining us today. Thank you again, Matt. Thank you, Mark. Thank you, Tom. And we'll see you next Friday. You all have a good weekend. And please be safe. And cheers, everyone. >>Tom: Bye everybody. >>Mark: Take care.

Channel: Unreal Engine

Views: 47,310

Keywords: Unreal Engine, Epic Games, UE4, Unreal, Game Engine, Game Dev, Game Development

Id: i8Re7HPwh5Q

Length: 105min 30sec (6330 seconds)

Published: Tue Jun 02 2020