vMix – Creative Compositions with Masks

hello everyone welcome back to streaming alchemy i'm john mahoney and on today's show we're going to be taking a look at a capability in a lot of production systems called masks and the way you can leverage masks to do more interesting and creative composition of the elements of that you have inside a given production but before we get to that uh i'd like to invite everyone to please if you have feedback any questions just post them in the comments below and also you know we'd love to have you join us live on air so if you'd like to connect with us and become part of the show here in the studio just type in the link we have below and that's also in the show notes and that would be uh you know somebody here will get you on and we'd love to talk to you we'll try to get to all your questions here live during the show so rudy welcome thanks for joining us today so to get started i mean i think one of the first things is when when we think about uh masks it's a general construct it's it's basically alpha channel a way to take a map and lay it on an asset and have parts of that asset be transparent so that's essentially what mass are so the best way to cover this since we'll be using vmix is we'll use a couple of different cases that demonstrate different aspects of masking and how that can be something that you could incorporate so for our first example we're going to be taking a case where we want to talk with doctors that will be you know joining our production from different hospitals around the city so we can start with something like a map and then as we add doctors oh ken how are you thanks for joining here uh as we add doctors you could have them pop up and go away and as you'll notice what we have here the doctors are actually not a square cutout but it it has a bit of a skewed uh slanted rounded rectangle around it so i think if we want to look at you know the way we can do this type of stuff uh let's dive in and take a level down look inside of vmix so if i have this is the asset and what i'm going to actually do now is let me jump in and open this and talk a bit about what we have here so james thanks for joining us great to have you here so inside this asset we have a video plane and we have a series of layers so the video is the base layer and then we have a name for the hospital that doctor is associated with we have the frame around them which blends in with the map position pointer that we have here and you'll notice that everything in the background here is transparent so usually that checkerboard pattern is a way that vmix tells you that the background element there is transparent and anything that you layer this on top of you'll be able to see that through this so let's start with taking some of these things off here so we'll remove these extra layers and now we'll take a look at how we edit the key so the key here what i did is i jumped to the color key and down at the bottom there's something called key fill input and what the key fill input basically does is it accepts an alpha channel mask something that's composited of different grades from of pixel color of pixel luminance from zero black up into 255 which would be white and it uses that to create a mask basically say anything that is inside the white space is going to be fully transparent anything behind the black spaces will be fully uh will be fully visible for the white spaces and fully transparent for all the black spaces and then there's a gradient in between that so things will become more or less opaque based on how white or how gray the pixels are in this mask so for us what we are using here is slide this over this mask right here which is a hard cut mask with a white center for the solid image we see here and black all around that for the transparency we want and the way we built this then is this was just an image we went into photoshop created a rounded rectangle skewed it and put this together now as our mask and so all we would need to do is we go to the key fill input and select any mask that we have set up and that becomes our alpha channel for this video and when we put this all together uh we're adding in the layers we now have sort of this full composite one quick note is that this little map pin uh that you see here this actually is also alpha channel it's a png file which has built-in alpha channel and it has a little gray gradient down at the very tip which helps this image blend in with the map so if we switch this back and look at the map again you can see how all of these elements go together this is all transparent all around it and we have this little map with a little pin point sort of showing the map through it so you get that sense that the two are combined and even though you could conceivably do all of this with just sort of cropping an image uh aesthetically this makes it a little more dynamic because there's motion in it just by adding a little bit of a slant it makes the text we want to put on the side for what hospital they're associated with a little more readable than if it were purely in a vertical position and overall i think it lays out nicely with multiple elements that you could have late on this map so that basically covers the core concept of what a mask is and how that that that can be leveraged but let's take another use case here that actually starts to leverage how you can now place things on a com composite you're doing now that you have mask to work with so what i'm going to switch over to next is something which is going to use puzzle pieces so let me just set this up here so if i were to if i were to think of the layout that we had last time we had the doctors all in different positions around a map but since you are cutting things out you can actually create voids inside of your image and then add new things to this so say you were this is something we put together for mobile app development and we were talking about you know it starts with having compute assets in the cloud you have software that works with it you then composite data and other content-based elements and that goes back and it's delivered to a mobile device which then returns data and feedback back to the cloud in a loop so this is something that we just put together here but because we're using alpha channel now we can merge all of these videos in sort of a great layout for communicating the connection between things and because we have this overlap it's sort of smooth and seamless but the puzzle pieces also have an implication it gives you a little more artistic control not just of the layout but communicating that these are parts of a whole so fan films ah great to hear from you thanks for joining us i appreciate it so uh so this now it can be used for things that may be brand based so maybe you have a logo that could be a great alpha channel maybe it could be things you want to do that are more creative between different elements where you can have sort of cutouts that overlap them in different ways like the puzzle here but all this is doing is it's using the exact same elements so if we switch back to the desktop you can see that i have all of the different uh masks set up for the different videos we have and you can actually see them here applied to each of these videos and then all we're doing to put these together is we're building them using layers and so if i just take a quick jump in down here you'll see that i've just taken all these puzzle pieces set them up as layers and inside we scale them down the half size and positioned all of them so they line up here inside the uh the layout with all of the video playing so again another way to take the core construct and start to look at it uh so josh thank you for joining us uh so yeah i mean we think this is these are sorts of things that once you think of the the technology you can move beyond that and start to think about the creative aspect of the application and we think this is this is one of those areas where you can do that you know masking is that type of tool so these are two very straightforward things but uh one of the other things to consider even though what we've shown here so far has all been around uh static mass is that masks can also be dynamic so let me just pop this around here move this over so one of the things that we can do is we could create a little call out so say we're at a seminar and we want to say this is the next round table that we have going on we could do something that would push something mobile as a mask into the upper corner and say what's coming up next so you can see that we have not just the motion inside the video we're playing to advertise that the cloud computing round table is up next but the mask around it is also now moving and this can help draw somebody's eye to something you want to show on screen so though this is just a single example the fact is you can use a dynamic mask to uh to create the sort of motion inside the asset that you're displaying on screen so if i uh uh so wolfgang thank you for joining us this is great so uh if we jump back to the vmix interface you can see that what i'm using let me see if i just call this up this is my mask and we're playing this dynamically and what i did here to to generate this mask is i found an image which had with smoke you know and there were plenty go things like story blocks or whatever they have lots of these images i upped the contrast on it to make sure i had a clean white center here and then the sort of the the grays fading to black on the edges and set this in record it as video and set that in as my mask so again this is another way you can take something that would normally on its own be like interesting but now start to leverage that as a modifier for something else so the two motions we think are really uh pretty cool and lots of different ways you could do this you know again things like other logos and graphics that are brand based you know could could be great fits here as well especially if there's a dynamic aspect to the brand so the last case that i want to show involves a couple of vmix things that sort of ties some of the things we've talked about together into a more unique composition for this type of structure and that will be let's take a look here and let me put this out cut over to so say i had a medical conference going on and i had different doctors talking and when this doctor was talking say i wanted to push something out which could give you some of their background so this would you know this would be something where you know i could i could have the doctor talking and then i want to do something which sort of pushes in and gives you their video so this is stuff we've talked about with merge and with with doing layers but what if you could actually just create a reflection under them so this is something we definitely can do and so with this type of thing now what was a more static flat shot now can actually have a little more life to a little more spatial sense to it and you get that from having this reflection right under the image and this is leveraging a few things inside of vmix but i'd like to just go through this so you get a sense the first thing uh is how do we how do we actually create this sort of fade out and to do that you know we've mentioned that gradients uh you know are perfectly valid uh masks they have you know a white to black with a gray fade in the middle and that is exactly what we're using here to create this type of look so i have two copies here of the the video coming in and this could be a camera so this is something you can create live as somebody is is talking so in the first one which is the video we have up here in the corner there are there's no mask it's just we have the video here we could have done a mask we could have done something but this is for this purpose we just set this up as a a rectangle space with the guest video for the video reflection we did a few things so let's let's let me jump in here and show you so the first thing we did is we added this reflection gradient to the image to the video coming in and that creates this fall off so you see it coming you know starting here and becoming fully invisible uh what we then did uh is we actually took and well let me actually go here we let's go to the layers uh we actually jump over here and go to the layers and then when we edit this here this is a layer so if i jump to the layers you'll see that the video reflection when i edit this i actually rotated this image around 180 degrees so that makes it upside down and what we need to do though when this is upside down for the reflection illusion to work is we actually have to mirror it and so in here i've taken this image you rotate it 180 degrees but now it's sort of working opposite what the image above is it's a mirror image so i have to flip it back and now this gives you jawad thank you for joining us this is great to see you here so so now between adding the mirror adding a uh the gradient rotating 180 degrees uh and i did i actually i did that over here in the in in everything we have set up here so this is just how we have things set up in the positional thing i have this rotated completely upside down uh now putting all these elements together we create a reflection and this is dynamic again i want to emphasize that you know when this is playing uh let me just you can see that this reflection is happening as the image plays and if this were a camera coming in uh you would see a reflection here uh of the camera live so this could be something where you have somebody up on at a podium or whatever you could do exactly this in a live situation and these are the types of things that if you've you know if you've worked in things like after effects or build virtual sets these are tools you'd use but now using gradients and some of the other features inside vmix you can leverage this to do these live and dynamically in a given setting and so a great case for this would be say i had like take the case where we had the three guests that were talking in previous shows or i had the three of them along a straight line imagine now if i add a little reflection below each of them this adds another dynamic element and sort of gets people out of the uh the world we live in where everybody's appearing to us in little square boxes of rectangular boxes and uh you know this is something i think that will help make you stand out i think a big piece of communicating a message is getting attention having somebody look and pay attention with a lot of distractions so these types of production tips i think could help make some of the things you do stand out uh from the simple sort of multi-box shots that a lot of people are using today to bring in guests from different remote places so this was everything i wanted to cover today uh hopefully you found this useful and interesting uh so wolfgang uh yeah thank you uh this is uh you know this is something that uh you know we love to we love it when you know we can bring something new that you know maybe you hadn't used before or make you think about different ways to use tools you've already been using but sort of bundle them up or combine them in different ways so that's the great thing about productions there's so much you can do with the tools that are available kim first thank you for joining us and uh yeah so i i'm glad you know this might get you started with using you know some of the key and phil and alpha channel uh a lot of stuff in there that i think can be useful before you get to things like virtual sets which is something else we'll spend a little more time on as well but so uh joey thank you thank you for joining us and uh greetings back from trenton new jersey so not as sexy a place as london but uh we call it home here so but uh so thank you everyone and uh we will be staying around after the show to to talk answer any questions again if you'd like to join us in the after show uh please you can either do that through the comments or uh give us a call in again and you know just have you on the show and talk to you but uh so we will be back again next week with another topic uh and you know any anything you can do to to spread the word we you know we'd love to to grow the community and uh we are grateful for everybody that takes the time to watch so until next week if we don't see in the after show have a good week bye hi welcome back everyone so when we were doing this this was uh uh rudy thanks uh yeah i mean there's rudy's point is that you know the there's there's always something to to learn when you when you're using all these tools and the fact is uh uh jawad gerard just mentioned that he went to new jersey for for college 30 years ago so uh no that's that's great it it is a really small world uh and uh you know 30 years ago it wasn't but uh today there's there's a connection we all have that that is just incredible so it's uh it's great it's great to find that out uh so uh when we put oh chad chad hello thanks uh for joining uh we did we didn't talk specifically about uh drop shadows but we did talk about reflections and uh they are a similar uh feature when you want to uh when you want to do these types of things where you are creating that type of inverted mirrored uh asset that follows a another the same basically a duplicate of the of another asset to create the either shadow or a uh or a reflection type effect and uh so uh so i think rudy just hadn't like you said what was the reason you separated the red border from the boxes so the the only reason we did that is because when we layered the the mask in uh we didn't want to the red would have some luminance value and if we made that part of the mask it could distort uh the image around the edge the by using it as a layer that allowed it to be full red and sort of cover the the outside edge without being interpreted as a mask element in and of itself because in doing that we're what we're basically saying is just use this as a mask for another image so to get that red to show up that's why we did that and we did that so it tied together with the push pin at the bottom for highlighting things and maps and that that was why we we we did it as a separate layer but you know there are you know there are things that uh you know vmix has borders in it but the borders don't follow mass they are more rectilinear so based on you know how you have that sort of cropped and positioned you can get a mask a border around it but that doesn't follow masking right now sorry so okay uh so james is asking about the kind of processor and graphics card uh to be able to do everything we're showing here so masking is actually uh fairly lightweight uh it it is something that would be done in uh sort of on a standard gpu as a sort of a pre-multiplier effect for for luminance so you you get these types of cut throughs so that it doesn't require anything massive to do this uh more dynamic things probably require uh more processing so if you for instance use the uh dynamic mask uh where it's changing constantly uh that may require a bit more uh processing capability but i don't think computationally again that isn't because each frame is being generated individually so it isn't going to be something that requires something huge we i mean we run on a on a higher end system here because we do all sorts of different things but this would be something i think could comfortably run on a an i7 uh you know a newer i7 laptop with you know uh the gtx uh you know 1650s or you know any of the rtx line and i think that would work work fine but it doesn't have to be a massive system now one thing uh so let's see joe could you switch back to the vmix desktop for a second i just want to point something out because we used so many different i don't know whether you can see this here but my gpu memory here is pegged at 100 which is something you never really want to see uh and that's only because for doing the demos here we had lots and lots of videos that we had queued up for all these different things and vmix leverages uh a cache of memory inside the gpu that uh is what they listed as the gpu memory and when that gets filled you can then end up with things like drop frames and other things but that's only because of the level of video that i have sitting in here for the demos this would normally not be an issue that you'd have but it's something to just keep in mind if you're going to be using you know a ton of videos and manipulating them in all different ways that could be one constraint that you'd have to think about because i know that's something that you're trying to find the right gpu cpu combination can be can be tricky based on what you do but uh you know the other thing is if you're doing a lot more with ndi that's a higher uh cpu as well as gpu load so just something else to something else to consider if you're sizing a system but masculine of themselves are not really a a high demand type of feature which make them great to use because you can get some really sexy looking uh compositions with with them with with not a lot of processing overhead so let's see love to know more about virtual sets with integration of unreal engine to extend the zoom capability okay there's there's a lot to unroll in there so uh virtual sets uh virtual set's an unreal engine on your unreal engine is is basically a a 3d compositor that allows you to create virtual cameras inside a 3d rendered space and composite elements with a sort of a 3d feel to them lots of different this is not a this is not a simple a simple discussion uh one of the things that uh that make unreal engine so powerful is something that they're calling virtual productions now where they'll do things like and use an led screen uh as the entire set and uh generate that in something like unreal engine uh and so they do things where they track the motion of the camera so if i move the camera or tilt it up or down uh you you get the uh you get that reflected in the video that's being shown on the wall so you could do things where you have sort of a virtual newsroom and as i tilt the camera over around the operator the back wall will move as if i had a physical set behind me and so what you'd see behind that operator now would be the right component of the virtual environment so this is something that i just started looking at i mean it really is it really is an interesting option uh for creating virtual spaces uh i think that some of this based on what you want to do some of it could be computational overkill if you're using this just to bring in sort of remote guest as as a framework but in terms of creating something that would be incredibly immersive for an audience uh i think of use cases in education where you could actually be creating spaces that show scientific theories in action or you know basically sort of let you zoom in to elements that then become almost like augmented reality when they become a part of the set even though they're not physically there so lots of interesting things i i wish i had more direct experience with it uh but i'm definitely looking at it it has piqued my interest and it is possible not sure if we'll get to it this season but it is possible that that will be a topic that we we will cover at some point uh we need to get the green screens and everything set up again to make all this work but uh i think uh it could definitely be uh definitely be a cool subject to cover oh gladia how are you thank you for joining us this is it it's always great and what's amazing is to to have people from so many different locations around the around the globe here that i get i get the privilege to talk to every week so this is this is something that i think is uh is great for me so i i all i can say is thanks to everybody for taking the time to join us so so are there any other questions any other points you want to make i'm not sure if i missed anything let me going through the comments here so so rudy ah thank you again for joining uh this is this was a lot of fun so if we've covered everything i'll just end by saying thank you if uh if you are celebrating easter this weekend have a great weekend with friends and family and i'll see everybody next week take care

Channel: The Streaming Alchemy Show

Views: 2,632

Rating: 5 out of 5

Keywords:

Id: Y9x-cS6IVo0

Length: 31min 41sec (1901 seconds)

Published: Fri Apr 02 2021