3D Keyer - Good for Green Screen? - In Depth Analysis

let's have a detailed look at the 3d keyer in davinci resolve i want to discuss what its strength and weaknesses are and whether or not it is suitable for chroma keying like for green screen and blue screen work in short it can be used for this and sometimes it can be a good and fast approach however it also has weaknesses and to understand this better i want to discuss what the underlying keying technology really is how it relates to other keyers especially to the hsl kia so some extent the 3d keyer is based on the hsl kia and then i will also discuss how it's different from the delta keyer which is like the premium keying solution for fusion so let's have a look [Music] i have in front of me a very simple keying scenario you know some cheesy stock clip of someone standing in front of the green screen and the green screen is a little bit uneven it always is but it's not too bad it's pretty okay uh there isn't much on the foreground so no no severe spill no hair detail so this is very very simple in many ways and you know not much detail to worry about so let me drag and drop it onto and you need to have the openfx overlay on in order to use it so if not switch to the openfx overlay here and then you can already use it and i will just show you how i use it for a very quick scenario like this with the first one is the pick the pick solution here i will just click and maybe drag a little bit so i am picking from the foreground you can then add more so you can add more strokes maybe pick down here to see it better i will do two things i usually do first of all in the usage options i will show path just so that you can see what i'm picking so it will just stay active and then here in the output i will temporarily switch to highlight background so i can now see the mask permanently and not just while i'm dragging and clicking so this way you can add to the colors that are being considered part of your background that will ultimately become transparent so black transparent white opaque so along these lines all the values that it finds along these lines it will add to the area that will be considered background so i can do this again for example here and you can also if you have problems in the foreground if you find holes you could also make red lines via the minus here and add strokes that are then doing the opposite right say this these color values these ranges should belong to the foreground but you can also delete strokes again here so this is great quick setup you always need to refine this further or almost always and there are two ways of refinement there the key adjustment and then the matte finesse key adjustment decides on how the algorithm actually works what how the key itself is is built i have been doing this now with this here already and then there are additional adjustments here i will speak a little bit about it later and then there's the mad finesse which works after the keying after the keying you have this black and white image and you're basically doing color corrections to some extent on this black and white image and for example you can try it with a clean black to get you know this cleaned out a bit more and always go gentle not too aggressive but then you can get your black and white separation here it goes quite quickly in most cases you don't want to use these parameters to get rid of everything especially like the corners often you don't have the lighting super even sometimes the screen may not be large enough or you have other props on the side whatever those kind of stuff you don't want to exclude with the kia but use the garbage mat instead and here we have two simple options a rectangle and an ellipse let me use the ellipse for a person and you always need to invert it when you want to select the person so either select the garbage or the person here i select the person and invert it and then i can you know enlarge this like this and adjust it and i guess yeah maybe like this that's kind of the best what i can get with an ellipse and for this example it's just good enough all of this now took i don't know how long you can check in the recording but it gave me a very quick solution and when i switch back to composite i kind of have something workable and this is exactly how i would suggest to use this kia if you have like very simple shots or if you are pressured for time or if you just need a quick preview in the edit page you're still editing changing some things changing the backgrounds arranging things or so just use something like this and very quickly you you get a nice key without having to build more complex fusion flows um but as you can imagine there are downsides to this especially once the shots get more complicated before i really dive into this for keying let me explain to you where this 3d keyer is actually coming from it was not built for keying examples it was created for the color page in davinci resolve and specifically for color selections as part of grading and not particularly for green screen and blue screen it's not optimized for green screen and blue screen and that does matter and i will show you an example which i have more in mind when i think about the kia i have more something like this in mind and think about let's say selecting here her shirt let me go over to the color page in the color page i have the clears directly here in these qualifier tabs it's basically the same what you find here in the openfx you find it here in the openfx the key the hsl key the 3d key right now i have the hsl key but you have here the 3d key it's the interface looks slightly different but it's basically the same parameters if you understand one you understand the other so i mentioned that this example here is actually what i have more in mind and to demonstrate this let me first start with an hsl key and then go to the 3d key because the 3d key kind of expands on the hsl concept in a certain extent so if you understand the first one the hsl example then it's easier to understand what the 3d keyer is actually doing so if i want to select her shirt i would sample here you can click and drag and then maybe turn on the key again either with gray or black and white to see what's happening and you see you don't get a perfect selection there's noise and especially her face and around the teeth nonetheless with plus and minus you can try to to get it better select something more no maybe too much you know the arm here select mine's okay not working so well so you see there there are restrictions and then you can try to go further either with these with the softness and with a range here so by using the plus and minus i am adding or removing from each of these ranges all at the same time and then there is the softness parameter which uh if you extend it you know the key gets a bit softer if you shrink it like this all of these goes directly on the ranges and decide which room part of each should belong to the key which part should be outside what should happen at the transition in this example hue saturation and luminance is actually really good because we kind of need all three to separate the shirt from the background if you look at the image again just separating based on color on on red would not really work because there's a red component in the skin of course and there's red in the shirt so the classical channel keying like what you have for green screen usually would not work here if i wanted to i could try the delta key on this but it wouldn't give a good result i can tell you right away feel free to try it on something similar but it won't give a good result because well there's red here there's red here it doesn't have this classical green screen separation but the combination of all of them does work and let's turn this on again and i can show you that you really need them so if you just use hue well you get a lot here and you know but then you use saturation to remove the face uh and possibly luminance as well um actually luminance is not doing that much here and yeah you might even have to extend it if you want to have the shoulder here and of course you see i'm not getting really good result and even the fine tuning here won't help much let's see how it is if i go to the 3d keyer instead and sample a point in with the 3d keyer set this again to the picker pick a point and now with the 3d keyer i have up here a color space and if hue saturation and luminance if i want to really compare it with the other one i can change it to the same color space the default is yuv which is luminance and chrominance but then you can change hue saturation luminance hue saturation power quite similar and lab so these are different color spaces all of them to some extent do this hue and or chroma and luminance separation so all of these have this in common and i will discuss in a minute whether or not this is a good idea for green screen but let's first use it the way i wanted to show it to compare so if i switch this one to hsl and sample somewhere you see immediately i get much on the one hand softer but also more complete selection of the shirt so it does look significantly better and i could try to to add a bit more here or to subtract a bit but already the initial selection seemed to be much better now i said it's kind of based on the hsl key and what i mean by this is it is kind of a three-dimensional extension what this means the hsl key you select three ranges kind of independently okay by clicking you're click selecting them all at the same time but each range independently selects okay these are the hue ranges these are the saturation ranges these are the luminance ranges and then all three get combined for the key in the 3d keyer you are picking a range really in a three-dimensional space you say this is huge saturation luminance and then please select all the values that fall into kind of a three-dimensional sphere around it if you can have a little bit of mathematical visualization think about a three-dimensional coordinate system one direction issue one is saturation one is luminance if you take three ranges to define something in that coordinate system you are actually defining a cube yeah so you have an interval with hard boundaries or soft boundaries but you have an interval and then another interval and the third one in the third dimension all of these three intervals basically define a cube of values in the three-dimensional space here we are getting a sphere so we with the plus sign and minus sign i decide uh what ranges of values are inside the sphere and then it in three-dimensional space it determines the distance towards these values um to determine what is inside and what is outside and you see like a 2d breakup of the 3d sphere in this preview here so you see like a blob up here and well a luma range down here because the three dimensional hsl sphere is being separated into a 2d chroma component and a 1d luma component even though internally it computes all the three together but because it computes all the three together and kind of determines a distance in 3d space towards your selection because of that it tends to be a bit more accurate often a bit less noisy a bit more precise in the selection it doesn't have to be of course but that tends to be the experience with this and that's why i'm saying extension on the on this one here because here three intervals each individually turn on turn off each individually makes the selection the combined selection we have here here in the 3d keyer select the points and then it builds like a three-dimensional sphere and all the values that fall into this sphere then fall into this here even if you can't visualize it like immediately um it is relatively easy to tune it so there is on the one hand here a tolerance slider by the way all the names you find if you hover over and wait a little bit so there's a tolerance which just increases your chrominance range like in the 1d version here you increase these ranges to left or right right and then here you do the same for chrominance in 2d that's what this slider does and then you have the softest slider which makes like this range that is selected harder or softer and then for luminance you have it separated by a low end and a high end pretty much the same as you had in the 1d version so in the 1d version luminance left and right like low and high and then the triangle shows the soft transition between inside and outside of the key and in this one here you have also low and high i think in this example you will see the high version if i bring this up you see this way i bring much more luminance into it extend the luminance range and then i have the low and high softness so i can make it either very hard or yeah soft so this is how this works in addition you have this x and y which here is called shift and tilt and then you have here rotate um so this is a 2d way of moving the chrominance around this actually messes with your original selection to a certain degree so you have first selected and then if you're moving it around you're actually kind of selecting something else so you're selecting a different hue so usually i would not do this if you have done a good selection um the only advantage of having parameters here is that they can be animated in principle so if you had a lights changing light situation over time or colored lights coming in and out or so you could in theory animate those parameters here or in the edit page same thing you could animate those parameters otherwise if you have done the selection here it will probably be very difficult to adjust these parameters even the low and high end of the luminance that's a bit more intuitive than than these up here but even those are rather hard to really manually adjust once you have done like the the plus and minus selection here nonetheless it's possible and then the matte finish just to complete it now and really discuss the whole tool so the matte finish let me go through the parameter even though i won't need all of them the pre-filter is there to get basically rid of noise especially if you have blocky noise like coming from compression artifacts like uh jpeg or h.264 compression kind of sometimes this can help you see here it does a little bit but i wouldn't overdo it uh there's actually also the clean black and clean white which also helps to get rid of like small speckles so if you have like noisy type of uh background okay this is already quite large but if i clean black and bring this up you will see that here these points go away so you can use this clean white does the opposite from the other direction and then clip black and white is basically the color correction to crush in the white crush in the black and will make the whole thing significantly harder so if you have grace and semi-transparent and bring the white down you will get more white and well with black exactly the opposite so you can just work through these top to bottom blur if you need a very soft mask which happens often in color correction then you can can blur your mask the in and out ratio says blur more towards the inside blur towards the outside of the mask that's what this does again here i might not want to do this and then matt finesse here you can shrink and grow the mask so you could you know grow it like okay you know just extend it now for compositing this would in most cases be rather crude but um yeah sometimes a little bit can can help especially if you don't need like super accurate examples uh shadow mid-tone highlight again color correction on this black and white image if you have semi-transparency like gray areas you want to bring them a bit down bring them a bit up go to the midpoint right it's called gamma slider and some other tools in fusion otherwise the same principle finally there's a post filter which tends to help with bringing back some edge detail in some cases so i don't know exactly what the mathematics of that filter are but that's how it's advertised that here i think it's not doing much right now in this configuration so you have seen this is a difficult example and i now didn't produce like a super great result i could probably tweak this a bit uh nonetheless often in color correction you might not always need the like super super accurate mask if you just want to make the shirt a bit darker for example you see you're kind of getting a result it's not perfect it does get the skin somewhere and doesn't get the whole shirt and so on so you could tweak this more but that's kind of the idea behind this solution right find these scenarios which are based on hue saturation luminance or based on chrominance some attribute in color in the widest sense and then you can do some minor adjustments on it if you do more extreme adjustments like change the color then you will see it breaks apart and i would need a better mask i have shown you this example because this is really what i'm thinking about when i think about the 3d keyer or when i think about the hsl kia let's look at a green screen example for comparison let's look at this example would you use the qualifier on this well you could you could use the hsl qualifier sample and you know switch to the key in the color page you need to invert it which you can do here via the key output invert and well maybe let's look at it in color and you see well i'm getting something and depending on where i sample with the qualifier i might get well kind of some result medium okay if i add a bit more i can kind of get there right but you see it's it's rather difficult and the reason it's difficult and the reason it's not ideal here is that my selection here doesn't really depend much on saturation and luminance i can show you because the foreground is also saturated and the foreground is also bright and same as the screen in many cases your foreground subject in a green screen scenario is well lit and maybe as bright as the screen so luminance in this case doesn't do anything right here so this is not helpful saturation where your foreground subject can of course be saturated your green screen will usually be very saturated again the selection isn't contributing much so basically in case of the keying scenario everything relies on that hue saturation and this is not ideal because the keying scenario was set up in a different way it wasn't set up for this hsl keyer right it's more set up for a channel keying to say the green channel is background the foreground red and blue have used the two other channels in the rgb separation and you can do rgb separation here by the way in the color page here is an rgb and if you select rgb let me pick you see immediately i'm getting a cleaner result from the first pick and then i pick the second and you see it's practically gone so right away on rgb i can get a better selection because now i'm really using all three channels um the foreground is separated in all three channels the foreground is predominantly red and blue and has less green that means that you know all three channels at the end contribute to the difference between the foreground and the background not just the hue and that's why the rgb keyer or rgb qualifier actually works better than the hsl qualifier and uses more information that's more attuned to this example now how about the 3d keyer well it is kind of based on the on the hsl clear just the more advanced version and then it depends on what color space you're using um but you do not have an rgb color space here which would actually be probably the best one for a green screen if there was one but you don't have it you have uh this your v separation which is luminance and chrominance maybe better than the hsl because there are two components u and v where the screen might give something and you see the the result select a bit more um it's it's getting there not as good as even the simple rgb clear um but you know it's kind of getting there um then hsl will probably be well i know i've selected so much i can hardly see the difference but hsl somewhat similar but might be a little bit inferior to the yuv in this example and then the others they're all similar so huge saturation power and then lab all of them separate kind of the luminance out and that's not really what the keying setup is doing so the mathematics underneath are in a way inferior i should say even rgb selection is not the best rgb keyer is not the best for for keying so what you ideally want to do is look at the difference between the different channels between the keying channel the green channel and the red and the blue channel and what the delta keyer does it looks at this difference looks at red channel versus or the difference between red and green the red minus green versus blue minus green and you know where this difference is dominant where the red channel and the blue channels are stronger than the green channel those are the areas where you have foreground and where the green channel is dominant where it's stronger than the red and the blue channel those are the areas where you have the background so the logic of the key is really different and that's also why the delta qr tends to be superior in this example actually let me let me show you if i use the delta keyer and just use we are just speaking about the selection process the keying process itself i didn't really mention all the refinement that you can do and of course there's a lot in the delta gear but if you just sample one point and let me put alpha channel on one side and color on the other you see i'm i'm getting a pretty soft result right it's not as aggressive as the color clears by default but in the soft results you already see the background uh relatively evenly tuned out and then you know if you want it harder you can immediately get there right and then there are a few few other fine art tunes but the processing logic underneath is superior itself let me go back and mention one other part where the delta keyer really has its unique advantages and that's when it really comes to cleaning out the uneven background and that's the second step in the delta clear so one step is the color selection process so in the 3d keyer you can do these different strokes in the delta keyer you just select one sample but then you have the pre-processing in the delta keyer the pre-met and that really makes a big difference as well and can really make the delta clear hugely superior compared to this gear let me go to a different example where i see hair detail a bit better so here we actually see hair detail we have uneven background but let me choose one example where we have even more this is from the black magic design certification training if you have done it i'm offering this course on my website or you may have gotten the footage from blackmagic design so they offered this as a keying example there's a reasonable amount of hair detail here and a reasonably uneven background to worry about so let's use this and throw the 3d keyer on it and pick the color i picked something here next to the hair and let me switch this to alpha and now if you are adding strokes it's really tempting to think that you are kind of locally cleaning out the background uh but you are not let me show the path so you see so i selected here and if i'm adding here well i am removing now at the head kind of the colors but actually what i did is i added to the overall color ranges of the three-dimensional color sphere of values that are being part of the key so this will have global impact everywhere right so also at the hair so i can select down here but this stroke here will definitely have impact up here as well right so it will impact uh whatever color is uh considered background if you look here at the hair detail and i can delete the last strokes again the last one you see it had quite significant impact up here even though i selected somewhere completely different and the other one from the head as well so yeah and that's really the problem here you are not working locally you're not protecting the hair detail with the delta keyer this works quite differently because with the delta clear if i set that one up in fusion let's quickly bring it in with the delta keyer i do select one value here or you know you can press control to average around it but you're just selecting one background color here to start from alpha channel so the starting point is is different uh and then when you start want to clean out the background what you do is you go into the pre-matte and then select the areas of the background uh like this and i need to bring it in go to the pre-med tab here and select from here while on the pre-met tab and now i'm selecting background like this but i am doing a pre-key i'm not doing the final key not the final key that will work on my hair detail but i'm doing an initial key to clean out the background that was maybe not needed i'm cleaning out the background and if i get any problems in the foreground then i have to counteract this by eroding potentially a little bit eroding maybe using a bit of soft range or even blurring a bit so with these sliders i'm kind of protecting my foreground while still cleaning out the uneven background and if i do it too much then i i don't i'm i'm losing some benefit but if i'm gentle and just make sure that i just about keep my my hair detail uh then i should be getting a pretty good final results if i switch to the final result you see the hair detail is still there now the final clear the second keyer works on the pre-cheat version right so the selection process is safer you are not selecting and at the same time ruining detail but you are doing a pre-selection just for the outside uneven background away from the detail and then only in the second step you're getting the final result so this is called the pre-met or clean plate approach so you can build a clean plate separately the same principle this is used to clean out the background without touching the foreground and then second step the foreground is being computed this is a reason why the data clear can also be a bit slower because there are two keying steps that happen but it is really the step that makes all the difference for the hair detail and that is a step that i just cannot do with any of the keying tools from the color page there are additional advantages of course from going to fusion you have already seen the garbage mat for example is like super crudial rectangle or ellipse infusion of course i can draw more precise masks i can put in a solid mask if i had holes in the foreground i can do that i can combine multiple keyers in the area of these spill i have way more options i have the delta gear for example has an automatically spill that happens internally if that doesn't work you can turn it off or go externally use the bit from the matte control or you can use color suppression you can use different algorithms to achieve these here you have just one slider in this 3d gear in the other case you don't have any slider at all so in all these options you're of course vastly limited also no edge treatment particularly there are a few things as part of this not refinement however you don't have for example a color correction on the on the edge whereas in the delta gear you have this fringe gamma and then you can build your own edge masks and so on if you want to move towards more sophisticated solutions but all these details these will edge treatment and so on this all comes later right this all comes on top of the actual keying process the actual keying process itself as i mentioned earlier is not adjusted for green screen it is more for these color selection examples from color grading in the case of green screen or blue screen a color difference here like the delta gear is really the tool that is adjusted to that particular scenario and then on top of it the clean plate the two-step keying pre-key final key and that's really how you get to the better quality solutions and yeah that's really the advantage of heading over to fusion nonetheless sometimes just a quick solution is all that you need the 3d keyer it does work quickly it's just one key with a bit of ranges that you're setting uh computationally it's faster and uh to set it up from the edit page is also very fast uh so in that sense i do recommend to give it a try especially for previews for fast work and so on and don't forget sometimes not every king problem is super sophisticated you know sometimes people work for example with with stuff like this so uh how much effort do you put into keying this well probably not much actually uh if whenever i see this i wonder why people didn't export an alpha channel to begin with but you see these often in stock footage with these artificial green screens those kind of scenarios are very simple and well sometimes the simple solution is all you need okay i hope you enjoyed this overview let me know and as always you know subscribe like uh thumbs up or whatever you feel like or whatever makes you and youtube happy and otherwise see you next time thanks for watching [Music] you

Channel: VFXstudy

Views: 1,368

Keywords:

Id: IGckfA8yYBk

Length: 31min 13sec (1873 seconds)

Published: Mon Dec 06 2021