vMix: Scripting A SMART MERGE Transition

hello everyone welcome back to streaming alchemy i'm john mahoney and on today's show we're going to be taking a look at a way to set up a smart merge to handle transitions that typically would have a lot of overlay on them where if you were merging layers it could basically distort things so uh before we get to that though what i'd like to do is i'd like to invite everyone to if you have any questions have any feedback please just post them in the comments below and we'll get to them here live on air uh also if you would like to join us uh for a you know a one-on-one here on the set you can also call in we have our uh our call-in number here and it's also posted in the show notes so you should be able to uh to get to us pretty quickly and someone will get you on air with us all right so let me talk a little bit more about what we want to do today with a smart merge transition uh merge transitions are really great they they allow you to create dynamics on the actual framing of the video that you're sending out on a production so if i just jump over here to what you see on the screen uh that's what we have here on the screen if i wanted to take and have two guests talking and then add another element to that and have those guests reposition this would be a perfect use case for the merge so if i come here and i have these two guests here and i want to change this over so it becomes a three box when i press the merge you can see everything on the screen just does a quick move and that can also work if i'm doing things with pulling out individual guests or multiple guests so in this case you can see the guests that were in the to box zoom in and become part of a four box and then i can push those out again so very very cool way to add a dynamic visual without any special like fancy effects this is something that's sort of very baseline just take elements that exist in one and manipulate them uh in another into the shape they are in in another input so we've used this before but one of the things that you will notice uh let me switch this around a little bit here so uh actually let's start up here if if when i do the merge here you can see that this guest it's very clean it just zooms right over the first guest and you get a very nice clean transition however if i'm using the first guest here and i do that same merge you'll see now that there's this whole frame and the other guest fading out on top of them so it just is a little bit jarring and especially if you have a lot more going on on the screen where so let me sort of flip this around and do it this way you can see now when this comes out there are a lot more elements and it covers a lot more of the image that you're doing this merge with so what we wanted to do was find a way without requiring a whole lot of pre-planning to make sure that whoever you are merging in a multi-box shot it'll actually still be a clean transition and probably the the best way to think of why uh in one case it worked in the other cases it didn't is because when you are doing merges what you're effectively doing is using the layering that exists in a multi-view inside of vmix and you are just manipulating things on those individual layers so when i had the two box so let me sort of jump back to that so we can cut that over and put this here when i did this merge with the two box the second layer was on top so it was a very smooth transition but if i went back and selected this guess they were on the bottom so doing this merge now it just had to fade off everything that was above them on top so what we're going to do uh to overcome this is we're going to dynamically rearrange layers based on who's in preview and who's and the layer that that preview could be on in program and this means that we have to give up one layer in our multi-view so for the multi-view we're taking layer 10 the topmost layer in a multi-view and we'll just say we'll never use that and from there any time we go to do a merge we'll have a different button that will set up that will run with a script and the script will effectively look for the guest that's in preview find them in the layer in program make the adjustment to that layer to move it to the top to layer 10 and then it'll go and execute the merge and clean everything up after we've done that so what you can see here we'll just show it in action uh if we go and look at what we have here with the clean merge the scripted merge you'll now see that you don't have that same overlay when i press the button here it's jumping this guess to the top layer and then doing the merge so they expand full screen so it can be much cleaner and that's especially visible when you have you know much larger numbers here so with this and i'm doing this now you can see he's just cleanly over the top and that will work for all of these guests whereas if i did that same jump with the old merge the traditional standard merge you can see everything's sort of fading off across the top of it so as i said the one limitation though is you have to be willing to give up layer 10. so let's actually jump in to the code here and we'll start to take a look so over here i'll put this on this screen and we'll come in this way so this is a very true actually before i uh i get into this let me just welcome everybody so we have uh rudy rudy thank you for uh for coming and joining us uh i always appreciate that uh james uh thank you uh great to have you here again uh so we have uh brula fu i i am probably doing a terrible job saying the name but uh thank you for joining us from scotland uh so i definitely appreciate that so uh james said if we could slow down the transition on the merge so we can see it better okay uh james let me what i'll probably do is i'll i'll play around with this as we go into the post show and we can look at it in more detail there and and try to see exactly uh you know what you're looking for uh and we have m12 pop m12 pop and john simon okay so both of you thank you uh amtrop thank you john simon for joining us great to see you okay so let's dive into the script here uh we have pretty much our standard uh our standard run of uh initialization so we are setting up all the variables and in this case we're going to find the variables we we need to know what's in preview and what's in program so we have all of that uh we have uh so that would give us the number the input numbers the keys everything we could could use we have another one called found layer which basically is if there's an input in uh preview and i need to find if that is a layer in program uh to handle the merge that's what we're gonna stick and found layer then we have layer counts which is how we know which layers are uh which preview and program which ones have layers associated with them and then all the xml components so we can start to tear apart and work with it uh so the real processing starts as usual by uh we have the vmix xml piece that we always load up to start and we're digging into this now to find out a few things so the first is the in the xml we look for vmix active so that is the input that's associated with program out they call it active in vmix for some reason but it's in a traditional uh framework that would always be called your your program app uh and so from getting that program out number we go and get all the xml associated with the input that's in program and then uh we get the key value for that so that tells us the specific input uh that we have to work with and we do the same now identically for preview so now we know exactly what's in both of them and what their structures we have the xml for each of those inputs so what we're doing as a quick check is we find we get the xml node list and if you remember when we've talked about node lists what they are is a list of xml uh blocks of a certain type and what this lets us do is it lets us determine if what those inputs have different layers because what we're creating this node list out of is the selected nodes overlay and if you look at the vmix xml overlays are what a multi-view is so when you set up a multi-view it has the input and then it has the 10 overlays on top of that and so that's how they call multi you know their multi-view layers are called overlays in the xml and so we can basically say give me all of the overl an xml tree of all the overlays in program and all the overlays in preview and that's exactly what we have in this section of code here and we are now just also you know we're coming back and saying what's the count of all those uh layers and if it's if they're it's zero or there's nothing there there is an account uh then we know that it's just an input and it's not layers so that gives us a better sense of how what do i have to manipulate if it's just inputs then i don't have anything uh to worry about and i'll just do the traditional merge otherwise i need to start doing stuff so the first thing we're going to do here is we're going to come in and say if the total number of layers in program is less than or equal to zero which means there are no layers in program but there are layers in preview so they're greater than zero then what i'm doing is i'm basically coming in saying this is almost like a reverse merge i'm taking a single input and i'm making it part of a broader number of inputs in a layered structure because that's what's in preview so when i do the merge that's how we're going to do the ordering so all we're doing now is now that i know that i'm checking and saying if and this is a very contorted thing you have to do in visual basic but the only way i can find if there are any layers here because it will come back if i do this count up here and say give me everything for count if there is nothing there if there are no layers it doesn't come back and give me zero it'll come back and give me a construct called nothing so the only way i can ask for that is say is to say if not found layer mode is nothing so this reverse thing to basically say if there's something there then i want to do this code uh and so very very simple set of steps we're going to do we're going to go and basically go through this xml to say find the find any one of the layers here that match the layer that's in this case in program because we're doing it the other way around and so that's all i'm doing i'm getting i'm getting the index of the layer that actually matches the found layer and i am using api calls to move that up to layer 10 and that's the move multi-view overlay and when you do a move multi-view overlay it doesn't just move the input to that layer it moves all of the other attributes so one of the things you see in scripting is that you can't really adjust the attributes inside of a layer there's no calls to say here's the cropping here's the the sizing so you have no functional no sort of scriptable functionality that will let you uh control that but you can copy it all from one layer to another with this and this will let you shuffle the layers around with all of their settings intact so when i move that to layer 10 we actually have to do a little bit of a sleep we do a 200 millisecond sleep to just let that transition happen because these things are it's not like traditional coding it's with scripting you don't have that same sort of step synchronization so this gives the system a little bit of time to perform the move then we actually do the merge function and that takes longer because the merge itself is is is scripted to uh to just be a one second transition and so we sleep for another second 100 milliseconds after that which lets the transition happen and then we just move that layer back to where it came from so we're doing the move multi-view overlay uh same for the input that's in preview but now we're moving it from layer 10 to the layer we found it in originally so that's all we're doing in this block of code so very very straightforward functionality but what that is doing now is it's getting that layer that's going to be the focus of the merge on top so whatever happens with that visually it's covering everything else below it and then when it's done we just move it back to where it was and all is good nothing has changed in the vmix session so you have no worries that you've shuffled things around which could impact things later it'll work exactly as it did before uh and this this is the case where where if uh there was nothing there if there was no layer uh in the uh programmer preview then it's just gonna do the merge so we'll do standard merge so that's what this whole else clause is but all of this logic here is basically a duplicate of what we're going to do if it's the other way around if i have a single input in preview and multiple a multi-layer inside of program doing the exact same functionality in reverse i'm just saying if this input is in one of these layers find that layer uh move it to the top do the merge and then move it back to where you found it and that is exactly what we're doing here with the script so really very very simple scripting for all of this and you know for everything that we've been we've been looking at here this is really something that uh is very very powerful uh and very generic because it gives you those clean transitions you're looking for but you don't have to set everything up with specific inputs ahead of time we have this just set up as a a button that launches the script and when we do this you end up with no types of of issues no matter how i set any of this up so if i jump back here and uh put let's let's just do this merge here i can you know swap this around and it doesn't really matter uh who who i have in place there switch somebody else same sort of thing so nothing has to be set up and i could have you know i have multiple uh frameworks here so this would work now there's no overlap so this just works as a normal merge which kind of looks like a fade but uh if i do something where let me sort of sweep this back and i do another multi-box here this will again it will just sort of work as a regular merge would work but if i take any of these people and just throw them in this stuff all just works and you don't have to when you're thinking about this set anything up uh special in advance you just have the script and it just does a better merge for you uh so that really covers everything i know this was you know a fairly short show up front but uh before we wrap this up i just want to again welcome everybody else we have pocket video school uh from mark from newcastle so mark thank you very much for joining us uh we also have uh axa films uh so hello uh thank you very much for joining and we have uh frank uh manky from rome frank i hope i pronounced your last name correctly but uh thank you for joining us this is this is great so uh hopefully uh this this all makes sense you could make things more complicated you could do it so it works with multiple layers so if you're doing things where you wanted to do you know move if there were two elements move them both to the top all of that stuff uh will work oh so i have uh gosh thank you all for for for joining in so i have uh sri srikanth uh hopefully i said your name thank you for joining us from india and we have dennis uh thank you out from from rhode island always great uh love uh love the love the northeast up here so uh this is great uh we have ad hoc uh ad hoc thank you very much uh hopefully you you know you're finding these things useful but uh for everybody that is uh is is joining you know if there are specific things you'd love us to cover here or you know want to learn more about please just uh you know send us a comment send us emails so you can send it to john at streamingalchemy.tv and i will get it from you so ad hoc is asking where can i find the scriptings here so we actually have a github repository that's online it's listed inside of uh the streaming alchemy uh group in facebook so that would be something that uh you know we can certainly uh you know published to you in a in a reply uh to the comment or if you make a comment after show because the show comments go away on us after after the live stream so uh if you make a comment after the show we'll be able to uh we reply to you and and get you all that information if you need it so uh so why don't we wrap this up we'll have a pro show so i welcome any questions then it doesn't have to be about what we covered here it can be about any uh topic related to streaming video production and if there's stuff i can answer uh i'd be happy to answer and again we'll be taking callers so if you'd like to join us live on air we'd love to have you so uh if i don't see you in the post show we will see you next friday thank you all for joining otherwise we'll be here in a few seconds and kick off the post show so thanks everyone hello welcome to the pro show so this is uh uh this was this was fun putting together this we we're actually expecting to to have an interview the show uh and uh for you know a variety of reasons we weren't able to coordinate coordinate the time and so we we had this show sort of scheduled for later this month so we brought this in but we thought it was just a cool topic because it's the sort of thing where you start to see the differences in refinement for the types of productions you do and i think that's something for a lot of us when you're you're trying to project your image so to sell your brand those little things even though if you looked at it it wouldn't seem like a lot those little things can make a difference uh and so this was something that we definitely wanted to uh cover here and see if we could uh sort of help out and add a little extra uh polish to some of the things you may be doing and we know for a lot of people this is something that is is actually a big part of a lot of their productions they're going from multi-boxes with remote guests or guests and slides and they're bringing one or the other full screen and this gives you that smoother look when you do that so now i know that we had uh we had an ask to see if we could change the speed that we're doing the merge at so let me see if i can change this up here we'll try to do something let's say if we can do this at four seconds here so let me see if that did that take okay so we'll give this a shot here so we set this up as four seconds and we'll do the merge so it doesn't necessarily seem to be slowing this down for the merge so yeah this is not sure if this is going to get you where you want to get in terms of visuals here but hopefully so hopefully the uh what you have set up here let me see if we do this yeah okay oh good i was pressing the wrong merge here so so what you can see as you look at this move is the layers on top are fading out on top of this but uh when if i take the top person which is basically the equivalent of what we're doing and i set them over here and then oh sorry mess this up a bit here so we'll put this here cut that over and then we'll to the top person here and when i do this because this is on top you can see that what it's doing is covering up everything that we have there so this is the that's the actual difference is that the way this thing operates always guarantees that whoever is doing the expanding merge will cover up everything else below it so so let's see we have a few questions here so uh srikanth uh said he's new to scripting uh i've been watching uh for a time so what's the correct way to import scripts into vmix so there is there is no one correct way but i'll make a couple of suggestions here the the first is that don't edit in vmix because it's it's a terrible editing environment uh what we use here is uh see if i get this here let me just so we use a visual studio code which is a free editor from microsoft and this will work with visual basic and do everything you know you need to do but what it does in addition is it gives you a much easier look because it's color coding things based on the structures that you start to type and so as you edit a script here what we typically do is we will i mean sort of see if i can swap these both so we'll go in we'll do select everything and then control c sets control a control c and then when we go into vmix you'll go over into the settings and scripting and then to add this you can just go in and paste the script in so it's a very simple way to move back and forth the other thing that you get that's really nice when you use visual studio is that if there is an error in here vmix will give you a line number and come back and say you have an error here also highlight but it gives you the that line number so it makes it very easy if you sort of jump back in here because all the lines are numbered and so you have no problem just taking an error message and going there and making changes and we usually do this uh either on a single screen so we have sort of a wide screen monitor that we use for this and so what we'll do is we'll pin them to each to one half so i work in the visual studio paste into vmix run things test it and if there's something i go back in and keep working in visual studio that's probably the better way to handle this if we are uh if we're doing sort of any sort of complex script uh the other thing is never add it in vmix edit here even if you're making changes like that little fix or anything else you want to do do it over here and move it in because otherwise you lose this is really your master copy because you can save these you can put these in a source control system you can do all the things you would want to do with maintaining code and unlike vmix where it's sort of buried in with that session this gives you that freestanding copy that you can archive and make sure you have and what you don't want to do is be making changes in the one because you thought oh that's a quick change like i just spelt the net the name wrong the name still spelt wrong over here in the uh in in the visual studio version you'll save that you'll bring it in and things don't work and you're trying to figure out why or you've made a change and you had think it's good you come back you make another change copy that over and you lose the first change so lots of things so part of the process is not just copying them in is is also building a discipline around how you manage the code as you're developing it and then after you're done with the production bringing that you know keeping that in a way that you can leverage it in the future so uh are there any other questions or anything else you'd like to see us jump into here so from uh so uh james has something what are you doing to take the script uh oh okay that's a good point i didn't actually cover that so let me jump back into uh to vmix here uh so when you set the script up you have a name up here that you give to the script so in this case this is just called smart merge and there is if you go into the shortcuts in vmix there i just will edit it right here so i have this shortcut tied to a vmix to a a stream deck so that's my my key press and there is a function under scripting called script start and so this will let me take and based on the name in this case smart merge start up any script we want and there are multiple uh sorry there are multiple scripting related functions so there's start script and stop script uh there is a stop all which means if you have multiple scripts that are running uh you can stop all of them and that's without having to give individual names but where this the start script is it's great for these types of things but where stop script would be useful is if you have a script that's just running in the background all the time and at some point you want to turn that script off this will let you start it up so it runs and does something and then when you're finished having it sort of watch your session you just tell it to stop the other is something called script dynamic and what script dynamic lets you do is i'll just set this up here you can actually start to write code in here so you could do api dot function and do a whole set of function calls for vmix uh and you can put them all into one single batch of code here and that lets you do sort of small scriptlet types of things i want to take and i want to move this to preview and then i want to do a transition and then i want to move the next thing to to preview so i'm scheduling that's perfect uh use case for this where you don't really need a full script but you want to do three or four commands in a sequence and you can also do things like sleep in there so it will give you uh the ability to space the timing on things so very useful for that but that's for working with these scripts that's how they are all invoked so it's it's very very very useful for for doing these types of single button pushes that you want with a lot of complexity behind them that's exactly how we do it here and you know the start script dynamic this is a case where you know i mentioned it we just do the merge function so in this case we have that so i have another button here that does the traditional merge and that's what you see api function quote merge and that's how we we do the scripting functions so uh let's see so we have uh der luffy uh from germany hopefully i am saying that correctly so thank you for joining us uh great to have you here uh so let's see uh that's that probably covers everything that you know we we we did here and you know hopefully the the concepts here are generic enough that you can start to think of different ways to apply it so you know even though we did this with merge you're basically this is a general template for i want to find what's in one input like my preview and and find what's in program and then based on what i see take a series of actions and you know especially for finding something in a layer so if i have an input and i want to find what's in that layer i mean this could be something which we haven't done but i could look at different layers and based on that just pop layers out dynamically so you could go and say i want to pop what's in layer one out and you could find what's in layer one and then go and put that in preview and do the merge dynamically so there there are things here that we didn't cover but if you start to think about different uh different shortcuts you may want to use that could be that could be very very useful so uh oh uh paulo paulo from italy thank you for thank you for joining us follow this is great i am i am i'm really humbled by the number of people uh from the number of different countries that are taking the time out of for what a lot of you are friday evenings to come and join us so we everybody here that's works on the show is really grateful for that so thank you uh so you say i was afk for oh okay uh when i'm not in facebook uh where can i find your scripts uh so well first this there's a couple of things here first uh we just have this on it's not in facebook itself it's on it's in github and we'll post the link uh it isn't our it's it's the link is in our user group uh on on facebook but the we'll also post the link in in youtube as well as one of the comments so you can have that uh the other thing is these are we're putting these out for everybody to use so this is not something where we're holding these as proprietary copy them use them uh what we'd really love is that if you take these and do something great with them and you'd like to share you know please just reach out to us and we can you know share them in either you know if you have a direct link or we could put them up and with attribution uh for what you guys have done so please this is this is not something we we are holding close to ourselves as sort of our proprietary stuff take it use it change it share it i mean this is you know basically how how we'd love to have everybody uh you know everybody that that that's joining us to to feel comfortable with doing it's it's really open source for the community here so please uh have at it and enjoy so all right then uh i guess we've probably covered everything for today so thank you all uh for hanging out with us after the show and we will see you all for our show next week same time same social channels take care everyone bye

Channel: The Streaming Alchemy Show

Views: 1,226

Rating: 5 out of 5

Keywords:

Id: AOCXovJK5e8

Length: 37min 20sec (2240 seconds)

Published: Fri Jul 09 2021