Controlling Unreal with OSC (Tutorial)

alrighty today we are looking at osc in unreal so osc stands for open sound control um it's a bit like midi um uh it's sort of like an open control supposed to be used for musical instruments but we're using it not for musical instruments um so it works across a wide variety of devices um there's some you get some digital dj decks and stuff like that some sound control panels um there's plenty of apps android ipad ios even on computer and unreal can send out um osc stuff as well as receive it so you could even use unreal to control unreal through osc which is a strange rabbit hole um we won't be going down so today we are going to be looking at the touch osc app i'm going to be using it but um because it is an open standard you can use anything so there are plenty of other free apps on the app store uh but touch soc is really good and i recommend it so this is my preferred method over web control for a lot of things um simply because there's some issues with web control it's a bit tricky this one's nice and easy to implement and i like that easy is good so what you will need if you're using touch osc is the touch osc editor otherwise you're going to need whatever it is that um you use to edit your osc thing or um in the case of some like decks and stuff like that they can't be edited so we're going to go through some of the things that you can add to touch osc here um i'm not going to go through all of them because they all work the same so it's a bit silly to go through all of them because it's going to be a lot of repeated stuff um so to osc can do booleans integers and floats touch osc only sends out floats so it makes things a bit simple but also i think a little unnecessarily difficult in some ways so we're going to start by simply adding a push button just in the middle here um and that's it so over we have a few things over here um so this send i'll press send on release this is going to send a value on press end on release and that's going to fire off an execution pin in a blueprint twice essentially it's like pushing and then releasing again so we're going to untick send on release so for the push buttons we so the push button sends a value range from zero to one um because it is a push button it's gonna be zero or one yes or no pushed or not um in unreal we're not even gonna bother with that we're just gonna let it trigger the execution pin so yeah so we can just ignore that um so because we don't want to give it send twice i'm going to untick the send on release and then the last thing we have is the osc address which is this here so it's forward slash one forward slash push one you can change it if you want i'm just going to leave them as is on their auto values and this is what you're going to have to find the osc address in your own app if you're all dec or whatever if you're not using touch osc which should not be difficult so once you know the address for the buttons you're using then it's going to be exactly the same so we're going to go ahead and sync this magical layout to my ipad so to do that we're going to hit the sync button and then on the ipad we're going to go into the menu and under layout i'm going to go add and i'm just going to choose my computer i already have untitled one so i'm going to overwrite it and then that's it so if we hit done there is a nice big button in the middle so i'm going to hit stop sync and we're going to move over to unreal and start so first make sure you have the plugin enabled um i'm just under built in just type osc and it's right here open sound control version 1.1 it's not in beta so it's pretty stable um now we're going to create a blueprint to use this um i like to make it its own blueprint to keep things organized um obviously there's no blueprints anyway but um organization is always good uh we're going to choose actor and i'm going to call it osc receiver like that we're going to open it up we don't need to add any components to it because it's going to be a code only blueprint and we're going to go ahead and first start the osc server so we can receive commands so i'm going to delete these two and just use the event begin play i'm going to type create osc and we want the server so under receive ip address i'm going to choose zero zero zero which if you remember from the web control stuff that means anything coming any um ip address coming in from the um like the the outs you know from the local network to this computer if you type localhost or 127. um 0.0.1 then it'll only receive local from on that originated from this computer which we don't want so that's why i'm going to keep it like that for the port um just choose you can use anything so unlike um web control that was bound to point i think was 8080 you can choose anything i'm just going to choose 8 000 because i know there's nothing on my network that uses port 8000 um i'm going to tick start listening no need to add a server name so the server um we're going to drag out and we're going to promote it as a variable so i'm going to call this osc server i'm doing this because i talked with someone to someone at epic to because i was having some issues with this um apparently because this is on the event begin play um essentially this server can just be like this value if we just to use it without making a variable that would eventually go around and get collected with the garbage collector and it would cease to exist and so i get these random disconnects and they reckon that's why so and that's what we're going to use so first what we're going to do is we're going to unbind all events from osc bundle received this is just going to double check to make sure especially when we restart the game or if something else happens that's going to help us start from scratch next we are then going to bind an event to the bundle received so make sure you do it in the right order um so we aren't oh sorry we want message received not bundle received my bad um good thing i made notes like that now for the event we're going to drag back from the event and type create we're going to type create event and under event um you can make a function we're going to make an event instead and i'm going to call it osc message received like that now make sure this will say error um if you just compile it it'll go back to saying a legitimate thing um so what's going to happen is we're going to make a server we're going to then make sure that we're starting clean we're then going to when we receive any message at all to our osc server from any ip address at all then we're going to fire off this event so this event also comes with some information it comes with the osc message itself it comes with the ip address that's centered so we can filter out and have you know if you want an ipad and a knife two ipads um they're gonna have different ip addresses so you can have them do different things while having the exact same interface so if you wanted if you had two cameras in your scene and you wanted two ipads with the same interface to control different cameras that's how you could sort it it'll also show you tell you what port um the server received it on now technically could only ever receive it on port 8000 in our case um so that's not going to be problem um so what we're going to do is we're going to drag out and then we're going to check our notes so we're going to get the osc address address message address there we go message address and i'm going to plug the message in here so this message address is going to be um this right here the slash one slash push so we're then going to convert this message to a string so convert address to string like that and so we're going to get this full path out and for testing purposes what we're going to do is we're going to type print string and so we will now just print that straight to the log so i drag out the osc server go ahead and hit play and nothing happens then i hit the big touch button and nothing happens because we have not connected the touch osc to our under reel so we go ahead and what we need to do is get our ip address of our computer because that's where we're sending things to so open up command prompt and type ipconfig there's other ways to get this but this is the quickest so this computer is on 192.168.1.22 so under host i'm going to type that in one nine two one six a one dot 1.22 make sure i didn't screw something up um port we on 8 000 that is fine so we're going to hit done dun dun dun dun dun now if we go back to touch osc and hit done um we can close command prompt now now if we hit play and we hit the button so as you can see that push that slash one slash push one matches up with this so if we added a second button for example um that's obviously got a slash push 2 so if we go ahead and sync this new layout to our ipad um layout add there yes i'm happy with you to override it so we now have two buttons if we go into unreal we can then that's push one that's push two and as you can see that doubles up because we did not turn off the send on let go um so that is going to so if you hook this up to say start the take recorder and you have that on then it'll immediately stop it again because you'll when you release it but that's all good so we are just going to use this path to determine what to do so i'm going to delete that and we're going to use a switch on string note so this is where you may want to think about a better naming scheme than this slash one slash push system um i'm going to keep it as is because we're not adding too many things so i'm going to add out some pins uh and the first one i'm going to do is be forward slash one forward slash push one so now when we get a message we are then going to fire and and it comes from push one button then it will fire this pin off and because it's a sort of a press button um we're not going to bother getting the value all we want to know is when it is pushed so if i add a print string to this and say push one pushed push one pushed and compile and save it now when i hit play and we push push two nothing happens we push push one it says push one pushed so we can start hooking that up to things which we could do going to do in a minute but what i'm going to do is go through and do two more things so there is this which is an xyz wait a minute x and y pad there's no z there um and the last thing is a fader or rotary fader um so rotary fader looks like that it goes around in a circle and a regular fader just goes up and down um they both work the exact same way in touch osc so i'm only going to do one but just know that you can use either or so as you can see it's given us automatically some names we have some options we can invert it we can um center it in here we can invert some things whether we're going to outline a background color all that sort of stuff um we're going to leave all that as is one thing to know these value ranges here um i would recommend you can set this to whatever you want so if you wanted this fader to go from negative 12 to positive 12 or something you can do that in here my recommendation however is leave everything at 0 to 1 and do that inside unreal that way you always know no matter what that the values you're receiving from osc are going to be zero to one makes things a bit nicer so we're going to go ahead and sync this new layout because it's so pretty um layout add yes touch osc done so what are we going to do these controls on well what i think we can do is we're going to use the fader up and down to set and rise the sun the button i'm going to trigger an explosion um and the xyz pad i'm going to move an object around um so what we're going to do first is we're going to add in the rest of our switch on strings um stop there so we've got theta one and i think the other one was x y one so one one one i think this one was x y one one x y one let's double check yes it was good alrighty so now when we do our three different methods of moving we are going to be able to trigger them next what we are going to do is so our what we are going to do the the first one is going to be a trigger an event the next one is going to set the sun and the last one is going to move an object so we're going to use um i think let's say two types of blueprint communications in this we're going to use a direct communication and we're going to use a interface communication i'm not going to bother with casting because that's that's only really if you're spawning objects in so what we're going to start with is the button to trigger the explosion so i'm going to create a new blueprint called explosion delusion um i think i did this in the web control tutorial but i liked it so i'm doing it again so in the style of content under particles um actually you know what why don't we do fire instead like that so we're going to start and stop fire at the push of a button all we want to do is we want to untick auto activate like that um and then we are simply going to um so next what we need to do is so that we're going to use what's called an interface communication method for this one so to do that we need to create an interface on that's under blueprints and then blueprint interface i'm going to call this osc control interface open it up and we're going to create a new function um it's already asked us to create one i'm going to call this trigger explosion so um an interface like this is going to be really handy if you have to trigger lots of things to do the same thing at once if that makes sense so with an interface i can put as many of our fire i'm going to rename that around the map as i want and then when i hit one button they'll all be triggered at once but i don't have to specify every single one um so in our fire we're going to click on class settings at the top here and under interfaces we're going to add our osc int interface here we go so go ahead and compile it so they show up and then if we type osc osc um trigger trigger explosion um is that what i called it wait a minute trigger explosion oh okay so if i just type trigger explosion here we go so we can add an event called event trigger explosion and so you can see the little interface logo here that means this is going to be called through the interface in a different blueprint and then when that is called this is what's going to happen so when it is called i'm just going to use a flip flop and what we are going to do is drag out from our fire and then i'm going to type activate so the first time it is going to be activated like so uh yep and then the second time it's going to deactivate like so so now i trigger it once the file's going to start trigger it again the file is going to stop so now to trigger this interface what we're going to do is we're going to get we're going to go back to our osc one and we're going to get all actors with interface so we're going to select our osc interface and so now this is going to bring out an array of objects that have that interface in it no matter how many objects are in our scene and so that's why this is really handy because i don't have to make that list i don't have to do anything like that so long as the objects are using the interface then we can get them like this so what i'm going to do is i'm going to drag out the array and i'm going to type trigger oh wait a minute um trigger explosion so you can see here it says message and we have a little like you have mail you have mail um thing that means this is going to be sent off through the interface to everything that implements it and you should be able to drag the array of actors straight into that so all the actors that have this interface we are then going to do we are then going to trigger the explosion event in them so you can have multiple events in an interface you can have as many as you want in this case we're getting everything that has this but we are only triggering the explosion once so if there's you have other things in here with like fire explosion lightning you know but we're only because we're only triggering the explosion only the explosion event will be triggered so that's all good so we know this fires whenever we hit it um so this should work straight off the bat so i'm going to drag fire out and i'm going to drag a bunch of them out as a demonstration for how this works um if you do magic five okay put there save i'm gonna hit play fly back now we hit the button and we just ignite the world on fire and because of the flip-flop we can make it all disappear if only the bushfires last year could have done that as well the push of a button but it's for another time so yeah we have just implemented and because we're using the interface i can drag more fire out does not matter we can have a lot of fire if we want and i don't have to do absolutely anything they'll all just work except apparently the lighting which unreal freaks out about for some reason with that many maybe because of dynamite i don't know so we now have we can now set our world on fire at the push of a button which will be everyone's dream i'm sure if you could snap your fingers and set someone on fire i'm going to delete a few of these because that's excessive i'm gonna have three all right so that is our interface communication method um which is again really good for these like lots of repeated things if you needed um you wanted to hook a button up so all the alarms in a level go off at once this is your best way of doing it stuff like that next we're going to do direct communication we're going to directly tell the sun here it is maybe i can line them up we're going to directly communicate with the sun because there will only ever be one of them in our world so we don't need to worry about that as much so i'm going to start by changing these to movable simply because we can then move it without freaking out about light baking so in our osc receiver we so to make a direct reference we are going to create a new variable i'm going to call it sun and under the variable type we're going to type directional light and i'm going to make an object reference and i'm going to make it public so now if we click on our osc like that we now have an option to choose our directional light under here and so we're directly referencing this actor or this yeah actor or the light i guess it is um so what we're going to do i'm going to set that as like a sunset on this fader um so i'm going to choose i'm going to find my variables like the variables or things i want to use so i think i'll go um maybe if we go negative 90 negative 90 there we go um so maybe if i go negative 90 which is the top of this fader um to 90 that should be night time yep all right so go negative 90 to 90. all right so we are then going to get data from our message so what does that look like so we drag out from our osc message like so and we can get osc float at index so this is going to get us our data yeah like our osc data i'm going to put that there so we know for a fact that this is sending float data um we can say value from range and and touch ic only sends floats um we also know that um there's only going to be one thing so this this only sends one val so in the case of this x y pad it'll actually send two float values the x and the y this one's only sending one so we know that index zero is going to be the value we want so what we're going to do is we're going to drag off of this we're actually going to drag off of the variable and we're going to map range clamped so we know for a fact that this is sending a zero to one zero is actually at the top and one's at the bottom because i put this in the editor upside down so we know the range is going to be zero to one and so and i decided our range our actual range is going to be negative 90 to 90. um so i want the sun up to be all the way at the top um so i think i'm going to screw this up i'm going to put negative 90 and then 90. like i say file save we are then going to drag out a reference to our sun and i'm going to set rotation set actor rotation drag out there and split this and that was on the y axis wasn't it yes so i'm going to plug that into the y um x is 32 i'm going to keep these values the same and that's negative 120. like so so now i save save and we hit play and i drag this along you can see sun up sun down but that doesn't look very good why is that um you may know that the sky sphere has this refresh material which is required to um do it so what we're going to then do as well is make another direct reference to the sky sphere so i'm going to call that sky sphere and i'm going to call it out my shoulder we're going to use the i think it's bp underscore sky there we go object reference we're going to make it public as well so now in here we can then oh no in our osc here we go we can now choose with this guy because there's only ever going to be one of these as well that's why we can do this well that's why this is the best way of doing this um so we're going to do is we're going to drag out and it's actually a construction thing but i think we can still trigger it so if i don't refresh material we can call that function from inside this blueprint like so so now this is really simple we're simply going to move the sun using this set active rotation and then we're just going to make sure we refresh the material automatically after every time so this is um because um every time you have your finger on touch osc it's sending constantly at the pretty much at the tick rate of unreal um this is going to update the sky sphere like uh like 100 times a second um that's okay though because it's not very um heavy on performance so now if i hit play um the only other thing is the skylight i don't think yeah that that um we could go ahead and do the exact same thing that we did for the um the material here um and recapture the skylight every time as well but i'm going to just delete it save the problem uh so now as i there we go sign up sundown all right nice um golden hour sunset all the way up and so that's look at that and you get you get that really fine granular control because it is a um i want to say analog input but it's like it's an analog input on a digital device using digital it's like but you get the idea like we have a very fine control over it to get right where we want and that's just that's so much easier to set up your own custom system to do this than it was with web control because if you were to program your own web control to just do this that'd be a nightmare only problem is you can get osc working inside the editor like like web control without running the game but that is a tutorial for another day perhaps so the last thing we have is moving around an object using the xyz pad which is going to be the most complex so we're going to go back to our osc receiver i'm going to make a event so type custom event i'm going to call this move objects i'm going to do this just to keep things clean so from this one i'm just going to call the custom event which was move objects like that that way i can now build over here and then that's not going to get any messier so we're going to do the same thing here we're going to get message float at index float at index and i want two of those because we're going to get two things so um the second one is going to be an index one like so so um for the message is this like so uh i'm going to move that over a bit somehow neaten it up been attempting to make things neater just so it's easy to read in video so and when you download the project so what we're going to do is i need to double check these are the right values so we're going to print them um i'm going to add a print string at the end of this and what i'm going to do is add an append node and i'm going to make four um so i'm gonna say index one colon space and i'm gonna plug the value into b here and i'm gonna type index sorry index zero yep and then index one colon space and drag that into there all right so print string is your best friend when it comes to debugging things um so now if i go here we make it sunny um i'm going to go up on the pad and that is looking to me like it is index one i think it's upside down as well um because zero is top so that is index one is y because y to the sky and zero is top which means index yeah there we go index zero is x and zero is left x zero left all right now we did that little bit of debugging we can delete these so it is x and y um so we did get around the right way so we're going to use a more of those map range clamps map range clamped in this i however um i'm going to go so we know it's zero to one and zero to one in the out range i'm just going to turn it to negative one to positive one that way what we're essentially doing is making the exact center of the pad zero um so next what we need is what's called a dead zone um so if you play video games and stuff you know what this is but essentially we because it's going to be near impossible to get this in the exact center where it's zero zero we want a small area in the middle where you know so if we're around the center it's going to be zero zero and to do the dead zone um what we're going to do is drag out from our return value and we're going to do a greater than and a less than sorry that is greater than and then this one is a less than so if the number is greater than say negative 0.1 or less than 0.1 so it's between those two values and so we want an and boolean then that is going to be our dead zone and we're going to do the exact same for the next the other one so there and there and so that's how dead zone is done and so adjust this number to where you feel comfortable with your dead zone um and lastly what we're going to do is we're then going to find something to update the um location of i'm going to move that back to negative 19. like so and i'm going to update all right there we go alrighty so i think i'm going to go this chair so what i'm going to do is i'm going to make a new variable i'm going to direct i'm going to do a direct reference again um oh i don't even know what a type they're movable like so um so what we can do then is we can actually reference just a straight actor um so you can reference any of these like the sun and the sky sphere using a direct actor rather than their specific blueprint the only problem is we then won't have automatic access to um its own functions like the refresh material but because we we want this to be sort of a more broader usable thing we're going to just leave it as that so i'm going to change this chair to movable grab our osc receiver so i compile it oh i didn't make it public there we go compile there we go so i'm going to change it to you know shoot which chair was it chair this chair um i might rename this chair just for my insanity move chair there we go now if we go on osc and we find the move chain there we go so i'm going to go into osc so you can set this up two ways what we're doing is setting it up like an xbox joystick where you sort of push it and then you sort of pull it back to the center to stop things you can set it up how it was set up the stage mover was in the web control where it's like this is a maximum distance from the center sort of thing and so you can't continually forever go in one direction which is another way of doing things and that would be a case of so we're going to use an add relative rotate um add relative location um if you were to do it as like a hard stop then um you would use the map range clamp to map it to your maximum distance so say 300 a negative 300 and then you would just do a set relative location not an add relative location if all that makes sense um so we're going to do that we're going to do the add relative location we're not going to use the default route we are going to use our reference to actor um all right so it needs to be a scene component sorry it'll be a ad actor local offset um add actor world transform that's going to be the one split just like that all right so then what we are going to do is we are going to plug that into this like that i'm going to split out this struct pin so we said um y was a second value so i'm going to drag these down i'm going to drag out of y and i'm going to type select float and we're going to plug in our dead zone into this and so if so if we are within our dead zone we're going to pick a which is going to be 0 if we're not we're going to choose b which is the map range clamp and then we're going to do the same for this one so select float move this back a bit for room like so so we're going to then get our dead zone plug it in there and plug this value into b like so and so now that is going to move it around like an xbox joystick hopefully so now if i hit play and move back a bit let's just move the sun some somewhere nicer and then i can um ah what do we do oh we did i put it in z wait a minute that's supposed to be an x that would be y there we go [Music] um yeah that's so you can use this to move the camera around you can use this to rotate the camera around you know all the all that and some um if we want to speed things up um you can change the clamp but um what i'm going to do is just um go float by float and so if we type 2 then we're doubling the speed same here float by float um boop and double the speed like that um the dead zones are a little large as well so i might actually drop it to negative 0.05 0.35 same thing and negative and 0.05 um and there's there's no reason you can't use the like get forward vector world vector and stuff like that so this always goes forward there you go that speeds up it makes it a bit quicker well it's a bit it's a funky game controller but it is one nonetheless there you go so we we've successfully done osc control i'm moving a chair around with a stick um the future is now old man i can also set and raise the sun [Music] using this dragger thing and i can set the world on fire with a push of a red button we have our evil um mastermind plan is complete so to speak could turn back off again um and so using that functionality you know it's the same thing like i said the the um the rotary encoders um no not that one sorry the rotary faders here we go work the exact same way as this one so the easy to program same with the toggle button um that'll just be always on um the uh so things like the battery and the label um you can actually send data from unreal back to osc but i'm still working on that but then we can have things update on the app as to what's happening in unreal so like with our sun controller we could then say at the top what the sun's position is um but so keep an eye out for that um these multi-push ones um this is where those float variables come in so it'll be like float one uh sorry index one oh sorry index zero index one exact two and x three index four you're probably going to be better off just doing manually lots of buttons it's probably gonna be a bit easier same with like the multi xy um and the multi that's going to be index zero one two three four the there's one more thing on here encoder um if you've tried out you've probably gone like oh wait that's probably a much better joystick this sends a value of zero to one based on whether you're spinning it clockwise or anticlockwise which is not quite and then i think the number increases the faster you spin it in that direction it's like a um like a uh like a record deck um you know they scratch it whatever i know that's not what it's actually called but i don't know what it's called um you know it's but that's what it's supposed to be so you can't use it like a thumb stick i'm afraid which is really annoying but that's just how it is but other than that cool beans um thank you for watching if you would like to download this project for yourself head on over to my patreon there is also a already set up osc camera controller with things like the um exposure and the white balance and the moving the camera around and stuff with osc that's already on there they've had that for a week and a bit now as an exclusive so if you want something like that go check that out as well but otherwise thank you for watching and i will see everyone later

Channel: Aiden Wilson

Views: 4,446

Rating: 5 out of 5

Keywords: Unreal, UE4, UE5, Virtual, Production, How, To, Tutorial, Epic, Games, Indie, filmmaking, Composure, Vive, Oculus, Tracking, Camera

Id: 9CkKPCBys44

Length: 44min 43sec (2683 seconds)

Published: Tue Nov 10 2020