Introducing the new Input System - Unite Copenhagen

hello and welcome to an introduction to their new input system yes it's finally here it's real after you gooey and nested prefabs we present you long in the coming project number three the new input system i'm renee lead developer on the input team here at unity and i'd like in this talk I'd like to give you a comprehensive overview of the new system the goal here basically is to give you a solid idea of what's there how it may be useful to you and where do you find more information if you want to go and have a deeper look so let's go and dive in so let's first look at where we are as of today 1.0 preview is out now as a package what that means is that you can install it in your project separately from the package manager UI inside of unity it still has the preview text in order to see if you actually like you need to go into that little drop down called advanced and enable show preview packages and then it should show up when you install it for the first time you will see a little dialog box that ask you to him whether the new input back-end should be enabled you say yes you have to restart the editor once and that's it you're good to go so the reason here is that we developed a new input system side by side to live side by side they don't want for now means it's completely optional it's your choice and you can even like enable both at the same time you know that's not extensively tested you may run into issues there in terms of features one Penzer preview is what we consider feature complete and what we intend to land as a verified package in 2020 dot one meaning by that point it fully becomes like an officially sanctioned and integrated part of unity between here and verify there will still be bug fixes and some polishing on the docs and stuff like that but we intend to not like land any bigger changes between here and there by the way by feature complete we don't mean that that's it everything is there rather what we mean is that we think it's a viable system as one to ship a variety of applications on but we know there's things it doesn't do yet and that we wanted to do and I'll circle back to that later in the talk so at the moment the minimum requirement for the package is unity 2019 dot one we may move that up to point two but we intend to support the 2019 line with the 1.0 release okay so next before moving on to looking at specific features I'd like to take a quick look back on why we got here like why did we develop a new input system in the first place one of the big problems with the existing input system and unity is consistency like power of unity has has always been about trying to reduce the pain of multi-platform development and like with the old system like to be frank that pain was off not really reduced ball that much if any like you could take the same controller plug it in on Windows plug it in on the Mac and you get completely different button mapping stuff like that for the new bit system we wanted consistent behavior across platforms same device same result also if it's input it should come in the same single way and the old input system joist a gamepad mouse keyboard input is separate from touch and separate from sensors and separate from tracking data it's all input but it's cast in two incompatible API so they've just thrown their own separate ways over time we want all these inputs to surface the same way also we realize that with what's going on on there in terms of platforms and hardware and just sheer variety of use cases there's no way we can write one system that as this just fits the bill for everyone so we knew that one of the key capabilities of any new system had to be that it's adaptable adaptable in that like you can use its public API to Bent the system to your will the olden puts us to make this one really hard all the data that it had was like kept internally and processed internally and all you saw was the end result to the point where even bindings were only surfaced in the editor not really changeable at runtime so from the get-go we decided that we want to have only very slim platform-specific native C++ part in the unity runtime whose sole job is to just collect and forward all the data it has to an open source managed code part that sits in a package and the does all the processing out in the open all the source data we operate on you can see you can alter and even like make it up on the fly if you want to but thanks with a new system not by a nature now being a good deal more complex than the old one we still wanted something that's easy to use something that you can get set up with in mere minutes something that even though it offered an API that allows you to go really deep at the same time had an API that allowed you to get started really quickly so when we looked at all these things back then we came to the conclusion there's really no fixing the old system without rewriting it so that's where we are this is why the new input system is there and why it's it is its own separate thing completely segregated from the old input system ok that was a lot of words on the why next I would like to take a look at the what and for that I would like to do something a little different rather than me just like talking here through some slides I would like to just introduce the three key building blocks and just like quickly run through some bullet points and then for each one do a live demo instead I think this will will be a bit more interesting than just slides so first big building block is devices devices are the bread and butter of the new input system they're basically the foundation upon which everything else is built each of them is just a bunch of controls and I'll show you in a minute what this actually means oh that was this point by the way this like weird looking stuff up there on the right these are API entry points just for like in case you want to take a deeper look this is just meant to get your start and give you some point to look into and there's a couple slides that will have this weird text arrow in the upper right the way these devices are authored is based on a flexible data driven layout system that you can use yourself to extend and/or modifies device support however you want that's the various register calls that you can see there finally it's worth mentioning that input comes in as events events that you can listen to and intercept that you can put on disk or send over the wire and then play back that you can generate yourself if need be nothing is hidden away there also just to mention that there's support for sending data the opposite way meaning from the application to the device we call that device commands and we use it for Rumble and the kind of stuff but it's also completely extensible you can even offer custom commands if you want but let me actually switch over here and show you this in action so this is a blank project in unity in the latest beta of unity you can use whatever you want like from the 2019 line all I did here was I installed the input system from the package manager restarted the editor like here's the input system restart the editor and that's where we are so the best way to look at devices is through the input debugger and this is something that comes installed with the package you can find it here in window analysis input debugger so here we see devices layout settings metrics basically what this window is is a literally window into the internals of the system metrics is just a bunch of numbers about looks actually only three numbers but stuff that goes on internally settings is a dump of the values of settings that you can actually configure now here like these are the settings for the older put manager we have a new tab now with settings for the new system if you are fine of the defaults you don't need to touch this if you want to modify the defaults you can create a settings asset the settings are stored in the system in your project as a normal asset and then modify this here and then we have layouts this is was what I was referring to previously like this is the extensible layout system basically this branch here represents all the knowledge by controls and devices that the system has and we can go and even like look at I don't know in the gamepad's and maybe iOS game panel this layout completely describes to the system what a a gamepad looks on iOS but we can even go here in the editor and say create a device for this and bam we have a device which looks exactly the same as it does on the device on the on iOS you can double click that and get a debugger window specifically for the device which in this case however is pretty boring because there's no actual device and there's no one feeding events for this device so let's look at this actually with a real piece of hardware so I have an xbox controller here and I can plug in USB always upside down and there we go by the way this notification that just came in that made the debugger realize this you know device its public API you can listen to that and you will get a notification when a new device comes online or one is removed that kind of stuff so I can double click that we see a bunch of information up here about the device and then in here in the middle section we see the controls and this is what I was referring to earlier a device is just a bunch of controls and actually it's a hierarchy of controls so like here we have the device at the top the d-pad as its child and then d-pad further breaks down and a bunch of buttons ok and all these controls have names and you can see for example for the face buttons like we named them by the cardinal directions to avoid any ambiguity like between different kinds of naming schemes that different platforms employ so button South is guaranteed to be the bottommost face button on the gamepad and using that name like this is immutable this name always were first to that control but we can also see in the second column here that they have a display name so here because this is an Xbox controller we see a because that's what they call on this particular piece of hardware if I were to plug in a duel shark roll in here it would say cross and if we were to look at a keyboard for example all this would change depending on the keyboard layout because the display name depends on what the keyboard layout actually chooses to assign to that key so the display name gives you an easy way to display proper names in the UI where's the internal name gives you a stable identifier that always finds the right control so there's a bunch of other stuff here's mostly internal things and blah blah at the end here in the last column we have value and this is a life view like I can go and actually actually let's light them left stick sorry and we see the value and what we also see is here at the bottom you can see that the events come screaming in so it captures the events for us we can even look at one and see the value is coming in free we can see like this is the raw payload of the event troops don't know where this took a second to switch so you can see from here you can pretty much see exactly what is going on with a device and this this works not only locally but also remotely so I have a an iPad here which is connected to this computer and I can go on a remote devices and say iPad oops I need to unlock it it has locked itself enough so and now I should be able to connect and as always this is the part where it does not work let me just quickly see if I can fix that by restarting this thing if not well then you have to believe me that it actually like at least sometimes works so yeah now we connected I had to restart the application on the device for some reason we can now see the devices that this device devices that this piece of hardware here has we can for example open the touch screen this is the view on the state of the device that it has here on the iPad and I can go and I can generate events here by by operating the touch screen updates life in the input debugger disconnect that here and just to close this one out I mentioned that this is completely extensible so you can't go in like we have a sample here actually in the input system the samples are found here directly in the package window and you can install them from here we have a custom device sample you can import that in project just a single c-sharp file heavily document and commented recompile there now we have a custom device and it should actually show up in the input debugger which I should have left open here and then custom devices there's our custom device completely extensible okay so that was devices and actually it was live demo ok next building block sections so actions are this high-level concept where you try to move away from tying your application code to specific inputs like for example clearing specifically to the spacebar key or the left stick on the gamepad and instead like going a level up and operating in terms of logical stuff like jump or interact and making your processing logic less concerned about where an input is coming from but rather just that something is coming in there's somewhat reminiscent of the old access concept and the old input system but they are more flexible fully refundable and lots of stuff so actions come with their own type of asset along with a dedicated UI in the editor but that you can also make them up at runtime or like modify them any which way you want so like they're fully accessible through the API they're based on a flexible binding system which is fully reconfigurable at runtime and there's even like out-of-the-box support to help you implement stuff like rebinding screens that's the perform interactive rebinding you can see here which is based on a flexible mechanism to just listen to for input there is support for control schemes so you can group bindings into sets that make like in case your application can be operated in multiple different ways and finally the whole actual system is built around an inter extensible interaction model that supports patterns such as holds and temps and that kind of stuff and you can extend that and implement your own interactions as well so same thing as before let's jump back into unity and look at this in action so I said it's an asset so you can't create it same way you create any other knesset you go into assets create it's done your introductions or click here right click create oh not material pellet cannot delete okay I guess I had the wrong since anything selected in productions okay let's name this some way sample controls and I double click it and this is the UI for editing actions looks pretty blank now because the asset is completely blank um a bit of terminology here on the left you see action maps an actual map is just a collection of actions can have arbitrary many actions the only requirement is that these actions have unique names and also there can be arbitrary many action maps in an asset same thing only requirement is that they have unique names at runtime like for actions to actually do something they have to be enabled and like you have complete control over that you can enable/disable sets arbitrarily multiple sets single sets whatever even individual actions just from the action map instead of enabling entire map so let's go ahead and create some stuff here let's say okay we have a one map that represents our gameplay and maybe just for fun let's call it maybe menu and let's just come up with some simple set of actions that would could make sense in a real application so you could say okay let's say we have a simple game where you can move look around and interact with things okay so we have a move action look and interact okay and excellest also have this thing like they have brought behavioral types and right now you can see that every single action is set to button which makes sense for interact but it doesn't really make sense for look at move which is actions that we want to source values from and in particular we want them to source vector to values like planar to the emotion vectors so let's switch that we can set that here to value we say vector two and we do the same thing for look okay value vector two and they're like that's our like a sensible set of actions let's go on and actually bind discs to some stuff in particular that's binded to the gamepad okay and I said before it's based on a flexible binding system it's actually a path language kind of that allows you to very flexibly identify controls at runtime and you like we could switch this into text mode and enter some path here but it would not be really any better than the old system where you have to do exactly that it's that we have this really nice new control picker here where we can go and actually pick controls from so let's go say gamepad and you can see on the gamepad here it only shows us three controls because it only shows the ones that are relevant to the action and which we've already said to vector two on a gamepad these are the only controls that can give you vector two values so that's what it shows so let's go and bind this to the left stick move look let's bind that to the right stick and interact let's say this is the DA button on the Xbox gamepad for that we have a working set of controls for a gamepad now let's say we want to extend that you have keyboard and mouse support okay but we want to do that cleanly as a separate control scheme we can do that up here and say okay well we already have a gamepad scheme right we at the gamepad as a required device to that and now you can see it says globe fingers like a binding that is not associated with any kind of control scheme it's just active in all of them so let's actually fix that and say ok all of these they go into the gamepad control scheme and now we have this one set up we can go and add another one keyboard and mouse we can it the key work to it and we can it the mouse to it it's empty because we have no bindings yet for this so let's fix that problem is move already puts us in a conundrum like there's not no control on a keyboard I can give you a vector to what we want to do our verts we want to have WASD like all four keys to act in unison to give us a vector - we can't do this with a normal binding just plants to one type of swords only but we can do that with what's called a composite so composites are a way for one binding to actually source its values from a number any number of part bindings and it sources the values from there perform some computation returns a single value and this is completely extensible we have a couple composites that come with the system but you can write your own they plug right into here they're no different from the built-in ones the reason only one is shown here is that we have only one composite out of the box that delivers the vector 2 value again it does filtering and tries to display to you only what really makes sense so let's add this thing and we want this oh yes this one let's call it WASD ok and then these bindings are the part bindings that source the values that the composite then uses so this one we can combine to the W key we could go in here and actually like look this up in the list but it's kind of tedious instead we have this listen button here and what listen to us is just listens for controls that you actuate and that actually fit what you're looking for so good just hexam you some keys here and they all show up so let's hit W and bind to that so with that it should go pretty quick we say okay that's the S key left is the a key and right is the D key okay with that we have a working WASD setup but now let's say you you also want the arrow keys to do the same thing we could go and just add another composite here and set that up for arrows oh we could go just go and say okay well duplicate this and set this one to left arrow sematic get slightly different in how it works now there's a single composite instead of two but fully valid setup here and I could go and do this for the other ones but that would be boring so that was composites in a nutshell for look at simple we don't need a composite like we want to bind that to the Delta control on the mouse and that is already a vector too so just add a binding here go on a mouse select Delta that's it interact let's bind it to the e key on the keyboard and we're set we have a fully working control scheme for keyboard and mouse now let's say we we don't like the fact that when you hit the interact button it immediately triggers like in many games it leads to like you accidentally trigger actually just walk through the world and you haven't had hit the interact button and it goes off so what many games do is they actually require you to hold the button for a bit before the interaction is started so let's say we want to do that we could go here on the e key and add a hold interaction but that would only apply to the iki so let's do that at the level of the action here you can add a halt so now interact only triggers once the button is held here we have a default time of 0.4 seconds we can customize that by the way you also get a callback when the button goes down not just once the action is complete so if you want to have UI feedback and like this place circle that is charging up or that kind of stuff you get a callback then you can listen for and then kick off the UI feedback so that's interactions and nutshell again inter extensible system couple interactions come out of the box you can write your own and they all show up here and s before like you can go in the package manager like there's a sample for a custom composite as well a custom binding composite shows up there and also just to note that the the custom device we previously added also shows RP on the control pick herb is that the wrong does it not have a button I'm not sure anymore what kind of control it actually has oh I'm in the game a keyboard a mouse control scheme so it filters out the custom device because it is not a keyboard and not opes and not a mouse so in here I can go back and say here's the other custom device and I can bind to that okay so they appeared just as native as anything else in the system ok that was actions let me switch back here this thing and that brings us to the final key building block of an input system so fingers once we had actions after running we looked at the thing and looked how users are using it and one thing that became obvious pretty quickly is that it takes a lot of steps to set things up especially if you go from scratch and especially also if you have like more complicated cases like local multiplayer in case that doesn't mean anything to you local multiplayer we refer to you like a game that where multiple players are playing on the same instance of the game on this machine wear as opposed to like multiplayer where you have separate instance of the game on separate machines and they're wired together over network so we looked at that set up cost and we thought okay how can we make the system so that you can get started super quickly and you still have access to like like all the juicy bits and basically the power that the new system brings and that is player input player I put is a motor behavior component that takes care of a whole bunch of things automatically like for example automatic control scheme squid chain oops following behind here in single-player like let's say your game has a number of control schemes and in single player mode you want the player to be able to just freely go from one to the other like keyboard mouse you should be able to just pick up the gamepad and play and this system will detect that and it will automatically inform the application stuff like that so that you can update your UI hands for example and switch out the icons that you displace things like that um also one of the very reasons that exists easy local multiplayer it's simple every single player but instance simply represents one player if you instantiate multiple you know multiple players simple as that and every player gets a specific set of devices specific to that player um related to that for doing things like joins and split-screen and that stuff we also have another component that allows you to gives you a quick start for this kind of logic again let me switch over real quick and let's look at this in action ok let's save this thing oh I actually forgot to show you one thing well let me just real quickly mention that like this the actions that we previously offered they're fully functional but like you probably remember in the old system you had to then go and actually look everything up by my string names at runtime that was really brittle you don't need to do that a new system we have the ability to generate a c-sharp class that just wraps around the entire thing when you play that here in the inspector importer settings you get a C sharp file it's self-contained no longer depends on the asset you can instantiate that however many times and it does all the lookups for you there typesafe if you rename and actually your api breaks and you get a compiled arrow so that's what I forgot okay let's switch back to player input so let's say we want to create a player here okay so this game object should become our player so you just at the guy and we add player input and with that to the input system this represents one player this thing is not fully functional yet we have no actions the player needs to have a set of actions that belong to the player so we could go and actually drag the actions that we just authored in there but let's say we don't have them you're starting from scratch in the old system you have this thing where like you have some pre-configured stuff right that makes sense for at least some kind of applications so you're not starting from scratch with a blank thing we can do the same here and say create actions we just give them a name BAM and it gives us a pre-populated list that has a number of control schemes in a number of bindings in place just to get you started quickly and not have you start from scratch every time okay then over here we could select the default control scheme it would then start out with that and try that first but if we don't select any just goes through the list sees what devices are available so we don't need to make a selection here what we do want to select this thing as I mentioned before that actions need to be enabled to actually like actively listen for input if we don't do that and assign a default action may appear nothing gets enabled by default we can do that manually but if we don't want to we can just select one and the player will go enable that action map when it is itself enabled and finally we have the behavior thing here we have multiple ways in which we can react to input multiple ways in which play it we can tell us that something happened here let's just switch that to events look in there and we like I don't want to do any scripting here and I think that that's boring to watch but what we can do is we can just wire it up to the player itself it has a debug function on it so every time we trigger the move action the player there will be something in the console right so here and there we have a fully functioning player we can actually look at that guy in the debugger let me pop that back up that's actually docket here and we can see users it's also mentioned down here the play output implicitly creates an input user and the input user is what holes like the pair devices and all that kind of stuff we can see the actions that the user has all that stuff and if I actually go and duplicate the player here now we have multiplayer and we can see that the second guy ended up with the gamepad because there were no more keyboards and mice available so we now have one player controlling the game with keyboard and mouse and the other controlling it with gamepad now let's say this is like uncoordinated let's say we want players to join explicitly we can also hook that up pretty easily we can create another object here say that this is our player manager we put the other component on there now this guy needs prefab I can just drag this guy in here delete it from the scene and give it a prefab and now if I go into play mode I can join one player you can see it on the right and the debugger from keyboard and mouse join a second player from the gamepad easy joining behavior also just to mention that I won't really demonstrate that but salsa support for split-screen we could set up a camera on the player and then enable split-screen here and it would automatically subdivide the the screen area so that's it for this part like these were the key building blocks for the remaining slides I just like to real quickly go and touch on various feature areas and highlight just some key aspects for each ok let's start with touch right now we have two levels of support for touch the basis of all touch in the new input system is the touch screen device which is fully bindable from actions and works the same way as any other device in addition we have a separate polling API that is meant as to be a superior alternative to the old input systems touch api and support for touch recognition for touch history is easier finger tracking primary touches a bunch of stuff one thing we do not yet have though is gesture support this is high on the list of want to have things so we have to stay tuned there game pads and joysticks free game paths we decided to roll out with support for just the most common controllers as a first step meaning Xbox / X and put controllers and PlayStation controllers and on switch also the switch controller controllers the idea here is to first get everything solid in 1.0 and then go broad for these game pads we also support rumble dual motor Rumble in general and dual + trigger mode Rumble on the Xbox one site out for consoles support for NBA platforms meaning an Xbox Playstation and switch does not come packaged with the input system itself but rather through dedicated packages made available directly to license holders it's just the joy of NDA stuff besides gamepad support we also have support for generic HIDs in case that just a meaningless acronym to you HIV stands for human interface device and basic it's a standard built on top of USB and later adapted for Bluetooth for doing input devices if you're looking at an input device chances are it's an HID thing is hid oops HIDs have a way of describing themselves and we can use that information to automatically generate a layout even when we do not have a hand to layout specifically for the device unfortunately the descriptions are they they're not very precise so sometimes it's up to guesswork we have a fallback in place that does exactly that to turn hid joysticks and hid game paths we don't specifically recognize into devices in the input system muscle pen not that much to say here from Mouse pen and touch there is a new pointer abstraction that essentially unifies the three types of devices into a model that allows you to treat these devices interchangeably and then for pen and tablet support which also encompasses stylus support on mobiles we report things such as pressure until levels so like a heart the prayer pen is pressed against the surface and how it is tilted against the surface of the tablet or whatever device keyboard now there are actually a couple interesting things so one keyboard support and new input system is based on mapping keys by their physical location which means that if you set up WASD as we did before and we guarantee you that those keys are in the right location regardless of keyboard layout so if we talk about what we call the a key it is always the key to the right of the caps lock key but if you want to you can also bind by what character the key generates we support that as well which means that if the layout changes the key may be in a completely different location or may even not be on the keyboard at all and then finally we have a no heap garbage text input path that delivers text input character by character so if you for example do typing game you can do that without incurring GC spikes implicitly from just picking up text input um and yeah feature wise there's a bunch of other things that I just like to point out real quick and go through we have support for sensors when gyros accelerometers gravity sensors and so on and so forth these are normal devices so you can bind to them from actions you can make up input for them whatever you want we have support for XR devices like vibes and oculus is and whatever same deal it comes out as regular devices that looks like everything else in the system which also means that unlike in the old system tracking button access data for attractive next our device comes out the same single way instead of one on separate api's note that the the extra support for specific devices is packaged separately so for oculus for example you need to install the oculus specific package and that comes with support for the new input system this has been a very recent change and I'm not sure that their latest packages are updated yet but if that hasn't happened yet it's going to happen very very very soon then we have support for you GUI of course there's a new input module that replaced a standalone input module and has support for keyboard mouse and food gamepad navigation and touch and all that stuff it's based on actions so it's fully rebind able and you can customize it to your liking to use it simply select the event system that unity sets up automatically and on the standalone input modular component you will see a button to replace it simple as that tight to that we also have support for per user UI so if you have a split screen setup for example where every user has a custom menu just on that very the users on split screen we support that as well now [Music] then we have support one screen controls which are typically used to like for virtual joysticks on mobile titles but in this case it's not restricted to that like these controls they can be used to feed input for any arbitrary input device we have support for editor window code so like on GUI for example so you can use all the devices that the input system surfaces all the input there you can use that to tie that to editing logic as well and last but definitely not least we have also have support for writing automated tests involving input and we use that ourselves we have hundreds of automated tests that use that very same very feature it comes as a separate DLL as part of the input system package and a separate API called input test fixture which completely isolates the input system it puts it in an old state and then your tests can drive it however you want so we're pretty much at the end we've looked a bit quite a bit of what still so let's close this out and look at what's not there right first big-ticket items the old input system I mentioned that right now they live side by side and from the get-go the intention was to not think about deprecation of the old system until we have a system that we can rightfully say it fully replaces the old one we're slowly getting to that point and they're like we don't have a definitive plan at the moment how we're going to do it but ultimately the old input system will be deprecated and will be removed nice thoughts and ECS the new input system at this point has no dedicated support for ECS like any main thread api you can pick up input in your component systems or not date function and then route the input there from there into jobs we have something called manual update mode then you can switch the system into which decouples the input system from the play loop and you can just run input updates under your own control but beyond the 1.0 we will take a look at integrating input more natively with dots another big thing is broadening device support especially with respect to game pets and the like we still have pretty limited support so beyond 1.0 we want to broaden that support to point where many more devices have custom tailored layouts devices such as racing wheels and flight sticks and so on and finally there is stuff that didn't make it into 1.0 the aforementioned gesture support is one but there's also holes like stuff like multi display support multi pointer support so there is still work to do okay and I think there's less one yep just to give you a couple pointers in case you want digging deeper and learn more about this stuff here's a couple of resources first they already made an appearance in there in the talk we have samples that can be installed directly from the package manager they can be very useful to look at then we have documentation which can also be accessed directly from the package manager just click that view documentation link we're still refining a thing as you will likely find typos and you will probably also find a hole here and there we're still working on that but there's already a lot there then if you have questions or feedback we also have a section in the forum feel free to post there it's a good way to reach to reach us directly finally there's a github repo if you want to follow development of the system or even contribute when we accept PRS for the repo contains the entire contents of the package it's completely open-source and it's actually where all development happens like it's not just a mirror of something it's where we actually work oops also in case you come across bucks the recommended ways to just use the the normal unity bug reporter file a report through there and it will reach us ideally with the repro case that's always super appreciated also just to know that we have a kiosk here at you night in the hub area for the input system in case of questions you can head over there and yeah we'll be there and that is it so give the new system a try if you if you'd like can let us know what you think yeah another wise just uh yes have a great day and thanks a lot for for sitting here and listening to me thanks [Applause] [Music]

Channel: Unity

Views: 49,146

Keywords: Unity3d, Unity, Unity Technologies, Games, Game Development, Game Dev, Game Engine

Id: hw3Gk5PoZ6A

Length: 44min 48sec (2688 seconds)

Published: Mon Oct 14 2019