Developing AR/VR Experiences with Amazon Sumerian | Ep 3: Introduction to Hosts

streaming is off the air I don't know why we are having streaming difficulties it looks for my in like we are on looks like we're back ok we are back alright so this might be a new video so this is Jake's mr. creative content specialist for Amazon Sumerian and we are joined by Ben Moore who is our lead product manager for Amazon Sumerian hey guys round two on the introduction that introductions that is a technical error I don't know where that is coming from but we are gonna be talking about hosts today after we have a brief and exciting announcement and after that we're gonna get on to a workshop and to show you about Sumerian hosts and how they work and how to use them specifically creating custom speeches so we will be doing that here shortly let me change Kuan quick setting here for myself okay Ben I have a question for you you have a big announcement we have a big announcement if you want to get going with that okay cool well everyone I'm extremely excited to announce today that Sumerian is now generally available which means it is a open and production ready service from AWS we announced that earlier today around 11 a.m. Pacific time the team is really really excited to bring this to everyone who's developing on top of Sumerian so go check out some of the new content you know most of you that have been using it already will be familiar with all of the features but we're really excited to announce this yeah this is this is a big one we've been working for several months and and we're just really excited to get to this this milestone and to just start working with customers and seeing what users like you guys are going to be building we're really excited to be doing more of this sort of stuff on Twitch and just be doing a lot of like workshops and demos and a lot of interactivity and a lot of conversations with the Cimmerian community so yeah we're excited for that another question then is Sumerian hosts it's as I've seen in my experience it's one of the things that people are really really excited about is people see Samara and they're like oh wow Sumerians really cool then we show them host and they're like oh wow that's really really cool so can you tell us a little bit about what is a Sumerian host and what makes them so unique yeah sure so if you've caught the original two episodes of this twitch series you've learned that Sumerian is a in-browser IDE for creating VR AR and 3d applications and then you can deploy those to a URL so that they can be viewed from either compatible VR devices or using augmented reality compatible mobile devices but that also doesn't limit the fact that you can deploy these scenes to a 2d screen and serve it through a web browser and part of what we've done is we've integrated AWS services into Sumerian so that you can use some of our AI services so for instance Amazon Lex which converts text or sorry speech to text and then Amazon Pali which converts the text string to speech so that you have natural language and you can you can create a chatbot so Sumerian hosts are essentially a I driven digital characters that you can use Sumerian to define those properties and then provide an interactive element to your sumerian scene and the host can help guide you act as a virtual concierge as you as it were through the scene and so you can interact with them a little bit yeah yet they're they're pretty fun is that you'll see in this in this workshop in this twitch stream how to use them in just some basics and they're pretty powerful and really easy to use we do have a couple questions that I want to cover one question is when are we gonna have VR on mobile Ben do you want to feel that question you know we're working on supporting more and more devices and so as we have support for new devices we will definitely release that to you and give you the updates whether it comes through social media or any of the twitch channels but the best thing to do is go to aws.amazon.com slash Zoomer in /faq s so fa Q's and you'll see that there is a link there for compatible browsers and compatible devices and you'll be able to see what we support today so you know that stuff is coming soon and we will definitely look forward to supporting more and more devices as time goes on cool another question let's see let's see sue Tinky su Tinky not sure how to say that but you have a question no standalone PC version if you're asking if this is something specific for PC if you're talking about Sumeria itself Sumerian is not device specific in terms of how you edit because it is a completely web and browser-based and so you don't need to download anything so you can use any type of device well any type of operating software to actually edit yeah that's it that's a great point Jake so is it is it PC based no there's no runtime that you're actually downloading or client that you're downloading on to your local machine this is all run through the web browser itself so it does require connection to the Internet and you also need to be using the latest versions of Chrome and Firefox - one for compatibility with Sumerian because those have the latest support for things like WebGL and web VR and the web XR specification that's currently being defined by the w3c so yeah Sumerian is all browser-based it's a great way to get started with building AR and VR applications cool another question we have is can you upload your own hosts or model I've been filled that one as well so you can definitely upload your own models and then if you had a 3d model of a character you could upload that to Sumerian as well there have definitely been some proof of concepts where people are working with powering their own 3d characters through Sumerian and and some of them have have been put out in some demos and things like that but there was recently actually a blog post released by fidelity labs this morning and I'll post the link in in the twitch chat that shows how they're using Christine one of our Sumerian hosts that's available for free with Sumerian - to create an interactive scene and - to look at trading information and things like that's pretty interesting stuff I just posted it for you so yeah this fidelity use case is here it's a pretty cool video to use utilizing I'll just let you check it out fidelity yeah a cool customer use case and then also another question do you have do you need or is it do you need to have specific specs to run Sumerian will it run on Chromebook so Chromebook will probably run it you may not have quite enough GPU to get the performance that you want out of it I be completely honest with you I haven't tested it on a Chromebook at this point but if you are running the latest version of Chrome on the Chromebook you should be able to access all of the features of Sumerian and and test it out we'd love to love to hear your feedback on it there is a slack channel if you're not already aware where the sumerian developer community is sharing tips and tricks the AWS sumerian engineers also monitor that channel you can get there at slack dot sumerian dot world Jake's putting it into the twitch chat right now if you have questions about compatibility that's a great place to go and ask them again the FAQ is on aws.amazon.com slash Sumerian / fa Q's is a great place for finding out that information and then if you do find something works for instance on the Chromebook and you have good success you know share it with your peers and post it and we love to get that information in a developer community yeah what I've noticed is that like yes you can if you have Firefox Chrome and you're able to access the web on a tablet of some kind yes it'll work it just the performance could just it just depends on the processing power so I've seen actually tablets work really well I've seen some other tablets not work as well so it definitely is yeah it's definitely some some trial and error in terms of in terms of devices while you're editing but in terms of functionality yes it does it does work so but again like like Ben said please please post that to slack and let us like update us we'd love to see what people are doing let's see we have a couple other questions is it possible to customize some appearance of features of the host that come with Sumerian no you cannot that is just we're keeping those pretty well that's tag we're keeping those well been a lot waiting answer that question well so you know I think the thing to keep in mind for hosts today is that we're making them available to developers to help you get started and to help you build your scenes and so we haven't built in the functionality to customize them completely at this point and if you want to upload your own characters and start to work with them in Sumerian that's where you're gonna get more of that branded feel that you might be looking for so you might ask a question of well you know why do the Sumerian hosts look the way - they do why do they have the outfits that they have we have an art department with working with conceptual artists that are defining a target that is somewhere somewhere in the middle right but if you want to create something that's a more realistic looking host or you want to create something that's more Stein stylized that's something that you can either do yourself or it's something that you could work with a 3d model or an animator to get the type of character that you're looking to put into your scene cool this is great lots of great questions here and also do we have a way you can specify this is I'm guessing si si oh I games do you have a way we can specify what we are trying to get to work fight if I'm understanding you correctly are you asking how we can communicate how you can communicate with us you trying to get to work if that's the case then I would definitely point you to the slack channel and that's where a lot of the communication like I'm on there Ben is on there so you can actually reach out to us directly if you have any questions also a lot of our engineers are on there so it's it's a great resource what want on violins how does source control work for Cimmerian is there some sort of kind of get CI integrations so today there is a asset library where you're importing all of your models and those models that are unique to your scene the versions will be there otherwise there isn't source control in terms of what you're working on and there is no git integration with AWS Samaria or sorry Amazon Sumerian today so you wouldn't have that as an opportunity for source control either and then as a response to Sao I games I'd also like to say that you know I it sounds may be that you're trying to ask what maybe file formats we support with the models that you can upload so today you can you can upload obj and FBX 2017 so if that's how your characters are are exported from whatever 3d modeling software you're using you can pull those directly into Sumerian by dragging them onto the interface or you can use the asset library and import them directly from a file selector cool all right we have yeah I mean keep asking us your questions we'd love this whoever let me just get rid of all this okay keep asking us your questions we can field these but I am going to jump into the editor itself the IDE Sumerian and start showing you a little bit about hosts and how to get started with them so let me switch over to my screen I will be I will be checking out the the twitch the chat room as well so here regard let me just look around in this scene that we got so far now a few people have asked do you have to watch episode 1 in 2 to get going to follow along for this episode quick answer is no you do not what you see here is just something I've already built and honestly none of this that I have in here has much to do with what I'm gonna be building today it's more just background some aesthetics and it we it just demonstrated some cool features and I'll quickly run through what we're looking at here before we get into the hosts themselves and so let me just start with that so I'll zoom back out here so I've just created just a simple room and what it is composed of is two walls so if I click on this left wall here and then this right wall and this floor these are all just boxes so if I go to create entity and under the 3d primitives and select a box that's all it is and if I unfold the transform component it's just it's compressed and so it's just compressed price on the right word I've just made it very very thin box so it's very skinny and tall and wide but you know the dimensions for the Y value is like 0.1 so as you can see just a really thin box so that's all those are the other assets that we have in the scene if I go to import assets appear above the canvas we'll click on that and this will open up the asset library and so these are these are assets objects that we have provided to you that you can use this is also where you'd find the hosts so we will come back here in a moment but the couch for example is this this object right here the chair is this Essen number right here and what's cool about this is if you select these and you search for this asset number on amazon.com this will be an actual couch and chair and everything else that you see in here you can actually purchase it from amazon.com and so they are real they are real objects that you can purchase so they are they're not just made-up objects or entities they are real-life furniture pieces so that's where we have there let me just close this so this chair this couch this little table and this lamp those are all things that I pulled from the asset library and lastly this is a pretty cool one this rectangle or what looks like this rug all this is is a I went to create entity and I added a 2d shape so it's just a rectangle and then the texture which is really just the image that's displayed on top of that rectangle I pulled a an image from also amazon.com and I downloaded I saved the image for a specific rug you can find on there and then I just dragged it onto the texture image here and then I get again a real a real life physical object that you can find on Amazon and I just placed it here in my scene and so that's for the most part what we have here again this doesn't have a lot to do with what we're gonna be building just some some backgrounds and some consistency with the scenes that we've been working on all right I'll look over here at the the chat and see it looks like Ben is yep I'm working on the chat Jake don't worry about it all right Ben is working on it let's see so let's work on importing a host I'm gonna go import assets and today I'm going to be using Luke Luke is one of our new hosts that we recently added that we announced at the Sydney Australia summit so I'm going to select Luke and click Add and he's going to load original at first down here in the assets panel and so one thing it's important to keep in mind is when you are importing assets from the asset library they don't automatically populate in your scene you need to first find them down in your asset panel and I have speeches filtered to right here so I don't want that so I'm gonna toggle on all of my ass entity types so I can see everything so under the Luke pack you can see this like package icon you can see this Luke entity and the entities are denoted by this hexagon icon and so that's always the that's always the entity or asset type I should say that you need to pull from assets that's a panel into your scene so that's what I'm gonna do oops grab too much there all right grab Luke the entity into our scene and he should load fairly quickly momentarily into our scene and while he's loading one other thing I want to mention is an important component of you using hosts is the ability to tie in with other Amazon services and AWS services so specifically Amazon Lexx Amazon Poly so these are two services that one that we're gonna be using today is Amazon Poly and it is the seat text-to-speech make sure I get that one correct it's the text-to-speech so you can type in a speech and then our host will spit it back in a verbal format and we can actually hear him verbalize it it's cool it's a speech and to do that though we need to connect to our AWS services and in order to do that we got to do something called entering in a Cognito let's see how do I set kognito identity pool ID so let me explain how this works so to get to that select the top scene level entity so I call it intro to hosts and you'll see the year AWS configuration and there's this field right here for your kognito identity pool ID if this is empty then hosts will not be able to speak and because because they're not actually tied to Amazon Poly so we need to create this first and enter this in here before we get it to work so it's a very important step if there's anything I see people do is when they're trying to create hosts as they forget this step or don't see it is an important step and they skip it they're like my host isn't working so it's just quick it's a quick and easy way to the thing you can forget so just a reminder don't forget this it's really really important and it's really easy and I'm gonna show you how to do that I'm gonna jump over to this tutorial it's called Amazon kognito setup using AWS CloudFormation I'm gonna go ahead and post this URL in the chat room there we go you can also find this information on the Cimmerian tutorial site and that URL is underneath the one that Jake just posted yes so it's all there so in this tutorial you'll see two different stack types one is the IOT thing shadow that's for a different workshop or demo the one we want is this dynamic speech using Amazon poly I'm just going to go ahead and click stack hey Jake you got an interesting question about the free tier okay can you do all this with the free tier can you do all this with the free tier yes you can yeah you can so what's really cool about the pricing of Amazon Sumerian is there are no upfront licensing fees or anything like that AWS is a it does its pricing mainly on usage based so with the free tier you've got build up to a 50 megabyte scene and then do roughly a hundred views of that published scene so that would kind of equate to about five gigabytes of data transfer if you go over that then you start to get charged the the tier of the pricing which is storage so six cents per gig per month of assets that you have stored within Amazon Sumerian and then 38 percent a thirty eight cents per gig per month of scene traffic and that includes both during at a time and publish time cool well said it's a lot of numbers you just threw out there but yes it works so it's if you're just experimenting around with it definitely just get started it's it's an easy thing to get going with let me jump back to my screen all right so all I did was if I go to the tutorial I click this launch stack button this opens up a cloud formation stack that is specific to what we're doing here and under the details all I need to do is just create a unique name so I'm gonna call it the default right now for mine is Amazon Sumerian poly tutorial stack and I'm just gonna add on twitch and twitch May 15 so Jake while you're doing this do you do you have an answer for what cloud formation is in terms of the people that may not be familiar with AWS services and and why you're using it on on this particular tutorial that's a great question what clever I'm gonna give you a my version of the answer which is a cloud formation essentially gives you a specific stack and aligns all these different services in a way that you can just plug them quickly into your Amazon seen so it essentially is a is it a template that puts together all these different services and connects them in a way that works for Cimmerian so essentially one-click publishing for resources that you can then call from an ID which is what you're doing with the Cognito identity pool correct correct yep yeah so we'll jump back over here so I'm in cloud formation and I can't really see what I've added is just this twitch May 15th now obviously anybody on watching this stream is going to be able to see what I'm making right now and I will post it and make it available but one with one caveat I will be deleting it after this stream so you can use this as you're following along if you'd like but if in an hour or so you log back on or you try to use it it doesn't work that's because I've deleted this this cloud formation stack this is I've deleted this this kognito ID so just one thing to keep in mind I will make it available but it will not be available for all that long so I would encourage you to make your own it's and as I'll show you real quick it's not very difficult so I've made a stack name I'll select acknowledge and create and it'll start showing this create in project progress and when this is complete then I will be able to grab the actual Cognito ID all right while it's doing that I'm gonna go through a few things over back in Cimmerian and I'm not seeing Luke apparently Luke did not take to my scene so I'll add him again I might be doing it too fast where are you Luke see if I'm missing them so Jake what's the what's the part of the IDE that you're working with right now kind of walk through people who may be just joining us on what you're looking through and and how you're trying to find your entity that you've please all right so I've imported the entity Luke down into the asset panel so I can find the asset there I can find the entity here and what I'm what I'm doing is is I'm finding the entity which is the certain asset type and you can see that by this hexagon and I'm dragging him over into my scene some reason he is not taking what I might need to do real quickly this could be my network settings here I'm just gonna try Christine and what I just clicked on is the import assets button and this pulls up all the different asset types and so I'm gonna try look at Christine and when she appears and I'll drag her and also into the canvas the canvas is this center part here so this is where a lot of the editing actually goes and so when you want to see what's going on in your scene I'll drag the scene over if you want to see what's going on in your scene and what's playing and you're seeing that is what's going on in your canvas so it's the visual representation of your scene and shows you what the 3d experience will actually be and let's see where is Christine where is she I'm gonna try refreshing real quick the other the important of the other comport important parts is what actually may have happened is that I may have logged out so let's log back in on the left hand then the right hand side what you'll find is the inspector panel the inspector panel is a lot of the various components that are unique to whatever entity that you're working on and so we'll show that in just a second the various entities that you can be working on whether it's a host whether it's a anything with an animation those things you'll find over here in the asset panel and why is this not let me quickly just go back into the console and click on Amazon Sumerian pull this back up what I think I did here was actually log out and then try to jump back in okay so pulling up I'm operating another thing to keep in mind is a recent update is that we have made it regional specific meaning bent can you explain that sure so AWS operates in geographic regions around the globe and they have what are called availability zones which are separated data centers so Amazon Sumerian is available in six regions today it's available in Northern Virginia also known as IAD it's available in Ohio notice EMH following the airport code bit here then also Oregon so that's PDX Sydney London and Dublin so there are six regions globally and then the pricing information that I gave earlier today that price applies across all of those regions so one of the things that's unique to Amazon scenario is that there is a single price globally to use the service itself cool Jake did you get yourself loaded up I got myself loaded up excellent alright so here we are I'm so what I did was I found Luke down in the asset panel dragged him over here into the canvas and as I was saying the inspector panel is unique for whatever into entity types so when I select floor I'll kind of fold up some of these components you see a few components there if I select Luke we have a few other components and that's what we want to talk about today we'll start with I'll collapse most of these we'll start with the host component the host component has three different properties that we're going to look at well actually just two today the first is the point of interest the point of interest has to do with where your host is actually looking and so right now it's set to none and let me show you what that looks like I'll press play and what that means is that he's not Luke is currently not looking at anything he's just looking straight ahead so his point of interest is just a I would say blank stare but it is forward so as you can see here as I move my camera around he's not actually following me yeah a little bit more there you go point of interest none now we're gonna look at switch it to look at entity and let's it's like the target entity to be the floor and so what this does is that it allows the host to look at that specific entity because the floor is below him I'll have to stop and restart the scene because the floor is below him Luke will be looking straight down at the floor because that the target entity and what we have selected to be the entity to which he focuses on so press play and there you go there is Luke looking at the floor because that's not as useful I'm gonna stop and remove that target entity from the floor and I'm going to take the default camera and add it into the drop entity input and this one's actually really cool what it does is that it means that because the default camera is the camera or the entity in which we're viewing our scene within the the host will actually follow the camera with his eyes with his head as you move around the scene so as we play it here and you notice as I move the camera around his head swivels he watches the camera as it moves around so because he's tracking the entity which is the camera let me zoom in a little bit more and one thing it's really cool about this is that if you're new to like 3d some of these little nuances are can slip by it really easily but what's neat is that if I move just roll slightly his eyes are moving but his head is not but then as I move more then we get more movement from his head and his and his eyes so all these different entities that comprise Luke are all separated but all operate independently but yet together to make a really realistic experience in a Hugh lifelike human host and so that is what the point of interest component is I'll go ahead and stop and I'll leave it as is with defaults camera is the point of interests and I'll collapse that and then animate using speech all open up this these two settings under this are lip-sync and gestures these are really two useful ones also so when you we will shortly add in a speech and the lip-sync just makes it so he moves his lips around along with the speech the so with that this day Amazon Pali speech that we've created so how does Amazon Pali and the lip syncing work together that's a great question do you want to answer that bin sure so Jake well Jake I think I'll get into this when he starts building a script but there's something called speech markup language that comes along with Amazon Pali so as you have that text string that's being translated by Amazon poly there are there's markup within the text that allows us to identify specific words that are very common in the English language and associate gestures with them so as we're speaking to you it's hard to see in the frame but I am actually speaking with my hands I raise them up and move them around it's much more natural for human beings to interact with each other that way so to make the hosts believable we add those gestures into the speech and so those those speech marks that are written by the SS ml let the hosts know that we want one of these animations to happen and so for instance if the host says 1 there will be a speech mark for one and the host will hold up one finger or if they say something like my they will hold their hand to their chest representing themselves and so it makes it a little bit more of a believable experience when interacting with a AI driven digital character right and we'll see how that works real shortly so it's for right now we're just going to keep these two checked because yes we do want him to lip-sync and yes we do in him to have gestures this gesture hold time and this minimum gesture period what this has to do with is how long it'll hold that gesture and gestures depending on what they are they can be very long they can be short and so it just it will define how long a gesture will actually be held for so example like a wave can be held for a really long time could be several minutes or it could just be a couple seconds so this gesture whole time though is just set up to be ten seconds currently and that minimum gesture period is just saying how long a gesture needs to how long it needs to be before I will go over to the next gesture so right now we have it set really short to the point at two point one and so these settings also are by default are usually the best ones to use because different gestures can be placed at different places within your speech and we'll see how that works shortly these are some values you can kind of mess with a little bit and it definitely takes some tweaking to get your specific speech and the gestures and how many gestures you have and what looks natural and what you want exactly for your scene these values are important to kind of mess around with an experiment with but just for right now to start with these two settings are a good place to start so that is the host component we're gonna close that and we're gonna move on to the speech component Jake you mind if I interrupt for a second yes so one of the things that you're looking at is you're looking at what we would call a direct integration with an AWS service within Amazon Cimmerian so Jake's talked about the inspector panel and he showed the components and there is the host component and then we're gonna move into this this speech component here we've integrated Polly directly into the Cimmerian interface so that you can interact with it graphically whereas there are other Amazon or AWS services that are available through Sumerian that you can access through a text editor and use Java JavaScript to access the AWS SDK so that you can call and get data from other AWS services so the example might be for instance maybe you're collecting sensor data off of something and you want to move that into a scene you can do that through those through that text editor so don't think that the functionality is just limited to for instance Amazon Lex or Amazon poly there is a lot of other services that you can start using the the the data that you have in the AWS cloud to power your scenes cool great it's Amazon poly they've been mentioned that's actually what we're gonna be seeing here in the voice so though we have host the host Luke and we've named him Luke he's currently using the voice Russell and again we did not we did not create these voices because this is directly from Amazon Pali so we've selected by default Luke's voice to be Russell but you could select it to be a female voice you could select it to be any one of these voices that you see here and what's cool about these is that they all have different accents accent types associated with them and so again something I'll leave it at Russell right now but it's fun to experiment around with these and to see all the different voice types different accents and how they sound and some might be really unique to what it's a senior trying to build and might be better suited for what you're trying to do but Russell actually has an Australian accent and so we're gonna leave it at that volume is how loud he talks we're just gonna leave it at one and then this speech files input right here right now we don't have any speeches and so what we want to do is create excuse me press this plus button to create a new speech and so when I do that what it does is it pulls up this text editor Ben Mitch in this text editor and it's just a simple text editor this is the same place that we will edit speeches HTML 3d entities or HTML entities scripts and JavaScript so it's it's a basic text editor that is associated with in it's a tool within Sumerian and this is where we're just gonna start writing our speeches so let me start with something really simple hello twitch my name is Luke and I am a Sumerian host and save okay tonight almost made the same mistake I was about to press play just get back to the other here I was about to press play and make him talk but I've not actually never entered in the Cognito identity pool ID so I will go back over because it should be completed by now to my stack and the output is right here so this Cognito pool ID the value this is what we want so I'm just going to copy and paste that in here there we go and after we do that you don't have to touch anything else and so great so let's press play and see what happens I'm gonna give you a heads up it's not gonna work he doesn't say anything again neither another easy step that's easy to miss is that right now we have nothing in our scene that's actually telling Luke to start speaking and so that's what's really important about the state machine in the state machine is a really robust tool and component within Sumerian that allows you to add in interactivity and create behaviors and so you can start making some some interaction happen so what you need to do is under the state machine we need we don't have any behaviors we just want to create a new behavior by pressing the plus button again this will take us into the state machine editor and it's really easy what we need to do we just need to click add action to this state and I know I'm kind of breezing through the state machine there's a lot to say about the state machine so I would encourage you to go back and look at last week's episode if you want to learn more about the state machine but for right now let's just add one action called start speech so I search for it I find it here start speech we'll add that and under the speech drop-down menu I need to select speech because that's I didn't change the name of it and that's it this state is also selected as the initial state and so when I press play this state will fire and because this state will fire it has the start speech action associated with it so it will play the speech so that's the other part that we need to make this to make this happen so let's press play and see what happens again and this time Luke should start that speech that I just entered in here we go loading up this new scene so excited solo twitch my name is Luke and I am a Sumerian host there we go I don't have any speakers or headphones that it looks like he just said it so there we go that's the speech it's real simple just one sentence and if you notice he didn't have any gestures he's moving he's still animated he's still having that what we call an idle animation meaning he's he's breathing he's moving his hands back and forth he's not like a stick figure like he is when were in edit mode but he's not gesturing yet so let's press stop and let's look at how to do that if you notice over here in the speech component under there are certain speech that we have or the specific speech we just created what's actually let's edit this first and now that I have the text editor I'm gonna hover over the speech over here in the documents panel I'm gonna click the edit button and I'm gonna call this intro speech so we see the word speech a lot in the UI and I just want to make sure we we're all on the same page of what I'm referring to okay so now the speech changed to intro speech great now you see this little button right here this is the auto generate gesture marks this is really really cool what this does is automatically inserts this that markup that Ben was talking about into your speech so I'll click that and then I'll pull up the speech by clicking the edit button again and there we go we just we have some gesture marks so this gesture wave this gesture self which means he is pointing talking about himself because and the way this the this worked is that the hello' allowed this wave because wave is connected with hello self the self gesture meaning kind of points at himself is identified by my the word in my and let's see in the word i also generated another self gesture so or just just self gesture yes however one thing to keep in mind is that we have this is a really short sentence hello twitch my name is Luke and I am a Sumerian host and this automatically inserted three Jeff into this speech so not all of them might they might not all fire because they're all really really close together and one gesture might kind of override a run too long so the other ones don't have time to fire so that's again just one way you need to work with it and tweak your speech to make sure that all those are firing and the way you want one really important thing is to make sure you click Save so a lot of times if you don't see any changes it's because you haven't actually saved your your speech file so I'll click Save jump back over to I'll close the state machine editor and now with those gesture marks let's press play and see what happens should start with the way or twitch my name is Luke and I am a Sumerian Heist okay so you see he just keeps waving the reason why is because I have his I've actually not done this this made this exact adjustment so let's just see if this works so I'm gonna reduce this gesture hold time to a lot shorter and then we'll press play again and he should actually be firing that second there at least one of those self gestures and I do have a speech that I've already prepared I'm gonna insert it into addition here see my name is Luke and I am a Sumerian Heist alright so that exactly works so let's go back to it and let's take off this one I'm gonna do some editing here and I'm gonna move this gesture to the very end click Save then again I am I'm editing this without this specific part without any without trying it before so if it doesn't work exactly the way I want again this is what I was saying is the takes a little bit of trial and error messing with these to get them to work the way you want them to work hello everyone my name is Logan I am a Sumerian host there it is cool so yeah moving that around allowed him to actually to see both of those gestures because the wave is longer than the speech is and so it was just it kept going then the speech was over and didn't get a chance to actually fire that self gesture so that's that is the way that one works okay so let's do something else let's add another speech and I'm going to rename this one so we have our intro speech and then we have speech I'm just gonna call this long speech alright and I'm not gonna do all the markup here I'm just gonna I'm just gonna copy and paste it all right and this is a cool thing so let's try I'm gonna insert some names here let's see let's do just I'm just letting the chat window see who do filthy Charles I answered that question I don't I'm just putting his name in oh gosh a speech gotcha and Su Su Tinky great username oh this one's gonna be fun I'm gonna write this one in donuts and pizza our life that's a great one donuts and pizza our life did I get that correct okay I'm just gonna leave those three in there and see what happens we'll see if it actually reads it yeah yeah it might not read it correctly so maybe you could part in a look if he doesn't say quite the way it's supposed to be let's see how it sounds I'm gonna put on my headphones so I can hear it hello twitch my name is Luke and I am a Sumerian host oh sorry one thing I forgot to do my mistake this behavior that I created the only it still is tied I'm gonna go back into this I need to select long speech because that's not the still just reading the first intro speech that I had so it's played again I'm really excited to be here today with Jake and Ben and all you wonderful people here on Twitch I'm glad to see you've joined us today filled Charles seat inky doughnuts and pizza AirLife you learn a lot from this workshop because you'll soon find out that we Sumerian hosts are quite the characters that is cool I love doughnuts and pizza are for donuts and pizza our life that's great don't actually read pretty well cool all right so now we have two speeches I'm gonna stop this how do we get it to where we can play both speeches well what we need to do is I'm in the state machine here I need to add another state and I'm gonna rename state two I'm gonna call this one intro and I'm going to add an action and I'm just gonna do the start speech action and the speech I'm going to select his intro speech and I'm gonna set this is the initial speech because this is the intro speech and then state one I'm just the state one is just a name I'm just gonna rename it to long and then it already has the long speech attacks attached to it so that's fine and they wanted me to do is add a transition from the intro speech to the intro speech to the long speech so that it will actually fire the intro and then when it is done the speech is ends it will then transition to the next state which has that long speech attached to it so in this transition label right down here I'm going to click and drag and we'll get a transition arrow and we'll drag it over to the long state for the long speech and what you'll notice is that when he's in the when he's speaking the intro speech this intro state will be highlighted and then when it's when that's done it'll move over to the long state or long speech sir Jake maybe maybe I have even a little deeper into the state machine what if you add another state you have an input for a click or tap on the host itself to start it instead of just having its dirt directly on the scene load okay so he's putting me on the spot yes there are different ways to start this right now I have it set up to be so that it just fires upon when you play this when you play the scene and so that's one way to do it if you want to make it so you can initiate the speech to start you can definitely do that and in many many different ways a simple one that Ben just mentioned was like a click or a tap tap meaning if you're using mobile I'll click Add state and for this new state I'm gonna call it I'll just call it click and I'll add an action and one thing to keep in mind is that just renaming it click doesn't give it a click State or the click action you have to do that separately I just like to name States so they correspond with the action that's on them so that's why you see click here and click here alright so now I have click and this one is actually attached to Luke so when I when I actually click on Luke that's when the speech will start and so I'll add a transition here and I also need to make sure that my click action is set as the initial state and I'm gonna rename this actually I'm gonna call it wait for click because it will be in this state until I click on Luke so let's see how it works so we press play he will not start speaking right away another thing I'll add to what you just did to is this is only going to play once if Jake clicks on Luke once the the preview is playing in the scene so on that last speech if he had drag the connection around to the initial state it would loop that back around so once it ends you could click it again yep I just did that yes so right now as you see it's he's waiting like he's animated he's not gonna fire and we're in the wait for click state until I click on Luke and once I do then it'll go over to the intro speech it'll play that whole speech along with its gestures then it will transition over to the long speech which currently doesn't have any gestures we'll change that in a minute and then it will go back to the wafer click and I can click on them again and I can repeat that process so we call it our twitch my name is Luke and I am a Sumerian host I'm really excited to be here today with Jake and Ben and all you wonderful people here on Twitch I'm glad to see you joined us today filled Charles seat inky doughnuts and pizza AirLife you learn a lot from this workshop because you'll soon find out that we Sumerian hosts are quite the characters cool yep that's exactly how it works ok so yeah what we did is we added what's called what I like to call like an initiation action so a click or some sort of an action from the user or you know right now myself that starts it otherwise it'll just fire automatically when you press play let's go back to Luke I will make just a couple more adjustments to the long speech actually before we do that let me just click the auto generate gesture tags or gesture marks and then I edit so I click that Auto generate button and now I'm going to edit that speech there we go it automatically added all of these gesture this gesture markup so this it will definitely play or it'll fire a lot of these different gestures but it might take some tweaking to make it work the way exactly the way you want so I'm just going to go back in here let's add I'm just gonna copy and paste some some stuff that I've already had just so I know that it works the way I want it to work and I want to point out one other thing and let's see so I'm just copying and pasting and once I'm done with this I will explain some of these other markups you see in here actually it's pace their dad dad okay and then lastly this one okay all right thanks for bearing with me as I copy I'm also used to using a Mac so using this ctrl C instead of command C is very unique for me I must say there we go all right I got it all right so we have the the gestures and so one of them is I'll save it just so I don't forget this we added it in this generic see there are three different generic gestures there's generic a generic be generic C and they're just kind of generic movements that you can add there's this many gesture that you can add as well and you see that oh there's another generic a in there I'll see agreement actually that one that will not work right now and then you gesture we've looked at also if you notice there's these break tags in here and so if you want him to pause or your your host deposit views and Christine do you want her to pause you can add these break times in there so they'll actually pause for that amount of milliseconds and so I think that I have this all correct so let's click Save again and let's press play see how it works now we'll have to click on Luke again to initiate this and here we go click on Luke hello twitch my name is Luke and I am a Sumerian host I'm really excited to be here today with Jake and Ben and all you wonderful people here on Twitch there's a generic I'm glad to see you've joined us today sliggoo Lee and semester filled Charles seat inky doughnuts and pizza AirLife I hope you learn a lot from this workshop because you'll soon find out that we Sumerian hosts I quite the characters cool there it is so no I didn't actually copy all the ones I wanted to in there but you get the point you can add those in there we I can also in one of the tutorials let me find that actually there's a tutorial in here and it is right down here this using hosts in speech components I'm gonna post this one in here this lists a lot of the different what we really just hello my name is Preston I am a Sumerian this tutorial lists a lot of the different speech markup and gesture marks that we have and so if you want to access that a lot of those a lot of this stuff is there so go ahead and look at that if you want to get all that so anyway that is the basics of getting a host to work in a scene using the host component and also the speech component and the important part of using the state machine to get these speeches to fire the way you want and also I can't specify this enough the AWS configuration entering in that kognito identity pool ID on so you can actually connect with Amazon Poly Amazon Lex so that is that wraps up the demo for today is there any other questions that we need to field any other outstanding things people were wondering too pal thanks well so I think one of the things that we haven't actually discussed Jake is maybe the amount of lexicons or scripts that you could store using Amazon poly and use within your scene and maybe what some of the limits are so since poly is a different service than Amazon Sumerian and you're taking advantage of that service there are certain limitations that you have to consider in terms of the scenes that you're creating so for instance each lexicon and what you're putting into your script can only be up to 4,000 characters in size so each time you write one of those it's not gonna be something that you can go on reading for you know 30 days if you wanted to and then and then the other piece to remember is is that Amazon poly allows you to store up to a hundred unique lexicons so there would be a significant amount of language that you could put into your scripts and our site but it is it is key to remember that you need to look at the other AWS services that you're integrating into your Amazon Sumerian scene and understand how the limits of those services will impact what you're trying to build yeah another cool thing that we've seen users do is actually import multiple hosts and then you can create state machines that allow these different hosts to then kind of talk back and forth and fire multiple multiple speeches and you can loop those I mean so you can actually create conversations between hosts and so there's there's lots of different things that you can do involving speeches involving Amazon poly and also Sumerian hosts so yeah a lot of lots of things that you can cover I believe for next week we are going to be doing Amazon Lex and so creating an Alexa so we can actually interact with a elect or a host and so they can respond and we can have inputs based on voice or typing it in as well so really really cool stuff that should be next week yeah but again I can't I can't point out enough how much we would love for people to join the slack channel I'll post it again here it's slack that Sumerian world please find us there that's a really really great place that you can access you can get access so you can ask us questions you can communicate I'd I had somebody just even this morning asking me some questions about actually this demo that I had done a little bit ago and so it was really fun working with people and just helping people get their scenes up and running and doing it what they want to do it's fun and at doughnuts and pizza are real life just because I want to say your your twitch name on live thanks thanks for the congrats we're really excited that we were able to announce GA today we are extremely excited about all of the things that the community is going to be building so please share them with the slack channel that Jake just put into the chat we want to see what you guys are doing we know that you're gonna have some really creative ideas so keep in touch cool all right well Ben thanks for joining me today we are going to be back here next week Tuesday at 1:30 Pacific yeah we look forward to seeing what people come up with even this next week so thanks for joining us and we'll see you soon

Channel: Amazon Web Services

Views: 5,415

Rating: 5 out of 5

Keywords: AWS, Amazon Web Services, Cloud, cloud computing, AWS Cloud, Virtual Reality, Augmented Reality, Amazon Sumerian, machine learning, Twitch, AR, VR

Id: DLUDpl7wVyg

Length: 61min 36sec (3696 seconds)

Published: Wed May 16 2018