AWS re:Invent 2019: [REPEAT 1] Create 3D web apps with AWS (ARV201-R1)

hi everybody so so I hope you're here to to explore and learn and not just because it's raining outside because we've all been there but without with that being said I want to thank you all for the privilege of your time for joining us today my name is will Kennedy and I am a specialist Solutions Architect at AWS and I focused specifically on augmented reality virtual reality and 3d web applications for AWS and we have a treat later today Eileen from a1 control will be here or she is here I should say and she'll be discussing a little bit about her journey and her company's journey in leveraging the AWS ecosystem to build a fully functioning 3d web application the reasoning why was created the processes that were involved the challenges that were involved and ultimately the real problem that it helps solve which i think is incredibly interesting it's it's really powerful stuff life-saving stuff that it's absolutely amazing what can be achieved using cloud technology today and so sure she'll share her journey and then also provide a demo of the application that she has built so before diving real real really into sort of the nitty gritty on on how all this works I do like to pull the room a little bit and ask a couple quick questions how many folks today or are in this room today who believed or thought AWS had a 3d application before they either read the syllabus or found the topic on the application that you knew that we had a 3d offering to build something okay so not too many hands of those hands that were up how many then have been successful in actually creating even if it's a a non fully functioning application but how many have been successful in creating something with it okay fewer hands and so I've the reason I like to ask this question is it proves my point that folks want to build 3d applications that's why everybody is in this room the process of building it however is incredibly difficult and you usually had to have a set of really specific skills in order to be successful in building an application and so some of the things that were incredibly you know helpful for us to understand and why we wanted to create an offering that allowed folks to create at 3d applications was first understanding the barriers to building an application today when you build something ultimately you want to create something that's useful so having a useful answer sometimes can be challenging to define when you're working in the 3d application folks also said that we didn't have all the skills possible to actually build a 3d application there were a lot of dependent technologies dependent roles that you had to have either known or have a network of folks that you could work with in order to help fill in gaps and you didn't have that or you didn't know that that was something that was required to when you started you know so then it became an incredibly difficult task and it was oftentimes too difficult to complete and so we wanted to ask ourselves that there are a lot of ways to create a 3d application today a lot of tools a lot of engines a lot of services have come and gone from all sorts of different organizations and companies and manufacturers but we wanted to ask ourselves what makes an AWS 3d application what are the things that we do well that we will continue to build upon to enable you to build a really rich immersive 3d application and so there were a couple really important pillars that we believe all 3d applications will have to have in the future in order to be seen as successful in engaging applications first it needs to be contextually aware you have to know how it's going to be used and where it should be used natural language processing is something that we fully believe will be the natural interface when working within the applications doesn't mean we're all going to be building towards this today but we do believe the final resting place will be an interface that allows me to use my voice or you to use your voice to actually interface with your 3d web application building the integration hooks into that can be challenging today we also believe that an application will be globally available and scalable like everything that is on AWS the reach that you have to have today has to be of a global reach and so building for that using today's tooling and for the 3d applications can be challenging we also believe in this concept of multi-user synchronization that 3d applications will not exist in a void they will not exist in a silo that when you create an application it will have multiple users who can interface with this application maybe sometimes at the same time and so being able to build in the hooks that allow you to know where my user is what the past history might cut my user has with my application become very important in order to provide a really curated and catered experience for that user and lastly and this is a very important concept we believe that 3d applications are a stepping stone if you today built a 3d application and you wanted to let's say add an additional element of interactivity be it VR virtual reality or AR augmented reality you would generally have to start over and so all that capital and all that investment that you put into building your 3d application wouldn't necessarily matter you'd have learned a few things but the interactivity would completely change as you build this new type of application what we believe is that 3d applications can be a stepping stone that will empower additional interactivity into the application itself so that if you want it to build an augmented reality or a virtual reality interface into your 3d application you don't have to throw away the work that you did creating your 3d application to start with so these pillars are vitally important to how we believe AWS can best help provide real tooling for you all to build compelling 3d applications but all of this doesn't matter if we don't have one thing and that's simply adoption we have to build stuff that people want to use and so if we build something that nobody uses it doesn't necessarily matter how passionate we may be about that it is not going to necessarily be a great exercise and so we have to ask ourselves well what are some of the things that are really important with adoption and the good news about this is and I would like to see a quick show of hands how many people today believe you have used a 2d web application ok everybody's hands should be up because either you're not paying attention or the 2d web application you're not familiar with what it is nice to any web application if you use the web page you've used a 2d web application if you book your flights you've used a 2d web application so the good news about this is we've already have a working model of something that has been ubiquitous you bigger ously adopted we have this workflow that we can study from you know so we're at reinvent and you know we're all quirky and and kind of fun so what's one of the most quirky ways that I believed you could represent what adoption could be and the answer is math so this is my formula for what I believe adoption is it's not too complicated because we're gonna walk through the basic components of this and this becomes the framework of what we're going to be talking about for the next hour you have to have a great developer experience plain and simple you have to have a great user experience but it cannot cost so much so that it undercuts all those benefits you're going to iterate on that experience over and over again and you're gonna try to make it better a better developer experience a better user experience that lower costs at some point in time you may say whoo we're not achieving what we wanted so we're gonna have to course-correct so in an ideal sense you want to have a great developer experience a great user experience low cost with minimal oops or minimal deviation from each round that you're iterating on your project and that's adoption very simple to explain and we're going to dig into the specifically the developer experience and the user experience so the developer experience can first be digested or research but if you look at how things came to this point and so there's a really interesting video that I'd like to play for you all now what you're seeing on screen here is is if you search for something called a rant ablet but this is a rant a blight interface it is one of the very first graphical user interfaces for writing code now this was used by biometrical our biotech engineers and financial services firms because this is how they worked the developers in the 1950s worked off from a flow charting sort of process in order to actually go through their programming or their designs and so when this was created initially the input was thought to have been a keyboard but nobody knew how to type so a user interface was created that worked the same way that people worked in their current roles and and that's a really really really interesting point because they didn't have to learn necessarily anything new they became immediately productive in this task in this environment now why don't we see these everywhere today because one of these costs one hundred and forty thousand dollars back in nineteen fifty was incredibly expensive and so we have to ask ourselves how can we achieve a workflow that allows me to work the way my developers would like to work give tools the way my developers would like to work at a cost that becomes affordable so let's think about what how we work today when we do a couple things differently first we all know how to type thank you it's that's a huge benefit but secondly we also have a graphical user interface we work with we do a lot more clicking today we also have a unified interconnected fabric using common tools with a common language the common tool that everybody is used today is the web browser a common language that powers that web browser is JavaScript so we're asking ourselves what can we do to bring a set of capabilities for folks who know how to use browsers who know how to use JavaScript and give them the ability to build 3d applications traditionally 3d applications required advanced knowledge advanced math skills and while those skills are beneficial to building a 3d application we wanted to lower the barrier to entry and make it so that anybody who was able to create a web application would be effective in creating a 3d application so we have to ask ourselves what tools are available today as a developer to allow me to build what things exist today that give me that unified interface that allow me to take native JavaScript allowing me to take a browser that allow me to to work with web tooling to create that experience and the answer is there's there's plenty of tools that exist out there as independent buckets but there isn't necessarily a unified interface so it allow me to bring in assets work visually script as necessary make easy things simple and hard things possible these are the sort of things that we would like to have solved for and so when we realized that there wasn't a unified tool to do this at AWS we solve problems by building our solutions and so therefore we created Amazon Cimmerian [Music] [Music] hi Christine can you book me a hotel okay I have you down for a three night stay in New York starting on the third of September 2018 [Music] [Applause] good evening welcome to Sumerian concierge experience [Music] vous pouvez Monteux non-commercially keep the clothes on 8 Arizona to envelop Watkins s2d a lady blue hello my name is Preston and I am a Sumerian host how can I help you today [Music] what would you like to build today so there's a couple really important things that are all under one umbrella when you start working with Amazon Sumerian and that is the specific concept of being able to create and integrate and deploy your solution all within one platform when we say create we're saying give you as a developer the ability to take entities and assets that you're used to working with maybe you already have 3d entities out there you're working with maybe you don't that's ok take that information and pull pride a common platform that allows you to compose what you'd like to work with all on the web all powered through the web integrate the AWS ecosystem into that 3d application natively because Cimmerian leverages JavaScript as the native scripting language if you know how to write JavaScript you know how to work with Amazon Sumerian and if you can write native JavaScript then you can use native JavaScript libraries to power an additional experience case in point if I wanted to make API calls to any back-end EWS service I can instantiate JavaScript objects right within Sumerian and use those objects to make requests if I wanted to if I wanted to bring in a custom library let's say I was working on and this is a true story I wanted to I'm working on a call center use case and I want to integrate the Amazon connect client-side JavaScript library and make my 3d application make phone calls you can absolutely do that so the imaginations your limit on what you like to achieve based upon what level of integration you require and then to be able to deploy your solution by just simply publishing to a common web link is going to provide the ability for you to share this experience with anybody who has a web browser and that's a really important decision in order to make your 3d application or maybe an AR or VR experience easily deployable to public it also makes it easy for you as a developer to make changes to your application because if I deploy through web link I can then immediately update and redeploy through web link and that doesn't require my user to redownload an updated application to use my web link so we're leveraging native web code in order to make this possible we're leveraging WebGL to drive the 3d experiences but you do not need to be a shader expert in order to use Amazon Cimmerian you just need to know how to point click drag and if you like to script JavaScript so let's so let's show you a little bit so hopefully everybody in this room has seen this so if you've not seen Amazon Cimmerian on the council before it does it is on here loading an Amazon Sumerian scene for the sake of time we'll put you into an environment that looks like this ok a little bit bigger very good so this is a web site this is a web interface this isn't a client that had to have been downloaded we created a brand new scene within here I have total 3d control of my scene within here I have the ability to compose exactly what I would like the scene that look like so I'm gonna do that you can see I can create basic shapes within Cimmerian I have a basic ability to do scaling and different transforms rotate and movement with these shapes so I'm gonna make this a little bit bigger I'm gonna move this up just a little bit and then what we have on the asset panel is a set of assets that I've pulled into this environment and so we're going to import one of the available textures that Cimmerian has now when you look at this asset import panel you notice there's a couple things here I find them interesting you saw the host being used a lot in our videos and our customers we've worked with really enjoy working with the hosts because our hosts are essentially created entities that have built-in rigging so structure and bones built-in animations can work natively with poly and can work natively with Amazon Lex as well so these can become the face of a concierge experience well he's also give you to come these could become the face of your Lex BOTS if you would integrated into your body in here it's also a different in here are additional assets that you can work with different materials that we provide for you to help you get started of course if you wanted to bring your own assets into the environment you certainly can but what I'm gonna do is I'm gonna take this wood ash and apply it to them on the cube I made so now I have a block of wood not real exciting however we're gonna do a trick and we're gonna change the culling of our wood so that the engine knows to render the inverse of this object effectively creating a really really quick room okay very very very quick room inside out room I'm gonna take another entity that I imported earlier and I'm gonna drop it in here and that's a lamp and I'm going to raise this up let make it a little bit bigger and I have a nice large lamp in this room and I'm going to fast forward to what this looks like because I built this earlier so I'm all done and I have my Sumerian host inside of my room now there's some interesting properties about the host that I would like to have us explore a little bit on so when you select a Sumerian host specifically you have on the right-hand panel here additional options about the specific entity you're working with so if I selected a basic cube you're gonna have geometry materials but then you can add differing components if you need to - the object my host I have the ability to do some interesting things with them I can make him animate I can make him gesture I can make him talk I can make him integrate into an Amazon Lex pot and you do all this by leveraging the actual components on the right hand side for the bot so you don't need to write code per se to integrate into the Lex service all that needs to be done is indicating what voice you'd like the Bob to say and then indicate the name of the Lex bot that you've created and what you published to to enable this experience I'll load this up real quick so this is the Lex bot I created and this is the bot that is going to be leveraged for my hosts so very easy to get started working with integrating your hosts into Lex into Poli and making them the voice of the 3d experience you would like to drive something our customers have really found valuable and getting started the other thing that's fascinating about working in Cimmerian specifically I mentioned earlier is I didn't mention this earlier I should say is this concept of a state machine so our visual state machine and you might have saw it in the video that played earlier is a graphical coding interface that allows you to effectively connect the dots of differing states that happen within your scene so when my scene starts in this example I want to only move forward when the AWS SDK is ready I want to then have it say something and then I'm gonna ask it to do different things so let's see what happens here so we're gonna preview and play this scene effectively it's going to start with the AWS SDK ready component so we have this bot here pressed in asking me for what he'd like help with and all of this now is going to execute my Lex pot I'm gonna ask Preston turn the lamp on and so he's gonna make the request the what and now the lamp has turned on so it's a way of being able to talk to your use AWS back-end services and turn it off he's probably being blinded turn the lamp off okay so very very very simple to get started working with your entities and your 3d environments but let's take this a step further because this is interactive and and I would like for you all to be able to participate as well so this is the time which everybody needs their cell phone so as we've talked about publishing these on a web browser but but this isn't the only place you can publish so we're going to pivot out of this and I would like for everybody to navigate to this link because this will and I'll leave it up on screen for a little bit well load a 3d scene and this scene will be rendered on your phone so there's no client you had to install nothing to to download this is using your native web browser to render a 3d scene when it loads these aren't containers in the traditional sense these are shipping containers in the traditional sense when this load you're going to see a scene with a bunch of containers in it now we mentioned earlier the ability to integrate and so you have to ask yourself if I make a 3d web application we're doing want my data to live that I need access to is it really smart that that data lives within my 3d world probably not from a well architected perspective because you want to prevent duplication of the information that you have so you're gonna keep your information where it is but you're gonna want to have access to it so if you load this scene and we'll get back to it when you tap on one of the containers nothing will happen but something should happen and the reasoning for that is we made a real simple mistake in our code so we're not gonna review exactly what this code is doing but we are gonna demonstrate that this is the built-in JavaScript editor that allows me as a developer to work with in Cimmerian and so if I take my line here and I uncomment it and I resave it so this will effectively update my scene and then I republish the scene so once this finishes saving then I will republish and then I will ask everybody to redownload or not redownload reload their browser and so we're gonna republish my scene and this is effectively how you share 3d web applications with everybody there are options for private sharing but we're gonna publicly share this to everybody and the actual deployment we take care of for you we do the undifferentiated heavy lifting of the actual infrastructure so now this finished so go ahead and reload your web browser and raise your hand if you see something different happen and I'll load it up on here as well and so you're looking at one of the containers I'm gonna let this thing load a bit and in case you're curious two fingers on a cellphone will pan one finger will rotate so now when I click the container I see that this container has a set of toasters in it and I'm gonna put that back and let's move over to this container and this container has a set of oranges in it so the concept here is you're gonna have business data and that data has to reside somewhere you're not going to want to have this data live within your 3d web application you're going to want to be able to integrate your 3d web application and make requests to where that information lives once that request is made then you're going to want to be able to represent that information in your 3d web application so we've talked over some of the quill quick benefits of working with Sumerian the hosts the monitoring the integration the support for WebGL and we hope that this gives you some information on on what a good developer experiences at a minimal cost but we need the shift to what a good user experiences a user is an incredibly challenging piece of the equation to fully understand developers are easy you throw us a t-shirt we're happy users not so much so one of the best ways to really study users and this has been done for 2d web applications is to kind of get out of that ecosystem get out of the bubble that we sometimes find ourselves in and so when research was done around what makes a good graphical interface the first one of the first test subjects believe it or not were children and the way kids interacted and work became lessons and learnings from that experience became effectively the things that we use today so what was discovered is a real big concept that people want interactive experiences they want to be engaged but being engaged and being interactive is not the same as gaming and you don't need a game engine to drive a fully interactive experience and so what I'd like to do next instead of talking about customer stories is actually have a customer come up and present her story and talk about the work she is doing for diabetic stimulation training and the exciting stuff that Amazon Cimmerian has been able to help her achieve so I leaned from a1 control please come up thank you [Applause] thank you well I'm really excited to be here today to tell you about a one control we are a diabetes education software company and we provide personalized diabetes technology education and we partner with diabetes tech companies to improve outcomes associated with their technologies while decreasing educational costs let me start off by just telling you a little bit about myself I am an endocrinology diabetes nurse practitioner I'm a certified diabetes educator and I'm a PhD candidate at V Ohio State University it's a bit pretentious I'm sorry and I've spent the last decade helping people use technologies to improve their outcomes and really for a number of years I operated under really a pretty simple sort of formula or philosophy you might say it was take somebody with diabetes like Kevin here and add a diabetes technology like an insulin pump or continuous glucose monitoring system called a CGM and like magic I would get instant diabetes outcomes success so really the only problem is I could see it is that we just simply didn't have enough patients on diabetes technologies well recently all of my diabetes dreams have come true we've seen this huge increase in the number of patients using diabetes technologies over the last decade the number of patients using insulin pump therapy has nearly doubled and over the last three years the number of people using CGM has risen 10 fold that is astonishing and it's absolutely the secret to improving diabetes outcomes except it wasn't those same large population-based studies that show this surge in the utilization of diabetes technologies show absolutely no improvements in outcomes I mean how is that even possible well you see the truth is diabetes technologies don't improve outcomes people improve their outcomes and technologies are just this really useful tool that can help them get there and when we look specifically at the outcomes associated with these diabetes technologies we can see that they're actually related to the way the person uses the technology and to their underlying understanding of their diabetes and while diabetes education can absolutely improve outcomes today's diabetes technology education is outdated it hasn't changed in over three decades it still involves a lot of plastic food it's inaccessible to the most people in the US with over 50% of people with diabetes in the u.s. saying they've never even had a single session of diabetes self-management education and it also happens to be really expensive so this problem is receiving increased attention recently for a couple key reasons and the first of which we've already touched on it's that we now have really strong evidence proving what I'd say we've already known in clinical practice for several years the diabetes technologies in isolation they don't improve outcomes the good news is this is finally shifting that focus to include more patient centered initiatives the second reason is that diabetes technologies are now dramatically and rapidly changing up until this point you could essentially say that all diabetes technologies were the same species they might look a little bit different they may sound different they might be different sizes but really patients use them instantly the same way they incorporated them into their lives in the same way going forward that is no longer the case in the future all diabetes technologies will essentially be drastically different species with the advent of closed loop systems and these artificial pancreas systems these technologies will operate on very different algorithms and they'll require distinct patient interactions for successful use and while this may be kind of exciting to some of you here in this room it absolutely terrifies me because it means our current one-size-fits-all approach to diabetes education is completely ill-equipped to deal with this changing industry so with this sort of problem at hand I began to ask several questions it was could we potentially improve the way that we're providing self-management and diabetes technology education and could those improvements chillie improve outcomes associated with these technologies could we develop a technology that was agile and left to deal with this changing industry so what I did first was I began working with a small group of individuals from diverse backgrounds and we started off by just talking to people we did over a hundred and 50 interviews we talked to patients who told us that they hadn't received education many of them in years some of them over 20 years some of these patients told us about ER visits her hospitalizations that they had that they were convinced could have been prevented if they had remembered their training we talked to providers who said that even when they knew their patient would benefit from diabetes education that education was often inaccessible to that patient or was hard for the patient to find the time in their busy schedule to make it a priority we talked to people in industry who said diabetes technology education was a huge expenditure for their company knowing a little bit more about the problem we then began to look for solutions and we landed on educational theory and research and what we were specifically interested in is how do people learn really complex tasks because diabetes self-management is incredibly complex and what we discovered is people learn best when they can experience something when they can see how a specific concept or task how it actually fits in their everyday life the problem is it's really hard to teach people this way in real world environments but it has been successfully deployed and researched in simulated environments or virtual environments with this potential viable solution we next partnered with our local college diabetes Network at Ohio State this is a group of really motivated college kids with type 1 diabetes and we have worked with this group from initial concept all the way through our current development in this group of students they were motivated to work with us because they wanted to ensure that the technology we developed together was solving the problems they were identifying in their everyday lives we worked with OSU engineering students and we crafted two prototypes and did usability testing all of these experiences all of these conversations they sort of crystallized into a good idea what it was that we needed we understood that we needed to create a really high-quality simulation experience and that that our technology had to do three things really well we had to be able to train patients to fully utilize that diabetes technology but at the same time we had to provide baseline essential diabetes self-management education and we had to also be able to assess patients we had to be able to take that data and find out how we could connect it to their real-world experience we could also then use that data to package it and give it back to the patient's provider and diabetes educator so that they could follow along with their patients progress we we understood that creating a technology with this sort of sophistication and creating a high quality experience then we were always going to have to partner with an experienced talented game designer we always knew this what we were less prepared for I would say is that in addition to the game designer we were also going to need to work with a game artist and not just a game artist we were also going to need a programmer and in addition to the really experienced and talented game developer and the game artist in the programmer we were also going to need a game animator cuz apparently I learned an animator and an artist do two different things I did not know this and to round it all out we were going to need a sound guy we needed a team and we knew orchestrating and managing this sort of project was going to be incredibly difficult for us we also lacked the connections in this particular industry to really assemble this kind of talent and as if those issues weren't daunting enough we didn't have the capital to orchestrate this kind of build this was incredibly defeating because we understood this problem we had this timely patient problem we'd co.design we'd crafted these solutions and yet we were stuck we're frozen so it was about this time when I'm sure we were still wallowing in self-pity and misery that one of our team members had the opportunity to go to reinvent 2017 and she began to blow up our phones she said you are not gonna believe this she said AWS is launching this new product called Sumerian you've got to check it out it's an absolute game changer for you we couldn't believe it as you can see here was this technology that felt handcrafted for us sumerian offered this high-quality artistry and animations built into the platform it had these avatars called hosts that had built-in sound capability and what we soon discovered is that we could connect Sumerian to other products that we needed for things like security and analytics Sumerian broke down all of these barriers standing in our way and just allowed us to begin developing our product we were incredibly fortunate to be able to work with an AWS partner mobic Witte for our initial build and their really talented engineer Randy Sanborn and before will comes up and tells you a little bit about the architecture I'm just going to show you a little taste of what it is that we've created so our platform provides diabetes self-management and Technology education in a virtual environment this virtual environment and it allows the patient to practice diabetes self-management using their specific brand of technology that we've built directly into the platform you're gonna see in a minute here that we've tried to personalize everything in the platform from the doses of insulin that a user sees to the pace of Education let's get started really quick by showing you some of the early content that a user might see if they get a diabetes technology for the very first time and their programming their technology hi there you are doing great now let's set your sensitivity this represents the number of points that one unit of insulin is expected to lower your glucose for instance if your sensitivity is 25 it means one unit of insulin should lower your glucose 25 points some people have different sensitivity settings based on the time of day allowing them to receive more or less insulin to correct high glucoses throughout the day click the pump button to access utilities in your insulin pump so we've built fully functioning technologies directly into the platform and you'll see an insulin pump here and the users encouraged to put their actual insulin pump settings into our pump and the reason is these settings really dictate exactly how many units of insulin or the dose of insulin that a patient's going to receive either for carbohydrates or to bring a high blood glucose down it was really important to the people that we were code designing this technology with that the insulin dose is a user sees mimic their real-life experience next let's take a look at some of the simulated content that a user might work through hey I was planning on working out in a little bit I can't wait to get started would you like to come hi there sure that is great you are trying to work out exercise is great for your heart and helps you use insulin more effectively great let's get going so the user then has multiple choices and they platform like they do in all the simulations and if they check the glucose they can see it's 106 and for those of you that don't know 106 is a really really good blood glucose so it might be easy to say you know what let's just have Malcolm go ahead and exercise without doing anything else but the thing is physical activity actually speeds up your glucose metabolism and it also depends on the level of physical activity and and we've built all of this directly into the platform so let's see what happens here if we have Malcolm just do a moderate level of physical activity without any other intervention okay let's get going I feel more in shape and better looking already okay that is ridiculous we haven't even left the house yet well let's get going these muscles aren't going to get bigger on their own so time passes how are you feeling now not great I don't think I can workout right now I'm shaky and I just don't feel good so Malcolm has experienced symptoms of hypoglycemia which is a low blood glucose and if we check the glucose in the pump we can see it is indeed below that 70 threshold for hypoglycemia so the user then is again faced with multiple different options and the correct one would be to give Malcolm something to eat and specifically we would want to choose a fast-acting carbohydrate something like a jelly beans for instance maybe eating something will help bring my glucose up great idea Malcolm so you can see we've built in to pull different algorithms into the platform algorithms for insulin for glucose appearance glucose metabolism carbohydrates different levels of physical activity and so the whole reason is that so every user action within the platform produces a physiologic meaningful reaction it's something that they can learn from next let's take a look at some of the emergency content that a user might work through by their are you okay you look nervous I feel anxious but I'm not sure why I'm sweaty and shaky - that doesn't sound good what do you think should be done so Malcolm is again having symptoms of hypoglycemia low blood glucose but this time the user doesn't do the next best thing which would be to check the blood glucose instead they decide to have Malcolm take a little nap because he's tired and just a little hint taking a nap is always the wrong choice in pretty much every scenario maybe if I lay down for a few minutes I will feel better will you wake me up in a while sure but I don't think that is the best idea right now hey Malcolm are you okay Malcolm wake up seriously man wake up okay I need some help here oh my I am going to give you a glucagon shot and call 9-1-1 wow that was really scary and it could have ended very badly so Malcolm experienced an emergency event a severe hypoglycemic event but all of the content in the platform even the emergency platform it's it's all based on really tough situations and mistakes that patients commonly make and that I see in clinical practice many of the people that we Co designed the platform with shared many of these experiences themselves these mistakes the thing is and the beauty about using a virtual environment to learn is that we can make those mistakes and one of the most powerful ways we learn is by making mistakes the problem is real-world environments the stakes are too high we've built a scoring system into the platforms so the scoring system really encourages users to replay the content so that they can achieve optimal performance and the scoring is based on several different things it's it's based on trying to achieve a glucose in target and then keeping that glucose within target range it's based on the choices that the user makes them a sequence of those choices and then it's also based on optimizing that technology at the end of each scenario the user received personalized feedback on their performance let's take a look at that Wow look at you you should be so proud of yourself you decided to start exercising which is so important for your health you also made some really smart so Christine then goes through the choices the user made the rationale for those choices the physiologic implications and the feedback is really personalized based on of course the outcome in the scenario but also the glucose levels the choices that were made she then goes on to really stress the importance of expanding the use of the pump to fully utilize that diabetes technology in the before you had a pump you were very limited in your options to prevent hypoglycemia during a workout yeah I usually just heat extra carbohydrates before and during my workout it was kind of frustrating when I was trying to avoid gaining weight yeah I agree well now that you have a pump you can also reduce your basal insulin using your temp basal feature this will allow you to prevent hypoglycemia while eating less carbohydrates man that is fantastic what do I reduce it to so Christine then goes on to explain exactly how to set this advanced feature called a temp basal how what units or percentage you might want to set it to and then application of that based on certain different levels of physical activity then the user is encouraged to go back into the scene and replay the scene but this time they're actually going to use that advanced feature set it for whatever percentage they want and then choose the level of physical activity of the exercise they plan on doing this allows them to then see the glucoses play out in response to using that advanced setting this is really pretty advanced diabetes tech training and is training that patients currently aren't getting so it really shouldn't be any surprise to us that we see in research and clinical practice that patients aren't currently fully utilizing their diabetes technologies too quick aspects from our database that I want to show you so the first of which is that we can track the user performance down to their specific actions within each scene and this is really important so that we can gather that data and use that to personalize the experience and it also allows us the opportunity to package that and give it back to the provider the second element is really just this really awesome nugget of ingenuity I would say so it allows me to make changes to existing content or create entirely new scenes entirely new content all from a Google sheet and then within the database I can instantly push it to the platform where it goes live and we'll kind of showed you a little something similar to this but the difference is I don't have to hard code anything to do it and this is really cool for a startup like ours because we're constantly Co designing and creating new content or modifying existing content sometimes based on our usability testing and this allows me to do that without an army of engineers behind me and so while I have to admit this is actually the brainchild of Randy Sanborn I actually believe that I deserve sole credit for it because I think it was the opportunity that Randy had to work with me and meet me there really inspired him to create something so so very simple to use before I hand it back over to will I'm going to just say really quickly that it's been an awesome journey thus far in just over a year we've raised nearly two hundred thousand dollars in grant funding we're currently conducting NIH sponsored research with usability testing and outcomes among rural Ohioans and West Virginians these are people that would otherwise not have access to high quality education we are being incorporated into OSU's diabetes Education Center and this gives us a large initial pool of users we've received interest from several diabetes technology companies and leading endorsements we're really excited about the future and we're really excited about this technology and what it means for patients for their families these people that have been desperately seeking solutions and with that I will turn it back over to all [Applause] Thank You Eileen it's it's absolutely amazing to think that today you know anybody with the drive the vision the passion is able to create real life-saving technologies and applications so I just wanted to go a little bit into what the architecture of this looks like so I only mentioned earlier that there essentially were two users of this application there's the administrator and then there's the actual patient or the actual end user in this point and so taking a peek at the administration interface you know when the Cimmerian scene was created it was published so that it uses a kognito token and Cognito inheritance a role that allows Cimmerian to make those back-end requests so we saw a little bit earlier of a very simple example that used a dynamo DB lookup the application that a one control created is a far more sophisticated application that makes multiple requests and the requests that are made are not necessarily they're both right and read requests effectively we're gonna write patient data and we're gonna read the dynamic changes that the administrator make or the administrator made and so when the Cimmerian scene was created the actual language the dialogue wasn't hard-coded into the actual 3d application the ability to dynamically do a data dip and look up that information I'm back in AWS services enabled that capability because it was published using cloud front anybody anywhere with a simple URL was able to access the application and log in with their information through whichever Cognito Fedder or identity pool was created during initial setup from a patient experience same sort of workflow we mentioned patient logs incognito identity token inherit it's a role same sort of deployment framework logs in with a URL Cimmerian at this point takes advantage of its built-in poly components to do Texas speech based upon what data dip returned into the Sumerian scene so Sumerian real data dip pull the information and then it gets echoed back using a built in poly component patience stateful information we mentioned earlier that multi-user synchronization component gets written to a DynamoDB table and then analytics are done leveraging using pin point so being able to effectively notify a user using traditional pinpoint tooling SMS and whatnot is is something that could help enable an additional sort of integrated experience and so with with that we need to go back to kind of ask ourselves do we better understand the actual adoption formula that we started with and I would argue we do because it has become clear that a good developer experience that works the way today's developers work plus a great user experience that can be achieved that's studying the way users actually interface and want to work at a reasonable cost that works under an iterative model without too much deviation but deviation possible achieves a high level of adoption and so looking at our application framework of what makes an AWS 3d application I am going to repeat the important things that make an AWS 3d application possible your ability to create integrate and deploy your applications are contextually aware they work with natural language they're globally available they involve multiple users in the application and you can elevate that application you've created to be a spatially aware 3d application may be some VR AR application in the future maybe at some point you'll say you know I would like my users to put on a VR headset and have an experience but you're not going to lose the actual capital that you've invested creating that initial application and so you say great how do I get started doing something like this because this is so cool I didn't realize that as a simple either web developer or somebody who's just generally curious what can I do to get started the answer is get your hands dirty get your hands messy because that's the way most people are gonna best learn and so we have a set of resources that I would encourage everybody to use and you know contact us if there's something that you would like to see that isn't on there specifically the tutorials are certainly going to help folks who have existing 3d engine knowledge and want to bring that knowledge onto AWS and Amazon samarium the articles will actually the demonstration that we just showcased with doing a dynamodb lookup there's an article written specifically for that so the articles will work with traditional AWS practitioners who want to be able to integrate how they've worked either with machine learning models or data dips or other database lookups into a 3d world and then our documents also expand upon that now there is one parting gift for everybody in this room so if you look underneath your seats you'll see nothing I'm not Oprah but you do have a QR code for a free book so I'm gonna leave this up on screen I would encourage everybody to scan this code and get your free book I'm sorry I'm not Oprah she's way better but this is our gift to you so this was put together by one of our partners that goes very deep into working and actually building 3d applications VR and AR applications and that's our gift to you for for staying and and learning about how to work specifically with AWS to build your application and I can put the slide back up at the end as well and so with that it wouldn't be a reinvent presentation if I didn't ask everybody to do your surveys so please do your surveys let us know what work let us know what needs improvement and I do want to thank you for the privilege of your time for joining us today at reinvent 2019 thank you everybody [Applause]

Channel: AWS Events

Views: 442

Rating: 5 out of 5

Keywords: re:Invent 2019, Amazon, AWS re:Invent, ARV201-R1, AR/VR, A1Control, Amazon DynamoDB, Not Applicable, Amazon Sumerian, Amazon Lex

Id: SCW5EGM2OmU

Length: 55min 46sec (3346 seconds)

Published: Fri Dec 06 2019