609 sd building ar experiences with reality composer

you good morning great to see you hope you're all having an awesome dubbed FDC we're super excited to be here today to chat with you about building a our experiences with reality composer I'm Michele and with me are my colleagues Pao and avi and today we're gonna cover what reality composer is and how it can help you create an air app after this session you'll know what's provided with reality composer how to set up your experiences in it and then how to take and take what you've created in composer into your app in Xcode so first what is reality composer as you've heard earlier this week rally composer is a tool an app to help iOS developers get started in working with 3d and they are some of you have already begun and it's been really really amazing to see what you've all been making posting and tagging with reality composer so please keep doing it if you're not familiar with 3d the first time you ever open a 3d app can be pretty daunting and it can be kind of time-consuming to even get started in AR but with reality composer it's really easy to just previz and lay out your experiences using the built-in content library or if you prefer you can use your own us disease and if you're not familiar us VZ is just a 3d file format that's supported in reality composer once you've got the experience laid out you can add simple interactions and animations with the behavior system before you even commit to writing any code so it really speeds up the iteration time on the experience itself and then finally everything that you've made in reality composer you can take directly into your app the next code and so much about building for augmented reality is in seeing how the experience feels in physical space looking at it on the in and advice and seeing how your audience might see that experience so rally composers available on Mac OS iPhone and iPad and you can edit on iOS so that you can develop on your regular dev machine and then edit on iOS in order to continue working on the exact same project on iPad or iPhone alternately you can also just start the project on your iOS devices and take it the other way so today I'm going to take you through building scenes Powell will cover adding behaviors and using physics and then obviously an app so for building scenes first what's a scene so as you may have heard in the introduction to reality kits and reality Composer talk a reality composer scene is essentially a collection of entities and it contains an anchor objects behaviors and a physics world every scene has exactly one anchor and when you start a project or when you start a scene any scene you'll be asked what kind of scene what kind of experience are you building if you're building something for a room a stage or a tabletop you'll choose the horizontal anchor if you want it to be on a wall pick vertical if you're augmenting a book or some packaging a brochure or a movie poster choose the image anchor and for selfie effects like glasses or hats choose the face anchor projects can have multiple scenes but I think you'll find when you're first starting out one or two scenes is usually sufficient okay so in this case let's assume that you've picked the horizontal anger and so here we are the content is our focus in reality composer so it's laid out exactly as you would see it in a are in the main viewport or the workspace and this pane on the right here shows you the properties of whatever you selected and shows you options for configuring that entity be it the object itself like this text that's selected here you can see that it's selected because of the ring you can just select an object or a scene I could show you more slides but why don't we check out composer together on iPad oh I want you to remember this scene though because we're gonna be building up this little tutorial over the course of the session together it's a Seasons and solar system lesson will build up any are okay so for our demo I'm going to show you three quite different RC projects today the first is a little primer and it's gonna show you kind of just five really basic examples of things you could create and reality Composer probably just in a couple of minutes the second is a much more richer and more complex experience a full experience of what you could create just in reality composer alone if you didn't want to write code at all so this is great if you're not a developer or you're just kind of starting out and you want to just try out the experience before you fully commit to it when you make your app and finally we're gonna build that Seasons scene that we saw together okay so here is the primer in reality composer and I'm gonna take us into AR because that was how we intended to see it okay so these are the five examples I mentioned and let's press play so that we kind of see the interactions so when we started we're just so hi everyone when we just started this spiral started spinning and here if I get close in the second example I can make this chest piece jiggle I can tap this one for physics and audio it's kind of a fun sound and then someone had asked if I'm an animator and I have my own models I have my own animation can I bring them into reality composer and yes you can and you can trigger them on user interactions like a tap and if you're not an animator you can use pre-built emphasis animations on any object in the scene here we've applied them to some pretty basic shapes but you can apply them you know to this dancer as well so that's the siene are next I'm going to show you this more complex experience richer experience oh and you're probably wondering how I'm moving the content around the scene moving the camera so for panning all I'm doing is taking two fingers moving across like this so I can see the content exactly as I want if you were watching State of the Union on Tuesday this island might look familiar to you and we wanted to build up the experience a bit more so if you imagine you're making like a museum installation or a tourism Center experience that's something that we kind of envisioned here so let's look at this here in AR and I'm gonna press play to start so you should hear some waves and I were being prompted to tap on the helicopters let's do that you can see we were also asked to tap the location to learn more so I'm going to do that tap on these markers now and you can see I'm getting more information about each location as I tap the other one speed off [Applause] and so I'm just gonna take apart what we did to build up this scene so there's actually not much custom content in in this scene itself there's only two artists created custom Geocities is the island and the helicopter you can see here we imported them you can import more if you want the rest are just built in from the content library that comes with reality and poses you can see we have a lot of stuff here text basic shapes and that's kind of what you're seeing in this scene these markers are just basic shapes we've grouped them to create my two pieces as one marker we have a text here the other question we get asked a lot is I work mostly in 2d and so I have a lot of photos and you know company graphics can I use them in reality Composer and the answer is yes of course you can you can bring them into the experience as photos using the picture frame which isn't in the content library that we just saw finally you're probably wondering how did we create those little interactions so if we go to behaviors by clicking on this behaviors button in the in the top right you can see the behaviors we use to create the flow of that sequence and those interactions so we're we're billboarding things to look as you might have noticed as I moved around the scene those items looked at me so we're using the look at actions here to make everything look at the camera at the beginning you saw just the island and everything else was hidden so you can see that we hid everything else and then we showed them to you in sequence so we waited for 0.2 seconds and then first we showed the helicopter text then we showed the helicopter itself and it's tapped to start and then we faded the markers and from below so that's what this move from below is doing and I can preview that to you now just by pressing play on the card I'm going to show you also a little secret these birds you might have noticed I'm just gonna look at them here so we asked them to orbit orbit the island as a group and they're not really Birds at all they're just primitives that we're using or just basic shapes that we've put to move cards on to flap their wings so instead of investing time in creating Birds all we did was we took a couple shapes and put two action cards on them to make an animation and if you tried to animate something in 3d before you know how time-consuming so simple thing like this can be and this takes just a couple of seconds so that that's an example of a full experience that you can make in reality composer without any code at all and the really great part about this is that once you have this reality file you can just share it with anyone and preview it in a are quick look and the experience will be exactly as you've done it in composer so that's that second project so finally let's build up the experience of a seasons less them together so I'm going to add three celestial bodies the Sun the moon and the earth so for this I'll just block in the scene with some placeholder objects to start us off so here's the Sun I brought it in and now I'm going to duplicate it to create the earth and I'm just using pinch here to zoom us out a little bit to move the camera so let's grab this arrow to move the earth over and then next I need also a moon so let's duplicate this guy one more time bring it over to about here and then I can kind of grab anywhere in this ring to move it up in this plane so let's move it over okay great and now I want to scale the moon down a little bit so I'm just gonna grab the ring and move inwards to scale it there we go move it a little closer okay so I have these three celestial bodies let's scale the Sun up so I'll grab the ring again just drag it to scale scale the Sun so we've got our three three items there next I'm gonna I'm gonna add some text so I go to the content library again let's grab the text and it's a little bit below where I want it to be so I'm just gonna grab this green arrow and drag it up and you can see it pop up to the ground plane and it's doing that because I have snapping turned on so if you look in the left corner we have this little magnet and I can turn snapping off if I want to move stuff freely but actually I just want to pull this blue arrow to bring it forward and I want it to be about mmm half a meter or so from the Sun and I'm gonna open the pain that we saw on the right to change up the text so let's just make it say autumn that's super easy to just add these elements in composer so finally I think that's pretty much where I wanted to be so all I'm doing to move the camera around is using my my finger and touching the screen next I'm gonna add an arrow and it's not pointing the right direction I want it to be kind of rotated so I'm gonna move the camera so I can see the the green ring which is sitting on the ground plane this ring rotates with the camera as I move so I can always be working in the plane but I want to be working in if I don't want to move the camera to have to do that I can just tap on the arrow and I'll lock into the corresponding or you can see the ring got a little fatter there so I'm just gonna rotate the arrow and it's snapping to 15 degree increments again because I have that snapping turned on but if I didn't want to have it snap I can move it freely so here I'm dragging it and it's moving always and always in the horizontal plane for this green ring here but I actually do want it to snap between the Sun spear and the autumn text so I'm gonna turn snapping on you can see it's snapping into place so it's just super useful super easy then I want to modify this arrow a little bit so I can go ahead and click the icon next to modify our next to snapping to modify it I'm just gonna zoom I send a little so we can see better and I can grab this handle drag it I'm also just gonna make this just a touch smaller a little flatter okay and then I'm gonna move it just a little bit more okay so now everything is kind of blocked out and we do that in just a couple minutes I actually want to replace these celestial bodies because they're all pretty indistinct right now I want to import some custom us disease and since I already have them placed exactly where I want I'm gonna do a replace so I just tap on the selected object I get this edit menu hit replace I'm gonna import the Sun and it'll just replace it and if you can see it's the same scale and it's exactly in the right position that I wanted I can do the same with the earth so let's do that really quick there it is and the moon I'm just going to customize it a little bit here with some let's see I want the matte material and I can make the moon like any color that I want but I think I'm just gonna pick grey for the sake of safety and if I want to see any of these items a little better I can just double tap on them to frame the object it's a really easy way to get around the scene or if I want to see the scene pull back I can double tap on anywhere in the grid to see the scene those shortcuts are here to frame scene or frame frame selected okay so we're just about ready I think to add behaviours I could add all of the other elements but for the sake of time we won't do all of that now there's a few tips I want to leave you with before we finish this as part of the demo that replace that you saw if we had behaviors applied to this already like we were doing any animations on like let's say the Sun if I replaced it those behaviors would stay so it's a huge time-saver if you're just like swapping in new versions of content it's really powerful and then also we mentioned having multiple scenes you can access them here with the scene selector in the top left and so if you wanted to add any more scenes you just add them here if you are working with these entities in Xcode you're gonna want to name them and you'll see more about this later on in the session and you can do that when you open the pane there's a configure panel and you can just name the entity here and I name it Sun finally if there's something you didn't want to do like let's say I move the earth and I didn't want to do that like undo that action I can also long press and redo and it'll tell me like what the undo is that I what the under is that I am about to to Teen Vogue okay finally some folks have asked what's the easiest way for me to share my RC projects and you can airdrop them between your devices or like we talked about earlier you can use the edit on iOS to go easily between your development machine and your iOS devices okay so that's our scene now ready for pay views so you just saw how to build scenes and reality Composer so we took you through how to lay stuff out quickly using the content library we also showed you how to bring in your own custom use disease and now we're gonna move on to adding interactions and behaviors so to take you through all of that let's welcome my colleague Tao Thank You Michelle so now that we know how to bring content to the scene it's time to bring all this content to life so you may want to play some music when the user taps on an object or maybe show some text when the user gets close to an object or maybe you just want to play some USD animation so in order to do that and more we need to take advantage of behaviors in reality composer so let's get started as we seen before a scene in reality composer contains an anchor objects behaviors and the physics world behaviors are always seen scoped that means that you can only target objects within the same scene so let's take a closer look at what a behavior is a behavior is composed by two items the trigger and the action sequence the trigger as the name suggests is simply the criteria for the action sequence to begin so let's say we have a scene with some model of a record and when the user taps on it we want to play some music and spin at the same time well in order to do that you just need to create a new behavior use a tap trigger and then create an action sequence comprised of a music action and a spin action so the first thing when creating a new behavior is to pick what trigger will what trigger will begin the action sequence renta composer comes with five different triggers that you can use with the start trigger the action sequence will begin as soon as the scene is loaded with the tap trigger the action sequence will begin once the user taps on an object that you define with the proximity trigger you define a threshold distance from the object to the camera once the camera is close enough to the object the action sequence will begin with a collision trigger you define two objects and once those two objects collide the action sequence will begin and the last trigger is the notification trigger this trigger is what allows you to begin action sequences programmatically from your app and we'll take a look at this trigger in the last section of this talk so now let's move into action sequences there are three main ways that you can modify the execution of action sequences those are groups loops and exclusive action sequences so let's start with groups by default in reality composer groups play sequentially so the actions go one after the other but you can combine multiple actions if you want to play them in parallel so in this example action two and three will play together right after action one finishes an action for will only begin once all the action in the group are over action sequences can also be looped so let's say you want to have some music in your scene and you want that music to loop forever well you just need to create a behavior with a music action in the action sequence and Mark that action sequence to be looped and the last part is the exclusive action sequences related composer you can mark an actions was to be exclusive only one exclusive action sequences will play at any one time that means once enacted exclusive action sequence start or any other running exclusive action sequences will stop and action sequences that are non-exclusive they will play like together with other exclusive and non exclusive action sequences and that's pretty much all you need to know about action sequences so now let's take a look at some of the actions available to you in reality composer with the visibility actions you can bring objects in and out of your scene using animations so we can support multiple types of animations and you can use the basic things like fade in fade out but you can also use more playful animations like a scale up and then in scale down reality composer also called supports animating objects in place with the emphasis similar to the show/hide you can animate the object with different motion types and animation styles so you can use a basic pop but you can use also use a more playful pop animation you can also spin or orbit your object around other objects and if you're using a usda asset in your file you can use their transform and a skeletal animation now with the move actions you can translate scale and rotate an object in the same with the move by the movement will be relative to where the object is right now like in this example while the object always goes to two units forward and one left and with the move tool you can always move an object to an ax specify location in the same like here where the object is going back to the origin of the same with the look at you can make an object always face the camera like in this example where the horse will always face the users device another we have all the animation actions it's time to to bring some sound to your experience with reality composer there's three actions that you can use to bring some audio to your scene with play sound the audio will originate from the object itself like here where the horse is doing a horse sound with play ambient the sound will be scenery on them and with please play music the audio will play without any transformations so now let's take a quick demo and let's take a quick look at how to bring some behaviors to the scene Michelle was building before here we have a more complete scene to compare to what Michelle was building so we have the moon the earth the Sun and we have some text and some sun rays showing the different seasons so let's bring up the behaviors panel by clicking on the behaviors button on the top right corner and as you see here there's already some behaviors so let's hit play and see what is going on in this scene so you see here there's no movement for the moon the earth or the Sun but the different rays and seasons texts are appearing in a sequential order so let's go ahead and create a new behavior four-twenty made the different moon earth and sun objects so for that let's start by bringing a new behavior we hit plus and then here we can see all the available presets that we can pick from but for this demo we're going to start with a custom trigger now the first thing is to pick what trigger we want to use to begin the action sequence and because we want to start animating these objects when the scene is loaded we can bring up the library and choose the scene start right now it's time to start with the action sequence for this action sequence sequence we want the earth to orbit around the Sun then the moon to orbit around the Earth and at the same time we want to spin the earth and the Sun so let's start with the orbits so we can bring the action library and then select orbit at the first action first thing when creating a new action is to define what targets are the target of the action so in this case the affected object is going to be the earth and the center is going to be the Sun great we have other properties like duration revolutions aligned to pass an orbit direction and for the direction we want to use counterclockwise so now we can easily preview this action by just hitting play on the play button in the action card but that was a bit fast so let's change the duration to something more suited for this for this scene let's start with 20 seconds you can change the duration and try again and yeah I think that duration looks right now let's bring the second orbit so we can bring again the library and select orbit now the affected object is going to be the moon and the center will be the earth the duration we want it to be the same as before and for the revolutions because we are not trying to be physically accurate which is one to have something that looks nice in the scene we are going to use for I think four is a good number now because the moon is tidally locked with the earth we want the moon to be aligned to the orbit so we have to select a line to pass and now the direction will be also counterclockwise so let's see if that a try yeah that animation looks right so now that we have the orbits out of our way let's begin with the spin actions so for that we bring another spin action and let's start with the Sun same duration as before so 20 seconds and four iterations I think four will be a good number and same direction so counterclockwise if we hit play we see that that's a good speed for the spin right so now let's bring the last action in the sequence which is the spin for the earth so we create another spin we select the earth same duration and because we want to have a higher number of iterations compared to the Sun let's use 10 for this example and direction again counterclockwise see that's a try and yeah that looks right so now our action sequence is completely laid out so let's try to play the whole action sequence and we can do that by just pressing play on the play button in the action sequence header and what we see here is that only the earth is moving and that's because the action sequence is playing all the actions sequentially one after the other what we want to do here though is to make all these actions run in parallel and for that we can use drag and drop and drag the different cars on top of each other and just by doing that we create a group with the four actions and now let's just go ahead and track thanks so now let's go ahead and hit play and see how all the behaviors play together and what we can see here is a now all the orbit and spin actions play together at the same time as the text and the rays appear in the scene when the earth is passing by right let's go back to the keynote so that's how easy is to bring some interactions and behaviors to your sins and reality composer now let's move on to physics and so in order to create amazing and immersive VR experiences you need your objects to fit right into the real world you need your objects to move like real objects and for that we need to take advantage of the powerful physics engine available to you in reality composer so let's take a look at it this is an example of a scene that you can build in reality composer using physics as you see here the sphere is falling because of gravity then it's colliding with the pins and right at the end it wants it bounces with the green box is we is like winner text so let's see how we can build this kind of a since there's three different ways that you can influence the physics simulation in your sin a variety of materials will define how your objects interact with the sin and with other objects with the force action and the gravity you can determine how the forces will play into your world and with the collision trigger and the collision shapes you can define how the collision mechanics will work in your physics simulation so let's start with materials in Valente Composer there's six different materials that you can use you can you have to define the materials for the scene and for the objects so let's say you use material for your object use ice as the material of your object this will make your object slide through the same if you use rubber instead your object will be more much more bouncing now let's move on to forces all your scene will have gravity enabled by default and the value will be set to the earth gravity but you can change that to any value you want with the force action you can give your object an initial impulse you define a direction how much velocity you want to give to the object and you can build this kind of behaviour where you can add a force once the user taps on the object and now that your objects have materials and can be affected by forces aside it's time to make them collide with each other so by default in reality composure all the objects don't participate in the physics simulation if you want your objects to collide with other objects you have to set the set the collide option if you want your objects to be moved by the physics simulation you need to set the simulate option so here you can see how default collides and simulate options affect the objects in the scene so the coalition mechanics are determined by the shapes of the objects in the scene a reality composer there's three different shapes that you can use and those are box sphere and capsule those shapes will determine how your object moves in the scene or how collides with other objects and the anchor and with the collide trigger you can build this kind of behaviors where here we have a collide trigger with the sphere and the box and that way we can detect when the user is winning this game and show the winner text and that's pretty much all we have to say about physics and reality composer so now I would like to hand it over to my colleague Abbie to talk you through how to build apps using reality composer hi everyone my name is avi I'm super excited to be here today so earlier in this session we saw how with reality composer we can create amazing 3d experiences and then bring it to life with behaviors now let's see how we can take those a our experiences and integrate them into our app so to get started you provide three different ways the first is create a new reality kit AR or game template right in xcode then second is to create a new project from reality composer and the third is to export a reality file from reality composer and then include it inside your app to get started let's talk about the two central files the reality composer project and the reality file and we'll get started by talking about the reality composer project so this is the project file for reality composer it's included automatically inside of reality kit AR in-game templates it's pre viewable it right inside of Xcode we have Xcode integration and Xcode automatically exports your reality composer project as a reality file during the build step of Xcode so here we can see our reality composer project in Xcode 11 we're able to preview it and we're able to preview each of our scenes as well and in the upper right hand corner we can see an open and reality composer button and this will launch reality composer right from Xcode as I mentioned earlier if you have a reality composer project in your Xcode application during the build step we automatically generate a reality file for you and it's included inside of your application just like any other resource you can access it exactly the same way so what is a reality file this is a central object for how you will use your application and work with your AR experience it contains all the data required for rendering and simulation and it's optimized for reality ket like I said earlier it's exported from reality Composer or it's automatically exported by Xcode during the build step of your project you can use a reality file either by referencing it directly in your application or previewing it in air quick look and we encourage you guys to go check out the advances in air Quick Looks session tomorrow to learn more about that diving a little bit deeper what is a reality file so because this is a file exported from reality Composer it'll contain each of the scenes that you created in reality composer for example we have two scenes here and it also contains every object that you've created in the application when we think about a scene in a reality file we consider as seen in anchor an anchor entity specifically so this contains for example the horizontal anchor information that you set up in reality composer so we thought one of the biggest difficulties in working with with reality files was being able to access things with strings and we we both have the reality composer project and Xcode so we wanted to introduce a way to make this very easy to work with almost like just an object and so we decided to do that with code generation in Xcode what code generation will do is analyze the structure of your reality Composer project and generate an object a real object that you can use inside of your application so for example here we have our solar system reality composer project and we automatically generate a solar system object that you can use in your app so the code generated for you is an application-specific API for your scenes for your named entities that Michelle mentioned earlier for notify actions and for notification triggers and we'll get to the sect those last two items later in this session the first step to get started with cogeneration and working with your AR experience is to give names to everything you care about and one access in code right in reality composer and you can do that here in the settings pane under configure so for example here we've given the Nate the Sun our Sun USD Z that we imported the names Sun and we'll use this name to access it in code so how does our reality composer project get converted into an object that we can use at the top level our reality composer project becomes for example a solar system thought swift our scene becomes an entity that has anchoring and this is really important because it means when you load it you can add it right into your scenes anchors additionally any objects you set up get converted into entities these are real reality kit entities that you can use you can work with exactly the same way in addition we also provide two different ways to load your scene one that is synchronous and one that is asynchronous so taking a look at loading a reality file synchronously here we have our solar system as our example again and we'd like to load the seasons chapter and we can do that with a single line of code so we can say solar system got load seasons chapter and will get returned an anchor that we can then add into our view it's super simple next if we want to load our SAR seasons chapter asynchronously or any scene we can use for example the solar system that load seasons chapter async and we'll get a result object and we can handle the success case where we'll get the anchor or we can handle the federal case and display the error to the user in addition if you want to access the named entities in our scene we can do so as easily as calling for example seasons Sun Earth and Moon and this will give us access to the entity that we named in reality composer it's just like working with any object so putting it all together with cogeneration in Xcode and reality Composer we can load our a our experience in our application with just two lines of code the first is loading our chapter or loading our scene and the second is adding it right into our view it's super easy so that's code generated generation in Xcode we also provide a way to access your reality file for example in cases where you're downloading content off of a server and you want to display that in your application to do that we have reality kit API so to load a reality file synchronously you first need to get the URL just like any other resource in your application bundle and next using the entity dot load anchor method passing the URL of the resource and the name of the scene that you'd like to load and this will give you back an anchor that you can use optionally you can also use any load which will load the entity tree without the anchor information to load your reality file asynchronously there's also a way to do that so the first step is exactly the same just get the URL resource of the resource use entity load anchor async passing in the URL and the name of the scene that you'd like to load this will give you a load request and using the new combined framework we can receive a completion and then receive the value and the value contains the anchor that we'd like to load so you can put this right into your a our view scene accessing entities you can do this through anchor dot find entity and then pass in the name of the object that you'd like to load for example Sun Earth and Moon so that's how easy it is with cogeneration and Xcode with a reality kit to bring your AR experiences into your application so next let's talk about how our application can interact with our a our content we can create amazing experiences right inside of reality composer but how do we actually work with it so as Pao mentioned earlier we have a great built-in set of actions already for you but there might be actions that you want to define yourself so to do that we've created the notify action this is an action that you set up in real composure it's called alongside your other action sequences exactly in the same order for example if it's sequenced it'll be called in that sequence there's a settable closure in your application code so you can define the implementation of this action and just like accessing an entity through cogeneration the notify action is also accessible through your object so in reality composer we can set up a notify action here and we can give it an identifier this is the identifier we'll use encode to reference our notify action and provide a custom implementation in the structure of our generated object the actions lives alongside your energies inside of the scene object so here we have actions and a property actions inside of the actions class you'll see multiple notify actions that have the name of the identifier you've given your notify action so you can you can access these as simply as the loaded Ankur thought actions and then the name of the action that you care about in addition we also generate and all actions array so you can do great things with this just like any other collection and Swift you can filter over it you can iterate over it anything you can do with an array in Swift you can do with this all actions property to actually set the implementation of a notify action you first need to access it through your lead anchor so for example here seasons chapter actions dot display earth details and then provide the custom closure so dot on action and the entity that you'll get returned inside of this closure is the target entity you set up in reality composer on the other side to actually kick off action sequences and provide a way to do this as well we have a couple of great built-in triggers such as scene start and proximity but there might be times you want to provide custom triggers for example selecting a segment of control or tapping on a button and for that we're providing the notification trigger so notification trigger is set up in reality composer it starts an action sequence or multiple sequences that have the same identifier it's posted from the runtime of your application and it's accessible just like a notify action accessing your entities just by name in your code you can set up a notification trigger right inside of your behavior and here we have given it the identifier show gold star and we'll use this name in code just like actions it lives right alongside your entities in the generated code inside of a new notifications object a class and inside of notifications just like notify actions you'll have notification triggers corresponding to the names the identifiers that you've given to those notification triggers in reality composer and just like notify actions you have an array that you can filter over or iterate over as well that's called all notifications you can post a notification trigger by calling the post method and this will fire an action sequence or sequences with the same name and you can also pass in an optional override set this is really powerful for cases where you want to copy an entity and dynamically call an action sequence on only the new entity the overrides will take as the key the name of the original target of the action in reality composer and then you pass in the new target so when the action sequence runs it only targets the new entity so let's pull this all together and let's check it out in Xcode [Applause] so here I have my application over the course of the session Pao and Michelle have built up the seasons chapter of our solar system lesson and here I want to implement a second chapter based on the sizes of different celestial bodies so here we have our Xcode 11 artists or based application and we have already included a reality composer project again you can create a new AR or game template or you can drag the reality composer a project right into your write into Xcode we can preview the project right next code here and go ahead and open it in reality composer let's take a quick tour of this and the application before writing some code so in our chapter we've set up a couple of energies and we've given our scene a name and we've given different energies that we care about a name for example this gold star is called special star this is the name we'll use in our code additionally we've set up a couple of behaviors so for example for the moon we want to know when the moon is tapped so we can store some application state and also hide all the other text labels and only display the moon's labels and all of these actions happen at the same time because they're grouped additionally we have a notify action and we'll use this to know when the moon is tapped inside of our application we've implemented the same action sequence for the earth and the Sun as well we also have a look at camera action setup when the scene starts so each of our text labels will look at the camera no matter where we're looking and we have a hide on start action sequence which will hide all the entities that we want to view only at specific times for example the chapter completed text and the text labels additionally we have some actual sequences that will be triggered in code we'll get to this later in this demo that will scale the special bodies to their relative sizes scale them back to the same size display the chapter completed text begin the earth orbit and hide all of the other celestial bodies and we have a cool little animation fired from a notification trigger that will show the goal star spin it and give it a bounce so let's jump back to our Xcode application now and do a quick tour so this application is pretty straightforward it's a single view application and at the top level we have an AR view a segmented control to switch between size and detail view a lesson completed button so a student can just say I'm done with the lesson and view the the chapter at the end and a detailed view that will show more details as we tap on the moon the earth and the Sun additionally we have some view models that will just drive that detail view so it just shows some text and then some basic application logic to kick off our loading and let's look at loading first so down here the first thing you'll notice is we're using code generation in this project so our reality Composer project is automatically converted into an object that we can use it's a real type so we can use it right here for the size chapter property so for example a solar system lesson about size chapter it's a real type that we can use we can go ahead and load our scene and I'll let let's walk through this code as well so what we're doing is loading the size chapter asynchronously and then we're handling we're giving the loaded Ankur collision shapes so that top triggers work and then we're adding it into our scene because we'll be using the size chapter later we're storing it in the size chapter property and we're setting up notify actions which we'll get to later right now it's not hooked up so let's go ahead and run this we'll see with just a few lines of code we're bringing our entire AR experience and it should work exactly the same way it did in reality composer so here we go we have our scene we see the content for example chapter completed is hidden and when we tap on each of our celestial bodies we'll see text automatically appear and it will it will face the camera no matter where we're looking and same with the Sun so that looks pretty great with a few lines of code we've brought our entire reality Composer project into xcode it's super simple what i want to do next is actually fill in this detail view you'll notice when i tapped the sun for example the detail view didn't actually get filled in so let's jump back to Xcode and fill in the notify actions so because our notify actions are automatically generated for us we also get in all actions array that we can filter over and we're gonna filter over those actions to get only the actions we care about which begin with display so let's walk through this code because we set up notify actions in our size chapter we can jump back to reality composer to look at those for each the moon the earth and the Sun right here display moving details display earth details and display Sun details we can access these in code and we want to actually filter over our notify actions for only actions that begin with display because we want to set the same closure for each of them so jumping back to Xcode that's what we're doing here we're doing size chapter actions dot all actions and we're filtering over it getting the identifier from the notify action object and checking that it starts with display next we're iterating over all of the display actions and setting the same closure for each using the on action property and we're taking the target of our notify action in this case for the moon it's the moon for the Sun it's the Sun and for the earth it's the earth and using that entity to display different view models so let's run this application now and what we'd like to see is different view models appear when we tap on each of our celestial bodies there we go so we have our content and that will we tap on the moon because we've set up a notify action and we're observing those notifications we're now seeing the moon detail right at the bottom here we tap on the earth we see Earth's details and we tap on the Sun we're seeing the sun's details that looks pretty great we also were keeping some state so when we tapped on all three of our Earth Sun and Moon actions we're just making sure that three different actions were called and we display a complete lesson button up here in the right hand corner so what we want to do now is using notification triggers fire off the size relative size and size to same size action sequences that we set up in reality composer using the segmented control at the top and the button in the upper right hand corner so we can do this with a segmented control value change method this is an interface builder method that we set up and hooked up in interface builder itself and using our notification action triggers we can fire the scale to same size action sequence which we set up in reality composer down here which will scale each of our celestial bodies back to their original scale of 100% and when the size segmented control is selected we want to scale our Sun to a relative size to the earth to do that we can fire our scale to relative sizes action sequence and finally when the lesson completed button is tapped this is an interface builder action that we set up and hooked up will fire the chapter completed notification trigger and what this one does is because there are three different notification triggers hooked up in reality composer will fire all three of these action sequences will fade out the existing content will then begin an orbit of our little earth and little moon and will show the chapter completed section so let's run this now and see our application alright so here we have our content we can tap on the moon everything works exactly the same as we set up in reality composer and now when we tap the size segment to control if we're looking at the earth and the Sun we get a really good scale of just how big the Sun is compared to earth it's pretty tiny so that looks great it's sized it back and let's complete our lesson might tap the lesson completed the complete lesson button at the top right hand corner and we'll see our content fades out and we see our chapter completed along with an orbit starting for our tiny Sun that looks pretty great and we can see how with notif Notification triggers and notify actions we can work with our application and our a our content there's one last thing I want to do we discussed post with overrides and I want to show you how we can use that to display dynamic content using application state so because this is a lesson application we want to display gold stars based on how well a student does for this session I think we all did pretty great so I'm just going to give us three stars and we can see how we can set this up this is the method that we're calling inside of our display details view up above where we're keeping state so this looks like a lot of code that I just pasted it in but let's walk through it so the first thing we're doing is accessing our special star nd that we set up in reality composer right here you've given it the name special star so we're now accessing it right in Xcode next we're triggering the show gold star action sequence right in code so that will display the first gold star but we want to display 3 so we can do that by dispatching after maybe two and a half seconds for this demo and we're going to clone the star then we're gonna position it to the right and up a little bit we're gonna add it to our scene and then using post overrides we're passing in the original target of the action sequence and then passing in the new action the new target of the action in this case the right star and we want to do the same thing for the left star so we're copying it again copying the star positioning it to the left and down a little bit adding it to our scene and again posting with overrides taking the original name and passing in the left star this time as the target of our action sequence so let's build and run this app now and what we'd like to see is three gold stars appear even though we've only set up one in our application this is driven entirely by application logic so here's our scene again we can walk through the lesson think I'm done now so I'll click complete lesson and we see one gold star appears we'll see a second gold star appears and there's our third star we did a great job so we've got three stars here so that's how easy it is to load your AR content into your application begin working with it with notify actions to provide your own custom implementations and then trigger action sequences that you set up in reality composer using notification triggers so let's take a tour of what we learned today we learn with reality composer you can create amazing AR experiences using the built in modular assets for example the sphere or the cube and then we can bring those assets to life with behaviors and simple interactions finally we can bring those amazing AR experiences into our applications using code generation and deep integration with Xcode we've already seen some amazing stuff being created by you guys with using reality composer we encourage you guys to keep creating we're super excited to see the amazing air applications that you can create we have some sessions and labs available to you guys today and tomorrow and we encourage you guys to bring questions your projects anything else we're happy to help you have a great rest of your dubbed FDC thank you [Applause]

Channel: cyhunter cy

Views: 2,310

Keywords:

Id: aL_um8ygllw

Length: 59min 5sec (3545 seconds)

Published: Wed Jul 03 2019