ARKit 3 Tutorial: Create Your own 3D Worlds with Reality Composer & RealityKit

hi and welcome to another very exciting tutorial in this video we are going to create this cool augmented reality experience that's interactive that uses different files like a USD Z file for the robot object file that we're going to convert into USD z file for the earth and all of that together with the new reality composer and reality kit and if you don't want to miss any tutorials such as lease in the future make sure to subscribe right now and now let's get started so if we have a look at our a really composed project in Xcode and this is the project that I've created and you can see that in Xcode we have the ability to just have a good look and what we have created and with this button in the top right corner I can actually open it up in reality composer I tried that and the problem was that it was really really laggy and didn't work well on my machine I don't know if it's related to beta or of my 5k iMac is too slow for that but I decided to actually go with the iPad app that really works pretty well and smooth and I just want to give you a quick tour of what we can do here in this app that works very similar to the Mac app but you do have the ability to actually have a look at your stuff in a are directly from the device but not going to do that so this is something a preview that we can have on iPad and I can actually run my simulation here tap starred and just move around and have a look at everything so this is really cool and this is obviously what we are going to create and having a more detailed look at everything you see that we have different components here we have a very simple button here to tell us where we can actually start our project or our app we could just tap on that later with a welcome message that flies in with this nice little robot here kind of a vintage toy that even moves and we have our earth that spins and these different components are mostly from different rhea from from different places so I got this this robot as aus DZ file from Apple they have a quick look gallery a our quick look gallery on developer.apple.com in the AR section there you will find an AR quick look section there are just a few 3d models in the USD Z format that you have to work with when working with the reality composer but all in all they look great and they have also some animations included as you have seen with the robot and I just went and went along and downloaded this robot right here even from my iPad and we can use that right away but as for the earth I didn't get that from Apple so what I did was looking online for a free 3d model of Earth for Earth it's quite simple you just need a ball actually and apply a texture so the texture is actually the more interesting thing here so this texture that we're going to apply and we're going to convert this object file which is very common for 3d model objects and we're using this texture and we're converting that to aus DZ file that we can work with so if you have such an object file it is very simple actually to convert that to a US DZ file you could even include more material to actually also get depth effects and correct reflections and so on but we're just going to use the color map here and keeping it very simple so I have just opened up a terminal in indeed we're using a terminal here to for the conversion and I'm changing my directory to desktop and Earth this is where my files are located and now all I actually have to use is the Xcode run command and I want to run the command u.s. DZ converter and I want to convert the earth object into Earth esteem and my color map is also located here at earth underscore texture underscore CM dot TGA this is the file we have here and once I hit return now it just takes a few seconds and we have a ready you go USD Z file and as a consumer can even preview it here move it around and this is a file that you can sent now to your iPad using airdrop or whatever you like and then we can use it in our project so these are the robot and Earth and then we have these two objects that are actually part of the library from reality composer that you can get when you hit the plus button in the top right area of our application but now let me go back one step this reality composer and I can press the top right plus icon to actually create a new project and it is going to ask me what kind of an anchor I'd like to create I'm going to choose horizontal since we want to place something on our table and this is what our plain and simple project now looks like I just want to show you a few quick things here so that we can have a more detailed look at my project and that we don't waste too much time here on creating all the basic steps here but let me show you for example when we press on one of these elements like this cube here on the right in this pane you can adjust the material and with that also reflection of this material we can set it to aluminum gold steel and so on just to have a special look if you like that you can adjust the bevel radius we can set the physics and so on but let me show you how you can for example delete an object like this so you press on it once press again hit delete and we're good to go now what we can also do is now add a new object and as you can see in this library here you get tons of interesting stuff for different purposes but sooner or later you will find that even the morgue Eric elements like science and speech bubbles might not suffice so we can import things like our earth but let's do that a little bit differently let me just cancel that for a second and just use a sphere here as a very basic object we can move that around with these little cylinders that we can grab in different directions X Z and Y and as you can see since I have turned on this magnet in the top left corner my elements are actually snapping in position and using one of these circles around my elements that can scale these up if I actually move my finger along one of these circles I can change the angle and the rotation of these objects but now I actually just wanted to show you that once you have created such a basic shape you can also replace it with another object and by that keep it at this position if you want that so I'm going to now add or import my earth that we have created earlier just select it and there it is just scale it up a little bit but it is positioned exactly where we wanted it to be so this was the one thing that I wanted to show you now the second thing is how do we actually add behavior to an object like this like the spin that you've seen earlier so I'm just selecting my object here and opening up the behavior section and then I can add a new behavior and you get quite a bunch of elements here and then we can work with but we are going to use a custom behavior for example you could use a tab and flip this is something very simple wait and show but we can also use custom here and I can for example add a tab trigger so once I tap on my earth I want to do something like for example and this would be action we're selecting now a spin and what we have to do here sometimes if we didn't select it in the first place we can choose an facted object so we could also choose this is a button here at the bottom but we actually just want to choose our earth and then in adoration let's say 15 seconds we just want one rotation and want to loop this using this little loop button here and then as you can see this is how we can actually run a spin animation now having seen how to actually create objects and basic behaviors let's have a look and the finished robohand project that i've created you can see that it is basically the same thing just with a few more elements and now the question is how do we actually move our robot and how do we actually let our robot make one step after another the cool thing in this case is once I open up the behavior area at the bottom and if we have a look at these action sequences that if I play one of these behaviors which is a USD z animation you can see that this animation actually comes from Apple in this case because it is integrated into this USD Z file and you can just add an action sequence at the USC see animation action and that's actually it and what I did now to actually move our robot forward is to combine in this USD z animation with another move and rotate element or with an move and rotate action as you can see this would be in the move action that I have to find just moving our object along a certain path very simple I could move that to the right also move it to the top whatever you like you can move rotate scale it and so on and the app is going to create everything else for you but now then I want to actually play my real animation here and this move animation together all I have to do is just select that to start dragging it move it on top of my other animation and then I've created a group and once I run this as you can see we're actually moving forward and this looks pretty cool now I just added a show animation here so that our little robot comes down off the ground looks really cool and at the end what we were doing is actually introducing our welcome and tap to start button what we also have this is the Robo start behavior group and there's something the trigger that we're going to talk about in a second but once we run this as you can see there is nothing happening because we have the hide at start behavior here which is triggered automatically when the scene starts and we're actually hiding all four options or objects that we have as you can see all of them are green and I can select and deselect all of these objects to actually make them conform to this action then if we want them to so this is the height and start behavior and then we have start earth all that happens here is once we tap on our button we are actually hiding our welcome text and the tap to start button using a hide action and grouped with a another action is our earth and that we have in this show action just to move our earth downwards our welcome and have to start button are moving upwards and then we're just adding this spin animation so nothing really fancy here and now what I wanted to talk about is this specific trigger that we have for Robo start which is actually a notification trigger if I just quickly create a new behavior here with a custom behavior and add a trigger then at the very bottom you can get a notification and this is a notification that you can send from Xcode just by calling the name that we have here which is Robo start and we're going to have a look at that in a second as soon as we have this in our ax cube project and for this I'm just going to save my Robo man project here and send it over to my Mac using the files application or a drop and back on my Mac I have my Robo man our C project file the reality composer file and you could use a reality composer to actually export a reality file which you can use together with AR quick look but for augmented reality apps and in our case an app with the content technology reality kit you need this RC project for this to work we're going to call that Robo Swift hit next and create that on our desktop and resize my window here a little bit and what I'd like to do now is first of all drag-and-drop my file into my project make sure that copy items if needed is selected and that we also add our file to our target Robo swift hit finish and we are done so there's also a demo experience right here also when you're using the beta version at the moment if you drag and drop something into your view and this is running then Xcode crashes so don't do this I'm going to remove this file for now because we don't need it and what Xcode does in the background once you add such an RC project here is ours it's takes it just a few seconds to render and I can actually move around here and have a look at this beautiful screen yeah what Xcode does in the background is actually translating the content of our file into classes and objects structs and basically a data structure that we can work with and as you can see here there was this experience file and we are loading or the template loads the box that is included here we're going to do something very similar so I'm going to remove this template code right here and what I like to do is create a proper team for our Robo anchor so I'm going to define this variable here as a global property here and this is going to be off type Robo man and the Robo adventurer this is the scene name that I've given to the scene that we were working with just in top right corner we can have a look at that just for a second so that's the beauty of this thing here that we can open that in Xcode as well so here is our scene and with a click or a tap on this button in the top left corner could actually select our scene and here in the right corner in the settings for this scene we can change the name to for example Robo adventure and I don't know interested in exactly that scene and that scene anchor and I'm defining this as a property here because we're not only needing that in viewdidload in viewdidload what we are going to do is use our anchor and fill it with the actual adventure and we have to use a try statement here and we're living on the edge here not catching any errors and not handling optionals just using try exclamation mark to use Robo man and then synchronously loading Robo adventure there's also an asynchronous operation that we could use but for simplicity's sake we're now just going to use the load Robo advanture function and as you can see this is all done by Xcode in the background just by dragging and dropping this robot man our C project into our Xcode project we get all of this for free and now that we have this anchor what we still have to do is using it and then generate collision shapes recursively and we need this because we have this button that we can press if we do not generate these collision shapes button presses wouldn't work so now the last thing that we do is using the AR view that is already an outlet in our view controller so opening up our storyboard we can look at the AR view that is just added by the template and seeing this storyboard here and a standard view controller is actually not very surprising even with Swift UI as a new technology swift UI should be seen as something that is or that is running parallel and you can adopt it step by step where it makes sense use a storyboard where it makes sense use swift UI in this case storyboards and view controllers make sense so we are using them and even template suggests us suggests that we should use that and if we wanted to integrate something some UI using Swift UI we could do that and also the other way around but that's just as a side note and we are having this AR view as an iboutlet right here so using this AR view access its seen and its anchors we can append our new anchor which is just our anchor hey we should call it Robo anchor let's do that just quickly refactor rename and call it Robo anchor alright so this is our Robo anchor and now if we run this we would see nothing because our scene doesn't start we still have to actually start that scene therefore let's quickly open our storyboard let's press command shift Elm and a button here and drag that to our view controller let's say start experience and add quickly an auto resizing mask here to pin that to the top center and once we did that we can also open up a view controller here and our main storyboard here on the other side let's create a quick action right here below view to load and then to call that dard experience there we have our button action and here we can now simply use our Robo anchor access it's not vacations Robo start this is the notification that we've defined earlier let's just have a quick look at that so here is my project opening it in the reality composer and if we have a look at our behaviors there is Robo start and here as a trigger for our whole sequence here for our whole project starting with the show animation of our robot and so on we have triggered that with the identifier Robo start and we can now use our list of notifications that are available to us which is just one which is Robo start and then just hit the post method and once we did that we can actually start our application and I am now going to run this on my iPad and show you the result so an here is our app I can press the start experience button we have our robot that walks we get welcome can press the tapped start button we get our rotating earth so isn't that cool and that just with a few lines of code and the reality Composer application actually there is a lot to be improved here we don't have very great surface detection the user can't really interact with the whole scene you should be able to move that around maybe but that's not the point you've seen how to actually create something really cool in reality composer and how to actually get what you have created into an Xcode project and put it into your application so I hope you enjoyed this if yes then give this video a thumbs up make sure to subscribe to my channel to not miss any new tutorials especially now with all this great new announcements from Apple I thank you so much for watching I'll see you in the next one bye

Channel: Brian Advent

Views: 55,367

Keywords: Swift, Xcode (Software), OS X (Operating System), Core Data (Software), IOS (Software), Tutorial, programming, iphone, ipad, Macintosh (Computer), Apple Inc. (Organization), IOS 8, ios9, CloudKit, iOS10, Swift 3, firebase, firestore, ios11, swift 4, ios12, swift 5, arkit3, ios13, xcode11

Id: 2bGa4bhOHeY

Length: 22min 1sec (1321 seconds)

Published: Mon Jun 10 2019