Blender for AR/VR Design (Beginner Friendly!)

hey everyone it's jake here today i'm going to walk you through how i use blender as a tool for ar and vr design so i'm going to start by saying what we're going to build today first we're going to build a video walkthrough of an ar prototype this is using footage we've taken as well as 2d ui and putting them together and then secondly we're going to be looking at the vr scene inspection tools inside a blender i should probably start by saying why i use blender over other tools and i think this breaks down to what design is good for at its core which is communicating things quickly if we look at tools on one end of the spectrum we have things like figma and sketch which explain things really quickly in 2d but you don't really get that spatial thinking if we go over here we have unreal and unity which are really spatial in depth and can explain ideas really well but because of that there's a big time sync involved so blender really meets in the middle and kind of has a spatial thinking but allows that quick iteration time if you follow the steps i'm gonna outline here so let's jump into it and start off with the ar video on how i created this [Applause] let's start by shooting some ar footage we're using an app called cam track ar which is going to save us a few steps because it shoots video and tracks an ar at the same time you can find the download link below we're going to kick things off by opening the app and getting a good scan of whatever we want to get video of so making sure these grid lines are showing setting a point with the bottom button below and then setting an anchor point with the second from the bottom button and this is really gonna help us later on once we have a good solid track we're gonna step back and record our video once the video is recorded we're going to jump over to the files app find the cam track ar folder and you're going to notice three files we're going to send those to whatever device we're using blender on for easy access later all right so now that we've shot our footage we're going to go ahead and move over to blender to bring our ui in so the download link for blender is below install blender and you should see something like this to give an overview we have tools on the left we have sections at the top we have the outliner on the right with everything in our scene and then we have this property panel on the left which is going to reflect whatever is selected we also have hints at the very bottom to give us uh keyboard and mouse shortcuts we're going to install two add-ons to help us today by going to edit preferences add-ons searching a line to find align tools click this little check box and then we're going to search images and then install this one called import images as planes go click this cube click this light by shift-clicking and then delete them with the delete key and delete this camera as well we're like i'm going to import our ui by going to uh file import images as planes find ui assets we have select them come over here to the settings and change material settings to emit and then import images as planes we're going to notice that these don't actually have the images shown that's just because of the viewport mode we're in so if we come up here to the top right click viewport shading material preview they should appear so next up we're going to put everything into position we have two options for this the first is using gizmos so if i come over here and select transform as well as these rotate scale and move tools i can visually do this the other option is to use keyboard shortcuts so if i wanted to use r i could rotate and then press r and x to stay on the x-axis this would rotate along the x-axis so rotate this 90 degrees and all of these rotate 90 degrees and you can also type in 90. so rx90 to rotate 90 degrees we're just going to scale with the s key so s4 and then we're going to come over here to this panel and we can set the actual parameters here so location zero zero zero there we go we have some ui so we're gonna notice there's a bit of a clipping issue here even though this is in front of this it's not showing in front of it so if we select this button here come over to material properties go down to settings if we select blend mode blend alpha hashed it's going to fix this issue and we're going to do this for the all of these just so we get the right idea of what's happening all right so next up we're going to install an add-on to get some 3d assets and we're going to be getting them from sketchfab you're going to have to create an account for this site so go ahead and do that now but instead of actually downloading all these assets and installing them manually there's a handy add-on which is also in the description below go ahead and click this code button on the github page click download zip uh it should download and then go to file edit preferences add-ons same as before but instead of searching for it we're going to click install at the top right find that add-on the blender plugin master.zip click it install it and make sure you install the zip and don't unzip the folder you'll run into issues if you do that and if we did that correctly uh just type sketchfab here to see that it it was checked uh we should be able to pull out this carrot uh up here this little triangle icon and we should see sketchfab is one of these options here so you're gonna have to activate the add-on up here put in your login information under the import section we select uh search we search for what we want so searching ramen and then click this uh image here it's gonna bring us all the options i like this green one low poly stylized ramen it's nice and small as well so we're gonna import this model we're gonna get a little bit of a loader it's gonna load and you see in the outliner it appeared but it's not in our scene that's because it's actually way up here so what we're gonna do is go to this property panel set the location to zero zero zero and set the scale to one for now and there we go it's in our scene we're just gonna position this real quick all right and now we have a 3d ramana uh one thing we're gonna notice if we switch over to uh viewport shading render preview the ramen is going to not have any lighting data it's going to be flat and dark we're going to want to add a light to the seam so if we press shift a light go to sun this is just going to add some nice even lighting for us we're also going to want to make sure that over in this render properties panel it render engine is set to ev this is just the real time engine for blender so we're not taking a really long time to render things so next up we're going to install an add-on to help us import that footage we took earlier so we're going to go to edit preferences add-ons again install again find that footage from earlier and there's going to be an add-on called blender hitfilm importer dot py click install add-on make sure that this is checked and if this installed correctly we can go to file import hit film ar tracking data find that tracking data again and there's another file in there that's an hfcs file click this group blue button and if installed correctly we now have a ar camera in our scene as well as that anchor we set earlier and if this worked correctly you click this little camera icon here and nice we have a virtual camera uh one thing we're missing uh you probably notice is the footage so to import that we're gonna make sure this camera is selected click this green icon with the camera come over to background turn this on add image movie clip open and we're going to find that footage we took earlier in the same folder as the other two files open clip and go over back to the material preview is going to be pink don't worry we just have to skip to the first frame because there's no image data on the first uh frame 0 of the film but we're going to change some of these settings down here to put the opacity all the way up and set frame method to crop and there we go we're now walking through our scene in full 3d all right so you might have noticed that we're kind of just walking through the scene and walking through things we want to put these in position so we're going to want to do one thing first if you notice uh we kind of want to group these together but if we move these they're all moving separately we have to do is parent them so if we want this roman this button and this ui all to be together we have to parent so click the button click the ramen and click the ui background last press control p which is going to set up parent and then choose object keep transform if we did that correctly you should select the ui and you should see it all kind of move together we can do the same thing with this destination icon and the map control p object keep transform next up we're going to use that a line add-on we saw earlier by coming over to item seeing align tools down here opening that and what this does is uh it you select two things or more and everything aligns to the last thing you selected so we're gonna select this ui select this empty point up here it's anchor zero in the outliner you can also select that and then choose align location over here in the align tools and click all this is going to very nicely kind of align it without doing that eye work of trying to figure out exactly where it is we're going to put this in position real quick so that's how you place things in the world as you can see when you move the camera they're going to stay locked in the world but if you want things stuck to the camera i'm going to show you how to do that now so say we want this little map down here attached to the camera kind of like google maps ar view so we're going to select this map we're going to come over here to object constraints choose copy location choose target choose ar camera this is kind of like parenting but it's going to give us a little bit more control we're going to choose offset and what this is doing is taking this object properties data of the location and adding it to the cameras location um so as you can see if we zoom out it's kind of you this blue line in relation to this so if we set this back to zero zero zero you can see it's going to follow the camera as it moves and we're just going to move this in front of the camera a bit uh and you can see the rotation is not great we can also add another constraint here to copy rotation set it to the y and the z axis and if we did this correctly it should kind of stick to the world and not follow the camera exactly and as we look up it should look away so let's see if that worked so as we look up and to the right the map just kind of stays there and out of view so this is a nice emulation of that with just a few clicks so if we want to render this out we're going to notice an issue if you come up here render image you're going to see that it only renders the ui and to fix this we're going to do some compositing so we're going to do two things first we're going to come over to render properties go to film and turn on transparent this is going to add a transparent background then we're going to come up to the compositing section up at the top click use nodes and we're going to add three nodes here with shift a first we are going to add the movie clip we're going to add scale and we're going to add alpha over we're going to connect these with through all the image ones just connect those render layers image to this alpha over and the alpha over image to the composite image we're going to go over to this movie clip click this little icon and you should see our movie in this drop down here we're going to change the scale to relative or sorry render size crop and what this is doing is basically taking the movie putting in the background and then rendering the ui over top so if we did this correctly we should click render render image and there we go it rendered together so to render this as a video we're going to come over here to the output properties which one this one uh change the output location to where we want with this little file icon and change file format to avi jpeg and then if we come up here to render render animation [Music] and you're going to see it render frame after frame after frame after frame until it's done but this is kind of boring it's pretty static so let's add some animation to this so come back to the layout section we're going to scale up this um button so we're going to find a point where it's good to start animating say frame 160 as you get closer we're going to find the scale of this 0.72 and to add a keyframe we need to be at the frame we want hovering over this uh property and then press i it's gonna turn yellow and you see it adds a keyframe in here um and then whenever we select something else the keyframes are gonna hide so this is only showing keyframes for what is selected so say we want to come over to uh make this over 20 frames frame 180 change this again to it changes to green meaning it's not keyframed press i as another keyframe come back here to our beginning frame and we're going to choose a starting size so let's say 0.35 it turns orange that means it's changed but not set if we come over here and go back a frame it's going to reset on us so what we have to do is press that i key again to actually set it so and once we do it's going to change back to yellow so we're going to see now it's set so i just cleaned the things up a little bit we might want to animate the opacity here and you're not going to see a property for that over here so to add that we got to add a transparent shader to the material that might sound like a lot of words but it's pretty easy so we're gonna select this button come over to shading you're gonna see these noodles come over to render preview just preview it and what we're going to do is add two nodes here so shift a mix shader shift a transparent shader and we're going to put this transparent into shader and this bsdf into shader and the shader out and what we're going to get is a slider for the transparency this is basically taking a full transparent shader and a regular shader and saying okay we're making a transition between the two so we're going to put this all the way down for transparent all the way up for opaque we're going to need to copy these two nodes select the raman since we want to animate the opacity there and do the same thing take this bsdf into here take the shader out into surface and we're also going to get shader for this and if we come back to uh layout we are going to be able to keyframe this by going down to the material properties opening the surface section and we're going to see a surface mix shader and this factor this just means uh this is a simplified version of that shader view we saw earlier so if we slide this you're gonna see that it's um changes the transparency so if we press i here go to the frame where we want it fully opaque just before the scale ends change it to zero press i and if we press space bar to play this we're gonna see that it animated nicely there's a bit of artifacting here but it's gonna go away when we render it out so same thing with the button go over to this fac in the material properties bring it all the way to up for transparent go over to our end frame bring it to zero to be opaque and there we go now if you really want to get fancy with animations uh things like curves and all that if you're into after effects come over to this animations panel and you're gonna see two views down here one being the dope sheet and the other being the graph editor if you don't see this graph editor don't worry come over here to this drop-down and choose graph editor and it appears here i just like this two-up view to have keyframes on the left and dope sheet on the right and you're going to see something familiar here which are bezier curves for the graph editor you can edit easing and animation uh through here and one thing we're going to note is uh this little icon here only shows selected this is only going to show selected things so once we unselect this this is how you find everything else to to animate if so things aren't showing make sure to unclick this one but there's some handy things in here if we select a point and go to key there's easing type with control e which is ease in ease out ease in out interpolation mode so how gradual it is and then the handle type so these three properties are going to give us most of the things that we've had seen in after effects and other tools all right so i'm just going to clean this up and i'll show you what it looks like as a final render [Music] all right so for the last part of our tutorial we're gonna keep this one pretty short and sweet so we have the scene we built here by placing the ui elements we showed before uh and what's new is this virtual environment uh you can download it below in the sketchfab link that i've uploaded to view this in vr and kind of get an idea of scale and size we gotta make sure this apartment is set one to one scale then we have to enable vr support by going to edit preferences searching for vr and then 3d view vr scene inspection right here and if that did work correctly uh we should have a vr tab here make sure a few settings here are set so positional tracking we need to turn that on uh we need to set floor and i'm going to turn on mirror vr session just so you all can see kind of what i'm seeing in headset so if we have uh oculus uh open or steamvr open i have an oculus link with a quest here this should just work so we're gonna click start vr session and there we are you're seeing what i'm seeing um and a cool thing about this is everything we update in scene here will update on the headset so i can go ahead and press a select everything and move it around and you can see it's being updated uh in the headset as i'm updating it so that's it for today um hope this is helpful if there's anything else uh you want to learn about uh designing with blender for ar and vr let me know in the comments below alright thanks guys [Music] you

Channel: Jake Blakeley

Views: 19,208

Keywords: Spatial design, vr/ar, xr, mixed reality, product design

Id: lKlPCRn7W4A

Length: 21min 2sec (1262 seconds)

Published: Thu Mar 25 2021

Reddit Comments

[u/evakuttichova]

Cool video! I am glad there's showing up more of us doing AR/VR design videos. I love seeing development in this sector. Keep up the good work.