Using ChatGPT and GPT-4 to Generate 3D Content in Omniverse

hello I am Mario viviani from the Nvidia Omniverse team I'm very excited to be here to give you a demo of the new AI extension for Omniverse called Rune generator with this extension I'm going to show you how combining Ai and apis can really accelerate scene building the AIS that we'll be using in this demos are gpt4 and deep search let's imagine that we're mocking up a warehouse here let's start with the floor plan I want to generate the content of a few rooms in particular I want to create a small Warehouse here and here I want to create a reception room for visitors of the warehouse so let's focus on the small Warehouse I want to drop some items in here so let's start with creating a new curve from pencil I am going to draw an area here and this area is where the items are going to be contained here we go I click enter and the area is created now I need just to select the curve that I create go to generate room and here I want to add the name of the room in this case we already put in warehouse and then I want to add a prompt that says add common items that can be found in a warehouse I just need them to click this button here and the area is generated and is ready to go for dropping items in it all I have to do is to go and click generate so now gpt4 will generate a list of items connect to deep search and load those items in the scene and here we go as you can see the items have been gracefully disposed in the scene and we have actually a pretty good variety of items in here we have things like uh palettes we have shelvings we have a lot of different units in here and we could always go and change things around and we can always move those items but you know the layout looks actually pretty good for me all right so now I want to create a reception room so let's go back to our floor plan here what I'm going to do is I want to create in this area here a reception so let's go to pencil and again I am drawing here e area where I want the items to be dropped in again this time is pretty big here we go click enter and then I am going to select this curve here and I go to generate the room and I'm going to call this reception and then I already prepared a prompt that says this is our reception area where we greet customer at the front desk and they wait for the products in a comfortable Lounge so all is set and I just need to generate the area which is ready to go and then I go here and I click generate and we're ready to go so again it might take a couple seconds for the AI to generate the items and here we go uh I see that the AI did actually a very good job here into creating the items so let's see what we have what we have here uh it generated quite a lot of items uh pretty accurately as well there is a front desk and now there is of course it's a reception room so it added relevant items yeah the two sofas and the coffee table and I see that he added a couple of armchairs and a informative display stand so that's actually pretty uh pretty good uh we have a lot of different items in here it is interesting to notice the spatial awareness of gpt4 all the items are correctly disposed and there is a logic to the order of the items and also all the items are disposed inside the boundaries of the area which is actually quite impressive the last thing that I want to show you is deep search and how you can use it to change this even more deep search is an ecosystem of services that allows Omniverse users to search for Content stored in nucleus server using natural language queries it uses AI to search for the right assets so for example you can go and search for a chair and get good results as you can see here and start navigating through those however if you are a little bit more descriptive for example putting in office chair you can get even better results deep search can be trained on your own data and since GPT uses natural language too you can see how powerful it is to combine these two technologies and since we took advantage of deep searches abilities to use natural language for search it made sense to use it into another extension to make selections from the results so let's see what that looks like so let's say that we don't like one of these objects let's say for example This Modern sofa modern sofa one we don't really like it so what we're going to do is we're going to zoom in on this one and we're going to use another part of our extension which is called Deep search swap and what I'm going to do is we're going to use deep search to select another sofa and this is done automatically using AI it will again connect to a repository of items and looking for another comfortable sofa uh using using AI so we just with the click of a button again is proposing as another a different sofa and if I click again I click I get another one and here we go and we can continue this way and get a lot of different variations as well and actually this is the one that I that I had before which is which is perfectly fine but as you can see just by using deep search we can quickly iterate through it and get a lot of different variations on so hopefully this demo of AI room generator and deep search gave you an idea of the power of combining Ai apis and Omniverse and building tools around it we are very excited about what the future holds for AI and can't wait to see what tools you're going to make with it make sure to check out the Omniverse extension documentation and learn how to make new tools thanks for watching foreign

Channel: NVIDIA Omniverse

Views: 55,140

Keywords: NVIDIA, Omniverse, RTX, PixarUSD, OpenUSD, Ray Tracing, Animation, Collaboration, Photorealistic, GPU, Connector

Id: mFazJsjUUSo

Length: 7min 27sec (447 seconds)

Published: Thu Mar 30 2023