glTF - Efficient 3D Models - SIGGRAPH 2019 BOF Sessions

welcome to the gltf Boff here at Chronos my name is Ed Mackey I'm from AGI I'm a member of the 3d formats working group our chairperson is Patrick cozy he's on a red-eye flight or something but he'll he'll be joining us later and I'm gonna give you a quick overview of gltf I've stolen all these slides from Neal Trivette of course so that's awesome but I'm gonna sort of breeze through the ones that that he presented a little quickly here you guys should know hopefully it does everyone here know what gltf is that's why you're all here ok it's the JPEG of 3d what it really means is that it's a model format that is designed to be quickly and easily delivered to end user machines clients customers it will get into the description of it a little bit later but it's originally it came from WebGL like our in the old days of gltf 1.0 we were specifically a WebGL delivery format so it was meant for very efficient delivery across the network and unpacking to WebGL and of course now with gltf 2.0 we've have a much broader ecosystem with desktop that Vulcan metal direct3d all that in fact speaking of broad ecosystems we have a huge number of participants on our ecosystem slide and with more contacting us every year to add one more logos on here which is a which makes the logos hard for you all to see but it's a great problem to have Microsoft Office is on here so it's very meta because this is a PowerPoint and I show this slide to people and a lot of people start picking through like they pretty quickly spots some big names and big players on here and then they start picking through and seeing the all the other little logos like I I think I know those guys and I've heard of these guys and then usually about three or four minutes in they go wait wait a minute you guys snuck a Minecraft logo on the what's that doing there is that an easter egg no it's not an easter egg for your Minecraft players you can sleep / gives play your name structure block and get a block that actually exports 2d LTF and then draw drag and drop it into your powerpoint but I digress I'm going back to kneel slides now I promise so the basic structure of gltf is meant to be very straightforward to load there's a binary payload that has vertex attributes and indices and then there's sort of an overview structure in JSON which is a text-based format that says you know hey this is how you slice up your your vertex attributes and this is how you load all this data onto a GPU and the goal of course is to get this to come over the network very quickly and then render almost immediately without a lot of fuss and processing and yet still we support modern physically based rendering workflows with metallic roughness and spec gloss we also have an extension called Draco mesh compression which will take a large amount of mesh data and compress it down these slides are all going to be available to download after because I don't have enough time we have like nine other speakers who are going to come up here and I don't have enough time to go through all these links for you but you can download these later I do want to talk about basis Universal which is a technology that has been open sourced as a result of a partnership between Google and a company called binomial and basis Universal why be compared to JPEG you guys are familiar with JPEGs if you see JPEG on my little chart here you can see that network transfer is on the darker green bar and for JPEG network transfer is really small and then how much space it takes up on your GPU in your graphics card memory is the lighter bar and JPEG fully uncompress --is and it takes a ton of space once it gets into your memory there are various forms of what's called GPU compression which are special formats that don't necessarily compress as well as JPEG but can stay compressed and be usable on the graphics card even while they are compressed the main downside of those other than the size being not as good is that different GPUs have different hardware compression formats that they support there's not one common format this is where basis Universal comes in it is what's called a super compression which is a compression of a compression and it transfers across the network about the size of a JPEG and then rather than decompressing it we'll transcode to the locally needed hardware GPU compressed format and here's some more information where we're still in the process of finalizing this polar quest that you see at the top there it's happening on github that will bring that will bring the basis universal format into gltf and we have several engines all of which are speaking today although Babylon and 3j us are speaking during the WebGL section next cesium and UX 3d will be speaking if I ever get off the stage here but please do get involved and if you want to have any say over how basis fits into gltf that will be happening very soon so speak now or never the other headline for this year is that blender 2.80 which is a major overhaul of lendor got finally released the official stable release of 280 was released yesterday morning and it includes a full Chronos packed gltf import/export function the export functionality of course came also from UX 3d and the documentation came from me so if you have problems with it you can come complain to me so roadmap discussions we talked about gltf in the open on github issues on our github tracker we talked about next gen PBR and animation adobe has a pull request open for better metadata get involved it's all kind of fun ecosystem robustness this is something I want to say a little about gltf is such a simple format we we don't have a single piece of software that you need to go through there are other formats out there that if you want to get in and out of that format you have to embrace their software you have to take it on as a third-party dependency and use their functions and accept their you know their patterns and their security vulnerabilities and whatever else but with gltf you're free to write your own or free to pick some open-source thing that somebody else has written we do ask though that if you're producing gltf files you run them through our gltf validator make sure you're producing clean gltf files that are interoperable and of course if you're loading them we have sample models and we have unit tests from the asset generator please make use of these tools that way we can make sure that all gltf readers and writers can talk to each other even though they were written in different languages on different platforms here are some resources you can google these if you have any questions feel free to ask the person who made these slides which is neil yes it's supposed to be funny you may have and i'm gonna get off the stage now because i've wasted and used up a lot tons of time but we have a ton of fantastic speakers up next starting with Rene from Facebook and I will get her presentation up here thank you thank you hey everyone i'm rene Roisin i am a software engineer on the spark a our team at Facebook and so today I'll be giving you a brief overview of how Facebook supports gltf across our family of apps so we support GL TFS almost everywhere we support on Facebook Instagram Messenger portal and oculus so the only odd person out here is whatsapp but most of our support comes from the spark a our creation tools and runtime so spark a our studio is where creators any creator can import GL TS into their AR FX and then distribute these across our family of apps and so those are the ones in red are the GL T of support provided by our spark runtime so here's a brief look at spark AR studio yeah so here you can import 3d object and pick a gltf or GLB asset and have that placed into your effect so I'm choosing the damaged helmet and it'll show up there and you can see in the preview window down below that the asset is rendered okay so Facebook supports 3d photos I know Neil mentioned 3d in the feed earlier but I am sorry to report that we no longer support direct upload of three 3d models to the feed but we support 3d photos so users can take photos with depth information so if you've taken it via portrait mode or you have an explicit depth map you can upload those through the phone or through the web interface and play the video there a trick to playing the video here well it'll you choose your photo you upload it click the button we detect that it has the depth information so you can click the button to make 3d it'll create a gltf mesh in the background and then you can see the parallax effects in your photo and post it to your feed Facebook and messenger also support AR ads so if you're scrolling through your feed and you see an ad that says tap to try on that'll take you to the camera and then show our use our spark run time to render the AR effect and here I am not sure whether this specific asset is a gltf but because we are on the spark platform we will support gltf assets and of course Instagram and messenger support the camera effects which are all the fun fabulous facial effects that you see everywhere this one is a star's effect by user geyser and so if you want to be someone's Sun and stars you can and this is using gltf assets portal is our hardware like home video conferencing device that we came out with earlier this year and that also supports the spark platform to power things like story time where someone not one end using portal can act out a story that someone can see on the other end and so this one is obviously using mostly 2d textures and effects but since its spark you can use gltf and 3d assets and of course oculus if any of you have tried on the quest headset our beautiful home environment is gltf model so it's this geodesic dome with this nice cozy interior large oculus logo and a beautiful cubemap outside so that's the support across our family of apps so thank you to Chronos and thanks to all of you who worked on gltf hi everyone my name is omar cuz i got programmer on the season team and i wanted to share some results with a prototyping the basis universal texture compression in our work so just for some context the demo I'm going to show is using a 3d tiles 30,000 open standard for streaming massive few geospatial data it's built upon geotiff and kind of extending it for putting data on the globe or doing stuff like progressive refinement being able to look like multi-gigabyte models in the browser so typically the having this texture compression allows you to like load much more much more data in the scene much higher resolution textures in your scene because to details already has a way of loading more than your scene can allow by either loading these varying levels of detail or unloading things that are not in view having this extra compression allows us to load more levels of detail sooner or without having to zoom in all the way so for this demo it's a 1 gigabyte model of photogrammetry captured from satellites of Cairo Egypt I'll be showing a JPEG version and then a basis version that was encoded with the basis you command line tool and then it's transcoded with a web assembly on the fly to bc1 format so this is not our biggest model but it is bigger than the damage helmet so hopefully should be good so it's gonna be alive then I love seeing the remember here so so you can see on the top that's the memory use I'll come back in a sec alright so we're studying in like a 600 megabytes just kind of baseline here and if I had this JPEG model and then I'm gonna zoom in to the pyramids excessive climbing up and as I move through the city it's loading in high-resolution versions of all these textures streaming in it's climbing up all the way so it's over a gigabyte already but then if I then switch refresh this let's make sure this is back here okay so we're gonna try the basis version now okay so we're zooming in now and yeah it's like barely breaking a sweat it's still you can't even tell that you added something if you're just looking at the memory and as I'm moving through it's still just around 500 megahertz vastly smaller as fast as you can you remember back here so quality wise I don't know if you could tell very easily on the projector but it's it you can't really tell it looks just as good if you take a closer look with this so this is if this is a different tile set with a variety more textures or so you can kind of see how it looks on different areas so this is the P in the original PNG and then this is basis so I'm gonna flip back and forth it's very hard to tell because they actually do look pretty good but also if you look here's the Delta if you look on the bottom left you'll see like a little pink area that house actually has a slight color degradation you can see the color looks a little bit more faded but oh and here's for comparison a JPEG Delta it looks like there's no difference but there is a tiny bit of a difference on my on my screen here so overall we've seen about four times less and GPU memory usage and this is a little lower than what most people have been I think presenting like usually seen like five or six to eight but this is also because I'm measuring like the total everything in the application so it was the geometry as well but this was kind of the differences I we saw on this demo and the basis version on disk is actually smaller about yeah a little bit smaller than the JPEG version which is kind of an added bonus which means that I'll set the data loads about 15% faster um it's just kind of a added bonus and finally the artifacts that we saw here you can mitigate that by being able to because base is so much smaller on in terms of GPU memory you can have a higher resolution texture to account for that and still and still have a benefit that's it thank you all right thanks Omar so let's continue with more basis stuff if you look at this asset it's something we can get nowadays out of the okay hey sorry yeah I'm trying to rush through this presentation in order to keep on a 5-minute say I'm max from DGG we are working on an optimization software called rapid compact and yeah you'll see what we do in a minute and we are heavily working on gltf tools in this context so this is an asset that we got out of that sighs real scan system which is pretty impressive because you can see that it already has all the PPR maps that we need and it looks reasonably good so this is around 100k polygons it's an obj file can also get gltf from them it has high resolution textures so the question is can we already use an asset like that for all kinds of presentation modes and the answer is no for many cases it's too heavy so if you think about an AR app where you have a lot of assets if every shoe is 100 K polygons that's too much and if you want to show it in an online shop you will want this asset to be loaded in less than three seconds so this is basically the same story as if you are a professional photographer you have a really good camera take a raw image and then depending on where you want to use it and a catalog or on a web page you will want to have a different resolution in a different format right so for 3d we have automated the process of like mesh reduction simplification and so on using our rapid compact software and this is only one part like reducing the resolution of the actual mesh and textures and then the second part is compressing this information for efficient transmission and storage which is exactly the job that like gltf does in this context so JPEG for 3d fits quite well and we have draco on for quite a while now and as we have seen there's something new coming up which is basis compression via Katie x24 textures and we have performed a small experiment with the babylon team that provided the engine and render us or tweaked version of babylon j s and we have evaluated different aspects like quality and performance and here i am just showing a close up and you can see more if you talk to me later or come by at our booth so this is an in like inset of the like cloth model that you just saw on the previous slide close up and you can see that there are some blocking artifacts that we get through the basis compression which is not surprising but we really found that with a reasonable resolution these will almost be invisible because you will not never zoom in that closely in the final application of this cloth at least and regarding the asset size so if we reduced the original obj by half and measuring texture resolution with rapid compact we and just convert it to plain gltf we go down from 16 megabytes to less than 4 megabytes which is already good but then if we apply basis compression by KTX and draco for the geometry on top of that we go from a four megabyte gltf to 1.5 Meg which is I think pretty great and as you can see in this case the texture on sighs so this is transmission size not GPU footprint was reduced by half just by applying basis compression through KTX to and here's another example the well-known flight helmet and this one has already a low resolution geometry which is quite optimized and you can see we can gain a lot going from say PNG to JPEG from 50 to 12 megabyte and then again from 12 to 6 if we apply KTX so basis compression through KTX - and i should say that it's a bit apples and oranges because it depends on a compression parameter so we have used 90% our quality setting for the JPEG elibrary for the base color for example and we have used 128 of 255 as the quality setting it's a bite for the basis in here and I think if you're interested talk to me and I can tell you the details so we also had a look at encode and decode time so the decode time is really neglect Abel can directly push it to the GPU but the encode time differs quite a bit like if we compress these textures to JPEG and store them it takes us one second with KTX it took us over a minute and this despite the fact that we already ready multi-threaded the basis compression so I think there's maybe a bit room for improvement although it's acceptable that KTX were just like encoding with bases we'll just take a bit longer I think this is not going to change and I would like to conclude this now and just point you to a case study that we're just performing so if you're working in e-commerce and you have assets like this come talk to us or visit T 2g 3d comm /e comb and we will convert some of your assets you can use them for arbitrary purposes we'll provide you with a report and we would just like to get your feedback so if you're interested in automating workflows like that you should check it out and I would like to thank the monopoly team for having created basis which is a regreat texture super compression scheme and obviously very useful I would like to thank the KTX tools team from Kronos around mark hello for providing the base SDK and the Babylon Jas team around Gary sue and Trevor Baron for having worked with us on his case study so thanks a lot [Applause] [Music] so we both work at uber you might have used some of our frameworks will be presented to them before like cabe related the gloma or nebula ZL there's a role on github we recently started using gltf we made the blog post explaining the details and what we did so if this presentation doesn't cover everything you can go read that in it's in the Cronus blog so we have all these frameworks where you can draw lines points circles any any basic geometric shape you have to represent data so this is an example where we have airplanes going in and out of airports and we represent those with circles and it's okay but you don't really know this is airplanes and until I tell you so so we thought okay how about we we use it the LTF model to do represent that so we went online and until TF is like everywhere so so there's so many sites where you can just download the LTF models so we picked one we imported into blender blender is really awesome for important and exporting gltf and we did some small edits for example we put our visualization logo there at the fraud and at the back and it's a little bit more awesome so when we import this you get something like this you have all the airplanes rendered us as the LTF models and then I made an animation here where you can see them come in there are three airports in this picture there is SFO Oakland and San Jose and this is a time-lapse with using real data where we have this this demo in our repository you can just download and run it and it pulls real data so this is a real airplanes coming and landing in in SFO so how did this happen we first started by taking the reference PBR Seder in converting it into a lama GL Seder which works very well with to your coordinate projection system so you can put them on him up and we also take the same hymn viewer and converted all the codes to luma sin and by doing this it was very easy to have the photorealistic rendering and have Diesel's similar to the other apps we also have the editing framework nebula GL in and we did this demo where you can use nebula GL to create get gear json points and you can actually render a gltf model on those points this is the avocado from from the sames and you can see that it's all 360 you can place the points in the avocado syrup so you know if these are good places to buy avocados you can just render them on a map is very nice and all this is like 80 lines of code and we have even better plans for the future like what we have support so you can experience all that in beyond so I'm going to talk about a new member of the BCL suite of visualization framework so that's loader STL and it all sprang out of the work we did to integrate gltf into the this GL sweet and so loaders GL is a suite of framework agnostic loaders for a data visualization I'm gonna talk more about that so when we started out working on this we kind of quickly saw the need for a portable gltf parser as george's mentioned rendered the the rendering in VL TF was actually surprisingly easy and this was thanks to the fact that Cronus has published a PBR a shader reference implementation that we were able to simply porting Ustad multiple light sources to but the passing of the deal TF data required a lot more effort and if you have looked at the format or it maybe even implemented it you know that it's a binary format that needs to be unpacked into several variants you have to deal with you know integrating the Drako encoder and you know it's if not harder at least fiddly and and takes a lot of discipline and so as we went through all of this work we realized that all the resulting code was web the WebGL framework independent wouldn't it be great if there was a in a reference implementation that we could have used to you know when we started with this as we basically published what we did there's a new module loaders gltf and it basically you know you point it at a URL and it returns a you know a file JavaScript object that has all the typed array views and then you can easily put that into your WebGL framework of choice and so this has grown quite a bit solo Terrestrial now has you know a growing family of loaders these are combination of folks from open source projects like three J's plus a plus IO and so on some of them are newly written by us and we're thinking about this as a way to possibly increase the community collaborations this one example we have others but you know the way we think about it WebGL is for you know practical purposes of multi framework world it's great lots of innovation but when it comes to things like you know tables takes features like gltf we all sit and duplicate the same effort and we you know as a result we're less innovative and unique and so we were just wondering other pieces like loaders that we can factor out the frameworks and share between you know groups and companies and and in the ecosystem and so this is really the day of loaders GL and what we're super excited about is that we have worked with cesium and they have also you know they like this idea very much and so we're doing a first collaboration with loaders jl will be bringing a portable implementation of the 3d tiles standard with both uber and cesium engineers contributing so this is an early screenshot and obviously 3d tiles depends you know builds on top of those gltf and draco and so it's been incredibly helpful that we're able to leverage those technologies we are looking for more potential partnerships and you know basically if you have any interest in this yes that means you please contact us good morning my name is Mona special I work as a GPU soft engineer for UX 3 in Munich and I'm going to talk to you about our plans with TL TF in the following months so first up we are currently developing a middleware that is gltf native and runners would welcome so it's a voltage multi platform run run that is low profile and can be embedded into your existing applications it is also gltf consistent meaning that we can guarantee that the output of the middleware will be the same as that of the sample year that is all thanks to chroma specifications for example IDL function lights and PDF parameters next up you probably are most of you are probably familiar with the gltf tools extension provisions to decode you think that it's a great way for developers to work with gltf and adjacent format of course some sort of knowledge about the goty specification is necessary to use we and house users for example to debug develop new extensions and tools and also example blender exporter but of course there are people that are not as accustomed to working with text editors and would much rather work an entire application so that's exactly what we are currently also developing and so gltf editor especially made for our creators so you can use a property inspector and edit properties of tio2 F files and then it gets live updated in the viewport and that all works energy OTF native environment so what that means is that any action you take in the editor always produces a valid gltf and you can play around and always gonna be safe we support all crona's extensions and that will run on Mac OS Linux and windows in the gltf pipeline and the existing one we see that it fits right in after modeling programs so as usual your models would be created on Macs blender Maya and the GOG obscene editor or are you expedient editor will then be used to for example inspect and verify the pious gltf false make modifications on them if something goes wrong during export it can be used to optimize these gltf files and also of course to compose multiple assets so for example if you have multiple models and want a compulsion to a bigger scene you can use the scene editor for that and when you're finished of course everything gets exported to glt african so these files can be used in the common renderers and engines so we will make those all available in early 2020 as a public matter and people have a closed alpha and beta program later this year so if you're interested in that just feel free to email us yeah thanks that's it yeah hello and welcome everybody my name is David and I work at s3 and I'm talking I will be talking about how we use tree OTF in our platform and so first a 3a for those of you who don't know it's like a company which is doing which is most known for software called arc Jia's Pro and it's the software for doing em geospatial data authoring analysis and visualization but now I see has grown much more and it has a really wide portfolio of products and a platform for bringing data to the web and it's it's a vast system and part of that is CGS API which is about visualizing and analyzing to some extent the data on the browser and this is a team I'm working with so as I mentioned artists prod and also their City engine this is another product that we procedurally generate cities and all these different products they of the general day travel which is shared across all these products so gif they are for is for us it's really interesting and it's becoming now in a really important part of it and one part of this said we have this thing called Maps tiles you can think of it like a clipart library and so we have icons but also we have like 3d models and people and our clients they use these 3d models in our chairs Pro to propagate the scene like everything like like dress setting in real effects kind of and we now we were using different data formats to represent these files internally and now we are switching to G RTF and so and this is really cool for us because it allows our products to grow and functionality so now for example we can allow users to bring in custom content much more easily so actually has Pro is able to import G RTF city engine will import and export the area TF n ggas API is able to reach the earth here as well so and here i am i have just them this demo let me now show how we can basically am build more compelling applications for users so this is m demo application may be sure how citizens can basically propose things to urban development and basically we have this thing that we can basically now use the Sketchup which is awesome and basically we can browse the the content and like bring in oh that's interesting her to check that okay so anyways so basically it will import the model and then you can basically place it at the scene so and these use cases are really good path and also being able to do physically based rendering it's also very attractive because we are now also moving to rendering I will be talking about this in the WebGL session and now I but most important for us is now a level of teacher because we have these best scenes and we use basically three at F to propagate the scenes and level of detail is an extension tutorial TF which we use to basically be able to render vast amounts of symbols on on the on a map so here you can basically see that we have all the trees and all the trees they have level of detail and all the cars and everything and what also is really cool for us is Satya TF m allows to store metadata so for example for the trees we store metadata how to for example to do like some fake ambient occlusion inside the tree and this metadata we basically this is not this is what we will be doing because we store this may be updated ever in a different way right now but we will be moving towards drawing additional metadata like this and this is also and probably I would like to talk about that I think we would like to see better tooling around a lot level of detail so here we have this image and what we see here that if you compute the level of detail levels and the threshold levels at which you want to switch between deed between level of detail then if you compute is automatically then you may get scenes very I really have a level of data in places where you don't want to have it like it's much switching too early for some for some assets and what we did is basically we so we we analyze or we kind of evil like the level of details for our tree RDF models and so we have these these thumbnails where we can see how the the model renders on top of the pixels it actually occupies depending on the distance from the camera so and then basically we have this thing that we can very very basically can tune the LOD levels of our model so here you see basically the model at different distances and and then at each distance you basically have different depending on to distance in a different LOD level and here basically we changed the LOD level and and this is basically what we do to internally tune all these elytis thresholds and this is maybe an idea for fidelity tuning also what is also interesting about the extension is that it's basically the LOD thresholds are given in screen space coverage and which is like independent of resolution but we actually use a different metric which accounts for resolution so if you change the resolution in the browser our energy levels will switch so and this is something we also would like to see too because we cannot use the the thresholds in gltf right now because we have a different metric so this would also be something to discuss and in general we are really excited about this possibility to to store metadata so as I already showed you the tree has the metadata for the lighting and but also we are thinking now about like because we have city engine for procedural content creation so we with the trees for example we are now planning to store things like the humidity and the amounts of a set the tree needs to grow in certain areas and this is really interesting because now we can see both sea use cases where we say okay give me all trees of from from sketchfab which may which can basically a grow in in my in my desert environment for example so you specify the environment and then you get models which are able to grow at the moment this is this is something we would think is really useful and finally being able to sort of end me now first potential death model we assume a certain coordinate system and yeah but for GIS data if you if you want to really be precise and want to make sure that your positions match up you need to be really careful about the projections and the coordinate systems and we also would like to see that the metadata there's maybe a standard extension for for brings this information visit er their models and that completes my talk thank you [Music] [Applause] hi everyone I'm al bonk hundred sketchfab looks like I don't need to do my demo anymore since David just made so for those who are not familiar skate 5 the platform to publish and find 3d models on the internet over the years we've become the largest library of free content and what's interesting beyond the sheer volume of content is that it's extremely diverse you can find very low poly content but also very high poly some of these static some of it is animated some of it is made with professional tools like 3ds max I know it is made with Freddy captured holes both photogrammetry and depth sensors some of it is PBR some of it is not it comes from a community of very diverse use cases and backgrounds over the years we've been mostly known for uploading and publishing content and so we passed skate 5 7 years old we've passed most at that time amazing content and we're already just entering phase 2 of redistributing such content where it needs to be used and we are using gltf as main vehicle to do that we support more than 50 formats as input and then we convert to our own formats and then back to gltf were able to support PBR animations and so on the main use case initially for downloading content from sketchfab have been via fix and game dev but also more and more conversion which is interestingly with the multiple simplification say that viewers and add sets more more people users just to get a clinch LCF for their own native pipelines and then obviously a yarn we are we have free entry points to get content from sketchfab the first one is a community on skype fab.com we have a community of almost 3 million members and we provide free Creative Commons content then a year ago we launched store where you can buy and sell content it's a great way for community members to monetize their work and for people who need content to get higher quality content royalty free of use and then we have our download API which lets you integrate a sketchfab library wherever you may need it quick numbers as a cc library recently passed 200,000 freely models and so I think it's the largest library of LGF assets and the largest library of free DCC models so store just best fifty thousand assets also all available in gltf and we have about 500,000 downloads a month from about 100,000 unique downloaders and so we distribute the content in in two ways for our API one is building our own importers historically we've built a lot of exporters that you could publish from like blender or Freddy's Mac's straight to sketchfab and now we're closing the loop and letting you import from the same tools our vision is that all our schedule patterns will be a two-way street so you can both export and import export to scat have an import from sketch having all those tools we've just released see for the importer that you can announce tomorrow and we've also just added a feature to import your own models so here is just a quick demo of the unity importer and so you install the sketch fiber and for unity it's above export and import and then it pops up sketchfab browse straight within unity you can login to your Skype account you can filter by just your own models and then you're going to find all the sketch fab filters right to social layer like the lights also like staff picks as a poly count then you can search for whatever you want and it's going to pull up the entire CC library and then in one click you important model and it's going to show up in unity just as it shows up in sketchfab with like keep your materials animations if it's animated and so it's normally pretty fast it's frustrating unity and then you also have a set of native integrations which means Sketchup library shipping with the software we have close to 20 on more than that actually native integrations I just put some of the bigger ones two biggest ones in terms of volume volume usage are modular herbs and spark a are and so typically we will power the library of spark AR and so if you don't have assets you just click on yeah a our library and same thing is gonna pop up the Sketchup library and here I'm searching for my own models I have this beautiful bow tie then I can import in Sparky are in very easily deploy in IR filter in Instagram or Facebook and then on the right it's just an example of you see woodsy lav Europe's platform it's all web-based and same thing I can pull up the sketch February search for anything and then bring on things and and invite people and that's it thank you hi everyone my name is Adrian I'm from Google I'm an old salt from the game industry and I'm here on behalf of the team that released at i/o the view in 3d view and your space capability and search we were a pretty good match when that feature needed doing we were the team that made scene form which released a little over a year ago see form had a lot of challenges using trying to use traditional file formats like FBX and obj and and gltf has been just a revolution for us and it's wonderful so a scene form for a little bit of context is a means for Android developers to add their activities to their existing applications has a very high-level Java API for scene management and a tool pipeline and a plug-in that's available on all bestop platforms all of them built on top of filament which is totally awesome and she's seen form has been moderately successful it has a lot of has a lot of first party a third party clients Google Maps they are walking Neve uses it for example but boy we've had a tremendous amount of trouble trying to get by giving people a workflow to take some obj that they found on the internet or FBX and just get it into a PBR renderer you can't like obj is super subjective and I mean if you have stingray materials in an FBX you can get by but you if you're getting content from the industrial design division they don't know what stingray is it's it's a real hassle and on top of that loading FBX and obj on device is especially on like on mobile platforms it's not practical we don't have the ability to get live FBX Sun on Android so what we ended up doing was making a a packaged file that would be computed at import time it would be packaged into your apk or downloaded offline and then you would render that but like that still had a lot of problems because people don't really want to take some custom format from some custom tool chain and throw it in their CDN they just want to have they want to have what gltf is they want to have a known quantity and they throw thousands of it into this thing and then they serve it someplace else so when we got the opportunity to make to add gltf viewing support to search it really solved us so many problems for us we don't have to have a packed in file anymore we don't have to have a binary a binary archive that's tied to the filament version so if you were trying to deploy two mobile devices you need that like have every version that you've ever released available for sending we don't need to do any of that we just send the GLB we have a handful of pre-compiled shaders that we download once that cover the permutation space well enough we built a fuzz sandbox loader which if you've never done is totally fun and you should do it and gives you a lot of cred and it's been it's been really great for the next steps for gltf at Google we're going to be taking the sandbox buzzed loader that we made for a our search we're going to put it into scene form so that all of our existing clients can load gltf five or ten times faster than they can currently using our java pipeline that has to go round trip to our intermediate format on device we're going to be working with partners that were excited about if I had to pick one ambiguity that that would that we're dealing with the filament uses like a glass transparency model and the that there are certain we're interested in in in additions to the gltf spec that cover glass transparency we have some corner cases with our implementation but we have like this known quantity that's small and fast and could deploy it pretty much anywhere and it can render gltf sin in filament and gltf has made my life a lot easier and i'm really grateful for it and think that's all i had thank you [Applause] while we wait for this presentation load up give me another round of applause for all the people who've been contributing to gltf it's been amazing like it's been incredible head to us I'm from wafer and we you have like tens of thousands of gltf assets and it's been a lifesaver for kind of not worrying about cross spam platform issues but today I'm going to talk about a new initiative called 3d commerce and geo the up kind of does a good job with you know trying to figure out one format or figuring out consistency issues but we need to kind of do a little more than that so I'm feeling so dodgy I I run an R&D team at Wayfarer and I'm also the chair of 3d commerce group and so how many of you used a 3d experience in your shopping process like a viewing room like IKEA place has or way for everyone or Target cool that's that's pretty good how many of you be responsible building one of these experiences like ok that's that's good it's fine like there are more people here they're building these experiences than actually using them which is you know since we're all I guess developers but one of the issues so if you think of like number of 3d experiences are out there it's rapidly growing in the retail space I think 3d is evolving as a new merchandising medium for products for shopping right and so when you think of all the different experiences are out there like listen it started off with like a 3d configurator that was available online then you have these AR apps now you have I guess VR platforms that are kind of helping you shop and then you have now you have ads that are showing you 3d models of products you have like Google just showed the search results that are kind of powered by they're showing you the products in 3d and you can you can use AR to see these products in your space you have Facebook ads you have Facebook's little posts so it's growing like 3d as a medium is definitely growing and one of the issues there is that you think of like the simple use case where let's say like I have a product like a blue couch right that I built a model for as as a retailer and I use like a tool vendor like a 3d tool when there to to build this model and then I see I use it I use like Google's platform or apples background whatever to kind of show the product in your space right that's great now what if efforts take this model and hand it over to a social networking platform or an advertiser to show this model as an ad and when you're seeing these models you're seeing different retailers different model from different retailers at the same time right how do i as a retailer gain the confidence that my product is actually going to look exactly the way I intended when I built the product itself so that is the problem that it resolved how do we kind of align all of these different people in the 3d industry it kind of come together and work on a set of guidelines a set of standards to make sure that the make sure that we can resolve all these issues that we are facing today so so it all started with we initially basically we got together like last September most of the retailers and some of the technology platforms and we kind of sat together when you try to figure out like hey this is we try to identify that this is a problem right so once we did that we Kron was basically kind of helped us out and we we we created an exploratory group in April of this year and then today we're basically we've launched this as a full-fledged working group that is going to solve this problem and we have over 70 companies that are a part of this effort you might recognize most of these companies here and so it's it's it's a pretty interesting I guess oops it's a pretty interesting it's different because I think one of the things that's happening now is that we're actually can connecting the end user the retailer's to to the to the technology platform just kind of hasn't done has been beaten hasn't been done before so we're hoping to find hoping that this kind of leads some good results and we can actually solve this issue so we think of like what exactly we're going to go after right one of the biggest things we wanna go after is the consistency issue like that how can I be confident that my blue couch looks like a blue couch on any device on any platform right the other thing is if I have to if I if I see this one couch or one shoe or one shirt or whatever is if it's available in different colors or different options different sizes how do how does a third party reproduce these different combinations in the right manner that the retailer initially intended to so that's another kind of area of work the another thing is how do we actually show information about the product in a consistent manner so let's say there's you know you have a name of the product you have a price you have like a like an image or a Texas description how does how do we standardize that matter information that goes along each 3d product that shows up correctly on the end-user platform that's what money really the other thing you wanna do is I want to make it easier for people to create this content right it's not enough to kind of just say like sure this is the right way to do it here is a set of specification here are set of standards that you can that you can kinda just release and expect people to follow it right that's not try to pressure I guess we have to figure out how to teach people to create these assets in the right way so that's another area of work for us so overall I think we have we're hoping to accelerate the adoption of 3d in the commerce industry and in general so and so I don't think we're gonna do this on our own so we're we're basically going to partner with the rest of the Chronos working group such as like gltf or WebGL open X or Vulcan open VX to basically I think it's it's we have to partner with these with these different working groups to figure out a solution so we don't be working very closely with these groups to create a solution so at the end I guess M if you're interested in this effort and if you think this is a problem that you want to you know you're facing and you want to be a part of the solution then definitely just reach out to us help us kind of figure this out this is an important point in the in the industry today where we're basically there's a huge opportunity for using 3d as this new merchandising medium and if we all work together and we all align and I think we can get there faster thank you [Applause] [Music]

Channel: The Khronos Group

Views: 9,429

Keywords: Khronos Group, glTF, Sketchfab, Google, Wayfair, 3D, 3D Commerce, Commerce, Esri, UX3D, Uber, DGG, BabylonJS, Cesium, AGI, Facebook

Id: _MOp_SfGJx8

Length: 57min 14sec (3434 seconds)

Published: Tue Aug 06 2019