2017 glTF Webinar - Feb 14th - The Khronos Group

hi everybody thank you for joining us for the JLT f weather nice my name is Kathleen Madson I do marketing for crannis and I just want to go over a few notes before we get started first we are making a video of this so you don't have to take the million notes you can just review the video and you will receive an email and a couple of hours telling you what if you got it download that next we are very interested in any questions you might have and we have set aside space at the end of this webinar to answer your questions what you can do during the webinar if you find that you have a question look for the question mark icon on your GoToWebinar panel click that type in your question and then at the end of the webinar we will read and answer all of those questions all right well that's it I'm going to turn you over now to Margo header and he will do the presentation Thanks value-wise marker oh thanks for the introduction and welcome to the gltf webinar we'll start with a short introduction about what gif is and what it is developed or who first of all Grif is the three earth transmission format it's an open standard developed by a Chronos and it's designed for the efficient transfer of 3d assets over the web mainly over the web this imposes several requirements for the format it should be versatile to represent different different kinds of 3d assets should be compact for an easy and efficient transmission and it should be easy to process is process it on client side in this image you can see that it's sometimes referred to as a JPEG for 3d data the idea is to develop a format that is universally accepted by what brought variants of clients so that you have one format that you can distribute and can be sure that every client has a capability to me view and treat this 3d asset a short outline of the content of this presentation and it's roughly based on a tutorial series that will be published on the link you see on the slide the rough structure of this tutorial is to to explain the concepts that can be represented with appear here as it and it's a step-by-step tutorial so it's really showing each each feature of gltf within a small simple actual reality as it so there are examples for simple geometry and Nations materials textures and vertex pinning and you can have a look at each of these functionalities or features of gltf in these simple examples and this tutorial is targeting GF 2 dot oh that's that's really cutting edges are just being finalized can see a link to the pull request for the order VL tf2 specification here and in order to to make clear what what the purpose of gears is is let's say look at the the content pipeline as it is now or today there are several different data sources and for example the Navy laser scanners or other forms of 3d scanners that produce geometry data usually raw geometry data consisting of possibly millions of vertices and edges and faces and there are different authoring applications that produce lots of figures sophisticated 3d scenes with additional features for example animations or lights cameras and sophisticated materials and all these different data sources store their 3d assets in different file formats for example for raw 3d data there are obj files or STL files and the authoring applications also have their dedicated file formats lender nya and 3ds max for example and on the bottom you can see different runtime applications applications that are supposed to to show 3d assets 3d data and these applications are built on different graphics API Smith on one hand the family of of TL a B is OpenGL WebGL and OpenGL ES but also work on or DirectX and now the big question mark is how are these 3d assets brought to the applications that's one solution that is currently used and that has several of X the common strategy is now to create importers and converters so every application select the set of file formats that it wants to support and as you write an importer or in converter for any existing file format to be able to lead and please play with 3d asset and because this does not really scale well we cannot write converters between all pairs of possible file formats and additionally most of these file formats are not intended to be pure delivery formats for 3d applications and now we see how it could look like with gltf if and a different set of converters for file formats to gltf and for example for the authoring applications there is already a common performer colada and this can also be converted to gltf and PRT F is the format that is intended for the last mile for really delivering a 3d asset to a client to a to an application that is supposed to only show this 3d asset it's not a general and exchange format for this purpose there are things like COLLADA that's really an exchange format for different offering applications where Colada tries to to retain as much of the scene and structure information of a 3d effort as possible whereas three RTF is really intended to deliver to the client what is necessary to the state 3d effort so this comes together with several goals during the development of clear here as already mentioned before it should be versatile compact and easy to parse and on this slide can see an example for example this this plain geometry this heavy model example as an obj file it provides 90 Nega bytes and that's that's really a lot you can can hardly transfer this over the web in real time so the reason for that is that obj is not intended as a delivery format it's a compact representation but it still stores the geometry data as plain text that it has to be parsed to and client-side and convert it into the form of 3d data that can actually be used for rendering contrast to that the same effort as a DL TFS it contains the consists of only 19 new device similarly you can describe complex scenes with gltf on the right hand side can be a example that is as a colada file it has 5.3 megabytes and the GOC FS if it's less than a half of that so g OT f is capable of representing plane geometry in a very compact form as a less complex into the animations and materials and really complex 3d ethics and in both cases the asset can easily be consumed by clients so the core of BL TF is a JSON file that basically describes the same structure and the entities that appear in the asset and the geometry data the actual geometry data and vertex attribute is thought in binary form so there's no decoding necessary on client side the client can just read the data and use it for rendering as it is so here's an example of the adaptive the level of adaption of gltf that already exists there are already different libraries several different libraries that support reality f is one of their input formats for example free J assets very common bubble Andrei s primordial cesium field real and GL booms and all these already natively support Realty fs1 delivery formed for the assets there's also a link to a comparison of different orders and viewers we can see how well each of these libraries support the different gltf features additionally BFS reach the broad support or acceptance in the industry there are already different applications that employ GL CS one of their presentation formats or as their main presentation format and it really has a large large scope of different possible application cases for example you can imagine that it's used for all figurations or product presentation or as a format for general 3d asset libraries for example for cultural heritage digitization or can imagine a museum or an exhibition of scanned scanned cultural heritage artifacts that are displayed which can view online the browser can also be used as the basis for for example architecture planning we can combine different elements of housing or furniture and interior design to to plan the layout of of thousands of let's it's also used in view information system with helium and we can use the 3d effort as a basis for for City Planning and general view information systems and BioDigital is using gltf as a presentation format for for educational purposes in there in the area of medical education and health education and additionally there is a large list of supporters who have announced the part of gltf so you can see that GAF really has a great impact on the industry in its on its best way to become a form of edit that is generally accepted by different client and supported by different different applications we recently using the best work wordpress plugin for GRE episodes hopefully it will soon be really as broadly accepted as JTAG for example coming to the technical part the basic idea of quixote f or the basic file structure of a gr TF asset is that the core is represented as a JSON file at the main reality F file and it contains the information about the scene structure and entities that appear in this scene for example cameras innovations possibly lights sooner or later and this element these entities refer to for example geometry data or image data and these are represented as links to external files so the binary geometry data is stored in binary files and the images are usually stored as jpg or PNG files because they are already compressed and really compact and easy to decode on client side additionally external resources can be embedded into the Jason s data uris then it's called an embedded gltf file this has the advantage that you only have one json file that you have to transfer without any external references comes at the cost of a slightly larger file size but may be advantageous in some situations basic structure of the jason part is really stable it consists of several entities that are shown on this slide you can see basic scene structure consisting of nodes and these nodes form of scene hierarchy income to net infinite and lectures and cameras are attached to these nodes I will not go through all these elements right now but they will later be explained in detail you can see on this slide that there are really few basic elements that are used for assembling or if you can't complex scenes as well as simple simple plain files with geometry in them here is an example of the VLTs effort this is supposed to be the minimal gltf effort that can exist at all it's a simple triangle so you can see that there's a complete valid TLC ffo on a single slide can actually copy and paste this into a viewer and it should be able to display it so gif - dodo dammit yet many viewers but we hope that there will be broader support Lucy coming back to the contents of the actual Jason part I already mentioned that there's a basic structure described we are notes and these notes define a hierarchy you can imagine if you live in a small sort of see graph really a basic scene structure and each of these nodes can have a local transform that can either be given ever as a matrix or as translation rotation and scale so we can concatenate different nodes to achieve for example moving groups of geometry data or you can also attach cameras to these nodes and with the local transforms of the nodes you can represent camera flights for example this is the basic scene structure and now the elements that are attached to the nose there are for example the cameras different different forms of cameras are supported here you can see an example of a perspective camera and an auto graphic camera they are both attached to notes and are observing observing a mesh that is attached attached to another node you can see the rendering of the resulting rendering of this box in the lower part where the perspective camera has a view frustum that causes a perspective distortion and the autographic emerges to be an orthographic projection actually the only three data it is if start in buffers in binary buffers that are already mentioned at the beginning these buffers are stored in external files in binary form so the account can directly be loaded by the client and this raw data for itself does not have an inherent structure or meaning it's just rather nary data and the interpretation of this role binary data is accomplished online site by using buffer views and successors both of you just defined part of a buffer just cuts out the slice of the buffer and the accessor defines the data type and layout of the data that is contained in the buffer view for example whether it's 3d vectors of float floating point values or your ABCs as integer values for example and the advantage here is that you can take the data from the buffer view and send it to the graphics card and the result will appear on the screen it directly in the format that will be used for example in the calls to GL vertex every pointer so you don't have to decode or parse or interpreted the data in any way you just read it as it is and can send it directly to the graphics card which is very important for client in order to to efficiently display the 3d assets and another exams of the evidence that may appear in the scene is animation an animation basically reads data from an accessor in this example you can see an X so for the times which are keyframe x and an accessor for the rotations these are routine cretons describing the rotation of an object at the active keyframe time so and the example shows the rotating triangle that is consists of a simple mesh attached to a node and the animation read the input data from the accessors and forwards this information interpolate it accordingly to the rotation property of a node result will induce you onto the rotating triangle you can imagine that again forward animation information to the two different properties of a node and therefore model more complex animations but this is just intended as a basic example showing the concept of an it in May issue reading from a class off and following the keyframe the interpolated keyframe values to the rotation property of a node the actual representation of the geometry data and a gltf as it is also based on access or in this case an example shows the physics also shows a simple mesh which mesh consists of a set of mesh primitives and these mesh primitives assemble a set of accessories that provide the vertex attributes for example here is a as shown a triangle that has true attributes position and role in texture texture coordinates attributes and if access our data there are accessed via indices so example for example you can have an index mash or and mesh without indices and each of these successors provides the data for one attribute and again this is the data in a form that can directly we sent to OpenGL or to any other graphics API because it's just the raw data consisting of floating-point values already in a format suitable for being sent to the underlying graphics API so again there's no decoding involved just can be sent to to the rendering engine in this case and more complex topic law materials and each Nash connects primitives can refer to the material which Engram it shows the material that the respective mesh primitives should be rendered with and the material itself only solve basic set of parameters and there are different possible parameter sets in the example you should have you see some spheres that are different renders with different parameters and these parameters in this example are the metallic nerf and the roughness this is based on a physically based material description and this physically based rendering will be part of gltf 2.0 and the interesting thing is that this is the first time that physically based rendering his own way of being standardized in a coordinated effort this really the first time that that a group of experts came together in order to standardize the representation the underlying representation for physically based rendering there are different possible workflows for representing the physical properties of materials for rendering and gltf now defines one standard representation that is supposed to be supported by by a broad range of clients so you can have one effort and transfer it to different clients and with this very compact material representation you can achieve the same rendered result for all the clients I think this is really the first time that such a standardization effort for physically based rendering has been undertaken for for legacy or you shouldn't call it legacy but for for existing OpenGL based or WebGL based implementations of renderers there's also concept of K weeks this is for DL he F to know this will be an extension it's specific specifically designed for rendering using the GL family of AP is WebGL OpenGL or be lef in this case materials are represented with techniques the techniques are description of general description of materials so that you fast forward that is summarized on this slide is that material can be considered as an instance the technique the technique refers to the program which is period and that OpenGL program and the program refers to GLSL shaders and I have an example showing what the technique actually comprises the technique contains a set of parameters and these parameters corresponds to the attribute uniforms for GLSL shaders so each technique contains a description of the inputs for the GLSL program containing information about the data type the data layout for example in this example you can see the positions which are 3d vectors the colors which are forty vectors or the light direction which is single 3d vector and a description of these parameters is given in the technique so the technique is the description of these parameters that can be read programmatically can look up the technique information in the JSON file and then you know how to how to set the inputs for your shader program short side note the shaders react knowing the actual GLSL shaders are stored in an external file similarly to the binary data and image data so these external G&L shaders can be read into strings and directly compiled with open vm the concept of materials and weeks is that i mentioned before there materials are instances of techniques so one technique defines a set of parameters and each material is then an instance of this technique where several parameters values are fixed I've added an example here can what have the same technique and one material where the parameter worth values are the three positions a color that represents an RGB a color red color and certain light direction and many rules using this material foreign as primitive rendering less primitive then the result will be a left field similarly you can have a material that is an influence of the same technique but the input values for the positions are no positions of the Box a different color blue in this case and different light direction and can see that using the same technique and instantiating different materials with that you can render completely different objects using the same shader programs and the changes same GLSL implementations another element that may be contained in a the Jason yeah a reference to an external image file is the texture a texture is basically the input for center to be uniform variable in the context of technique based rendering of energy an excel-based rendering and each technique refers to an image the images are stored in external files and can be read and then the image is used as one input for the technique so the technique parameter says that the input type required there is a textural center to the input and one can instantiate a material and assign a certain image as this input and on the right hand side you will see the resulting rendering the textured square in this example somewhat more complex topic is vertex feeling this is the point where multiple elements of G on TFS come together I try to summarize the basic idea of vertex pinning on this slide the basic idea is to have a skeleton consisting of joints and the implicit connections between these joints are considered as the nodes of the skeleton you can define the skeleton hierarchy using use any nodes of the same concept of nodes that is also used for structuring the scene and this skeleton is then combined with a mesh with the vertex positions of a mesh basically to obtain something that is called a skin mesh when can imagine that the vertices are attached to the skeleton based on certain rules and the result is that when you apply transformations to the joint nodes then the skeleton will deform and the vertices will be moved accordingly so this is for example used for character animation or far more sophisticated deformations that are not just rigid body transformations try to explode the the the elements that are required for projects in here in little more detail in order to to define the rules that are used for deforming the vertices based on the skeleton pose each vertex needs information about which joins affect this vertex and how strongly these joints will affect the vertex and this is accomplished with also with accessor data mainly with joins and writes information so on the left hand side you can see that the right how strongly each joint should affect the vertex are known for joint zero for the upper vertex and high for joint zero for the lower vertex along the right hand side you can see that the influence that joint one has for the upper vertex is high and the influence that joint one have for the lower vertex flow and the effect will be that when you apply a transformation to the to the skeleton bones or to do the to the Royal nodes that are the joint between the skeleton bones then there the next geometry will move accordingly together with the skeleton and the upper right hand you can see an example this is also one of the examples may be now using the tutorial for explaining all this more detail but the basic idea is that you can describe more complex character animations in a GL CF asset using vertex channel is a general overview of different resources for ESP F main resource is basically the landing page by corners and the github repository where you can find all the most important links there's also a repository containing the central models there are several simple models for learning the basic concepts and they will also include the central models that are used in the tutorial where you can examine each concept under having minimal example identity for each concept but there are also more complex models real world examples for testing for example your your notice of us there's an effort validator that is online on github and there's also a - an online tool where you can simply drag and drop your efforts to make sure that they comply to the specification and additionally there are several converters for existing file formats to convert them to gltf most importantly maybe colada for voice converting when you have an offering application that like blender or three years next then you can always export colada files and with colada - gltf you can convert these colada finds into gltf assets there's also converter for obj files because that's a very common export format for chain geometry data and you can also just drag and drop obj file into an online converter and convert it to gltf this will be the link for this online converters also mentioned there's the provided bias idiom now this also includes viewer so it can directly drag and drop your peers your Colada files for example and see a preview of the result engine at the FSA to also download the resulting gee LTFS when you want to get started Grif there's no some impossible roadmap by here sorted by increasing complexity or difficulty the basic overview can be acquired by sign is created owner and although you image roughening the style of the Chronos reference pages where you can see the connections between the entities that they may appear in the VAF effort and what each entity consists of and how it's connected to other entities it's really intended as a rough overview then you can explore each of these concepts based on the sample model so when your example particularly interested in vertex pinning when you can have to look at the skinning skill examples that are shown in the central models repository and the details of these concepts will then be explained in the tutorial we can that are based on the latest example models and when you really want to know the details then you can look them up in the specification where the actual Jason's format is specified and all the valid parameter values and all that when you intend to to write a DCF loader of Uwe then I try to summarize the concept of gltf here roughly sorted from from simple to somewhat higher on the right-hand side a basic scene hierarchy traversal just consisting of the scene denotes is rather simple can just usually recursively traverse the scenes and nodes in the node hierarchy and the the next step would be to process the element the simple elements that are attached to the notes for example the cameras and animations and later you may try to tackle the compound elements where you have to combine the information from level several gltf elements for example meshes and mesh primitives or you have to to establish the connection between between the mesh definition and the buffers that have used an accessor that contain the actual geometry data then the next step would be the rendering it consists of techniques and materials and programs and shaders depending on which which approach you are using for rendering when you intend to write a GL based renderer band we will use probably use the technique extension when you are writing a book on base renderer for example then you will directly use a material definition that contains the parameters for a physical base material in DRC f20 and the most sophisticated example or concept is probably spinning because it combines all the elements of the gltf asset it defines meshes that are deformed based on the skeleton hierarchy this consists of nodes usually people deduce these skeletons will be animated and finally the resulting geometry has to be passed to the render and the form that so the renderer has to be aware of the fact that there's one explaining going on so the dedicated renderer that can interpret it interpret the input information that is provided by the body techniques this case that for example that the weights and weight and joint information for the vertices and that defined the information how to transform the mush that usually happens in the vertex shader as like so the skins really are the most sophisticated concepts appearing here to get started I recommend to have a look at the existing loaders and viewers there's already a long list with different languages JavaScript C++ C sharp go rough taxi and Java and there are already different libraries that offer the basic functionality of loading and unity access it so that you can directly use these libraries when you just want to render the efforts for example when you want to contribute made from GL here from the TSS ecosystem for example you are going to write notice exporters of yours then you can modify US and the loaders or your work will be listed at the main landing page on the github repository you can also contribute sample models that will be added to the official central models repository so that others can use your sample models for for testing certain features you can also write tutorials that will then be published in the Khronos github repository as well and when you're already using gltf then you can drop us a note and show it show us what you have done with gltf already so this concludes my presentation and can have the the fear set of final links here the general landing page when we want to download the specification or tutorials can sign up for the chromeless newsletter to always informed and up-to-date with the latest developments can have a link look at the progress members page learn about the details of becoming a Cronus number when you have questions about the slides then you can just write to the email address that is given there and finally I would like to thank all the LCF contributors and thank you for your attention and now we at rabbit and Patricks Cody will join to answer your questions thank you very much Marco that's real that's really great if you don't screw presentation we do have some questions here Neal and Patrick are you guys able to unmute yourself third let's see if I can unmute you there we go all right so go ahead and take it away guys okay well that's I can start there's a couple questions here on the ecosystem and the timelines for 2.0 and when people should switch to 2.0 as opposed to gltf 1.0 so I've actually typed in some answers but just to go through it very T so the plan is to have a final review version of gltf 2.0 at game developer conference that's the GDC coming up in San Francisco in a couple of weeks and the idea is to have a complete final specification and as always it's being developed in the open on Kono's github so you can watch it come into being in real time but at GDC we'll have a finalized version where we're expecting and requesting last round of feedback from the developer community and then the plan is for the working group to finalize 2.0 just as quickly as we can once we have received and integrated that feedback and as a related question there too if you're watching parsers and using gltf should you support 1.0 or 2.0 in general where you think that you two point oh adds a couple of good things for your systems while adding additional functionality like PDI the physically based rendering it does simplify processing actually in a number of places quickly around animation and so in many ways it's is a cleanup in Turkey and easier the process 2.0 in some ways so we're encouraging the position to move towards 2.0 as quickly as we can it's not a lot of work it's two points so we hoping that we can converge on 2.0 as a chemistry pretty fast okay Patrick are you with us are you able to take on the GLB question and a Vulcan question sure so for GLB so that that's currently the binary gltf extension for gltf 1.0 we're actually building a binary gltf into the core 2.0 spec so basically what that means is there's a buffer view for images that is allowing you pull images out of the binary blob as opposed to a separate image file so GL B will be pretty much core in an gltf 2.0 I'm still trying to find the Vulcan question does welcome question that's some anybody saluting gltf miles it's Vulcan couldn't the consistent Gianna so here's a whip with GL GF 2.0 PBR will be core so you want necessarily have to have to deal with the GL GLSL you can generate shaders directly from from PBR I have not written a Vulcan renderer myself but there are a handful that are out an open source so we could if we can post any notes with the webinar here we could we could send those app there also in the readme and the main gltf github repo it may be worth mentioning that many of the changes that have been done for gltf 2.0 has been aiming at a broader or since the support for welcome so many of the changes have been inspired by people who wrote work on based renderers so it might be a little bit simpler for for implementers to adapt to gltf 200 in welcome and the earlier question that we got was about those notes to the note and does a note index start with one what suppose to them now so in Jill Jeff 1.0 the top-level objects were object top-level properties were objects so they all had a unique name and 2.0 they'll be a raise so that because they're faster - faster easier to access and those will be zero zero base to raise alright another question was how would blend shapes be represented for animations of matrixes or matrices the only form of animations supported in the answer to that one Marco do you know off end and absolutely no I'm not not entirely sure what what is referred to with blend shapes I think that there have been some considerations think alexei did some work there but I'm not sure about the unstated middle yeah it may be better to over an issue on the github repo so we have the right person answer this in brief these animations support in gltf is keyframe animation you know for translation rotation and scale and then in 2.0 put an end morph targets for things like facial animation actually the the second part of the question are matrices the only form of animation supported then this refers to to the matrices that are built from the translation rotation and scale properties of a node I don't think that there clear support for for more than that but again I think that Alexei did some work in this regard this thing would probably be to discuss this negative issue so there's been an update to the question here saying it's morphing like heartbeat cycles so yeah so this is part of gltf 2.0 with the morph targets and there's a discussion right now it's pretty close to converging on on the spec and that all public in the github repo excellent thanks guys I'm not a question if it recommended to set custom properties and data values or other specific things for games and apps to refer to the gltf file yes you can so ders you can add an extra Zog to any of the other objects in gltf and then inside that object you can add any properties that you want there application-specific the other thing that you can do is you the nimbin gltf with your own format so you might have like a binary blob that contains gltf and then it contains metadata maybe for some of the objects in it or maybe it contains environment data you can kind of do it either approach is there room in a format for custom properties yeah it's kind of the same question so custom properties can be added to any extras property you can put whatever type of data you want in there whether it's array other objects intrinsic properties and so on okay does PF have support baked animations so I might not be completely as familiar with the terminology but I think this is similar to the to the morph targets where you could have multiple meshes and then blend between them and if I'm wrong please comment here and we'll try to get your edge question answered I can we not use accessories as better here - you mean this question is not entirely clear but these are different concepts the buffer view is only a part of a button and the accessor contains the reference to the buffer view and additional information about the structure of the data that is contained in this part of you little whether this answers the question otherwise it might be rephrased a little bit all right we have one more question are there any examples of a 3d line primitive or is it kind of covered with skeleton and the mesh itself it is basically a very very simple description of the geometry data and it also contains the information about the rendering mode so for example you can have a mesh with an accessor four positions vertex positions and the information that these should only be 3d lines instead of triangles is contained in the rendering mode of mesh primitive now currently I think they are counting no examples in the official repository I have seen an example in I think the three j/s repository maybe we can add this example or a similar one to the official repository as well I have an example for three long primitives it's fairly unrelated to pollutants from that alright well we have no more questions Marco or Neil or Patrick if you have anything you'd like to add well I've had the we are going to be at GDC so if anyone on call is going to be a kind of at the conference come come chat with us where you have session on the Tuesday on the GDC program we also have a meet-up on the Thursday evening all the details are on the corners website will be great to see some of you there and there perhaps just to chat briefly about some of the tentative things that we are looking at beyond 2.0 some of you might have seen the Draco mesh compression which is a pretty awesome technology coming out of some teams at Google we are talking to those folks because gltf is well architected to enable binary payloads to be compressed its various techniques and it seems like there is a potentially a good synergy between our gltf and the Draco technology so that's a discussion that's ongoing we're really interested to kind of get your feedback at the github repository is the best place to be feedback on or she'd like to see in future versions of gltf so things like compression and screening for large payloads know is there something that we're looking at but we're interested to hear what's uh what's key for your work coaching after mentoring you can get an awesome gltf t-shirt at that session it all comes down to the product alright well thank you very much for joining us as I said at the top of this this presentation the video of it as well as the PDF of this presentation will be available on the Cronus website and you will always see the now shortly with the URL where you can get that likes very much buh-bye now

Channel: The Khronos Group

Views: 4,421

Keywords: glTF, Khronos Group, API, Graphics, 3d Graphics, webinar

Id: KALedPvtFHY

Length: 54min 51sec (3291 seconds)

Published: Tue Feb 14 2017