Release Presentation: FUSE

okay three two one well welcome everybody outside and inside of this [Music] call we're having here i have to check my sound here for a second all right i'm set now sorry um yes so welcome to this presentation that we're having tonight and i am very excited to um and honored to present to you three fantastic people and a fantastic project these three people were working on during the last month i'm david brewer for those who don't know me i'm the creative director of my own studio a studio for interactive media and personally also using bbv since a long time and vuv is [Music] one of the core subjects of tonight and also i'm founder of the node forum for digital arts a festival in frankfurt dedicated um to digital arts in general but also they grew out of the community and around the world toolkit and two years ago i founded with a partner in berlin the node institute and yeah through this in this function i'm i was asked um yeah by this group of people if i would be open to host this presentation and i'm very much open to do so because this is super exciting um yeah and with me is well a lot of people know them already but maybe and hopefully we have also visitors who don't know them yet in person so with me is natanzinigalia from italy you are connected from berlin yes and um nathan is actually an artist um uh with a strong background in music contemporary dance but then also real-time graphics and programming so he's this mixture of you know intermingling the fields in one person and one personality um nathan is also known in the vov community as somebody who like over the last decade contributed a lot of libraries and note sets that we can as a community uh we can use namely um in the latest um yeah months and years it was elementor and ui library and kairos when it comes to arranging time presets and narrating data within the environment so this is nathan and then we have kyle mclean kyle is also in this community since a very long time kyle is a media artist an independent researcher currently based in bhutan joining tonight from australia so i think the time difference is huge and probably you have been sleeping just before this presentation glad you did the um yeah his practice is um uh maintaining personal um uh projects and then consults also like award-winning studios um like graphic anadolu and and and other like fantastic studios who are actually working um on open source libraries and computational libraries that he himself as well as an atom is contributing the third person is christian rico hi christian you are based in schwerin in germany correct and you studied experimental media design at the university university of arts and media programming um at the university of appliance sites in berlin and you are focusing on interactive installations uh mainly in the past and generative system kinetic sculptures i know you have been working or still working with artcom a lot and um yeah so you're merging your design skills with code skills and you joined the vvv um community not too long ago i when i'm right we can talk about this in a little bit and yeah i'm very happy that um you're also here and we're gathering together to yeah basically celebrate the final um or not the final like the first release of the fuse so this everything will be around fuse tonight and fuse as you describe it is an open source library for visual programming on the gpu built to enable rapid workflows and module approaches to accelerate the graphic logic and computation so this is obviously like this opens up a field and maybe like before you start with the presentation and um i i will have a few questions to you guys um like nathan and kyle i mean you have been in the uv world since such a long time developing projects and contributions um which became as i just said um yeah sort of a standard in the industry why the vv development itself like the vv group have been working on releasing a complete rewrite of the toolkit named dvv gamma which has been released also not too long ago so was this the initial trigger for you guys to kick this development off or has this been there before so like what is the story um how this started basically well i think uh like uh i guess you could say that um with the 4v gamma it's been quite a while that they've been really working on the on the language and really kind of fundamental features that are super interesting but it's also been the case for some time that if you were used to using the previous version you had access there to a lot of kind of graphics and shading and things that hadn't been there yet on the on the new version so there was kind of a need to fill that gap a little bit for people to actually use it for doing graphics shading and this kind of thing um so i think i i actually had originally had some people approach me asking about porting like um instance noodles and some of these libraries that we have for shading um over to gamma and that was kind of the original scope and then we were thinking about it and thinking okay but what else can we do we can we can have the same things but we can also go further and then i was talking to nathan about this and we like kind of had the idea of of doing this whole whole fuse thing um and i think we were we were actually looking just the other day that kind of kicking that off was pretty much exactly nine months ago um so like to the day so um this is kind of the conclusion of the pregnancy term i suppose and then pretty early on in that process uh chris came on board um and has been like absolutely amazing and fundamental to the whole effort um and yeah i think for the three of us even though we kind of work in the same field we we all actually have quite sort of specialized viewpoints and skill sets and it's been um very complementary that way i think we've found that we can we can do a lot more together than any of us would have done individually um yeah speaking of it christian like what is the um what has been attracting you or what has been pulling you into the well first we work but then secondly um yeah especially the the fuse development then yeah so basically i mean of course i followed like vvvvvvvvv like pretty much since the beginning and i work with code but as i'm pretty much code based then i come off from processing um yeah it was kind of attractive but also i had my other tools to work with and so i never really got into it um and then like two years ago i think i had one first project with chanel ubuntu builder and like i i got in touch with with gamma for the first time and was like obvious okay this makes sense to like really code your stuff but also um be able to patch object oriented um so that was far more attractive to me and i was just really waiting for stride to come out because that was clear like okay that would be the game changer when you could really start build projects with it um yeah and so then as soon as strike came out yeah i i i got in touch and um i was working a lot with unity recently and so i was really like attracted by the idea to kind of use the patching language um yeah to to kind of patch shading like so that was really my initial can you elaborate uh you you refer to stride um how this is related uh to the fuse development and also to the gamma world in general for those who don't know yeah exactly so i mean the first part is kind of of gamma and as a graphical visual programming language and and this is like first part to to build like fuse you know because we were not creating this visual language we were building on top of it and the other part is then stride which builds has a beautiful shading system and it's a such a nice 3d engine and which is like kind of the other part that we needed to kind of build stride a fuse sorry on top of it and yeah of course we were quite early in the process like um i was really like figuring out this stuff as soon as stride was has been released and was really diving into shader programming and covering stuff quite early and finding ways on how to use shader x from vl first and then kind of diving into our own kind of flavor of how to patch shaders um [Music] yeah it has been really really really nice and um fun not always but but a lot and and also like it was such such nice so nice you know to get in touch with nathan and kyle i mean i haven't i didn't know these guys before yeah and like kyle already said it's like really the three of musketeers um you know every one of us brings its own skill set but but we also share um this main interest of computational design and like to get really deep into it and i think it's like the perfect team to yeah to to get get to really explore this kind of this is the spirit this is the spirit that's super cool okay and um nothing like i know we have i think it was december when um we had another like the first public uh announcement basically we we presented the roadmap where you guys presented the roadmap of what you you know have in mind and um and then the community also came together and uh the studios in the industry like uh backed the whole project and and there's a there's a lot of sponsoring going on so you guys can actually um also have an income from your development would you say that um like before you start the presentation where you actually are but would you say like the since december you are there where you actually have imagined yourself back then just to you know take this in advance i'm pretty happy with the current state i will say um i think that yeah um we are in a good state and obviously we don't have the entire library this is a pre-release and but i think that we managed to have that foundation that we dreamed since the beginning and so when back then exactly nine months ago we envisioned uh we envisioned this this moment in time in which we can easily build on top of a foundation and so much easily expand the library and i think that now we are at that moment so i would say that it's a yeah it's a great achievement yeah i think we're all super excited to to see it now for the audience um we thought we'd do it like this the guy is going to present it now and at a certain point in the presentation i'm going to share the link to the zoom call basically to have a more vivid q a afterwards so for those who have like real questions in the like afterwards we are very much invited to click on that link and be with us for a certain amount of time and put your questions directly i think it's nicer to have just a little conversation um please put your name this is important please put your name with your proper name and maybe a little code like fuse presentation when you entering the zoom call so we can identify you and um yeah differ you from bots that might try the same um yeah and then i would say give it a go and um i don't know who's sharing the screen from you guys but please go ahead okay here we go [Music] yeah we can see it all right so yeah on to the fuse um 0.1 preview release um [Music] i guess i start then um we haven't really checked like who's doing what so let me go through the first slide and then maybe someone else takes over yeah so just a quick summary of what we will cover today so we will start with um an introductions um of ourselves again just to give you a kind of idea of what we're working on normally if not on fuse um and then an overview on fuse then we gonna show you some the the features that we have so far and maybe um get into the documentation of fuse which is like one really important aspect also that we want to cover because it's also about sharing knowledge and not only about sharing building a library then we want to share our vision like what what next fields we want to explore with views and also give an idea on how we want to work together with the community so meet the fuselab um we already had a short introduction but here with some images yeah i won't really kind of talk over these things so like because it's nice to show some some outputs and things but we kind of already had a little bit of an introduction um yeah that's just some stuff we've made some more stuff from downtown and some more stuff from me um so you see we're all kind of working in the similar field you see a lot of particles and computational visuals so we feel kind of profound and unable to kind of build that library not only from development points but also from designers perspective and i think that's quite crucial to build such a tool um to kind of already get into the kind of workflows that that other designers would probably like so let's let's give give you an overview on fuse um yeah so what is fuse um fuses an instant and visual programming on language built on vl um to use a gpu and not kind of work shaders write shaders and and build buffers and whatever and but make it faster and play freely um so to have rapid workflows and modular approaches meaning that you as a designer do not need to hassle too much like how do you enable how do you put on a shader how do you create resources but that you can just directly patch um which which switch workflows which you are fabulous in nvl and in this way we want to give people access to accelerated graphics logic and computation and yeah we utilize stride and we v for that so fuse and bbv so to really say that quite clearly i mean we worked really tightly together with the vvv development team from the beginning as we used vbv as language and basic foundation for fuse was quite crucial to to get them on board and we had them on board and yeah they helped a lot with features and other stuff that we needed and debugging yeah we also want to strictly follow um it's always runtime model so every time you change something you immediately see what you get which is like crucial for fast design and programming um [Music] yeah and basically one have at least almost no compile time or not too much between you and your results yeah stride is like i already mentioned that at the beginning the other big part this is like a nice 3d game engine which is written in c-sharp and therefore couples quite nicely with vvvv gamma so it's pretty much fully integrated and we take advantage of a lot of features not only in terms of shaders but also the complete rendering pipeline and materials and other features where we build on top but a lot of this nice visuals are just also enabled by stride yeah then why fuse yeah because we want to make it easy and we want to have a visual and intuitive access point to the gpu which is normally quite complex and not so easy to get um we also want to have like not too many different systems i mean if you talk about unity you maybe know shader graph and vfx graph and it's similar in other i guess game engines as well so you have like one kind of language to make simulation and vfx and another one to make shaders and we really want to have a holistic view on this whole thing so you should only learn one kind of note set and all the kind of features should work probably anywhere and you should be free to put anything together yeah we utilize the object-oriented programming environment of vl so you are free to express and use the features from vl and kind of find new workflows on how to build shaders and systems and there's something we're really curious about to see like what kind of workflows will evolve all of that yeah and then in the end it's open source because we want to share also knowledge because we share knowledge also from other people we build on top of of guys like amigo quilas which a lot of you know yeah like shader toy and there's so much more and behind the vr community where people develop shaders and stuff that we use yeah maybe now time you get over some features now yeah amaze um yeah we got as as we said we wanted to have like a solid basement foundation for fuse very low level a set of nodes but then on top of this we would like to explore management rs because yeah indian fuse is about dealing with gpu resources and ways to operate on these resources so let's imagine okay we want to do some procedural geometry it's just one perspective that you take over this data so you you think in in terms of geometry and manipulation vertices okay this is one way to look at this world another domain could be the particles particle system is another way to look at another perspective over the gpu data where you see where we are interested in concepts like particles with an object which has some attributes and you want to instantiate millions of them and work with them in different ways so these are all different perspectives over the same material india and methods so given the fact that we wanted to have this unified approach to gpu in general yeah we identified still this main domains to start from at least so yeah we have a core as i said which is the foundation all the other domains then this procedural noise which is it's like a domain that includes so noise manipulation uh kylie will tell you more about it we have distant fields and we're marching geometry texture materials gp gpu particles through simulation these are all different ways different domains different perspectives over the same gpu yeah all right so let's maybe start with with the base thing um of use like how how it fits in i'm already set like we are kind of um working together with vl and stride we built on top of it and like one low level thing really is fuse.net because the base thing of fuse is coded in c sharp um and then out of that we have a first like kind of note set layer which is fuse core um which is next to vr stride which is like the kind of node set that the vvv devs have um built to use stride in the l and then we have various libraries on various levels like mars color compute and and more high level stuff like particles and drain marching and these kind of build diffuse universe and we will cover those now in the in the next slide and just give me let me give you a quick view and this is really pretty much the only time we're gonna hassle you with code um but it is there somewhere deep in the system um and this is like kind of how the c-sharp code looks we have a flexible template syntax where we can integrate shaders but also can kind of keep them generic to any type to not um nothing to fry too many code and what you also see is the kind of generated shader code that fuses generating so you see a lot of weird ids and code which is in the end working but the shader normally do not look so readable so it's more like your patch that kind of reflects the logic for everything um so and this is really just the first glimpse of a lot of nodes and there are many many more and these are like the fierce fuse core level and this is kind of for example logic stuff like um and or and and switch and if and like kind of split nodes and swizzle nodes and whatever you use really on this low level stuff so pretty much everything that you need in shaders is there also as node same is true for math so like all the hlsl intrinsic functions are covered here so we have all the math functions refract floor log whatever and wherever this is possible we try to make those functions spectral which means you can put in as many values as you want so if you add values you can add like as many as you want but also for the max function for example this would work you can put as many max as many values in the max function as you want and then get like the maximum out yeah so [Music] we're gonna start just with a couple of the topic areas where um it's kind of basically functions that you can use and applying to anything so you can use to create geometry you can use to do materials you can use to modulate parameters and noise is like one of those really fundamental topics that way so we've kind of designed fused really to use this functional approach across the board um so you can you can really use noise to render directly or to modulate parameters and it's also really integrated with the particles and the fluids um so if we if we go on to the next slide you'll actually see for like what what it actually looks like when you're visually programming um so i mean this is just kind of to show some some little samples of what some of the noise functions look like um but they're quite sort of high level node sets we wanted to show you this kind of machinery first of what goes on under the hood but this is what it what it's actually like when you're when you're patching with the library um there's kind of a lot of those standard noise uh basis functions implemented there and also some more exotic ones um one thing that we've kind of been working on a lot is really using tool tips to help kind of visually communicate what's going on so you can see um like in the patch there you've just got kind of these thumbnails and those are like live previews if we do a little bit of patching later um you can can see some of that in action um but that's been kind of really interesting and something we want to push a lot further um you can see like for example there's as well as the kind of standard 3d and 2d views there's some some kind of graph used there and that sort of thing can be really really useful for for debugging and knowing what's going on um yeah uh for sign distance fields um if you're not familiar with them they're basically just like a like a type of function that kind of defines an area of space um and again like the noise we've really taken approach where you can you can use them everywhere so you can do all sorts of things with them um they're also really well integrated with things like the particles and the fluid simulations and um we kind of want to really uh document everything in detail um and we'll talk more about that later but um i would just mention if you're kind of just getting started out with views that the the sdf section is kind of where we we wanted to try doing the documentation style that we would like for the whole library first so that's kind of the best documented part at the moment um so it might be something to to check out when you're getting started um yep um that's just like some of the some of the kind of primitives or source functions that you can use with sdfs um and yeah there's like oh all kinds of operations and things you can do with them and they all kind of have these dedicated tooltips again so that you can see what's going on um one of the kind of common uses for 3d sdfs is raymarching which lets you make kind of these crazy surfaces and geometry that would be a little bit difficult or even impossible to do with like traditional geometry workflows um but it's it's quite nice like you can you can use that as approach and mix and match it with with like traditional geometry meshes you can see on the on the screenshot on the right there the the sphere and the teapot are just normal 3d models um and the surface underneath is already marched one and you can see that they're quite well integrated in terms of shadow and lighting and materials it works quite seamlessly um we've got kind of yeah like pbr shading from stride and lighting from stride that works out of the box um and again there's quite useful tool tips that you can see kind of steps and in in the middle as well as your output to see what's going on um for geometry it can also take like standard traditional meshes and use the same visual patching system to manipulate them we've got a few more things that we'll we'll add to this kind of area as well like like splines and geometry filters and some of the content basically that we had in libraries for beta but we're still in the process of boarding over but um you can still do quite a lot with it already um you can also use all of these same nodes to uh patch like procedural materials um so you know i mean if you look in the the patch there you can see there's this noise node that we're kind of uh manipulating some of the pbr properties with um but again that's the same noise node that we could have used for ray marching a surface or used for changing behavior or particles it's really the core kind of tentative fuse that these things are applicable across the board yeah so we're also addressing uh compute shading and we have like several two levels basically um for compute shading so we have like basic level where you kind of as you see here these example patches have a resource and like like a buffer and yeah you have a buffer in and this patch thread and and this looks pretty pretty much like a standard like compute operations so you get values from buffer you calculate and you set them but we also have kind of high level system and this is not only allowing you like to to patch with resources but you can kind of create attributes and then fuse is not only building the shader code but also the resource system so for example like in here you see this kind of struct that we have of a particle is basically constructed by fuse and it kind of looks of attributes that you have in your patch and then says okay there's an h attribute the position attribute and whatever and then it kind of builds a struct um and also buffer and fills it with data for you and you can just use this attributes then to build your shading logic and this in this case you see a kind of a particle system but it would also just apply for any kind of compute tasks and you are free to have as many attributes as you want and um also like of any type so as you see here you can have integer float and whatever but also boolean yeah and on top of this we um have built um the particle system which nathan will now talk about yeah yeah this compute system is very exciting because that's why we spend a bit more time on this because it's really the foundation of so many other possible domains like particle system is one a very basic example that everyone can think of but you can really use the same complex system and imagine building very complex ways to migrate geometry or operate on volumetric textures and it's really powerful so we focused on that and then we decided to yeah to test it building a article particles domain um um i would say is we just started with this honestly um this is something that will evolve quite quite a lot in the next weeks i will say um and to now we have a set of high level nodes some emitters forces collision constraints and integration methods here you see some screenshots some of them are coming from users from the community that already started to to experiment with use and there's a lot of enthusiasm around this so if you go to the next we see here is just an example of a particle system patch um as you see on the right there is this render which is the real-time output and on the left you see the patch that generates that and the patch as you see is composed by different parts let's say you can identify different parts like the mission the simulation the drawing but in the end they are made of these very high level nodes modules this noise force browning force drag so very high level nodes that each one of them obviously has a certain meaning and does something with the particles um so we we built a set of desired level nodes uh but as i say it's just the beginning and we're gonna extend quite rapidly um the note set so if you move to the next here i just want to show uh that it's so so beautiful i'll say to have this unified perspective over or the gpu domain because it allows you to really solve certain issues that work in so many different domains in different areas so [Music] here for example we we decide we define some attributes for the particles which are given to already with the library that comes with the library uh but if you open one of them like the age of the party in this case you find that is nothing more than a patch which is made with one of the low level nodes so you suddenly realize that you can freely extend the system and add your own attributes to it and imagine really to be any sort of property of these particles and it's really easy to do it that's why i'm saying that in the next weeks we will see a lot of changes also because we are gonna use fuse for big projects the next days um and the same approach you can find it also for the for the forces you know the particles like this drag module there i put an image from the previous uh patch if you open that module you see that it's nothing more than that patch it's it's a complete shader somehow it's a logic logic but it's patched which means that it's so easy to come up with your own modules for your particles like [Music] weird tractor and you just can patch it and use it within the same ecosystem of the particles on the right you see some of the models that we've implemented so far but we didn't really spend time on that honestly that's one of the main things that i focus now and it's just amazing that it can extend so so easily now it's really mind-blowing fluid yeah we also have a 3d fluid implementation gpu implementation this is interesting because we needed for a project um so we took an implementation that was already made in in be gamma using stride but in that uh implementation the main pipeline of the share was just built with shaders um like proper shaders important as nodes so they those uh those shaders are not patched using uh fuse so we took those shades and we kept them there because it wasn't time to to replicate the same like pipeline infused but it was also interesting it's very new to show you that you can still combine different approaches you can use your own shaders [Music] and then combine them together with some fuse other elements in fact in this fluid we introduced some aspects some new components like a collision with sdf or we also the the vector field implementation was made with fuse and we injected those functionalities into the original pipeline our plan is to replace the entire pipeline of shares with fuse made shakers so that it becomes suddenly more easy to extend and change them explore the functionalities that you can yeah you can find when you work with a 3d volume yeah documentation are you guys i i can talk about it so documentation is a key element for us first of all because we want to share this knowledge and and i think that we have a mature language and a library to to express very high level concepts and and find nice and simple elegant way to represent it which is the key to understanding it and so it is the key to knowledge and that's why documentation is it's a key component of this library that we are really we committed to it and we are not the only ones also just we will talk also later communities joining um they're joining the forces to produce high high quality documentation so we provide like guidelines and templates for top patches so that people can contribute and maybe go through hundreds of nodes and prepare specific aimed documentation for those nodes and we can uh basically merge back into the the main documentation that i think i i kind of alluded to it before but we went a little bit depth first with the documentation so there's kind of example patches for all the areas of the library but the sdfs are kind of a little bit more thoroughly documented at the moment and they want to do the same thing with the other areas and one thing we've very very quickly realized is that it's actually a very very nice environment for learning about a concept as well as for learning a particular procedure how to use a particular node so we're kind of documenting both those things a little bit we're documenting some of the stuff that's very specific about how you do x or how you do why um but also really trying to make an effort to use it as a vehicle for knowledge sharing and it's quite kind of visual and interactive and nice for that you can really take something that might be very dense when you look it up on wikipedia or something like this and have it in a really intuitive way so yeah i guess this is kind of like uh maybe a little bit more conceptual or philosophical but read write culture and encourage collective experiments and creative tooling that bypasses the strictly for profit corporate model and embodies radical openness and gift giving together we can go further faster word yeah [Music] that's the word i think that's everything yeah and i think we really want to address that um we're working in a kind of ecosystem so as we stated um several times here already like we haven't we have built fuse on top of other tools um and we kind of enrich those tools like with fuse and of course we are like the three core guys who kind of initially initiated this um project but we already have so much help um like in terms of sponsoring and people just believing that this project um goes on goes off um yeah we have people doing documentation and helping us with communication and yeah it's really fun and it's motivating like to feel the spirit and feel the drive and to see like people build projects with it and like really vivid projects and like really nice visuals and yeah we're really looking forward um to extend those and yeah to see like what emerged um not only from from us but also from within this community and regarding to this to also give you the freedom i mean it's also important like what kind of license we use so we have chosen the mit license so for you to have as much freedom as possible while it still maintain open source so you can use it for commercial and non-commercial projects um but that talking about community um we also ask you like if you make really highly successful projects um we would really ask you like if you make projects more efficient and you can make better visuals in less time um that you maybe take some of the stuff that that you benefit from and maybe share it with fuse because it's like also you share it you kind of finance you know the future development of us you kind of pay back also for the community um that drives us all um and and it would really help us to like um develop views further because um it's really helping but it's not covering all of of what the work that we put in it's it's still like motivation is still gaining knowledge working with other people and drives the community but definitely um having walls definitely there's there's still so much more we want to do and not even just in a linear process of like adding more features to the library that people can use but also in terms of like fostering more participation and building community and capacity around it um yeah you know there's there's like a nice kind of momentum there that we want to see grow um and so yeah we just ask that if you consider yourself a good person and someone who lives a life of ethics to consider helping the the greater good okay so last point pretty much coming to an end um is now the road map like what have we covered so far i mean today is our preview release so what will be the next steps um currently we have the core architecture and node set we have some high level node set for some of the domains which give an idea of how our fuse is working on those levels we also showed like with the sign distance field domain how nice a documentation can work to not only see how you use the notes but also gain a deeper understanding of the topic itself so you can learn a lot of scientists and spheres and for the next three months [Music] we will work with fuse now in some really major projects and within those projects we will um we will be able to discover like really deeper topics and and have more time to all the existing domains and also cover new domains we're also looking forward for the community but what they will provide in terms of documentation but maybe also um as functionality and maybe certain domains can also be covered by by other people not only us so if you're interested for example like a procedural geometry or things like this it's not something that we're gonna address in the near future that's something we are curious about but if you are curious as well and yeah just feel free go ahead and and we'll see if we can um join it yeah i'll be targeting the full feature release or not full feature i guess it will never be full feature but um kind of version one and the first quarter of um the next year yeah talking about funding we already had a lot of sponsors who supported us um got some money got a lot of motivation and really helped like over some long stretches of serious work also um to get things forward and also have some time so basically this allowed us really to address things and not always be busy with other projects um but to really focus on certain things and get it forward so thanks a lot to to all of the guys here to also making this possible it's really appreciated yeah so we come to an end thanks for for you for your interest um if you want to go deeper look at our website it's online the fuselab io you have some links there to the github page to some social network it's [Music] for the stride engine yeah and now we ready for some question answering and also maybe for some patching session to really source some stuff in action well thank you so much guys i mean this is this is massive this is i mean except like a lot of uh laughs and and amazed uh smileys in the chat we have we haven't seen a question probably people are very like uh stunned and um i think this is this is such a this is such a development for for our fields and for the community this is i mean this is a major major step um after the release of the uv environment to have now a library that actually brings us back on track um when it comes to visual computational render outputs um for everybody out there like we are open for questions at any time you can also like come to our zoom call here i posted the link in to the chat and also in the element chats and uh hopefully you guys are joining in for some detailed question and speaking of patching session i mean the i think the question is obvious um obviously like as somebody even with rev uh little knowledge but also um for the for the newbies like how would you like where's the first touch point how do i get on board to like how can i use it like what what's the maybe you can showcase this shortly like where to start you know like you open vb and then what's what's next and i mean we saw a couple of screenshots but i think it would be cool to like have a little journey through um where can i find what you know so at least we have a starting point if this is possible for you it would be amazing yeah kyle i think would you maybe give a first starting point on on first patches can you do some screen sharing uh yeah just give me a moment um i would say also just like if you're completely new to the whole thing and like you you need to know how to install 4v and everything like that um we did actually make a short video that explains this um and i've just now pasted the link for that into the youtube chat as well meanwhile simon joined simon holden welcome well there's a lot of congratulations going on in chat i don't know christian or not am i actually following it yeah yeah yeah i have the chat open i see that thanks a lot it's really amazing yeah but it's also like what got the feedback already from the community over the last days like um yeah was really motivating us to push this presentation and the website and like all the people were celebrating like every step so far and yeah really how to get it done yeah i think there's um some like an another obvious question that comes to mind so is it is it like you're patching like generally speaking i think i was reading just some on the on the chat here um so it's actually generating shader code in the background right it's there yes but at the very high level the the idea of it um is that you're kind of uh programming visually so you're using this kind of patching metaphor um but under the hood it's actually generating this shader code which is kind of you know that very scary screenshot cushion was sharing right at the beginning that that's kind of what the what the product or the output of it is um and that's been kind of quite integrated with the the 4v gamma stride implementation um so i mean a lot of it stuff that you could also do on the cpu but the advantage of the gpu is it does all of this stuff much much faster um i've just started doing a screen share i don't know if you can you can see yeah okay so this is kind of yeah what what the actual what the actual patching environment looks like um i mean this particular patch doesn't have like a render output but i just wanted to open it first to kind of show uh particularly for people who aren't actually familiar with 4d um when you open it you'll also have this help browser window open up as well um and it's kind of like we won't really cover today like just the basics of how to patch in the 4v environment that's kind of a big topic um but if you as soon as you open it up you have this this help browser you can go into and to learn and it really has a lot there kind of explaining the the basics of how everything works there's example patches there's there's videos um so all of that is kind of really good for getting started um this patch i've got open now is kind of like one of the ones where it's just also explaining a little bit like um the the point of a particular concept or thing um and yeah you know you can you can see basically i can i can sort of zoom in and change things and you get that kind of instant instant feedback um one thing i'll mention about all of uh these tool tips is that uh they kind of run like a little bit separate from the the render output um like if i open another patch that um has like a render output as well i can show you what i mean um but basically they they kind of run in the background so that if you're using like real time changes the the frame rate on the little tool tips might not be kind of quite as fast but they're really designed not to block the frame rate of your your actual render so that'll that'll kind of always run like at a much higher performance of what you want um so we can see here like uh this one for example is a patch illustrating like often people use noise and sdfs and so it's kind of explaining how you can do that but also some gotchas or things to be aware of you know like uh the more noise you add to an sdf the less accurate it is and so this is like a way to show that you can push it to a point and if you push it too far then you'll get some artifacts so really useful things like that that you can then kind of apply in in the the patches that you make um i might just show like uh very quickly before passing on the baton um maybe another rain march one that is just kind of shaded a little bit more nicely um i apologize if our presentation's been like very text heavy and not showing too many visuals we've been like in a little bit of a kind of mad rush to to get the the library ready so i know it would have uh it would have made sense to do like a little bit more kind of teaser videos and things like this but uh i'll show like just quickly one of the patches here that um i think used in one of the slides for example so yeah i mean it's it's really pretty simple like the whole material is just patched here with a single noise uh the actual kind of surface on the bottom here is basically this function here and then we just have like the kind of other other geometries um if i uh look another ray march node here i'm just using this one as like a debug view but i just want to show you um i mean you don't need to necessarily worry if you're not interested in shader code about the shader code that's generated um but maybe for some people it's it's interesting so for example this this kind of surface here that we've made if we uh if we wanted to actually look at what's generated from it we can do that um i'll just switch to this like debug and we should get like the i think for the shader code you need to take the other pin uh yeah so it's just as a second pin of the ring here exactly [Music] so again this isn't something that you really need to take care of her or worry about as a as a user but it's um just kind of showing you what what what's actually like happening under the hood so basically this this shape that we've patched here and the array marching that all gets compiled into into a shader like this basically uh so it's it's really cool i mean like obviously doing this is text based is the standard way and is totally valid as an approach as well um but you're very free to sketch very quickly and fluidly like this and you can really kind of reconfigure the way these actually very large shaders work just on the fly um so that that's just kind of i guess like at a at a high level the the workflow or the the output or what it actually does so users do users have to actually install strides um probably yes but then does it does it is it necessary to to have it running in the background as a game engine or is it like building on top of it and just accesses it well strides been quite well integrated into 4b gamma already um so i mean again for the for the really particulars of how to install it and set it up i'd probably just kind of refer to that that video that we've linked already um but basically uh if you um yeah you're not having like a hacky process where you know you're running three different tools in parallel and they need to communicate with each other somehow or something like that strides really well integrated into into 4v the environment i'm using here and then fuse is just a library that you use in 4b so it's quite kind of um yeah tightly tightly coupled as first so can you show like how to reference the library maybe for those who don't know v so much i think we have a couple of people maybe from different worlds so when you install uvb it's not coming with it it's like you have to reference it and download it yeah um so you've got uh like in basically it comes in the form of a nougat um nathan do you maybe want to explain the process for this i i would say really like the the the kind of best step-by-step ways to just follow the video but um basically if i if i go like uh in managed nuggets here uh you can browse like on nuggets online and you can look kind of like this one is called bl dot fuse um and once once you've got them like installed then basically on on that machine if i make a completely new fresh patch i can just go to dependencies oops you're saying it's actually online now yes yes so you can and and then we can just select like basically any of the ones so if i want to make a new patch using fuse i can just click here and then i've got all of the the fuse nodes should be available already so everybody out there can already directly start playing with it yeah yeah they would say cool so this goes out to the world so let's say somebody you know is into it how can people hook up to the discussion about it like how how can i reach out to you guys how to give feedback how how do you plan to actually you know have user feedbacks going into the development so i would point kind of first to the website just because everything else is like linked from there um but in terms of like feedback and working together on it uh we've got like a like a element chat which is is probably the the best place for that sort of thing um and i think that's also linked from like the the read me um i might just grab a like i'll i can show in the browser quickly a couple links that maybe it's uh let's show the website yeah so let me just uh resize here so this is uh the fuselab dot io and um here you you've got like uh yeah you know uh we've got a bit of info about the whole thing but there's also um links to to the to the repo and the social media things and if we if we hop on the github one for a moment um then you've got uh you've got kind of links here like for the for the nugget page um this is the the chat i was mentioning before um if you go in there there's actually like quite active discussion and feedbacks going on um this also has like some some of the basic kind of instructions again on on how to install does um does anyone else want to show like a little bit of a little bit of kind of uh visual output or anything i know probably been a little bit dry in that department so far uh i can show a little patch yeah yeah we can i can chat if you want a moment to open something see any any um like questions or anything in the meantime well there's an obvious question is it windows only at the moment and i think the answer is at the moment yes as i understand it um but that there's like some some development around this can you start sharing oh yeah uh do you see the script okay i just opened one of the hot patches just to show you yeah another example of how nicely you can integrate uh different words like sdf um there's this this one is a a model which contains this node we created into this head patch which is uh some which is what you see here it's a sine distance field that describe these slowly moving surfaces so yeah you got a assignment and here we are using it in a particle system that we made here with some nodes with a meter some forces and plus this collision nodes take care of so if you enable the particles now you see that they're simply just colliding with this chinese exterior and doing this with another approach like today or whatever it would be so it's very convenient you can get crazy imagining any sort of sun distance field and manipulating whatever parameters and particles simply react to it [Music] this module then the particle will be evaluated and we could see the expansion of the collision industry so yeah it's it's very easy to play so much fun and really come up with ways to use them into the particular systems the fluid it's really really really cool that everything comes together when you play around with [Music] also some people in the zoo call where are they simon and christina are you with us joining just to listening or do you have a particular question maybe it doesn't seem so um how about physics have you thought about this is this something on the roadmap in the future or is it am i missing something did you think about this well this is [Music] such a wide domain and you can already implement something [Music] sort of physical simulation uh with the complete system that we have because in the end it's just imagine like a rope simulation it's nothing more than a bunch of points connected with certain kind of relationships to each other and this is something you can already patch in our system i mean particles and fluids can like combine and collide with sdf surfaces already now out of the box um but yeah i guess like if you're more asking about like a fully fledged physics simulation system like like bullet or something like that um yeah i mean it's something we've thought about but it's probably kind of not first up on the agenda that's a pretty pretty big undertaking in itself yeah and i would also say there's like really advanced stuff from nvidia on this which is integrated into other game engines or whatever and that might be also be integrated into stride so i think that by it would probably be a level where it would be getting too deep and where it would make more sense to utilize the work from other people but that said it's it's more oriented i think into world towards the vfx direction so i think like physics really regarding meshes and um like jail simulation or stuff like this what what you really see there on and i mean if nvidia is providing it they are directly aware of their hardware and how to implement it best then i think that's that's difficult to compete with so i think our interest is more to make it really playful flexible and um yeah just just make people um enable them you know to create nice visuals very good any more questions anthony is asking can one use the low and high level compute shaders patching for audio processing [Laughter] many years ago trying to to come up with some gpu generated some generators and now i see that in the end this complex system is really nothing more than enabling patching complicated so and this by the way is the image that we use for the website it's just a patch dispatch here with a sine distance field part that creates this shape here that directly changes a full simulation that collides with these and that particle system runs through this uh through simulation and also collide with this conditions so this is one of the examples that you can find [Music] coming back to the question i well the compass system um it's nothing more than patching compensators is a very multi-port domain so you can yeah you can generate buffers back in the days supernaturals yeah created some shaders they were generating uh maybe an f50 and then i was reading back because there's also a way to read back data basically from the gpu so we can generate something very complex on gpu and then read it back in the cpu [Music] chip and then you could use that to generate some order the core thing is that what we wanted to achieve is a system that is really not goal oriented it says done for that specific purpose but it's more like we just keep you uh so yeah yeah i think there was also a question about emitters attractors and collision detection um so if let me answer that um i mean we're using sdfs a lot um so basically using sdfs for collision detection um so basically this is what you see here currently in the patch um from nathan um and we do this because sdf you basically have a box you have a plane you have a sphere you know you already have a lot of kind of geometry covered and so we basically we just need to provide one collision um node and then you can provide any kind of sdf and as you saw with this like repeat kind of sdf you know where the particles were moving through a lot of like this kind of um boxes or pads or whatever you want to call it so you see that's already a flexible approach for colliding and we have the same approach also for attractors and there's a node um like conform um to sdf and this is then kind of um attractor but also allows you to kind of keep them on the surface um i don't know if nothing can open this yep and this conform to sdf is far more flexible than a usual attractor because you kind of have it all in there you can attract you can move away or you can move to the surface and you can also mix it so you can also create like really nice variations of attraction and attracting to surface with this this particular node and i think this is also kind of how how we see the system i mean building working with these attributes it's really easy to build your own forces or whatever you want to apply to your particles and you can change the color you can change the size or whatever so it's not only bound to forces but you can also change change particle attributes for example on the fly based on velocity or whatever and i think we will more provide more more yeah complex forces like this conform node or the collision or also the fluid but then leave it up to you to yeah like build your own really custom forces where you address like your own special needs yeah as we say as a salesman presentation you really didn't spend time this morning but now it'll be quite freedom towards people it's it's 15 years that we do this kind of stuff with particles so we know exactly what are the algorithms and the logics there's a little discussion going on in the chat about exporting what you do in fuse can you say some words about that like is it possible to reuse what you've been doing there in other softwares meaningful or have you tried that or is that something that was a question about like exporting it to import somewhere or about like making a exe well it wasn't really specified um there's a little discussion going um yeah i guess if you want to export the mesh for example then this is currently not possible but there's something um yeah where we're also interested in um to kind of sample the sdf and build a mesh from it um so there are algorithms called already and but they're not yet implemented but these are like also kind of the challenges um very good also if you have if you want to support this if you want to give it a go and implement it yourself and kind of see if you can export meshes from sdfs then yeah you will come to look into it and then we can see if we can integrate it into a few so i also support you and this is like yeah valid for any other topic as well and does it and i guess as far as exporting like an exe or something um that hasn't kind of been our focus so far like that's not what we're really paying attention to but actually it it should it should be something that that you'll be able to do ultimately since the uv is coming with an exporter yeah exactly yeah yeah yeah i i basically the answer is yes but it's not something that i would say is kind of ready to go out of the box it's not something that we're like currently testing or something i imagine there might be some some things to kind of iron out with that or something like that but yes and in the in the big picture you should be able to do that um i i did see some other people who like use other tools and things also asking like if this can be used as like a shader composer that you then use like the shader elsewhere and um the the short answer is no but having said that i mean it does actually make like a valid shader code so you can take that and then use it where you want um the only gotcha is that you know you you then from your side need to take care of things like setting up the draw calls and providing the inputs and even though it's valid shader code you know that that still needs the context of the other things around it to really make it work um so i mean yes you like like i showed in the patch you can just grab the code that's generated by it um but it's not like super trivial to then use that somewhere else you could but you would you would need to like build a little bit of support around it yeah also because i mean this is the thing that this kind of level of visual programming hides from you a level of complexity is sometimes mind-blowing so you see just a simple simple like particle system a graph with some logics there but under the hood is quite complex and you you could come up with a massive shader with a lot of functions and inputs that simply in your patch you you you don't perceive as complex because you just perceive that the simplicity of the logic due to this nice simple representation right that's the the powerful thing of the the the whole point of this uh this library is mainly that to be really yeah to be free from this complexity and freely explore some higher domains some higher way of dealing with data without being concerned with implementation problems and technicalities yeah that was the intention well any more questions outside world moore is basically going back to the to the export question like referring to a use case like could i export alembic caches um of a simulation like would this be possible like potentially like let's say to use in um in houdini for instance it would absolutely be able to implement it's not kind of currently there out of the box and probably what would do much sooner is just like a obj export um just because that's much much simpler to implement um but yeah for for something like alembic it could be something that could look at adding as a feature down the track um that's probably a good example of one where it'd be it'd be kind of cool to have like a project that has need of that and use that as a way to to kind of support yeah so it seems like uh reusing what you do in other pipelines is one of the topic that's a major feedback i guess um yeah i think if nobody else is asking questions say anything from your side you want to add if you recap maybe what we forgot um from my side and pretty much through with my questions i think the main thing i i'd just like to restate again is uh that we're all like super super grateful to all of the people um both like studios and individuals who were really generous and giving a lot of trust in supporting us to do this kind of initial development um and yeah super excited like this is kind of just the beginning and so yeah i think over the next periods we'll we'll be able to also see a lot more kind of participation in terms of people helping out with making more content for the different parts helping with documentation everything like this yeah and i think an obvious follow-up and um we can plan this uh very soon as doing a proper workshop getting people on board to actually use it it would be amazing so i promise to push you to do that yeah please do and yeah so um maybe maybe you guys can show once the um url for the riot chat i think that's an important channel to know on the website in the context down there okay yeah there's all the information there all the channels to reach was just uh the bfslab.io all right then thank you so much guys i just saw someone saying they're going to have to install windows in order to play with this i'm sorry for that apologies but hopefully in the next future there will be some news about this yeah yeah there's something on the background well then i'm thinking you i mean how can we thank you for doing this i mean this is obvious that everybody should be super thankful especially with the license model and like this is massive and yeah we're curious what the future brings and hope to get as many people as possible on board it to everything and yeah we'll talk next time and we'll be back hopefully with the workshop and thank you so much for hosting us again thank you yeah yeah welcome appreciate it all right then see you next time bye

Channel: The NODE Institute

Views: 1,847

Rating: 5 out of 5

Keywords: vvvv, visualprogramming, fuse, instancing, particles, node

Id: 4xDShgbKTsQ

Length: 86min 45sec (5205 seconds)

Published: Wed Sep 15 2021