Interactive 3D Visualization in Jupyter | SciPy 2018 | Maarten Breddels

hello everybody my name is Matt vadas and I'm going to talk about well interactive 3d fish in in Jupiter so I'm a trained astronomer I was working on software for big data visualization thanks now I'm a freelancer consultant working on data science work in the Python and Jupiter ecosystem I'm also a quarter which is developer and author of what I'm going to talk about I buy volume and in fact you can find me on the internet at Twitter gmail email github assume probably also get lab and on the web so today I want to talk about like a bit of the history why I created I pipe volume submit motivation behind it and I want to give some live demos some more live demos some more and I'll end with a live demo and so after that I'll give some future future plans so a little bit on the history so let me start with a pre demo demo actually let me start with a pre pre pre demo demo what didn't work with self an I just want to show that it works with effects so this is like a I think hundred fifty million data points but I want to show a data set where we really needed 3d so what this is is a simulation of Milky Way halo which is like the spherical component of our Milky Way and what it looks like is like a smooth component well if you look at other features I'm not going to go into be like the science behind it you see that it's actually like a lot of clumps and if we try to like isolate one of these clumps you see that it's not so smooth but this is just like a 2d view so we can look at it at different angles but I'm gonna show you that you really need 3d to to take a proper look at this so our universe is 3d and our galaxy so before I pass all you I had a proof of concept just on a like a web page so let me show you actually what it was I thought like can we do this actually on a web page and just played around with WebGL and some sliders using jQuery and that that looked pretty nice and I thought okay this is something I could do so I had a like a really ugly solution where I inject JavaScript plus the data into the notebook which you shouldn't do and also the sliders I mean I was like reinventing the wheel and I thought okay this is not really a good approach so instead of reinventing like the whole machinery of the communication slider I built it on top of I buy widgets and that proved to be really like a key ingredient in the in the success and also realized it was really filling a gap when I was starting working on this because you couldn't find any interactive library for the Jupiter notebook at the time at least that I could find later on I found that there was one but it was like impossible to find that sometimes happens so now I pad folium is for it's like niche market it's pretty popular on github and something which also I think makes it like much easier to use you don't have to do the MB extension enable install dance anymore it's the easiest to install software on the planet just pipe install and done so I don't want to make this a reef you talk but just to give you some overview of all the packages so if this package doesn't work for you you can take a look of one of these what's important to note is that for me what I'm I'm gonna go back to that is that by three years 3GS is going to be coming in like a bit of the backbone for 3d face I think so why would you use it well you get in like really interactive visualization so you can rotate and it's like really fast it has bi-directional communication so you can actually get back at the kernel what the the statuses of the camera and do actions on that it's also interactive outside the notebook so if you can save it to an HTML and bring it on your tablet to show two colleagues performance really good can up to do up to a million million points and it's also all the animations are done in the shader it shows some of that and something which is important to realize is that that mostly we work at least I do like local with a local notebook server but the duplicate system is really great for bringing your code to the data so you just log into a Jupiter hub etc work with the data and then having being in the browser you have the visualization on the client where it's really fast so so there's really no setup while try doing that with remote OpenGL so if you're gonna try that you're probably gonna cry okay so this is kind of my lucky slide so I've never had a failure after this slide so I hope I have better luck okay so for me it's really important especially especially in the notebook to have a really simple API just a few lines and get a plot and that that's also why like matplotlib is so easy to use just a few lines and you have something so let's do the import create numpy arrays and just call figure scatter and show you should have something so you now have a 3d scatter plot and because it's a jupiter widget you can change some of the properties i'm now going to change from a sphere to a box and inspired by BQ plot if you change something it's animated and it's really useful if you change something in your like a model parameter and want to change like okay how does this affect the data produced it's not a jump you see how it's like really affects the the data and it to answer a question from the previous talk you can just like save it to an HTML file and put it somewhere okay so it's built on top of a PI widgets from take this figure again so we're now going to change it from a box into a diamond but instead we can like take the IP widgets and have a like a taco button oh and change it to sphere box or diamonds so it's really easy to create like small gooeys - to see what you want or the size of the scatter points gonna create a slider and tweak the size until you're like happy so maybe it's fear looks better this size it's really easy to build a buoys on the fly with this so it's gonna be a bit dry but I just want to show you like all the things you can do with the high volume so cuivre plots is basically a similar to scatter plot except their arrows and they have directions and everything's animated so also the angles like if you change the direction you see what's going on if you change something colors are also interpolated so I'm now changing to green and again you can change use the color picker of iPad widgets to change the color to what you like which just use numpy arrays with RGB colors for every individual points meshes so basically you have XYZ coordinates and the triangles that that refer to these so let's make a tetrahedron so I really like the simplest you can do or if you have like something simpler like a surface you don't have to define the triangles you simply claw plot surface and again everything's animated so if you change the color etc so for meshes you can also do like texture mapping so here are like a model of a brain and like you have three fuse of this this brain and you can put a texture map on this this or this could be like brain activity or something which could also be a movie I'll show you some of that later on just to complete like everything you can do line plots again everything animators mm-hmm but like the main point of this is whether this package was the volume rendering so I have some like 3d cube and like similar to in show I want to have full show so just show me what is this 3d volume look like so this is a spherical harmonic as you may recognize and now you can like play with the transfer function and set what to explore this oops this this dataset isosurfaces this actually comes from SK image so this is done on the under colonel side so if you change something like you change the level of the isosurface it looks a bit funny that's because it's doing this interpolation but the number of triangles are like moving all over the place and if you change like if it if it's constant it makes sense but you can always like disable it I put it time animation to zero and that changed directly so what about performance so let's see if we can do a million in epithelial yeah so interactive and again still all the animations like changing the size or changing like the data is all animated up to like a million you can do 210 million if you really want to push it so back to our galaxy so we have all the ingredients to like enable what I wanted to see this like stream in 3ds so let's go up so that was this one so what does it look like in 3d so what I'm showing here is a stream you can't see because I'm over plotting like the me velocities on top of it so instead of recreating the plot we just tweak the parameters a bit tweak the levels of the opacity the transfer function a bit and now we have a much better understanding of what a stream looks like in 3d and it's really difficult to understand what the stream is for instance that they that there's this gap between here tells you something about the the symmetry of the system but fix is all about like large data sets so let's take dark matter simulation here with 303 million rows and you can't make a scatterplot out of this so that's kind of the idea you reduce this to a 3d histogram in this case you show a lower resolution cube of this but let's say we want to look like pick inside it has like depth depth of where it knows kind of where the 3d sensor is of my cursor you can zoom in and it will do the RIC calculate this cube again and show you show you that like the high resolution so similar to the 2d like in data shader you can also do it in in 3d but not everything is like tabular data sometimes you have like huge cubes from this is an example actually I'm now showing it in Glu Jupiter which is a new project coming out of the Bluefish project which is also allows me to work too so it's a package I'm working on and also allows me to work on IP I folio and it gives a bit of a richer interface at least now to work with datasets so this is an astronomical to like data cube and well let's say we have a really bad internet connection so we like really can't transfer a large cubes but we still want to zoom in so it was sure now it shows like a reduced volume and again if you zoom in it's low resolution and then it sends you like the assuming region so you can explore like really large data cubes so shortly on selections it's pretty basic but you can simply do like a lots of selection and do I have this here oh I forgot so I have this here so what I'm doing here is I'm observing when the selected property changed so I did this selection there's something happening on the front end gets sent to the kernel and based on this information I can calculate what the mean x of these coordinates are so you really have this bi-directional communication which is really important so now I have a data set which is so X Y Z are not like a one dimensional array but they also have a time component so the first dimension is times a - not keyframes and 313 particles and if you send such such a race to the front end it will assume that the first dimension is time so you see these particles here represented as arrows maybe yeah this is a bit better and now I can change what is what I call sequence indexed so to the next so it takes the next time step and now you can and because of the interpolation but course time step it still looks smooth okay we can do better and just use I PI widgets so we have a play widget in a slide and now we can have a like a smooth animation so this is similar to the stream but now this is a simulation of a single stream around a Milky Way like you see this this object being red depart so maybe you want to have like spheres instead of arrows do not get a sense of direction and I think one of the like killer features is the cutter plot in 3d that's not all so if you have a Google cardboard and you want to do like three pets in 3d like fiar you just save it to an HTML file put it on your phone and use the Google cardboard and Jean cloning we have actually we have a like a dome so like for for shows astronomical shows as well so I just had to support like these projections so you can have like your cats on on a dome or if you want to make like a youtube movie like in 3d you can have like 3d movie as well so I talked about movies but let's say we want to like make like a real movie let's not make it 3d let's do it just like this ok I'm not lost no and there's a small like sub packaging I by volume ok should have used to Peter lap and oh no no no let's print out the figure here as well yeah it's a simple package but just to show what's possible because I built on top of Pi 3GS I have access to the camera the scene there's animation support so I'm gonna add some keyframes so here and here and here okay do the interpolation to smooth and start this so now we have a nice like demo we want to record so we use the I pi webpart to see package to to make a recording of this we just record a few seconds just to show that we made this recording and this recording you can like download upload to YouTube whatever or put in your tablet so actually if you I'm not gonna go in the DEP but just to show you what's possible you can like make it a bit more fancy so this is what we did for like after the release of a paper so this is using all of these components so it's using the animation animation of the camera and a-three 360 so if you do this on google carports you can take look at all directions still fan yeah he's down okay now it's gonna be it's freaky okay so this web I PI webparts you see packets wanna show you show you a little bit about it because it's fun and it shows the power of of widgets so you can have like a widget which is a video a widget which is a camera HIDA and this can be used as textures in i by volume and that's not because like hyper volume is doing its best it's just like in the browser it's such a powerful platform you can just like say okay use instead of an image use a video and it just works and the nice thing is that a so the WebGL come fast is like a canvas and a media stream itself so I thought okay does this work and I put itself in itself and of course you can but WebRTC was not designed for this it's actually for for doing chats so let's make a chatroom hi so fan Hey he's sitting there and so are you seeing my plot now so actually and now I'm not sending Sofia my my cam and my webcam I'm sending him the this my plot hey Sophie you're actually on the on the plotted spell so now it's your Fanny seeing its own on its own etc okay that's but this is just to show you like the power of widget this is not something you would put into a plotting library but because we're using the same framework it's kind of a nice to be able to combine them okay well my luck is not a few words on the future so the next version so they actually what I've shown here is master so it's like not so stable at the moment but the next version is going to be more about PI 3GS integration and Fodor is doing a lot of work on this and that allows you to control like everything like camera lights for materials and the animations and ideas to be able to use all of the PI three-tier stuff like make a mesh and put it into I by volume there's someone from the Netherlands as well working on getting multi volume rendering working and also in this process we probably can have like order independence transparency which is about a big thing and in the future I hope that that we can there's another like volume rendering packets called an array that we can use that into an eye by volume or vice versa like really mix those there's also google Summer of Code student working on for M&E and making whatever needs to be done to I pay for him to make it work and so they also talked about like other languages to actually now you can use I by volume in all the jvm languages so I have an example of doing it like a volume rendering closure and in the future we hope to have while C++ supports as well you can find me at the sprints tomorrow just try pip install I buy volume give it a star if you like it open issues come talk to me thank you [Applause] Thanks we have some time for some questions so could you load a desk large object like it and raid ask array and view it I don't think you should do that because it's get sent over the wire so I think basically what I showed to take like large data sets large cubes reduce them to something like manageable and put this into and because of this bi-directional communication if you zoom in you get notified like what has changed like the axis of the of the scales and you do your new computation upload this I think that's the best approach there are some limitations to the browser if you really want a cat large data set you should use like 50 km a AVI can you export the output 2 X 3 D or you 3d no but I'm I'm I think tomorrow maybe also with you we should talk about it yeah it would be nice to be able to go from I by volume to my avi maybe to mop top lip to have some way of not just having a common API but also maybe underlying models or be able to go from one to the other hi I was I was really interested in like the VR capabilities of it are you is there more functions coming to like exploring your data with VR or any more functions in that area I don't have any big plans except that I've some VR goggles lying at home that I I'm trying to find excuses to use them but otherwise I'm happy to talk to you and accept PRS or think about this but no big plans no you know but feel free to consume it that was great um can deal with like the last election can you do additive selections where you like you know select something and then I closed all the riches the short answer is yes the UI is horrible you need to know like and percentage and etc there are only like keys for it but there should be a UI so you can do like a circle selection subtract like a loss of selection etc so yeah okay thanks thank you

Channel: Enthought

Views: 41,411

Keywords: python, visualization, jupyter

Id: hOKa8klJPyo

Length: 24min 23sec (1463 seconds)

Published: Sun Jul 15 2018