Jeremy Tuloup: A Tour of JupyterLab Extensions | PyData Berlin 2019

hello everyone - Jeremy and today we're going to do a little tour of the triple add extensions so there is a ripple on github so if you want to follow along there is actually a binder link you can run the same presentation as I have at the same time or you can also check it out later if you want okay so before we start I'm currently working at constic lock on stock we specialize in the scientific computing so we work a lot in Jupiter ecosystem that means building tools for Jupiter and building on top of Jupiter we work for support on the Jupiter live extensions i buy widgets and also on steep response tools such as extensors and so the C++ columns so if you have more questions or if you want to know more about what we do if you talk to me at the end and check the website here if you want to find me your talk later you can also do that on github and Twitter all right so when we talk about extending Jupiter there is actually a lot of different ways to do so maybe not 99 but there is this blog post from Julie who is a quad core Twitter of Jupiter working on Jupiter hub and he took the time to go through all our different kind of extraction points we have at the moment for extending through Peter it's quite funny because the first one is the Jeep Arabic stanchion this is exactly what we're going to cover today but it's not only about this it's also about notebook extensions server extensions superior honors ipython magic and so on so as I said we are not going to cover all of this today only the GPL extension which is already quite quite a lot so let's get to it so about 2p lab quickly it's the next-gen UI for project Jupiter so you can think of it as the kind of the succession of the classic notebook it turned the version 100 early this summer so that is quite important because that means it's both ready for users which was already case like one and a half years ago but it more is also ready for developers so it means that if you want to make an exception for GP lab the API is now stable until of course we reach 2.0 where we're going to break everything but yeah that's the main point and that's why I wanted to show you now like a tour of these extractions that could be that can be built as of today while the main selling point of super drive is that is extensible by design that means that basically everything in the develop is an extension if you make an extension for GPIB has the same right is kind of lives on the same level as all of the others when you install to fill up from peep or coder you got your opening actually to get 60 of them have sis extensions but you don't need to necessarily see all of them that point you look at the year at the UI because we're going to show you an example of call centers afterwards yeah so all of these extensions are built in JavaScript typescript the using NPM to to publish them so this is really built the kind of the modern way of making JavaScript web apps as of 2019 which is also a huge improvement compared to the classical book all right so a quick a quick word on core extensions so we're actually currently looking at GP wrap and for example if you look at this file browser this is an exception in terms of HP wrap it just that ships automatically when you store it the same goes for the console so here I'll put the console can type code execute code this is also an extension that is built-in Shapira and of course there's also the notebook exception does two one IDs with the notebooks and that shows media notebook in the UI right here but today I want to focus more on the third-party extensions that means all the expressions that we as developers can build on top of you pair up give you a little idea of what we can do put a little video here so I'm going to play it and show what it means at the same time so this is running a simulation and on the left side you have 3d scene of a robot on the right side you have a notebook that actually controls the simulation so this is pretty nice because here the only thing we did was to extend the the GP lab interface and use these extension points to basically be able to do this so here we have a 3d model on the left side but technically could also be a like a real physical robot and the good the good thing about this is if we can actually reuse the whole you either already exists in a repeater such as two notebooks and consoles and so on alright so that was just a little intro to give you an idea of what we're going to talk about so we're going to start with the ten of them that's not going to be like the first batch of extensions we're going to look at then we're going to look at ten more but a little bit faster because we don't have that much time and idea that we show you a little bit what is coming up so exceptions that have been released that's the one that we're going to look at now and at the end we're going to look at the ones are going to be released in the future okay so why important thing when we work with notebooks especially in teams is being able to version code the notebook is not different on codes it's just coding stars and and again we get a JSON file that we need to put somewhere and share with all people so there's an extension in GPA lab coded fill up get extension and wait does it serve it cropped but yeah so there is an icon on the left side you can click on and you get this kids UI so this presentation is actually git repo so you see all of my comments here so the history can go here and I see that my notebook has changed I can stage it write a message and press commit can also for later strategies use it using this pattern as well as pushing new changes using this one so basically like simple git workflow but that is usually then enough for like a basic work with notebooks but the good thing is that you can actually use kids also in a turbine or and this extension is just hooking itself on the existing kid comment that you have installed on your machine so you are free to basically choose what you want if it's just hook quick and dirty commits in the UI that's perfectly fine and also a bit more complexed reverse and so on using the terminal this is definitely possible alright so that was about it but before we actually commit notebooks we often want to know what change is in the book and we want to be able to give them and fortunately there is an extension for that called anti dime so when bidam is actually a tool that you can run the other common line integrates rewire with get so you can configure gates so it actually shows and be damned if and not give thieves and it's very useful because if you take a classic notebook in the JSON format you're going to see a difference on files which is not very handy but NB time is able to recognize the cells and to a cell by cell diff which is very very useful and here this is even better because we can actually use it in super lab so for example I have this cell I'm going to change this value execute it quick to seven a notebook and I can click on this and we see that a basically a diff of the notebook we even see the video I executed before because now it's added to the output so it's also shown here so it could be used to really make sure that we are not committing outputs for example if the policy we have in our git repositories to never commit outputs then we can really quickly see that and also yeah have a better diff that and just just advice okay so steer about gates D thing and now let's say we want to also ensure that everybody is using the same code format that's great because there is an extension for this so it's called the code formatter this succession is meant to be used as a kind of add mystic code formatter extension so basically supporting multiple languages not only Python but also multiple formatters here I'm going to show you the how it works with black but basically you could do the same with Auto pipe eight and other four matters so this is very badly formatted cell and I would like to format it using black so what I can do is I can open the comment palettes and search for format and here since I have installed black it shows up in the list I will have installed another formatter there will be another entry so I can just hit enter and now the cell has been formatted so this this is really useful if you want to have first if you want to limit the number of dips you have with other other colleagues or something like this for support says I have not been modified they will stayed with the same code side in our inner repo okay next one is the table of context option so it's going to be pretty quick because it does one thing but it does it well it as this little icon here on the left side and you can navigate through new notebook so it's basically two kind of passing all of the titles of the Malcolm Malcolm cells the creates kind of entry parts for you and kind of anchor anchors so you can navigate through the notebook very easy so very useful if you have a very large notebook I want to basically jump around in a matter of seconds okay quick open is meant to bring you the functionalities of this control shifty shortcut you get in other IDs in his code sublime pycharm and so also you want to fuzzy search a file that's what you can do with quick open so you get this little icon on the left and let's say I want to open the readme file and search for Remy press enter and I get my file open okay next one is a sidecar so I think I would need to change the ID ratio because I don't think you were seeing the left side scream never mind yeah the sidecar so the idea with the this extension is that interpreted AB you have on the left side different entry points for extensions here but I don't know if any of you knew that you can actually have things on on the right side of your P lab so you can actually do that here by doing the right-click and switch side by side that will bring this little icon on the other side and basically move the extension on the other side and the sidecar extension does the same but programmatically so you can use it to display widgets on the right side here I took the example of going to take a pipe I can pass as an example we chat sure to put on the right side so I pack invest is being developed by Martha and you can also check it out so you will see in a minute how well it's doing so I'm creating a creating in canvas so we can see our canvas as a HTML canvas you get if you work with JavaScript and you can draw things on and here I'm going to draw a face it's a poker face not very happy but what's there in on the canvas so I'm going to scroll and it's already kind of already gone right I don't even see where it is so that's the whole point of using the sidecar exception is that instead I can put it on the right side I can start scrolling with notebooks and it's going to stay there even better now we can actually make the face happy and start modifying the widget and yeah so it's also useful if you work with maps for example you want to have a map on the side using a pilot and make some modifications so now we're going to keep that face with us the rest of the presentation now I'm kidding it's not true it's closest okay all right so speaking about drawing things there's an exception called draw IO for a GP driver so maybe you've already used this website called draw dot IO you can draw diagrams and then export them you can send them to XM eros or a PNG and SVG and that's the great thing about this trip Arab ecosystem is that you can actually reuse existing libraries very easily so that's - that's what happened with this extension so that this is basically wrapper around Road on Io and it means that if you go back to the launcher can create a diagram here and you get the same thing you can start creating your diagram and then save it to a PNG and potentially embedded in your notebook so you don't have to leave the HP lab anymore can just do everything in the same interface alright so one thing I've heard quite often is people want to to be able to use this face that you have here at the top of trippy wrap which is usually empty but if you use this top extension this won't be the case anymore because now we can start making use of it this is particularly useful if you want to be able to put some indicators one of them dies yeah it's quite use this to show the memory usage of system metrics so that's what the top bar does exposes the space for other extensions to place indicators or other widgets in there and for support this one is like little switch to toggle the theme very easily instead of going to settings and so on I can be useful to some people and here we have the ability to put some text so yeah just put the name of the presentation so we know where what we're doing at the moment another thing you can do is you can actually rearrange the the different extensions here so we can drag and drop them if you want okay so for those of you working with seeker and who want to basically run sequel queries very easily in trippy lab there is superdad sequel exception the same as before we can go back to the launcher and click on score and here we basically enter a URL to the to this database we want you to use so start to type sequel right that is because in my examples repo here I have examples an example database so testdb enter and now I can explore this database very easily in trippy lab can look at what is in the employees table and then I can go back invoices and I can also select folks up or select all from invoices where total I think is to that greater than five control-enter and I'll have all of my invoices that have a total greater than five of course if you know sick or against to a bit more advanced things like followed by tort or oDesk support and basically that's what you get so that's useful when you have sequel queries and put it in your Python code say you see correctly me for example we still want to be able to run those sequel queries kind of like like this like a one-shot query just to make sure that it work okay so last one for the first batch is called the sell tags maybe it's also like some kind of hidden feature of the notebooks but not vanish we have metadata about the cells the some of this metadata is all the tags so if you go here to the this section here did not book tours you see that you have possibility to add tags so let's go back to this and let's say this is a special special cell and now we have a tag called special I can click on the other side but then I can select this tag here and select all sides with this tag and I now I'm back to the other one I actually marked before so this is useful if you want to force up or navigate quickly in a notebook like this but it's also useful in the from sample if you create a pipeline on a CI system and you want to force up or format cells to matically based on their tags or to more advanced stuff that you can do programmatically all right so as I said now we looked a little bit more in 2004 in ten of the extensions and the time all are coming but we're not going to spend as much time just to give you an idea what is it variable so one once again it's like the whole purpose here today is to just give you an idea the extensions we have and the fact that you period is not you know liking behind the classic notebooks that extensions are being ported over to prefer lab and exit as usable as the classic notebook okay so first one is to go to the definition extension and as the name suggests so for example if you have used other IDs you know that you can usually do a right-click go to the finishing or click on f12 or GD using them and it brings you to wear something was defined so in case of fractions if you do that you will be able to come back to the where the function was defined yeah so again this is like breaking IDE like the truth to - perhaps we don't have to switch over between two different IDs if you if you are looking for these kind of features something very similar is the droopier lab LSP exception and LS P stands for language server protocol this is already being used in other editors to give you more like install introspection utilities for for your IDs and the good news is that we can actually integrate that with trippy lab so it's still kind of a work in progress but the results are pretty pretty good already and it's already possible to try it on binder if you want to just go to the repo okay so maybe some of you have heard about vara which is a tool that turns your notebooks into a two web application and dashpots yeah so there is also a period extension for for whare I will show you a little example so let's say we have this notebook here this is the basic notebook you get also in the documentation of water that can interact with it but I'm still seeing the code and I want to see what it will look like and start sharing with you water so there is this little icon here render with water I can click on it and on the right side I see the preview of the notebook rendered with water it is also usable here and you can actually stop et right or middle book on the left side the using the render unsaved checkbox here it will automatically reload your preview as you type okay so speaking of live previous one is more work for educational purposes is the Python pipe code extension so if you are interested in knowing how your source code is being you know translating to buy a bike code you can use this extension so it's the same kind of thing you type Python code and and then you get the bike code on the right side that is being automatically shown okay so my per clip is probably one of the most used blotting libraries and we've been using this multi beam line for years and it's been working real well that gives you static images so now there is this MPL I buy widgets they can use in GP lab and it comes with with this search period my particular expansion and where does basically add some kind of interactivity around the process you make with my club my Bradley so this is pretty nice because you don't have to rewrite your products you can use the same API as you've been using for a long time and my prototype API but still get some basic interactivity ok so after we also want to know what kind of variables we have in our color at the given moment and this is what the viable inspector extension is doing so here you can see that as a as we type we get create some variables and we have this right panel that shows what variables are being our life in our kernel at the moment is a bit similar to what MATLAB is doing with these survivors you have on the side I also has support for showing vampire race as a matrices it's pretty useful so the Stila but introspections and that kind of looking under the hood or what is happening there's this CounterSpy extension that you can use and it's especially useful if you're making an extension for triple-a is talking to a colonel or making a custom ipad widgets so what it does is physically intercepting all of the messages being sent to a color and being received from a colonel or so you can physically see what is happening with your extension as you build it okay so I think there are three left for this second batch so it's going to be a bit more random and this one is about integrating Heroku in trippy lab and this is especially useful if you want to for sample in one click deploy an application to Heroku using a notebook so again this is taking voila as an example but it could be also any other Python Python application it adds a little kind of UI sugar on top of here oh cool come online so you can do everything in Jupiter okay so let's say you work at a company that's kind of enforces a very strict way of how notebook should be written and this is where you can use this template library or the templates extension so you can create templates save them and then you can create new notebooks using templates that you choose here so let's say you what you have a logo of a company at the very top then this is two extension you can use for that okay and just to show that it's not only about features and extensions with also but seem so themes again they have the same kind of level as your other extensions if you filled up so we can create new theme using the extensions and this one is the matter darker once it's installed it's going to show up here and click and click on it and then switch to the new theme alright so where do we find all of this yeah I agree it's it's a very interesting question so there are three different ways compared to other IDs there is no marketplace yet for Java we don't know yet it's going to be the case or not but at the moment we can get up and search for topics so the true period extension topic that requires people writing extensions to actually put this topic but this is often the case because it's documented in the cookie cutter for extensions so you can go there is 140 at the moment there is also the exception manager which is now part of GP lab it's not enabled by default but you can enable it here and click on enable extension manager it's going to add this little icon here on the left side and then you can look at the one that you have installed and course also search in the in the box for new extensions and finally there is also a awesome contributor github repo so this is not only about Shapira but also other computer resources there's a section about triple extension so if you're looking for other computer related tools yeah and it's the difference with the rest is that it's curated so it's kind of e to the pro board so it means that usually we get some high quality content there all right so what's next as I said now we're talking about extensions are in the making but not released yet and one of them is a debugger UI for Jupiter lab and this is what we've been working on that construct with the folks from two sigma as well and it basically brings you a debugger interface to GP lab so it's a everything is being developed on github so if you are interested you can also check it out there's a rich text mode exception being bit by the interns at Cal Poly in the US so it's like a WYSIWYG editor for markdown starts same kind of thing as you get on using word package installer to install pip and color packages without using this tip magic or pip comments in under book commenting extension similar to the Google Docs style where you can put comments and then you can share the box with the comments with your colleagues and many more of course so yeah so if you're interested in this if you to check out all of these resources with that say thank you and if you have any questions just come to tweet at the end [Applause] hello so we have a time for one or two questions left does anybody have any questions for Jeremy yeah yes thank you so much for a great talk and great extensions as well and I was just wanting to ask um like if you're working with a puker you know our requirements don't text you can have an approach project or an environment Akiyama with Conda are you thinking about doing something similar for like Jupiter extensions or is there a good reason why you think that wouldn't be a way to go yeah so the question is about how do you manage all these extensions the similar way as as with a requirement of text or have a medallion board and at the moment so there are two ways of doing that so some some extensions are being kind of bundled in the Python packages so if they focus on put some extensions here that the mode require a component on the server side as well which is the case for the the get exceptions and here what you can do is you can actually install automatically the GPIB extension by putting it in the right folder and then rebuilding the GPU lab so that's one way and the other ways to install it by hand using the trip arab repeater lab extension install command like here and it's going to install the extension in your environment yeah so that's the two different ways at the moment but I agree they will be great you have some some help spec to be able to manage extensions and also there are versions of it bit easier way thank you very much chairman we fought a wonderful talk and I think you can enjoy the coffee break now see you later [Applause]

Channel: PyData

Views: 6,098

Rating: 4.9058824 out of 5

Keywords: Data Science, Python, Community, IDEs/ Jupyter, Visualisation

Id: 3pdrzhny9Lc

Length: 28min 58sec (1738 seconds)

Published: Thu Dec 19 2019