Alex Bozarth, Luciano Resende: Customizing JupyterLab using extensions | PyData Austin 2019

okay welcome to customizing Jupiter lab using extensions I'm Alex Bozarth I'm a software engineer at IBM Center for open source and data and AI technologies also known as kadai just as a starting point I'm making this talk an open Q&A so if you guys have questions at any time just raise the hand and I'll pause the talk the entire second half this talk is going to be one long demo so those questions are important I'll also be stepping away from the mic to cough every once in a while so pardon that so to start off a little bit about me I've been a software developer at IBM for four years now a little bit over four years now over those years I've also become a ppm C for Apache Livi which is an incubating project with Apache part related to patchy spark and I've given open source contributions to Apache spark Apache livvie the IBM model asset exchange the IBM model data exchange and most recently Jupiter lab which is topic here I'm currently working on creating AI centric Jupiter lab extensions trying to help out the data scientists that you want to use Jupiter lab given him more extensions to work with to give them a better environment I also put my contact information up there if anybody wants to follow up after this these slides will be available in some way yeah so that's my email my twitter handle and my github and my linkedin so that's me short summary you know a quick summary about today the Center for open source data and AI technologies just a quick spiel today aims to be to make AI solutions dramatically easier to create deploy and manage in the enterprise we are an open source shop within IBM pretty much fully dedicated to do an open source development that will both aid IBM and of course the open source community that IBM wants to support we're based in San Francisco and that's a nice picture of our lobby it's real plants so so yeah that's code a and I got this cool image from my teammate about IBM's open source participation throughout the year IBM is active in the open source community as both consumer and a contributor and has been for decades as you can see from all the things that various parts of IBM has been a part of we might be big and old but we're not not hip we are the big old fogies of the industry but we are keeping in touch so that's yeah that's just a little bit of an introduction so with this talk I want to kind of give you guys just some quick questions like what is Jupiter lab what are jupiter lab extensions why you should use Jupiter lab extensions when you should create Jupiter lab extensions and in order to answer those questions I'm gonna kind of go over starting up Jupiter lab installing some basic extensions and a demo of how you can very quickly create a very simple extension of your own and as I said before open Q&A throughout this talk so if anybody has a question interrupt me just raise a hand so quick question to start how many people here know what Jupiter lab is how many people don't know what Jupiter lab is okay so most people generally know what Jupiter lab is how many people have used Jupiter lab okay so it's essentially the same group so this will be pretty easy for the most room jupiter lab is the next generation UI for project jupiter yeah have a little picture up there of it nice simple it gives a web-based interface for interacting not with Jupiter just Jupiter notebooks but also other types of text files terminals and using extensions many many other types of files and the goal is to eventually replace the classic Jupiter notebook UI you can get all of this off of their front page pretty straightforward it is next generation Jupiter notebooks any questions I like this audience so Jupiter extensions what are they Jupiter lab from the beginning was designed to be extendable extensions of Jupiter lab allow you to create new file editors for different types of files and output your visualization in different ways so you can show a JSON file both in an editor or in an interactive drop-down environment it allows you to add buttons and menu items to other people's extensions and art and features that already exist in Jupiter lab and one of the coolest things is you can actually create back-end api's that run in Jupiter lab that other jupiter lab extensions can then call so if you want to create a new API and another extension that uses it you can do both and Jupiter lab itself is actually just a giant bundle of extensions they built it so everything you see in Jupiter lab is just an extension that just comes as the core jupiter lab people kind of getting along getting that cool so why should we be using jupiter lab extensions well one the core extensions are intentionally limited in scope they don't want a data scientist and an engineer to necessarily be getting each other's super specific extensions when they open up Jupiter lab they want it to open up run very quickly and have the gist the basics nothing extra so then you can customize your own environment by installing extensions you need because everybody has different roles data science can use it engineers can use it the layman can use it so the idea is everybody has their own scenario and you want to install extensions to your scenario any questions on the kind of the overview of extensions so I'm already into my demo so my goal with this demo is I'm just gonna quickly walk you through how to install an extension in Jupiter lab that already exists and then my step after that will actually be to show you how to create your own extension so I'm gonna just quickly swipe over please say it works yes okay so this is my terminal the best part about this you just type so I've already installed Jupiter lab that's pretty easy you just hit installed Jupiter lab but I didn't feel like having you guys wait while it's installing so typing out Jupiter lab I'm gonna increase the font size here I don't usually do this okay it's the same as every other app is that easier to see okay Jupiter lab it's really cool about this is it will immediately open up Jupiter lab in your default browser so this is Jupiter lab as you guys said most you guys have seen Jupiter lab before correct so you in Jupiter lab you can open up a notebook file you can open up a Python file and you can see it's fully editable in the Python file similarly with the decor functional you can open up a markdown and you can actually show a markdown preview all of this is included in core no extension required but extensions are pretty easy to install you can install them via command line but there's actually an experimental feature called the extension manager you just turn it on it gives me a giant warning in beta you just turn it on and for those that didn't notice there's a new little side button over here extension manager I have no extensions installed can everybody see this size-wize cool so - let's try to install extension so table of contents is a great extension a lot of people have talked about putting it in core because it's such a common extension so let's see TOC table contents there it is it even has a little install button if you click this it will open up in a new tab the git repo it's coming from but we're just gonna click install you can see the little blue bar go and it's installing wait a sec and then it's done and just asked us to rebuild real quick so in the background while using Jupiter lab it will rebuild extensions for you dynamically without you having to go close Jupiter lab and then once it's finished rebuilding it will prompt us to just refresh the page of course it is building does take a second or two but table of contents is a pretty small extension doesn't take that long to build and if you want to see oh it's already done click reload and on reload you will see there's a new tab over here called for table of contents and it'll show you the table of contents of whichever notebook or even markdown file if you click you see it goes to that tab that's an easy basic extension probably want to use you can all see how easy it is to install and pre-done extension that makes sense to everybody cool so let's try some stolen another extension where are you extensions there you are let's try get a lot of people like using get the jupiter lab team has a jupiter lab kit extension which they would like to remind people is not production ready as they reminded me this week when i mentioned that i was gonna use it in my presentation so let's click install in the get nope it gives us a weird warning it says it needs a corresponding server extension called jupiter lab - get well sadly you can't install server extensions through the UI like you can install the front-end extensions so we're gonna click cancel and we're gonna install that one via the command line so I have my little environment stuff saved here we are going to copy this so usually you just do a pip install but because I'm using Conda Conda doesn't like working with native pip so I'm using a Content stall from Conda Forge so you can see the normal pip install is very easy so we're gonna come over here I'm gonna kill Jupiter lab just you had a nice control see confirm and then I'm gonna paste in my content style it's pretty quick alright yes you can install them the same way via command line yes yes typically if you are a dev and you're setting up an environment for all your data scientists you would create a script that would just go through and install a bunch of them that's what my team does for our data scientists so so this is just a normal Condor or pip install and it's done and then we just have to rebuild Jupiter lab because I don't feel like typing I'm just gonna copy and paste Jupiter lab build if I just started Jupiter lab it would have then prompted me to build but I'm gonna do this it takes a minute again because all this is of course actually doing these things this isn't all pre done all I did was type out the commands to copy and paste them so as you can see all this is very quick this would take the same amount of time for you guys gotta love building but again not super long everybody here has had those 5 to 10-minute long builds right forward longer faster so if you look at it says build minimize so it's actually just building what's changed and the dependent parts of it when I am doing dev environment work I typically do full cleans and complete builds on every change just to make sure I don't miss anything okay see nice and quick compared to a lot of stuff I'm just gonna do this so then let's run Jupiter lock I'm gonna kill this other tab okay it remembered where we were and now inside the extensions manager gets there we also see that git has created a thing it's also noticed that if I looked at my file browser I'm not in a git repo so it is telling me you're not in the git repo and says go find a repo so if I had switched to a folder with a repo it would find it but I don't have that so we're gonna do this I can get clone right from in here and I am gonna get ahead for our next demo and get clone the repo we're going to use on in the next part of the demo which is the extension cookie cutter just copy that URL in there and clone no it didn't like it there was probably a space let's see what did it not like it really did not like that yeah that's worked last night yeah it's just the git clone in the URL it's not liking it today let's see if I can get it from the next no it's just the URL so once a dot get or not Wow I didn't like that so apparently it's not liking me today but typically you can do that we are just gonna go do that in the terminal then because we can so and you'll notice when I come back it's already seen it's there so when I open this up and I covered it the get how it really didn't like it remember when I said it's in not production writing example of not production ratings so as I said it worked last night and it's not working today perfect example of not production rating so I'm gonna go back to my slides real quick because I bet you anything there was a step that I skipped when building but worst case is we just move on yeah we're just gonna move on unless anybody has any questions about installing you notice that the extension did get installed though it's a good point to point out I did mention it earlier there's two types of extensions in Jupiter lab there's lab extensions which are front-end so like the table of contents in the UI side those ones can dynamically be added while running lab then there are server extensions which are typically the backend they are the API extensions that allow that back-end your for us your front-end extensions those have to be built and run before starting Jupiter lab and they and yeah so that is installing pre-done extensions so moving on to the next demo which is let's create our own extension so the repo I checked out was called cookie cutter it's a extension cookie cutter in typescript that the Jupiter lab community has already made it's really easy to use you check it out we're just gonna CD into the cookie cutter it's called extension there we go and we're gonna follow the instructions in the readme so if we go over to here we can open up the cookie cutter readme I'll see markdown preview can get a little weird we're just gonna put that there so these are the steps pip install cookie cutter so let's pip install cookie cutter done and then we use the cookie cutter to generate it's just cookie cutter and then the cookie cutter repo name this allows you to create your own customized cookie cutters and still use their scripts and it's gonna give me props it's warning me that I've done this before and so yes I want to delete my previous thing author I'm Alex we're gonna call this my button and it's about my button excuse me and it's asking for a repo that I'll eventually push this to I'm just gonna copy and paste it called it just in case I decide to push it and it created a whole extension so we're gonna come back overdue for lab where we can actually edit all this we fresh this and you'll see there's a my button in there now part in my asthma and so we're gonna open up the my button extension all of this is now generated we have the package.json with all our build we're gonna close all these extra tabs that we've had open let's open up the index file this is all nice and generated and right now if you run this extension all it does is when it starts it logs my button has been activated pretty straightforward but we want it to do more than this so I have a couple pre done lines already so we're gonna drag so a couple up here all the way up here we have this button dot ts file that I created beforehand it hasn't some button code that I'll explain a bit we're gonna copy this we're just gonna go back to the my button all the way into source and we're just gonna paste it and now it's in that directory all through Jupiter lab we're gonna close that real quick and then in order to import it I can just import it in typescript using my import line and then in order to use this button I have two nice lines of code that we put in this activate function here of course copy and paste never gets the indentation correct because why would it okay so what we're doing here is we're creating an instance of my button from that class then we're adding the widget the button widget to notebook files so whenever somebody opens a notebook it knows that my button widget is going to be run with that notebook and this is the button file so what I did is I created an extension uses all of these nice long models but this is essentially saying that it's gonna be part of a notebook file and I created a create new function which takes in the notebook and I create a button my buttons called my button when you click it does a nice job of script alert you did it and then I can easily add my button to the notebook toolbar I put it in position 9 right at the end called my button and then it returns the button itself so now when a notebook starts up it runs this it creates a new widget and they create in creating the new widget I've given it back a button to put on its toolbar as you notice there's a lot imports up here so we need to quickly copy and paste all those imports into our package JSON otherwise it will just not build so open up the package.json you'll see that JSON is open by default in a JSON viewer which is pretty cool but not super useful to us right now so we can do open with and we can open in the editor instead when we come down here to dependencies and we just paste in all these new dependencies make sure all my commas are correct and then the key is Jupiter lab by default doesn't autosave so you need to save each of these files to make sure they built right okay so we've customized that it's a pretty kind of following how easy that is to kind of create your own thing in general okay and then we are gonna open up the readme for this newly created application it has build instructions in the readme that was generated by the cookie cutter the build instructions are run JL p.m. which is a wrapper around yarn I think it's yarn or is it then yeah it's yarn and we're just gonna CD first into my button and then we're just gonna run JL p.m. and it's gonna build my button and hopefully it will succeed on first try as long as I saved everything and there's no missing commas so because of the fact that I did input a handful of packages that were not there before has to download all those packages from MPM so it does take a minute on the first bill so on your very first build you run JL p.m. and it also downloads all the requirements from NPM but as you're editing your widget you don't actually have to do JL p.m. the whole time you just have to do JL p.m. build which just quickly rebuilds the code without redownload in your requirements and what's really cool as we come over here we go over to our extension manager Nothings there's right let's refresh this see nothing's there so we must have missed a step yeah we did so the next step is Jupiter lab extension link so we just in this directory we're gonna link this current directory to jupiter lab so Jupiter lab knows that there's an extension in this directory that it should be running you can also link it and install in other ways but the simplest is linking local so we're just gonna quickly run that and it's thinking thinking thinking thinking so what it's actually doing in the background is it's actually gonna do a again a minimize rebuild of Jupiter lab to make sure that Jupiter lab is gonna run with this extension and okay and again this is all able to be done while Jupiter lab is running because it is a front end extension but of course it takes a minute or two because it's still a build why it's building any questions you would have to restart the whole thing because back-end extensions are stan cheated and runned at the initial runtime because they are server-side so they're usually written in Python whereas front-end extensions are usually written in typescript but not the not always yeah yes it's the it's the extension manager it will I believe it only searches for front-end so if as you saw with the get if the front and extension relies on a back-end extension it will warn you that it won't work without it I believe it's the I believe it's a pip I believe it's a pip sir it's NPM tax and so you and you can actually when you create your own and publish your own on github you can just add that or you when you publish it on NPM you can just add that get tat that tag and then it will show up in the search so you'll see over here that my button is now shown up after doing a quick refresh without restarting Jupiter lab you also noticed that it doesn't have this pretty nice logo this logo is for people for extensions developed by project Jupiter yeah so it's already there so now we're gonna come over here and we're gonna back out and reopen one of these notebooks you can see it always opens it in whichever section you're currently in so you notice oh it's not liking me let's try that again yeah my button it's right there on the end and when I click it you did it super easy to install and you can make that button do whatever you want and if you create a server extension to go with it you can have it call an API that then takes input from the user and submits it to a back-end that does something and return something or you can make it just say good job so this is how easy it is to create a front-end extension you can do this all inside of Jupiter lab without ever restarting Jupiter lab and you can do all your development in Jupiter lab which is really cool so that is the end of my demo I did include a bunch of useful links and a link to download this PowerPoint file so I don't know how we're gonna share that link yes they would have to install the extensions the extensions are related to their instance of jupiter lab not the notebook so the notebook is just a file and duper lab is closer to i like to compare it to an ide but that's not exactly the best comparison so their instance of jupiter lab would need to install those those extensions now if you're using a cloud-based jupiter lab so somebody else is running jupiter lab on their servers and you're just oh and you're just call opening it through your web browser you don't have to run Jupe lab local and so you're both running off the same instance of the jupiter labs then they would also have the same extensions but if you're running it local then you they would need to install all those extensions so as far as i remember if you are running a cloud environment and you are both interacting with the same jupiter lab instance in the cloud you will have the same extensions both server and front-end and so if you install one I believe in theory they should have it too but it also depends on who your provider is a lot of providers silo it per user so you don't have that yes they're lots of different companies are creating bundles of extensions that say these are all the extensions that you should have use this script to install them and according to my coworker they can use binder yes yeah yeah yeah if I remember correctly the person who spins up there's a setting you can disable the extension manager the UI one that I've been showing you guys which would then make sure users can't install extensions or you can or the essentially typically though it is cool that you can do this on your own if you spin up your own instance the DevOps team that is running in the instance of your jupiter lab in the cloud will typically have a set amount of control that they have and amount they want to give their users and that is usually up to them and it can range depending on how much they want it's very customizable I given I mostly work in the open and front-end I have not touched the DevOps side as much I have a co-worker who sits behind me does it but that's all secondhand knowledge and for me personally yes yeah so oh I didn't actually mention this so I've been mostly writing my own extensions which sadly I we have not published and I cannot share with you despite having them done for a while but I included a link to this PI data Berlin talk from this last summer there was a great talk where they just literally spent a whole talk going through 20 different extensions that are great in common to use and I have a link at the end it's called a tour of Jupiter lab extensions they did a great job and I didn't think it was worth three doing what they did yes so when you create your own extension you can publish it to any NPM repo you want so if you have your own internal NPM artifactory you can just publish it to that and anybody within that has access to that can do it or if you want to you can actually just run it locally you have all the code locally and you just install it locally and it's not published anywhere so just like how we just did the button right now that was all locally it was never published what that answer yes cool this is my team leads yes oh let's see I don't believe so you sort of have to migrate a couple of other labs like all types we deployed but most of the very the most user ones we have already seen people like alte migrating lab although so and it is pretty straightforward to migrate your own extensions this button demo I had my first time doing a button demo it was it was a notebook button so we had a button we just moved it over any other questions or we could call it early and everybody can get coffee okay if anybody wanted to take a picture of that link just in case it's hard to find it I'm gonna leave this yeah that's my last slide so I'll leave this up for a bit so people can take a picture it's IBM voxcom /v / AJ Bozarth @ - PI data 19 nice easy ish to remember URL yeah so if that's it thank you for joining I hope I helped somebody thank you so much [Applause]

Channel: PyData

Views: 2,772

Rating: 4.6279068 out of 5

Keywords: Python, Tutorial, Education, NumFOCUS, PyData, Opensource, download, learn, syntax, software, python 3

Id: PT9UIvLWUI0

Length: 34min 43sec (2083 seconds)

Published: Wed Dec 18 2019