Chalmer Lowe - bokeh: Data Visualization in Python - PyCon 2017

I will tell you who we can talk to about that to get some specific answers though there are there's a whole collection of folks who work on the bouquet libraries and they have been very good to me at least of answering questions matter of fact I ran across a problem this morning and I had an answer in 30 minutes and it was an answer that even worked thank you for you miss Carol how are you today well I don't know if they're a good crowd we have a crowd we don't know yet if they're a good crowd well welcome hopefully you'll have a wonderful week you're starting with okay or continuing your Davis website doesn't immediately I actually work a lot you actually helped out with some of the documentation for bouquet last year didn't you i yes I don't know the bouquet channels are very responsive yeah if you have any Wi-Fi issues tell them to use by g-time gee thank you appreciate it get her just one sec so as you guys are logging in and you're connecting to the Wi-Fi make sure you use the five gigahertz Wi-Fi we had some notoriously bad problems with the 2.4 gigahertz too many people trying to connect all at once and things got really wonky he was painful there was crying so use the 5 gigahertz Wi-Fi and there with me was it just as bad all right so there might be crying on both bandwidths excellent all right so take one of each of these and pass them around one of each of these and pass my round yes sir and we'll talk okay all right so we'll get started here for those who've not already gone there the material is up on github you can go to my repository Chalmer low no spaces no underlines and go to the bouquet tutorial we're going to go through a series of things today and let me get us started here okay all right so figure out where we want to start to print a book so and we don't like that alright alright there's going to be a part here where we install some stuff that is essentially part 2 so if you are on the github 's and you scroll down the page we have an install if you want to kind of get a head start on that while we're working through the introduction you're welcome to do that I suggest that you do so if you have any problems we can work through them hopefully a little faster alright so I'm going to tell you guys a little bit about Who I am get a little insight into who you are we'll talk a little bit about what boquete is right so let me see if I can blow this up just a bit more all right so my name is Chalmer Chalmer lo i'm on twitter Chalmer underscore and if you need to reach out to me about something you have a question Chalmers at the dark at dark art of coding comm so what do I do I'm a founder of a company called dark art of coding we teach folks how to write code also founder of a Python meetup group in Hawaii called ironically PI Hawaii I am a senior computer scientist for the consulting firm Booz Allen Hamilton people give me data I look at the data I try and make sense of the data I'm a past share and the co-chair of the Python education summit so tomorrow we are having an education summit I think there's about 150 folks signed up to come and talk about educating folks in programming Python etc and I'm the introduction to sprinting guy for this year so every year we have Sprint's at PyCon we attempt to try and get folks involved in contributing to open source software past two years we've run kind of a workshop to teach people how to do that and this year I'm working with those folks and I'm helping to create a workshop that workshop will be Sunday so if you have fun here and you want to learn more things about contributing to open source software come see us at the intro to sprint workshops I do want to take just a quick minute and have a couple people kind of volunteer and tell me a little bit about you know what kinds of things you've done with Python whether it be a lot or a little and whether you've got any familiarity with other visualization libraries like matplotlib or Seabourn and if you have some sort of a pet project you'd like to visualize in boquete by the time we get done anybody willing to take a stab and then say something go ahead yeah yep all right bouquets got tools to help solve that problem and bouquet the way I understand it's also got interfaces for streaming data so as new data comes in you can continually update what's going on right so there's something anybody else willing to volunteer that'll be awesome anybody else cool excellent and Boca I think also has a mechanism for interacting directly with Jinja templates and stuff so if you already have a familiarity with ginger templating if you're probably going to be in good shape what to look into that all right so let's talk a little about what bouquet is it is an interactive visualization library it's designed to target modern browsers for presentation the attempt here the goal is to try and provide some sort of elegant and concise construction right in the style of d3.js so you have lots of interactivity and to extend the the capability with the ability to look at streaming data or very large data sets right terms of today's agenda we're going to do some installation of Boquete and other pertinent libraries I'm going to talk a little bit more about kind of the history as it were the philosophy of bouquet we will do a very first basic graph we're going to figure out when the breaks are going to be don't quote me on either of those two things from there we're going to do five more graphs kind of the most common style of graphs that that people see or five very common styles of graphs that people see we'll talk a little bit about we'll wait how come that didn't work and what might go wrong some of the things that I've run into when I was working on using bouquet for various projects we will stand up a very simple bouquet server app we will do some conversation about where to go from here and how you can continue the learning process and look at some more examples if we have enough time toward the end of what you might be able to do with bouquet I will say that my intent here is not simply just to demo a bunch of bouquet charts and certainly that's what we're going to have there's gonna be lots of demos but I want to try and get as much interactivity as we can so that you are actually playing with the charts and changing the things and looking at them when it gets you doing some stuff it's a little hard because there's so many moving parts right I can't just say we'll go make a go make a chart so in many cases it'll be like here's some code now I want you to change it and see how that changes the outcome and how that makes the interactivity change or provide you with better appearance or provides you with better interactivity but also as we go along I'm going to try and provide you with hints and tips for how does one go about finding out more about a library that one is now using the focus here is on learning more about bouquet and I'm going to teach you ticks and tips and tricks to learn more about what bouquet can do but these are all the types of things you do anytime you pick up a new library alright a couple of things I tend to show this chart to just about everybody when I start a conversation you know your programming skills go up your programming challenges you can tackle go across and when you're really skilled but your challenges haven't really changed much you start to get to the boredom side if your challenges are really high and your skills are fairly weak you end up in this kind of anxiety phase the ideal spot generally is to be in this kind of white area in the center where your skill is pretty much fairly match up with the challenges you're facing but the little line is actually kind of where people's skills end up going typically your skills at a particular library or a particular language or a particular challenge don't really take off for a while you have to poke at it prod at it try it play with it and eventually at some point things will click and when they do your skills take off and then about a week later or whatever you're going to pick up a new library and you'll be right back down in that anxiety curve which are like I don't understand this new library I don't understand how to do this thing this worked yesterday but I can't get it to work today there's no real cure for this I have this boredom anxiety curve fluctuation about once a week my job sometimes once a day it's just a matter of keep trying so if you're new to programming here or if you're new to graphing and visualization just keep trying keep playing you'll get there how do you actually go about learning any new programming concept you change stuff and you see what happens and I'm going to ask you to do that a lot today I'll give you a snippet of code go in here and change these parameters and see what happens there's nothing to be afraid of if you break something you just download the code again off of github and you start from scratch right I am going to ask folks to buddy up I'm gonna all the tables are evenly divided in two into four I'm going to ask you to work with the person who sits next to you right and every time I say go do something I would really like it if you would work with your partner and you guys huddle around one of the laptops and you do that thing there's a couple reasons for that each you you will now have a built-in troubleshooting partner you will not believe how many times I've taught a class or tutorial where somebody is just struggling and the guy next to them knows the answer but they don't talk right and so this poor person is struggling and I've got five six ten people didn't try to help and that guy sitting in the back can't ever solve it but he's too shy to ask the person next to him and I don't have the time maybe to interact with everybody so somebody starts to fall behind they get really far ground all because they were too shy to ask so I'm going to try and force you guys into that mode if that makes you really uncomfortable and you're really stressed out about that you don't want to do that for some reason that's okay just explain that to your partner like hey this is not for me but I would appreciate if you guys would at least give it a try plus everybody now has somebody you can ask questions of you can discuss things when it comes time to do some experimentation you can suggest to each other hey why don't we try this why don't we do that one of you can type and one of you can essentially look over the shoulder and do peer review invariably when a student makes a mistake it is often because they have type something they left out of space they forgot to put a dot they forgot to put in the right set of parentheses or there's not enough parentheses on either side so your peer reviewers looking over your shoulder and he goes before you even hit Enter whoa you mistyped that variable name or you need to put another parentheses so it can save you from having errors and problems all right the next thing I'm going to suggest really strongly that as you're going through this process and you're entering code and you're typing things up think before you act what do I mean by that probably the number one error that I tend to see is people have mistyped some snippet of code an error flies up there's all sorts of crazy colors and there's lots of things streaming across the screen and they go I don't know what went wrong and when we scroll back and we look we're like oh you typed names and it was really named or you typed a name with a lowercase and it's supposed to be an uppercase right so if you're asked to type something in look at the command type it all out and before you hit enter double-check your typing and just make sure it looks good if you're like hey I'm not sure if there's supposed to be a space here a comma just ask and when I say just ask how do we do that so I've tried to give everyone um post-it notes and if you don't have some I'll send my colleague around and he'll make sure everyone gets them there's two post-it notes there's red and those green if all heck has broken loose and something is unfixable with you and your partner put up the little red flag will come around will talk to you try and get you back on track if I say hey go ahead and experiment for about three or four minutes try these next few steps when you're done with those put up your green flag as I start to see a sea of green then I know that I can pretty safely comfortably go on and we can continue with the lesson if I see a whole slew of red flags and I know that we've run into some trouble and we should take a break or something all right at the bottom of every lesson there should be some sort of navigation to take you to the next lesson if you're either following along on github or ultimately hopefully you'll download these to your machine and you'll be working in the jupiter notebooks themselves and we'll talk about some of those things as we go all right so let's get into the installing mini Conda so for this tutorial were going to be using Python version 35 or above I am asking that folks install mini Conda if you already have an akanda you're good if you've already got mini Conda you're good but if you don't have those install mini Conda and we're going to use the package manager Conda to install a number of packages and libraries etc and we'll walk through pretty much most of these steps why am i having you do this the version of Python you may use may not work very well with the version of Boquete that we're going to try and use today you may have some sort of conflict in the background you might have installed some library that doesn't mesh well with what we're doing so what we're going to end up doing here is we're going to install the condo package manager we'll create a virtual environment we'll make a little sandbox so you can put all of today's code and put all of today's libraries and packages in that sandbox and all of your stuff that's valuable to you you might use for work etc that'll all remain unchanged you not to worry about me cluttering up your hard drive with stuff right this virtual environment will allow you to play when you're done with it you can essentially blow it away and move on alright so again if you've already got mini condo or anaconda installed you can skip the installation step and you can move down to using it to create a virtual environment for those who have not yet installed mini Conda there is a link to the mini Conda QuickStart guide and will let them teach you how to install mini Conda ok and then when you've done installing mini Conda you can type in Conda list and it should display some stuff so I'm gonna let you guys go through that if you've already finished put up your little green flags so I can see if you're still working on that just keep cruising and when you get done put your green flags on so red so what Chalmer low your partner may know do you know not offhand okay but I'm looking at c3 six mini 585 ah so this will allow you to install a version three six three five three four actually even to seven so this this is the this is the most recent version and it will allow you to install any version of Python essentially that you want yep and there will be a step to teach you to install Python 3 five let's see what do we got here hit enter age are you on the internet icon 5g okay yes sir that should be it I thought I was missing something after then well when you get to my page there's links to all the repos that I'm in why don't we try one of the other won't we try PyCon to four they've been pointing people away from it but you know what would give both of them a whirl see what we can get so you're not quite logged on yet up could not be joined okay try again join up to five try to five again I mean essentially your computer's not connected in the internet so that's that's the thing to solve at this point and I'm not sure okay give it a whirl so yet well he's going to need he's going to connect the internet anyway so you be installing software now since you guys will hopefully be working as a partnership and if this fine gentleman is kind enough to let you type on occasion you guys can both study and learn together while this is kind of thinking all right for the record um this young gentleman in the back over here with the nerd life shirt that's that's my partner in crime he works with me at the dark art of coding so he's also available to help restore any any red flag problems so hit him up I'm going to go over here there's a red flag problem yes sir how can I help okay sure okay that should be fine but go ahead man till the environment - and Barney's crass there's comfort square you went through the process you installed all of the things that were requested there was like six things we wanted you to install so this is perplexing me oh did you close out of your terminal and reopen your terminal because sometimes when you when you run an environment you you reset the path your terminal might be looking at an old path variable on your machine and not your your most recent one no you should be able to close your terminal and try and open it activate the graphs and let's see if that works so let's close this out leave this out I mean it your path should update as soon as you start the environment but sometimes things get a little wonky so try and try and start a new terminal activate your activate your graph your your your environment and then we'll see we get so Conda is a package manager first stuff for stuff tip is a package manager for Python packages soap it doesn't necessarily install packages that are not Python as well as you want well here's the risk you take I tried to test as best I could all of my instructions to work in Conda I did not test a single one of them in tip so if you would like to go down the pit road you're welcome to give it a try that'd be cool so give give that a try but I can't guarantee whether or not anything Senora but the but if you wanted to Google there's a very good article by Jake Vander Plaats who who does a lot of data science and he works with the Conda folks regularly Conda was designed to handle the installation of data science tools that tip can't do and I don't know if any of the tools I'm asking you to install tip can do natively or not I honestly don't answer that do you use side PI do use numpy all of them with pip okay good enough then you're probably okay but I haven't tested it for this section yes sir all right if it works it works that's good I'm not using - all right are there any more red flag things p.m. the next step I've got web errors obviously it says retry will get me on my way but a retry create a new directory I mean whoa is there a way to get I'm sorry during the VM of graph it's open telling its dependencies for the virtual rooms trying to reconnect or man could create a new directory under here but then it's going to have to download the stuff it's already downloaded and that's just going to add to okay I'm going to read ok so it looks like it can't access the interwebs various things crap all right so you have access to the Internet and you said it there was a statement where it said you could do some sort of a retrial we will get you on your way yeah but what happened when you try me when I tried to appeal retry it to value error the prefix all right this because I'm trying to create for sure I so have enough experience on Canada to share so it has attempted to create this thing right now what we need to do since theoretically the the virtual environment has been made we now simply need to install and make sure all the things are installed so what we could do is TRACON de install and then python 3 and see if it'll go from there okay but we should activate your see if we can activate your your on Windows or Linux source activate can't type today activate I really can't alright so Conda has activated your graphs and now we would use Conda install to start installing all the packages and see if you can install Python 3 let's just start with that ok Virtual M and Conda the commands for making it do what it does are just a smidge difference you might have been thinking of the command for the one about bin flash I you might have been thinking about the command for virtual environments yeah come back give that a try see if that helps all right so perfect all right there's a red flag back there and I will come back in a minute but we need to move on just a bit if you're struggling horribly like I said partner up with the guy next to you or the girl next to you hopefully they have gotten things to work and that'll least allow us to keep pace right all right so in talking to a lot of folks many folks have been trying to activate your virtual environments and I've seen some success to activate them you would use source activate graphs on Linux and Mac or just activate graphs on Windows number of folks have gone down the road of trying to install the various pieces of software kondeh install and then you name it some folks ran into a snag they were trying to install or create a virtual environment and the random snag if that happens and you get something like hey there's an HTTP error I can't get out to the interwebs and it says try it again if you try and create the virtual environment immediately they would probably say hey one already exists from there since it's already been created it may not have anything in it yet because the the install process of getting things into it might have failed but it probably still exists so you can go and activate it and once you've activate it you should be able to say Conda install Python equals three and install these things so if you run into that kind of snag did that start to work did you get Python installed yay nay all right we'll talk now once you have your virtual environment activated you can go to the github x' and you can go down to clone or download and you download the zip file and you can open that up inside your virtual environment and all these notebooks should be there should be and theoretically you should be able to type Jupiter notebook and it should run the notebooks for you and that will allow you to act or interactively work with the code alright so if you successfully got Jupiter notebook up and running and you have access to the notebooks themselves put your little green flag ease up alright if you don't if you're not at this point you need help give me the red flag and I'll go back there and start with those folks where did my kid go what's up it doesn't want to connect to the internet okay okay all right so I have been told by the PyCon people that access to the Internet's have been horrible as of this morning so some folks have flash drives I can try and get you some things so let me see what we've got you said you had nanikana installer you have one - awesome so this gentleman has this guy's got an anaconda installer which allow you to keep moving forward this guy's got one too apparently all right so let's see where we are did you try for four just for for trying sake did you try PyCon - for the other internet' let's try that one and see if we can get you on the web with that icon - for okay let's try that and worst case scenario partner up with your awesome partner okay awesome all right I'm not in charge of the Wi-Fi so I honestly don't know the troubleshooting and why this is a problem gentleman in the red shirt over there he had the exact same problem you had he he had that same HTTP message right so you have a red flag I was still downloading okay so in the interim we have an installer that will that has purportedly all of these packages bokade blah blah blah blah so you should be able to get all of those sorry didn't mean to touch your hair he has one for Windows and one for Linux here on windows I got through that now it's just the other packages okay so the anaconda installer can you scroll over if you don't mind to see I'd like to see how large the package is yeah so that one has everything in with it it's got bouquet and it's got ipython and all the things if I have many funds I already installed this one it'll put it in a separate folder all right so the the Wi-Fi apparently is horrible my apologies right if I could fix that for you I would not good I'm going to throw this thing at you for future reference when you get done in a particular virtual environment you can deactivate it so you can move on to a different virtual environment or do different things and this will take you right back to where you were so it's talks just a smidge about virtual environments and what they're for they create a sandbox when you installed okay you install ipython you install Jupiter into this virtual environment it keeps it completely separate from all of the other versions of bouquet or ipython or python that you might have on your computer so that way you don't contaminate other projects that you're working on so it's the beauty of using these if you're not using virtual environments for your projects I recommend that you do so it is a valuable skill set to have in your toolbox and it will save you a great deal of headache later on when you're trying to get something to run and you've inadvertently installed some new package and your computer is looking for the wrong package of the wrong version of a package for those who care the rest of this conversation is a deep dive and it goes into more as to why you'd use virtual environments and what mini Conda is and what anaconda is and those things but I'm not going to go into that I'll let you read those things all right all right next so when talk a little bit about bouquet the word bouquet is a Japanese word it refers to blurring the out-of-focus parts of an image so you might have a picture of say a person walking down a street and the person is in high focus and everything kind of to the sides and behind the person is out of focus and is blurry it's just an aesthetic quality right it can enhance a photograph and photo geographers will use it to artfully focus or draw attention to something in particular and that was kind of the idea behind choosing of this name bouquet is the intent is to allow folks the flexibility to focus their viewers attention on the parts and pieces of the data that are most important without losing tracks necessarily of the context behind the data so in that picture of a person standing on the street you can still see the street behind them see the lights to see the cars but you don't really focus on those and they're not clear and crisp until you want them to be we talked about bouquet and how is designed and the principles behind it they want to try and capture perceptual approaches right that represent data to people who are experts in a particular demand domain so they can apply their intuition to the data they can instead of looking at rows and rows they can look at a picture and they can zoom in on the picture they can highlight aspects of the picture select things out of the picture etc they want to automate a lot of that work so that you don't have to do it and create a data transformation pipeline right they want to empower their users to have this ability to visualize things fluidly they want to provide useful software that incorporates novel ideas from the academic world on how to do data visualization and if you want to know more about these I've got a couple links in here on the technical vision behind bouquet there are two parts and pieces to bouquet there are client-based visualizations these are standalone objects that you can view in a browser you can send a file to somebody they can open it up in their browser and everything they need is right there right or you can create server based applications stand up a server they can surf to a web page they can see the data as they interact with it say zoom in on something the server will produce a new data set that is represented by the zoomed in area etc the limitation for the client based visualizations the ones that fit in your browser is essentially how much RAM and how much compute power you have if you don't have enough it bogs down your browser and your browser gets sluggish so that's the genesis behind having a server based app the server handles all the heavy lifting and it only gives you the data that you can see in the part that you're looking at of the graph so again if you zoom in it's going to give you a new batch of data for just a zoomed in part and in addition essentially there there's some some things that kind of happen behind the scenes that help to filter down some of the data that might be displayed until you zoom in on it all right so how does this thing work it's designed so that when you have a high-level object and you can hear me use words like glyphs or plots or axes those things are all created using Python boquete translates all that into JSON format okay j/s uses that JSON to create visualizations that get rendered in JavaScript so they can display it essentially in any modern browser all right and again kind of the nuances trying to display large quantities of data in your browser gets wonky when you're trying to feed it too much data so we're going to get started on our first graph this kind of presupposes that you've been able to install boquete and I know there's some some problems with a couple of folks for now just kind of follow along and we'll see what we get so we'll look at a first basic graph in this case I'm going to import from VOC a dot plotting three things I'm an import a figure I'm going to import the show function and the output underscore notebook function I'm going to pull together two lists they're just Python lists some X values I want to graph up against some Y values that I'd like to graph and I start off by creating a figure and I'm going to give it the name P in this case or give it a label P P has a bunch of functions that hang off of it and we'll see some of those in a little bit but when you set tell it that you want to run the line function off the P figure you can provide it with a series of X values a series of Y values and it will produce an object that can be displayed when I tell it that I'd like to output this to the notebook it knows that I'm talking about a jupiter notebook and it will prep things to display in line and when I say show P it'll do that so if you've not used Jupiter notebooks before if you have the Jupiter notebook running on your machine and you click in it you'll see that you have this nice green box if you press ctrl + Enter at the same time it'll run Analects cute you'll know it execute because you'll get a number up here to say hey this was input number one and hopefully you'll see something like Bocage is successfully loaded and we'll scroll up a little bit alright so let me walk you through kind of your first bouquet graph a couple things to note it automatically produced a set of axes for me that was a default behavior it graphed all of the points and put a line in between the points I asked it for a line graph it produces these kind of gray colored icons on the far right side we'll walk through a bunch of those as we go through the day but starting from the top down you have the color tube okay logo if you click that I believe that takes you to help it's got a low this up a little bit it's got the little plus sign with the arrow heads that indicates pandit allow you to scroll around if you notice that the arrow heads have a light blue kind of bar next to them that tells you that that movement indicator that movement tool is currently active so if I click on my graph you can see that I can drag it around so I can Center the things that are important to me potentially you notice when I put my cursor over one of the tools there's a tooltip that pops up so the user knows what that tool can do we have a box a zoom so I can click and then drag well let me click on the tool notice I've now highlighted the box zoom tool with the blue bar click and drag and it'll zoom in on the points that we dragged - if you're one of those folks with a mouse and you want to scroll you have a scroll wheel so you can zoom in with your scroll wheel that also works with Mac I think if used two fingers to zoom in and out if you want to save an image you can do that and it brings up a dialog to allow you to save your images in various formats when you've changed something you want to reset it there's a reset button click that alright and let's zoom back out there we go alright so that's pretty much the standard that's the default some folks may have gotten an error related to numpy did anybody get any errors related to numpy when you tried to execute that that cell temp got you kick known umpires did you gotcha that is a nuance we're going to cover that one in what goes wrong so Bocage is the the JavaScript library is hosted on the web and so you can always get the most recent version of Bo kjf so if your Internet's are not working well which apparently do not here that's that's a that's a struggle to the error related to numpy my colleague Aden was working with this tutorial on a Windows machine and he got an error related to numpy and he had to actually go in and modify the numpy config apparently this is also potentially a problem with Sai PI as well depending on like the version of numpy or sci-fi you might be using and there is some some discussion in that particular comment thread so if you run into that there is a reference to that particular error alright so here's where we get into kind of you guys do some stuff and that will give me a chance to also kind of walk around and look at what people are doing periodically we'll have a place for you to earn experience points so go into the cell above and do each of these steps change a few things add a couple of items to it see how the graph changes when you execute the new graph etc okay and I'll go around and we'll talk to folks and look at red things I saw your red thing what do we got sir internets I already put a commercial machine a virtual environment so I'm just installing the factory to do it so those are then waiting on pandas inside by but it's been it's been this just been like me trying over and over but that's that's the nature of this was there something different you know working for me so I'm gonna have the mech and a cond installer in a few minutes okay let me go eat your thumb drive back how's it going did you get anything installed on Doc's we get this right yep it starts with have to install again what is it doing now well so the anaconda installer should have downloaded everything to your computer and now it should be using that okay to do its magic that's the caveat should no way to confirm like which pond I was running ah that's an excellent question I don't know the answer to it because I have only ever done this through Wi-Fi when everything works perfectly I have never run into this particular problem all right it's going really slow when tech is that so it is appear to be doing this through anaconda 3 as opposed to mini condo because anaconda 3 and mini kinda have two different folders the default is probably to try and install from the internet so that it can get you the most recent version so when you do install Conda you can do - - offline and we did - the path how do we do that I have to make sure you have here's the syntax even stackoverflow the loading form which is kind of bad sign but I got enough so assuming that your condom that's all actually has well we know it's already looking it up in it you could see that it had it should be in the Anaconda folder all right let me go see what else is going on any luck so we're down to sequel like TK likely now it's working on Python three stick probably a period of time when it can't I love it and then it's time gather something wraps out and then you have to like when you do a Python install or sorry a Conda install Python again it goes right back to do all light so essentially we can get you the thumb drive and you can try and install there's a Windows installer wait here on Linux there's a Linux installer on the thumb drive and it installs all of anaconda which is like every library ok ipython the thing is though by default it wants to try and install no it should install everything but when you create the virtual whenever you create the virtual environment it wants to go to the Internet and try and get stuff because the internet always has the latest and greatest so there's a command like offline - - offline ok Negin install like for other things but all this should be much smaller in size so they're all libraries they're not entire languages so keep going you were alright so we do have gentlemen over theirs is downloading the anaconda installer for max and you could install all of anaconda the problem that you run into is even though it installs old anaconda as soon as you try and create a virtual environment instead of going to its local repo it wants to go to the Internet's for all of the virtual there should be a there should be a switch like tack tack offline there should be tech tech offline when you do a con to install and it should look in whatever repo you tell to look in but I don't have the exact syntax so let me just see what I can figure out all right and PLD 3 I think I've heard that Jake Vanderpump's all right number of folks are still doing some of the download thing and it sounds like some people are making progress but the progress is slow I'm going to go on and hopefully you guys kind of while that's downloading just kind of keep your eyes open and follow along and talk a little bit about how do you learn more about a new library you're using all right so in this case for example whenever you have an object and you want to know something more about it if you're using ipython or if you're using by extension Jupiter notebooks you can simply put a question mark after that in regular Python you'd say help parentheses the object close parentheses but in ipython it's pretty easy just question mark so earlier we made an object it was a figure and we called it P so we're going to go down here and in one of these cells I'm going to execute my cell and it's going to provide me with a help function or some some help documentation says hey you're looking at a figure and this figure has certain characteristics some of it's written in some gobbledygook there's a little bit of syntax highlighting in there for when you produce kind of online documentation but basically it says that this is a subclass of a plot and it simplifies the development of your plots by providing some default axes some grids and tools and such alright and you're going to see me in various cases we use this question mark to kind of interrogate objects and figure out what I'm looking at and what I could potentially do with them alright when you don't know what functions are available so I told you this key object had this capability of producing line graphs well could it make other kind of graphs I don't know so if you type P and you put a dot and you hit tab ipython and Jupiter kind enough to show you all of the functions that hang off of this object so we see that there's things like add a dynamic image there's an annular wedge I don't know what n DS are but there's a bunch of them right and we'll figure out lays to learn more about them as we go into we see something called P dot diamond talk about that later there's things about grids and extra Y ranges and X ranges PT lips and I keep scrolling down and sure enough P dot line is in there all right so when you look at it in your library you don't know what the library has available to you kind of functions and methods you can call this is a technique to start to look into those things all right but what if you want to know more about a particular library well you just use your question mark so P dot line and a question mark and I hit ctrl enter and it opens up this help space down here so it starts off it says hey whenever you want to make a P line you have to give it an X and a Y and you have the option of giving it some keyword arguments and then it gives you a list of potential arguments and it tries to define them as best it can says hey your X should be all the x-coordinates for your points on your line the Y should be all your Y coordinates and we saw that we had a list of X's and a list of Y's and then it says you've got these optional keyword arguments like JavaScript JavaScript callbacks a thing called line alpha whatever that means let's look the line alpha values for the line okay so maybe that wasn't entirely helpful for those who aren't into graphical design etc when you have something that's colored alpha is an indication of transparency so you're going to see alphas a lot and we're going to use them in a couple of places they also talk about line caps the line cap value for a line who knows what that is right when you look at lines at the ends of the lines they typically will have some sort of a cap if a line is very large it's very wide that cap becomes distinct so there's different types of caps you can put on either end of your line but there's a lot of them I'm not going to go through all of them but there's many and you should spend time looking at some of these one cool thing even though there's lots of them it's kind of a wallet text most graphs that you look at are going to use some selection of these so you're going to see these over and over and over and over and over again line cap is available in just about anytime you have a line alpha and colors and such are available anytime you have objects in boquete alright so for experience points what I want you to do is go up into this cell run that for me and just start reading a little bit take a few seconds read a few of these things explore them we're going to do this several times during the course and have you look at these things right and experiment and then we'll move on let me go answer a question to got you a little red flag up you're still downloading okay yes in the second half of the conversation we're going to do servers how goes it it's good to see you hopefully you get something out of it and we're able to get all the stuff downloaded my god it was like we didn't mess with if it's just good Rosa - but I was ringing it's because later on there is a place where you can tell it that something you're looking at is a date/time object so when we get to that part of the conversational I'll try and remember to point it out because I know I don't remember what's figured out I don't remember which which graph it was in but there is a place where it mentions that this should be considered a date time object there is a there is a place where you tell if the to x-axis values are date time something like that that the command or the the argument is something like X value type equals date time might be I'm trying to remember all right so we're going to look at another graph we're going to do it again but with more this graph is going to have circles it's going to have dashes and it's going to have a legend this time because you want your user to understand what he's seeing I'm going to I'm going to warn you ahead of time as we go through the different graphs many of them are going to have the exact same kind of stuff on them so if I've already kind of talked about something I'm going to gloss over some of that and move on to potentially the newer parts that we haven't covered yet if you want me to backtrack just a smidgen say hey can can you talk about a thing that's okay just ask and we'll do that just kind of an inch time I'm going to I'm going to cruise through and but this case we're going to do some more importing we're also going to import numpy and it is customary when you import numpy to use an alias and in this case we use the alias of NP I create a figure I give it a name more numpy has a library or a function that allows you to create a series of numbers that are linearly spaced between one value and another so I want a hundred values between point one and five and it does all the math to space them evenly so those are going to be my X values and then I randomly said hey let me use a range object from Python and I'll take from 0 to 200 and I'll skip by two the point here is that when I feed a numpy array or I feed a range into boquete it understands both of those natively and it's ok with that I don't have to convert them to lists or something else it's ok with using numpy arrays and ranges so that's all good my first thing that I'm going to create is I'm going to create another kind of glyph that goes on to my graph and this is going to be a glyph composed of circles so for X values I'm going to use the X's that we created and then I'm going to go ahead and use another numpy function called cumulative sum and it's going to take each of the X values and kind of add them and make a cumulative sum there's no real reason behind doing this I just wanted something interesting the important part here though is this I assign it an argument I say legend should say and I give it a string y equals cumulative sum of Y I can make this say whatever I want it'll print it essentially as written the second line we create this one is actually again going to be a line as opposed to a sequence of circles I give it an X and Y value I give it a range you know what I think I screwed this up let's change that to what 200 step by 2 all right there you go so I give it a edging to say where these values come from and in this case I'm now going to tell it make this line a particular color and we're going to get into more details about colors in a second this one is using hexadecimal representation for colors ok totally understands that in this case we're using Rebecca purple and the line format is going to be a dot in the dash bouquet has a default location that it puts legends but I want to override that so I'm going to say more dot legend location and I'll tell it where to put it and there is a set of key terms that you can use in here top left bottom right those kinds of things and I'll show you where you can learn more about those in a little bit but you can use these terms you can also use XY notation to say put it at this particular pixel location on my graph and then we run it see we get and so we see I have a graph with a bunch of circles we have a graph with dots and dashes this is purple that one was blue which is the default we have the same tool the none of those changed but now I've got an awesome legend right and it automatically uses the line type or the graph type that you you pulled out so it takes care of that for you and whatever you told it to display it displays all right so how do you learn more when you're in these notebooks I talked about some of these things use the little question mark right if you want to add a cell to your notebook and we never use one before at the top of your notebook there's a toolbar and you have this little plus button you can go in anyplace where there's a cell you click on it you put plus and it opens up an empty cell for you and now you can start to interact with any of the data that's been created prior to that so we had a I made a few moments ago called more and I can hit tab so don't fret about adding a few cells to this and using those empty spaces between some of the work that we're doing to experiment into play the other tools that I often use whenever I'm playing with new libraries and new examples I use Poorman debugging so I print my variables a lot what the heck is this and let's go see what it is right let's see the data on the screen or maybe I'll print a subset of the the data so if I have a list with five billion records maybe I'll print the first ten just to get an idea what's there so I recommend as we go through and you're looking at example libraries or example graphs print out some of the variables and see what is being fed into the graph if you have something and you don't know what it is I often use the type function to tell me what I'm looking at so let's put one of these guys in here so we had like you know print X and it says hey here's all those linearly spaced values that number I produced I can say what is X and tell me it's a numpy array those kinds of things and again just change something and see what the changes happen expression yeah we'll get to that in a bit when we get to the user guide the user guide is crazy awesome it's actually very very good total props to the folks who made bouquet all right let's talk a little bit about colors bouquet has rich support for colors again the idea is to produce quality visualizations so any of the hundred and forty 447 named CSS colors are available to you I have a link down here to take you to a list of those so indigo violet green grey and the various spellings of grey all those things work if you want to use hex values we saw a moment ago hex values work just fine if you're into doing transparency as part of it you can append the optional alpha values at the end of hex and it'll take care of that as well so it understands six character and eight character hex colors if you wanted to feed three tuple integers for red green and blue those values between 0 and 255 it handles that and if you want to do four tuples to include the the Alpha where your alpha is between zero and one that's all good as well so for example all right I'm going to create a new graph linear space stuff I may copy that alright this time though I'm going to add a title to my graph alright so that my user understands what he's looking at and in this case I'm going to tell it that my x-axis type should be log so I want to have a log graph right so those and didn't get along well with math log graphs just mean that they grow funny right and I tell it my why my y-axis should go from point zero one all the way up to 10 to the 22nd power so it should go pretty high now I'm going to tell it that I'd like to produce a series of circles I give it my X values and my Y values again I put my legend on but this time I tell it my circle should be empty in the middle fill color should be none so I'm gonna have hollow circles instead of solid my line color I'm going to use that color name from the from the web of blue violet and I tell it these circles should be bigger than before they should be like size 5 next I'm going to also do a line on this graph and we'll say okay line color will be dark red but my line width should be pretty narrow I want a width of two set line widths - all right so let me make sure I execute these guys real quick pink pink pink a couple things to note when I made this particular item and I made this particular item in the background a glyph was produced a glyph is basically just the thing that gets drawn on your graph it's got an identifier for it if you go back and rerun one of these cells it'll make a new glyph with a new identifier and we'll see a case of that and how that can mess you up so I'm not recommending you do it now probably already have for three-quarters of the room all right let's make another one this time instead of circles and lines we're going to make squares um line dash will be set the solid legend color it's all good see we got now I talked about kind of how we can tweak the graph so it's better for the user right I mentioned before that legend has various options top-left we're going to put this legend on the center left side but I'm going to change the font make it x for that title that I created about three or four cells ago I want to change that title I want to set the font for it I want to set the color of it in this case I'm going to go ahead and use a three tuple of RGB values we'll set the font to be courier in this case because we're geeks and we'll make it bold right but this graph it's going to be maybe on a white background or something so I want it to have instead of being plain white I want it to be a white smoke color I did not know that that color existed on the web but it does there's white smoke and should be kind of grayish it may not show up very well up here but it hopefully shows up okay on your own machines and I execute and show and there we go and hopefully you see it's kind of a grayish color now my legend is centered off the left I've got squares etc and it's got this log over here all right color palettes so some of you may have used Seaborn and I was mentioned earlier when we were talking to the audience the principle behind Seaborn was matplotlib is very in the weeds in some cases and it's hard to kind of get things set up so what if we had kind of a layer that lays on top of that that allows us to do good design using harmonious colors and those kinds of things so bouquet takes into account some of those types of design principles again the idea here is to help people do beautiful looking designs that put the folks in the right places so bouquet actually incorporates a series of pallets from a wide variety of sources and they have essentially picked particular color values that are harmonious from very commonly used pallets and I'll show you some of those in a bit but how do you get to them this is a palette that I'm going to use here in the next example it's the purples palette and something to understand about the palette the palette is essentially a dictionary and it is referenced by number so this dictionary has a key of 3 4 5 6 7 8 and 9 if I say I want to use palette number 3 out of the purples dictionary it gives me a choice of three colors if I want palette number nine it gives me choice of nine colors and it's already pre designed or preset to show all of the colors that are there so for example I go into bouquet palettes and I import purples I'm going to go ahead and print Purple's 3 and purples 5 and I'm going to look at Purple's 5 and choose the very first color out of Purple's 5 and I'm going to use that for my line color all right and so we execute and you see we get there we go so the 3 palette number three off Purple's has these three colors palette number five has some related colors but they're not exactly the same so if you have a graph and you want to use some you know arrangement of purples this is a way to get that and I'll show you in a couple minutes some of the other pallets etc that are available to you so let's add one more line to our graph we set our line color a second to go to one of those purples it was seventy five six b11 so we're gonna use that line color we're going to use a function out of numpy to calculate square root of x we're going to make it a dashed and dotted line and it'll be with the three so there you go and notice it updated my legend for me on-the-fly it's got a purplish color alright so there's a question it was hey where do I find all this stuff so let's go look at a few of those things if you google on the web bouquet user guide it'll take you to this place and there's a ton of data here absolute crazy amount of data they have a QuickStart guide they have you know getting yourself set up they have some key concepts that be important to cover like what is a glyph those kinds of things they walk you through plotting with basic glyphs and making some high level charts etc so bouquet has kind of three levels they have a charts library of plots library and a models library and depending on how deeply into the weeds you want to get you'll use one of those many of the things that we'll be using here come out of the plotting library and the charts library there's a fairly high level you can get to a statistical chart fast and easy without having to do a lot of poking and prodding this is no longer accurate this particular sentence this actually points you to data shader and whole of use instead of matplotlib and Seabourn but the user guide walks you through a bunch of the things we'll see but in greater depth like how do you add an annotation or how do you change your layout etc let's go see styling visual attributes so we'll click on styling visual attributes and if I scroll down on the left hand side you'll see that I can set certain visual properties like the sill or the line properties you'll see that I can do things like setting borders and backgrounds and titles I can tweak mine I can adjust the axes and the labels and the boundaries of the axes etc there's a lot of detail here they mentioned palettes so let's real quickly go into the palettes part right it gives you an example of how you might use one there's a variety of them if I want to know more details I should be able to go to the bouquet palettes page and we'll see that so go to bouquet palettes if you're into graphical design all the color Brewer palettes available to you all the categorical d3 palettes are there mat plot lid palettes like magma Inferno are there and some other palettes that are designed for folks who have potentially color deficiencies and you need to be cognizant of usability for those folks but I'm going to scroll through these you see there's a wide variety of these from various organizations and sources that have kind of standardized on various palettes and there's a ton of them all right now I don't spend a lot more time on palettes the gist of this conversation is this the user guide is incredibly good there's a lot of detail there and as we go through it I'm going to refer to various places where there's info and the youth in the in the users guide when you have a question this is an absolutely fantastic resource to go to a couple other places we'll look at there's some example galleries of graphs that they've made and there are a lot of examples inside the user guide as we go through it as well alright so next so we're gonna take a break here I think this is probably a good place to take a break we've been going now for about an hour and 20 minutes give you guys about five minutes because I know you guys going to take ten so if you don't want to go take a break that's fine too can stay but I'll answer some questions while people are kind of moving around what's that Mike is a graph you say download yeah the say button say only gives me a PNG ah option to do say in green when you say you can save the output and I believe there is a mechanism to set it like programmatically you write 11 line of code in your script I believe so what's the look yeah SVG would be crazy good let's go see the Sun I'm trying to get internet through my phone and my phone is not wanting to give me internets good stuff thank you it's not letting me it's not letting me dang it I can't connect my phone with the crap for 40 up there yeah so what is your status hey Matt did you guys get to the point where both gauge is is your hang-up that's a good question now once you've downloaded the book AJ is library your browser should take care of it because it's a JavaScript library for your browser and you should need a new one periodically they update it and you want to go you know your browser would just natively go and get the more recent versions it should be fine yep so the console for me I already loaded this once and then I hit fight out of this notebook little bit up another one and now I'm still gone maybe it downloaded it to you were you in the virtual environment or miner goes it was like it's lying it was done if not fine somebody else's so how's this going started catching up hmm your model paint dry sweet well usually good painting catch buzz hopefully hopefully I get somewhere so how are things going here everything all right okay good so how are things going back here progressing along okay good so how goes it so far okay I take note I take no credit or blame for the Internet we're about ready to get started you get plugged back into all right so we'll do some fun and interesting plots hopefully we're going to go through these pretty quick and as I said some of the things we've already covered I'm not going to kind of repeat we'll just highlight some of the newer things we'll look at a scatter plot we'll look at a histogram with probability density and cumulative density curves and we'll look at a graph that has some interactive sliders to allow you just choose parameters that you want to have displayed on the graph we will look at visualization tools and the types of tools we have and we'll look at a geographical map where you take some latitude and longitude data and you put it onto onto a graph all of these things that we're going to look at are available in the bouquet library what I've done here is I've added some commentary I've broken them up in different ways and I've said a few things there's a couple places where I might have taken some stuff out of these particular graphs just to simplify them so that we can focus on the new issues and not be overwhelmed by some certain things like the interactive slider graph if you go look at the gallery they show you a graph with four or five sliders I'm going to show you a graph with one slider again so we can kind of hone in on the things that are important and highlight the differences between what we've seen before so there is some sample data available for you for your amusement I'm going to caveat that the sample data kind of comes in three different three different ways there is a certain amount of sample data that just comes with bouquet and it's already pre downloaded for you small quantities of data there are other much larger data sets available as well and if you were to run for example the command bouquet dot sample data download bell install and then if you get into the server examples the server examples actually have kind of a third set of data and you would download some of those independently for the servers there in some cases certain scripts that come with the server exam that you would download all right so let's start off with a scatter plot bouquet comes by default with some color data related to irises and irises are these little flowers they have petals they have sepals and the petals and sepals have different lengths and different widths and they come with different colors so from bouquet dot sample data iris will import the flower data set that's going to come in and it will be a panda's data frame just so that we can get a sense of what the data looks like I'm going to use some of those techniques I mentioned earlier where I'm going to print the type of object that I'm looking at maybe how many records are in this object maybe take a sample pandas datasets if you run dot sample on them it'll give you a random sample of the data so let's go do that and we see that I got a panda's data frame there's 150 records and then my random sampling so a couple things to note from the random sampling sepal length sepal width petal length petal width and then species there are three types of irises that are represented in here versicolor virginica and Satou sa' don't ask me if that's pronounced correctly and we're going to kind of try and see what this data really means depending on what type of species we're looking at so I'm going to start off by making a color map anytime I see say Tosa I want a color of red anytime I say versicolor I use green this is just Python stuff this isn't really bouquet I'm going to use a list comprehension to map the species to a color and I'm going to create a list and that list will be a hundred and fifty records long and essentially what will happen is I read through the data frame if I see virginica I give a color in my list if I see say toast I give it a different color in my list and so I'm going to ask Python to then tell me how many colors I made and give me like every tenth color and so you see I've got a bunch of reds a bunch of greens in much of blues again that's not really bouquet that's just Python doing its thing all right so first off we're going to set a title it's going to be iris morphology I'm going to provide a label for my X and my Y axes and in here we see that I've got my figure P I have my x-axis there's a lot of things that hang off of attributes that hang off that x-axis the label is just one of them if I want to set the fonts for my label I can do or for my axis I can do that as well in this case we just give it a name petal length petal width when I go into a panda's data frame and I say give me the column petal length and give me the column petal width it's going to use that as x and y values I made a list of colors there's 150 of them they should map quite nicely to all 150 rows in the data frame the fill color is going to be point or the fill alpha sorry will be point two so it's going to be kind of transparency and the circles would be size 10 and so we rendered the glyphs and let's go look at our graph that's crazy awesome right and so right off the bat cytosis were red and virginities were blue so we see there's a very definite difference in size of petal width and length for the various types of flowers right we still have the same standard tool tips over here if I really want to zoom in on some of these it looks like there's a little bit of overlap here let's kind of get more detail and notice we can really get in here and start to see kind of where these things these two different colors might overlap or these two different species might overlap notice as we zoom in it automatically expands my axes on the left hand side as my zoom increases now when I go into the next few examples I'm going to follow a workflow I will break down each of the examples that are online into a workflow and the workflow that I tend to suggest is as follows start by importing your libraries generate just your figure and maybe some of the attributes of your figure your title etc create or load your data and your Richman's do as much of that as you can kind of in one spot so that way you can focus on that part of the problem then start to generate your glyphs typically can't generate the glyphs if you don't have the data to put into it right once your glyphs are generated go about the process of fine-tuning them change the the fonts and set all the annotations if you want to have an interaction and we'll see a few of those you can add those at this point and then you generate your outputs and your outputs could be in the notebook like we're going to show here or you could output to an HTML file or to a JSON file those kinds of things right now for simple graphs you know maybe not all these steps are critical or maybe you don't like break them out concretely maybe if you have a very sophisticated kind of server app maybe this is not quite enough you want to break it down a little more but this is just a starting point and if you follow this model you'll get pretty far so let's look at a histogram I start off with my imports we'll import numpy Syfy and bouquet from the layouts portion of bouquet we're going to import this thing called a grid plot so we'll see that later okay I create my figure I give it a title this case I'm going to tell it I only want to use essentially the save and I tell it that I want the background fill color to be white smoke again so let's run this nothing happens there and I've created a figure right again every object in boquete has its own unique identifier so I've got a figure now I need to start doing stuff with it we looked earlier at using kind of the question mark to figure out what we have and get some help about it so if I type H question mark and we saw this before we saw that you can look at it and says hey there's some details about figures great but there's a nuance down here at the bottom and it says in addition to all the bouquet model property attributes documented below the figure initializer also accepts some following options but then it kind of dies and it doesn't show you all those parts and pieces with ipython if you have an object and you ask for essentially the help documentation you can do that with the question mark if the item is written in Python and you want to see the source code you can use two question marks so we'll do that and now we're actually going to look at the source and you see there's a little place here it says source that initial doc string is right there at the top and that's all good but we could potentially start to scroll down and you'll see that there are a number of objects in here that they reference like arc and Astro seas and things like these are all things that you can put onto a figure the circle graph is in here etc better data so even if you don't have access to the Internet you can start to kind of scroll through a figure and see what kind of things could I overlay onto a figure I want to overlay a line or cross and maybe you don't remember the exact name but this might be an easy way to kind of refresh your memory all right so just be aware that you can see the source code with the double question mark and sometimes that gives you something if a single question mark lets you down all right so let's see picking back up from where we left off we've got a figure in I didn't scroll down far enough but one of the options for things you can put on to a figure is a histogram so we're an end up putting a histogram together we're going to use a tool in numpy that does this for you gives you the data at least I'm going to start off by creating kind of the midpoint and the standard deviation for a nice kind of bell curve we're going to look for some random samples from a normal distribution again using numpy I'm not going to go in the details how that works but I'm going to get a thousand random samples associated with a normal curve and I'm going to ask numpy to create kind of the data for a histogram and it's going to create two outputs something called hist and something called edges so what the hist is going to be is this array of y-values for my histogram points the tops essentially of all of my vertical bars and the edges are going to be essentially what you would call the sides of each of those bars so if you had a bunch of vertical bars where would I start the first bar and end the first bar and wherever the second bar start and end etc in this case I told it that I want 50 bins which means I'm going to get 50 items in the histogram data and 51 edges because we start on one side and then work to the other side right so there's going to have one extra value get that extra edge we create some X values a thousand of them between negative two and two do not worry about all of this crazy math we're going to skip right over that just just gives me values that I can plot curves it's going to be cool and trust me on that one all right this is again just straight Python I wanted to see what does this data look like what do I actually have here so we printed a few of these things out I generally took small samples of each of them so these are data that's measured and these are the histogram values and notice I have 50 histogram values and I got 50 one edge values and these are the points that will be used to generate my probability distributions and cumulative distributions etc all right but before we can draw these rectangles on my histogram we need to know a tool that we're going to use to do that with and in this case each of those little rectangles is going to be used or is going to use the quad object to create an individual rectangle so what I'd like you to do for your experience points is use the question marg function and figure out for the quad function that hangs off of my F or my H figure tell me what is its default sill color tell me what does the Alpha characteristic do and how is the Alpha characteristic different than line alpha so start to explore this if you have any problems put up your little red flags and when you get done answering those three questions put up your put up your red flags when you're done put up your green flags excellent see that was a test for you see if you guys would notice that and nine alpha and alpha which is equal so that everything to a different value who's winning who wins all the alphas but maybe these heaters override them I haven't tried it we should try it that is how I would think someone would program it if they were wise and smart yeah but you could try it you could set an alpha of like point zero one and then for a particular line set the fill line or put the line alpha and put the line alpha for that line to be 0.9 I'd be curious to see what you get so like in this case I'm setting the Alpha right here when I'm making um okay so so for example in in h quad if i said alpha equal to be 0.9 and then and then I were to say line alpha equals something different I guess what I mean is in this same function if put a comma and put like line alpha I will send it into construction and I wanna see if I can do it later when if I change my plate one as you seem to have a good so I exit the cell that you access excuse now yeah then if I see a toe tag it should show me any wine so two names it should be an instance of Q well it's it's a it's a glyph renderer in two different instance yes so can we can we override property that's maybe a property values I could be some property values so you probably not good mess never tried this yeah it's like supposed to do animation so that that's something fun animation you need to change absolutely whole change it over time right summarize to a fauxhawk the pizza cold or something that would make it sense that's obviously has waited now this can be set to a variable so you can see the variable into it so if you wanted to run a quantification of yeah how do we not have measure distance but then often the fact the leads should have properties you can change may be actually a special function to change of some element there might be a function that allows you to access it like set at her or something like that maybe maybe that's a show up the closest dynamic I don't know alright a bunch of green things I don't see any red things all right so what is the default still color gray excellent and with the characteristic alpha do sets the Alpha value for what everything and how does that contrast with line alpha line only affects the line so Mike and I were having a conversation over here is what is the precedence if i set alpha and then i try and set line off at is line alpha override alpha or is it vice versa and i don't know we'll have to experiment with that so i'm going to make a quad object in this case I set four key characteristics ice at the top of each of those rectangles using the histogram values that numpy gave me I want all these to start at the bottom so I set the value for the bottom to be zero and remember there's 51 edges but there's a only 50 bins so I need all the edges on the left side all the edges on the right side so I select all the edges and I have it step by one or negative one to get my left edges I do a fill color of yellow a line color of orange set my alpha there all right now I want to make those probability distribution curves so I set a line to appear on my figure I create a legend I make the alpha negative seven da-da-da-dah I tell it where to put this stuff and I give a label to my x-axis and y-axis and then we go alright none of those things that really my really knew there you go sure that was the same same thing that Mike and I were looking at how do you get back or how do you programmatically change it over time like could I change the Alpha over time maybe to make a dynamic graph to say let's make it fade in or fade out over time in a for loop or something and we have not yet figured out how to programmatically query for what it is or to set it on the fly so let's figure that out and by figure that out I don't know go home and play one thing to say about this library this is an amazingly powerful library so I've used it in the past and I spent a lot of time looking at this thing you know and getting ready for today's conversation and every time I sit down and look at it with fresh eyes I'm just amazed by how deeply into the weeds you can get into setting various characteristics and how sophisticated and we get we get into the sample library and you see all the types of graphs that you can make etc you'll be pretty amazed all right so that's our graph we got the title we've got all the colors that we chose notice our toolbar we tweaked this we have a single object instead of kind of the defaults that normally show up so we overrode those we've got X axis labels and these things you can arrange these in various ways so that they're tilted and horizontal vertical etc there's a lot of nuances here that you can change and we'll see some good examples as we go through all right next example in this case I'm going to import some new items from the library that we haven't seen before I'm going to import row object I'm gonna import a widget box object and we'll go into a lot more detail later on in the conversation about those guys I'm going to import custom JavaScript library or a function and import a slider function and a thing called column datasource so we're gonna look at a bunch of new things here what I would like for you to do is try and figure out if you can what we're going to get out of custom jsn slider I've imported these things I don't know what they do I've never used them before right so how would you go about figuring out what custom j/s and what slider do questions yes maybe that works I'm going to go on the web and I'm going to try and hunt these down on the bouquet documentation because as I look at these little question mark responsi things this seems kind of kind of limited in what it can do for me and do a little bit of search so I used a question mark and I got very little back hey custom Jess allows you to do JavaScript something I don't know when I went into the bouquet library and I said hey I'm going to use a guide let me do a search on custom je s I get this page back and if you look at the scroll bar on this thing as I kind of scroll up and down you know there's a lot of content here and you also notice as I scroll down through this thing there are lots and lots of graphs embedded in these and for just about every graph they give you the graph and they give you the code that made that graph and you can go in and you can actually play with all of these things and you can see the various components and what it's doing and how its interacting with this stuff right so the user guide is chock-full of living breathing examples of how to use these types of tools and what you can expect out of them so I mentioned that the user guide is really good it legit is really good alright so if the question mark fails you not all is lost all right so we're going to start off now we're going to we did our imports a minute ago we're going to do our figure generation notice here I set the plot width I want it to be 400 pixels wide by 400 pixels high so we set the plot width and height I set my Y range to go from negative 10 up to 10 let's I'm going to use numpy to create a series of values I want them to go from 0 to 10 I want them to be 500 of them and for the Y values the actual numbers that will fit on the y axis I've used num pies sine function and I create a column data source a column data source takes something that sensibly has columns in it or has something that could be considered column like and allows you to create a essentially a reference library that you can then refer back to based on those column names you can feed it dictionaries and each key would be the name of a column you can feed it pandas dataframes in each column then becomes a component in here so in this case I'm making a very simple data source to dictionary the X column has all my X values the Y column has all my Y values and I can then refer to it so how do I refer to this thing I can tell my line plot that my source of data should be this columnar table that I've created and I want to refer to the two columns called x and y notice I refer to them by as strings much like you would a key in a dictionary your data source can have multiple columns and it doesn't have to be just two and we'll see an example later with like three or four different columns in it so I make my line graph I use a column from my source for the Y values a column for my source from the for the X values I set the line width the line alpha top it all right and I use that custom j/s library and I tell it that for the arguments that go into this we're going to refer to those source tables right and this is all straight JavaScript I'm not a JavaScript guy okay you are on your own in terms of making JavaScript essentially we read in some of the source data we are going to assign an amplitude value for our graph to show how high the graph should go our X values are going to come out of the X column in the Y column of the data and we're going to have this complicated mathematical formula that does stuff and changes wise based on amplitudes that you tell it and when my slider changes the JavaScript will get updated based on the current setting of the slider and it will essentially change my Y values kind of on the fly I create a slider object and I tell it to use this JavaScript callback to help with calculating some of the values I tell it my slider should start at point 1 and go up to 10 the initial value I wanted to start off with v1 and the steps should be point 1 I want to lay this out so my slider is next to my graph so I use my row object I say put my plot first and then put a widget box to the right hand side and put my amplitude slider I could have just put the slider this graph was taken from one on line and they actually had four sliders on on the screen so they used a widget box to kind of put them all together when I stripped out the other three I just left the widget box and we'll see an example of widget box again all right so some crazy JavaScript can't say that I understand it it does magic and let's make our graph and horrible things happened did I not read in some of my my data somewhere like that go away there we go okay I must have must have accidentally skipped one of my cells to produce the data so I've got some sort of kind of sine wave curve ished thing I've got the standard tools I've got negative ten up to ten that's the range for my y-values and I've got this amplitude it's set at 1 which is what we asked to be set at and it can move by 0.1 so every time I slide it along it creeps up by 0.1 and as I slide it my graph changes in real-time if you go look at the example that's online in the bouquet library the gallery why am i typing in all caps okay gallery let's go see if we can find this example and it could take a while to see let's see where is it there we go this one's got three different or four different types of sliders one for phase shifting one for assigning an offset up or down one to change the frequency and notice you know I can use any one of these sliders they're all independent etc all of these sliders would be saved inside that same widget box so he's got the widget box and he lists all the sliders in the order that he wants them to show up okay there was a couple of extra lines of code in here in the JavaScript to change the frequency values you know the value of K based on whatever the slider provides it as a frequency value or a phase value and then there's some other magic down here to create a slider object for each of these and to call back appropriately when needed to change values in the JavaScript to calculate what the new Y value should be all right so a lot of functionality there we saw that there's a default set of visualization tools that go in your toolbar you saw that we could strip some of those out and get rid of them let's take a look at adding some new ones all right so go in we import our stuff that's all awesome and magic we're going to generate a bunch of random numbers the M the number random function makes numbers between 0 & 1 I want 4000 of these so I'll have 4000 X values 4000 Y values and I'm going to set some some radiuses radii for a bunch of circles so I'll make 4,000 various radii and I'll let you read all this this is just some fancy way of making more hex numbers all right so I'm going to skip that notice in this case when I create my figure I can tell it right off the bat which tools to use and it's fairly customary to use kind of this format all the tools have a simple names crosshair hover pan wheel and you can look those up in the documentation all of the tools also have a callable function name and in the callable function name you can actually provide certain attributes for those functions to make these tools do more magic for you we're not going to go into the depths of that I just want you to know that that exists but if you just want to take the default values and the default functionality these tools you just put them all in a single string with commas and it will add each of these tools to your graph so let's see what we've got we've got crosshair hover pan the wheel zoom they have a zoom in and zoom out buttons you should always put both they have a box loom undo redo they have tap which allows you to tap on the graph either with a mouse or if it's a touchscreen with your finger they have the Save button and they have box select right we'll just go ahead and run that why did I not like that yeah I didn't run that one all right so let's run that okay and now you see we got a huge selection of these guys so let's take a look at some of these tools a couple things by default the pan was selected it's blue the tap is blue crosshairs is blue and this little hover tool is is blue those are active tools I can turn them on turn them off if you notice there is a black crosshairs in the middle of the screen right now as soon as it covers over or crosses over the boundary for one of those circles a hover tooltip will show up and it will give you some details about where that crosshair is centered so right now it happens to be centered over two overlapping circles so it's in kind of that the union of those are the intersection of those two circles right in the code we asked it to show me the index what row that circle was in the data and give me kind of the X&Y value and then give me the position on the canvas itself and as I scroll around you know it takes me to the various things if there's a lot of overlap I might see three or four things show up in my hover tip I can turn that off if that's a distraction so now it's off if the crosshairs are a problem you can turn those off right if I want to select an item I can click on it and it selects it notice in this case when I quote-unquote selected everything else got kind of grayed out into this bluish color if I don't want the tap functionality on I just turn it off okay if I want to select certain things I can use box select and we can highlight just the ones we care about we're going to go and zoom in now when i zoom if you watch the screen closely you'll see that it zooms the stuff out and then it redraws what's there and you'll start to see potentially more details showing up kind of in between this cases keep going so as I dive deeper into my data some of the things that might not have been easily visible before start to become apparent and that's part of the idea of bouquet and I'm going to turn the hover tool back on and notice as I go into these places where there's a lot of overlap you know I get more of these tooltips when I go to a place where there's less overlap the tooltips start to diminish a bit okay all right so that's that let's go take a look at it do you can I think we do a little bit we're in the in the geographical over a sample so wait for a few minutes and we'll look at the the geography one and we'll see where we look at changing that okay for your experience points what I'd like you to do is scroll up to the previous thing and I want you to add an additional pan tool to the tools string and then execute that code you're going to get a warning and I want you to consider what the warning means look at your toolbar anyway to see what really happened and then go ahead and take the pan tool out and then I want you to add the poly select and lasso select to your tools strings make sure you spell them the right way and just see what you get and play with the lasso tool or play with the poly tool when you're done with those things put your little green flags up sweet each dot what there's probably these were to save a reference to enough time to make up again it is what if you have cool sweet thank you guys hey Mike so one of the guys back here was looking at that problem - how far under - thanks dr. alpha which is kind of convincing so he put but like ref equals h dot quad and then the ref object has all of these dots and you can you can change them on the fly so what did you do didn't see the effective are sort of some playing around with it but it should say again yeah thank you - cliff yep that's what they did cool all right and it doesn't have a lot but it has a Q cliff thought it has a line I got a line alpha sine alpha but that's not all it doesn't have this because fo seems like just a convenience thing - sure yeah who does know alpha he has also does my place now is the description that was in there in the help documentation that alpha was it like this helper right yeah it's a convenient but I try to play around with it but I couldn't see the effect I set the color to blue that shows you can see the blue one is what I'm getting here but if I play around this was our sauce I don't see any difference oh oh um when when you run one of these graphs and then you add more stuff to it it doesn't delete whatever was there you just keep laying right on top of the figure you have to generate the figure from scratch so if you go up like two cells and you you regenerate the figure NMU reruns or so here whichever one had the figure generation in it right here so pulley but just copy this one this is so all right cool so let's take a look I'm going to go up here and I'm going to add another pan to the front of this and what did we get okay we get a warning says hey you've tried to put more than one tool in there it doesn't kill anything it doesn't crash your system but it does put a second pan in there so some wonkiness can happen if you're not not careful and what happens when we add a lasso select select all right we get this nice little lasso over here on the right side I'm going to turn this craziness off make that go away all right so lasso select and it lets you do kind of little draw things bubble blah one coolness if you want to do some selection and then you wanted to start someplace else into another one you can continue to add selection by using the shift key so I'm going to go down in this corner press shift and I'm going to do a little lasso selection down in this lower corner and there we go so multiple levels of selection can be done using the shift key similar thing happens with the Polli we're not going to do that interest time all right so let's talk about geographical maps for here if you run the download function it will download some some datasets for you they've got some movie databases they've got some geographical databases I think there's some data from Gapminder if you watch the the talk by Hans on YouTube variety of census kind of data etc so there's a wide variety of things in my case I've already got it downloaded I'm not going to run this cell this is going to start downloading stuff in terms of my imports though I'm going to import column data source again to kind of make that column in our table of data we're going to import the hover tool and log color mapper to help us out with a few things we're going to use the viridis six pallet off of our bouquet pallets and we're going to do standard figure and output notebook and then I really care in this case about the unemployment data that we have and the US County data so we're going to import those guys as well alright when you go ahead generate my figure I put a title on it I have a set number of tools I'm going to use and I'm going to turn off the x-axis and y-axis I don't want to see them on this particular style of graph it's going to be a map of Texas so they're not really going to be pertinent and in this case bouquet always puts grids on I don't want the grid to be there so I'm going to set the color to none right so and I like to know what my data looks like so we're going to look at the first 20 keys from the county's data and we're going to see that it's a bunch of tuples in this case the first number is going to be a 48 the second number is going to be some odd number one three five seven and then in addition I'm going to look at just one of those counties I'm going to look at County number 48 comma 3 and we'll see what that looks like so we see there's a bunch of these identifiers for various counties and when I look at just the data for one of them we see that it gives me a bunch of latitude values it's going to give me some longitude values somewhere in there and it's going to give me a detailed name which is Andrews County Texas it's going to give me a state and then it's going to give me the generic name of the county which is Andrews right so that's what my data looks like for the unemployment records that I've imported I want to look at the keys and the values if the key is equal to 48 because that's the counties that we're the state that we're looking at I want to print out and see what those results look like so let's go do that and we see that for all of the the counties with a designators starting with 48 we see some degree of unemployment value and all this stuff's going to get munched together all right so we've got data great I generated a pallet earlier let's go look at that one more time to do I imported just the viridis six pallet all right and in this case it's a list that's all it is to list with a bunch of hex values so I'm going to reverse that list and we're going to swap the order no big deal there I'm going to use some generator or some some list comprehensions and those kinds of things to parse through the data go through all my counties of the county state is equal to Texas save off the code in the county go look in all my counties and capture the longitude and latitude for Texas get the county name for each of the the counties that are in Texas and then go pull out just the unemployment data for the various counties associated with Texas all right then I have this thing called a log color mapper again log if you're not into math don't let that scare you it essentially breaks out values and it will take a series of colors in my palette and it will map those colors against a set of log values starting at a low range going up to a high range right so as things get really close to the high range the numbers change very dramatically and or the colors will start to change dramatically in this case the palette doesn't have very many colors so it's not going to make much difference but the point of this is in any case we're on this map the unemployment is significantly higher than kind of the average ish you're going to see dramatic kind of spikes in color so to highlight those guys so we have a tool called a log color map where I'll let you could research exactly how it works and they Stav some example write-ups of how it's working in the background so we're going to make a column data source when I have four columns in it we're going to have all of my X values for my counties all my Y values will have all the county names and then we'll have the unemployment rates so I'm going to have four table or four columns in my table essentially we're going to generate a bunch of patches each of these patches is based on the and patches is a function bouquet to create kind of shapes potentially shapes of unusual layout they're not quads or squares so it's going to take all of the x and y values and it's going to draw these little patches right I tell it that I'm going to use particular fill colors it's going to grab some data from my color mapper to help do that it's going to set the we're going to set the Alpha etc alright so let's go look at the hover tool now there's a question earlier about tooltips so I'm going to use the tooltip function and I'm going to set a policy that the tooltip should follow the mouse the tooltip I can define the things that I wanted to show so in this case I'm going to set it to have the word name and next to that I'm going to use a name value in this case it's going to take the values out of the name column that's what the little ampersand means right we're going to set an unemployment rate and then we're going to get a Latin long and I think the expiry yep all right let me run these things make sure I execute the code and we'll see we get alright so I have a graph of Texas I have a little crosshair that follows my mouse and as I go look at various counties you'll see for example I'm in Culberson County right now and the unemployment rate is four point five it shows me my latitude and longitude if I go look at this yellow one up here the unemployment rate was three I go look at this dark one over here the unemployment rate is fifteen point nine so if you have geographical data you can create patches like this to represent you know a city maybe the the blocks or the areas of a town any place where you know the geographical boundaries are something you can you can set up this type of thing now your experience points thing is to get on the internet and go to the boquete user guide and look up hover tool and tool tips and figure out what that syntax means I give you a hint that the @ sign name looks for a particular type of data what is the dollar sign next to the X's mean right what other options do you have for getting data in so get on the internet and go look for the syntax for generating tooltips praying of course that the Internet's cooperate with us and when you get done put your little green flags up what happened there it worked okay I don't think I've run into it before how are things going just rerun this out we run this out so if you need to restart your kernel you can go to kernel and you can go to just restart and it'll turn off by then and turn it back on and try and give it a second go you'll have to go to the various cells and execute the cells that care about to lock up completely if it is locked up completely you can go back to your Jupiter notebook server and kill the server and start over from scratch so you get into your command line and your server will be running on your command line and doesn't seem to be working is it directing can you restart it restart can we keep that code you have no love man how's it going great I do not know when I told it to install it basically grab to the latest and greatest version of ok so I'm not sure what version we might have yeah if you didn't upgrade or an update it may help ok is under extremely active development they are making changes daily so now they have released cycles obviously they don't change it every day but the folks at Bo care are regularly making upgrades and changes so how's things going anylock back here good good how about you James good ok I'm all caught up that's a good place to be awesome getting caught up so overrated all right so let's see go to interwebs let's see tooltip give chance to its thinking configuring plot tools and see if that gives me any love do control F for tooltip and custom tooltip alright so with the hover tool if you have a column of data that you want to have displayed where as it's moving around it'll pull the right value out of the column you can start it with an ampersand if you have a field named particular known fields you can start those with the dollar sign for example commonly used in in boquete is the X&Y value so if you put a dollar sign in front of an X it'll automatically display the x-coordinate under the mouse position if you have x-values that might be slightly different all right and then they've got you know another good example of setting up some tooltips they show like five to six different variations and they show kind of this sample over here they show you how to identify colors and they give you some mechanisms to show the fill color as well as kind of the outline etc so there's a lot of good stuff here right all right so I'm sorry go back let's go back dude alright so I went to the documentation and I went to the search box and I typed in just tooltips and it brought me to this page and you know Google search right however tool pops up a tooltip div wherever the cursor is placed over the glyphs so I'm like okay let me go look at that and I clicked on it and then I just use ctrl F right to try and find where this thing is might be on the page and I noticed that a couple of the options on the left side you know these are first tool references where hey basic tool tips and custom tool tips they have very good kind of table of contents in the user guide all the the things that you care about in many cases are actually represented to them their own little kind of subsection so they do a good job of pointing in the right direction and then I just clicked on one of those to take me there I scrolled up a little bit to see more about the hover tool and then it showed me the basic tool tips did that help sweet alright next let's talk a little about why what why doesn't that work there are occasions where your little question mark doesn't work there's going to be certain functions in here that don't necessarily have help documentation the bouquet developers are fantastic I'll talk more about them in a little bit but you know not everything is completely fleshed out right they are a large project it is under very heavy development there's a lot of activity going on in the developer space for bouquet so not everything is fleshed out as maybe we want right so if they don't have docstrings built in read the manual Google for it post a question to stack overflow the bouquet github account has you know a place to put issues you're like hey I noticed this function doesn't have something you can post an issue if you're feeling really ambitious you can create a fix yourself and you can submit that as a patch to the bouquet people they would love you alright and you'll get your 15 minutes of fame if you're like wow I would love to help bouquet be a better tool how do I get involved in open source contributions well you can come see us on Sunday we're gonna have a workshop all about that alright one of the other problems that I ran into besides not having help was when I executed certain cells multiple times some things happened and I wasn't expecting them so sometimes your science depends upon like some iterative thinking and iterative processing like wow what if I change this what if I tweak that what if I manipulate this variable in some way and so you might be tempted to write your code show your graph up tweak the code show the graph tweak the code show your graph and you can get some strange results this first showed up for me when I was looking at the histogram when you go look at the example histogram online they use completely different colors and what I use I use like red and yellow and orange and they use something something different I didn't like it so I kept changing the colors to try and make the colors come out the way I wanted and I kept tweaking and running the same graph and what I was doing unbeknownst to myself was laying over on top of each other multiple glyphs on the same figure so I had for example made a line they were like eight pixels wide and I wanted it to be four pixels wide so I was laying a four pixel wide line on top of an eight pixel wide line but they were the same color so I couldn't see the four pixel wide line I'm like why isn't this changing this is broken these people are horrible how could they do this to me I have a tutorial to deliver so let's see how this works I'm going to go in we'll execute this we make a figure right so we've got a random figure and for your experience points we go in and we execute this once and we're going to just go look at it okay we're going to execute it again now this thing actually calls up a random choice of colors you don't have to change anything else tweak it it'll automatically tweak it for you that's why I use the choice so it's going to change the line color every time I do this so I'm going to execute it one more time we'll see that the glyph ID is going to change when I execute this a second time got a new glyph ID and when I show my notebook we'll see now I've got two lines which is not what I wanted I actually wanted a single line with maybe a different color so now I've got this kind of corrupted thing where I've got more data than I cared to have and so I kept doing this a bunch of times and now I have lots of stuff right and so chaos erupted and I was crying there's weeping gnashing of teeth all right so be cautious when you go tweak your stuff and especially if one thing overlays another may be very difficult to see this is an obvious example love Wow I can see what happened here but when I had a thin line on a fat line that didn't work out all right so be aware of that we're going to talk about servers so that I was running the servers or a server and they got very wonky I had some drop-down menus and I asked it to change some values and drop-down menus and the server kind of hung up halfway through and I could not get it to to jumpstart itself and go back to normal I was unable to figure out why and this happened about five or six times when we get to the server place I'll show you what I did and and we'll see what happened but so something about changing my data sources and feeding that into the server something got a little weird but again I never couldn't figure out why I was trying to run the code for the servers and there were some instructions in certain examples for the server's I mean the instructions varied a little bit depending whether you're using the reader the user guide or if you were reading the code if it comes with the server they have like a main dot PI file so I'm reading the main dot PI and I'm reading the server user guide and the instructions were just a little bit off so it took an inordinate of time for me to figure out how to make the the server code run so that was a bit frustrating internet access was required and a couple folks ran into this so when you first do this sometimes you need to get a copy of the Bocage a/s libraries and they're kept online and you can go get the latest and greatest javascript library and it's all good unless you can't get to the Internet in which case it's all bad when you fill out the survey on this class I am NOT accountable for the Wi-Fi ok just saying there are mechanisms though if you need to host your JavaScript code on a particular server like maybe you're in some sort of an environment where you don't have access to the outside world for various reasons whether they be security or it's a proprietary network and you know you're doing research on it and you don't to leak out etc there are ways to host the JavaScript libraries this is a problem that solved like with bootstrap jeaious and things like that you can host the the libraries themselves on your network so you can get access to them that's outside the scope with this class I'll let you guys figure that out output file you're killing me so just like updating your your your figures can lead to some heartache the output file led to a great deal of heartache for me you notice in every case we've been outputting to our notebooks and that was on purpose originally I kind of had it doing some things where let's output the notebook and have the graph in the notebook but let's also make an HTML file that we could give to our boss or somebody who is not computer literate and we don't want them playing with Jupiter notebooks the output file once you have initiated an output file function call there is something about it and it just hangs around in the background and every time you execute the show command it will try an output to that output file even if you're started on a new function or started on a new figure so let's take a look I'm going to we made a figure a few minutes ago is figure P and I had all those random colored lines on it right so I'm going to import the output file and I'm going to say I want all my results are stored at results HTML and I want to show P and when I run this it's going to make a very nice output file for my computer illiterate boss I hope he never watches this and by default it opens it up in this case it opened up in my default browser which is Safari even though I was running in Chrome and if you were to go to the folder this is in you would find it's called my results at HTML it is just an HTML file there's no Jupiter notebook Ness but all of the things that we want are all here you know the zoom and the pan and stuff they all work it's just straight HTML so again you could ship this file to your boss the JavaScript is embedded in it everything's good single file but that output file object is now floating around ready to rear its ugly head I go down here and I'm like hey I want to make a completely different figure now it's called figure H not figure P and let me show H now it's going to be an empty graph there's nothing to it but let me show H when I say show H it goes to my results dot HTML and it overwrites everything in that file with nothing right so they do make mention of this in the user guide I found it after the fact scratching my head but just be aware that that can happen right good yeah yep absolutely that should not be a problem and so with the point that he's making is is if you wrote a script and you said hey in the script import bouquet create a figure create an HTML output and you ran it as a dot dy file it should make your output and everything is done because your script is finished running we are in this kind of middle ground where we're in a Jupiter notebook and there's all these things floating around the background and because of that it puts us in this semi dingy same semi dangerous position where data we want to keep could potentially be overridden if we're not aware that this is happening so that's excellent point thank you all right so those are some of the things that that I stumbled upon as I was going through this this learning curve with bouquet right and they can trip you up to a number of these are likes to kind of mentioned in the user guide you have to hunt them down but be aware you're dealing with a very sophisticated tool stack you're dealing with Python you're dealing with Jupiter notebooks you're dealing with JavaScript and there's some coffee scripts thrown in the middle of that and you're dealing with JSON object no object notations and all sorts of craziness right so there's a lot of places where something can go wrong which i think is a good testament to the power of this tool and the developers were making it it does a good deal of this stuff seamlessly even though it's such a complicated system all right so let's go to servers we're going to do a very simple server application this is based on code in the bouquet github repository I stripped down a few things and tweaked pulled a few things out just in the kind of interest of time and interest of space so a couple things the bouquet server is an attempt to kind of keep your browser and your data synced more effectively so you want to have a vent response when somebody interacts with the user interface you want computations to be done on the server with the full power of the server and its computational engine right using Python you potentially want to have some server-side updates automatically pushed out if you're dealing with streaming data and you want to have the interface updated based on the streaming data or in response to timeouts this data is no longer valid those kinds of things the server can be run in two different modes you can have a kind of local individual use hey I have a lot of data and I want to see it I want to play with it or I'm going to stand up some sort of a server that folks from around the globe can access we will focus the conversation today on local and individual use that should get you started down the road and get you prepped for if you wanted it to create a deployable application so behind the scenes the bouquet server you of your application code when a browser reaches out to your server the application code will produce the documents for that particular browser in that session the document gets wrapped in this bouquet JavaScript the bouquet J's library which allows you then interact with it etc and you can have communications back and forth between your browser and the server document has been created for you in your session alright so you can have multiple sessions with independent documents when we stand up this okay server we're going to use the stock data example that they've got first thing we'll do is going to prep your environment oh god this is horrible now that I think this through Wow all right stir it you've all got bouquet you've got you but you got a Python you thought about pandas right the only thing I want you to do is do this let me let me change this on the screen okay we're going to do Conda all right you guys can just use your graphs just use the graph virtual environment you in go ahead and install Khan tornado 4.4.2 so here's the thing I was doing this on my Mac and everything was beautiful its glorious everything worked awesome and my colleague Aden wanted you know trying to improve the robustness of this particular offering let's try it on a Windows system and see how well that works so he dredged out an old Windows system right and tried to run it and it had a more recent version of tornado on it had like tornado 4.5 or something and yeah that was bad news so we checked my Mac was running tornado 4 4 so everything works great the Windows machine running tornado 4 5 that was an epic fail so this morning I reached out to the folks at the bouquet development channel and I spoke to them at about 11 22 and I said hey I'm trying to run my server some certain things are happening and I get some craziness and it's ugly please save me because I have a tutorial to do later this morning and within 22 minutes Luke kindly piped up and he said hey there's an incompatibility between bouquet and tornado for any version above 4.5 so the developer Channel is chock-full of folks there's about three or four hundred people in there tip at you know over the course of time they answer questions regularly they're just many of them volunteers so you know be kind be gentle but you can see just today since this morning they've had a lot of questions come in and they've had a lot of good answers come rolling back outright so alright so yeah tornado is probably in there but you might have a more recent version than you need especially if you're on Windows la blah blah blah blah area alright if you go to github and you download the zip file from them you'll get the server application stuff and we'll dig into that so you want to save all of that to your graphs folder save it to your graphs folder now here's where it gets interesting when you have unzipped that if you change directories and go into the stocks folder they actually have a script that will download some data and that script is called download sample data pie so run that with Python and that'll put that the data for this particular application in your graphs folder okay and this is critically important don't miss this back out of it otherwise a follow on commands going to fail so make sure you back out of the stocks folder go up one level into app and when you get all that done give me the little green flags and we'll move on how's it going horrible awesome so tell you what while that's downloading let's just talk right and let's go look at some stuff alright so a couple things um a couple nuances here I have the code to run the server and I've cut and pasted some of it into these cells these cells are not executable they are written in markdown so you can't like click this and make magic happen this is just to talk about the server script is actually a Python file and it's going to be downloaded etc so don't try and make these things run we're going to look at the code here in the notebook you'll go to your stocks folder they'll have a file called main PI the content in the notebook is greatly compressed and reduced for clarity I strip out some duplicate or similar lines I'm going to cut back on some details that we covered elsewhere in the workshop I want to focus again on the things that are pertinent if you do want to edit the code and play with it you can do that just open main dot PI as soon as the Internet's allow you to finish downloading it alright so main dot PI I open it up it's got some sort of a header in it and it talks about stuff hey choose stocks and drop down widgets and magics happen histograms and summaries update I snipped out some content in the middle that told you how to run the code and then when the code is actually running by default bouquet opens it up for access on port 5006 so you would go to your local host use your browser go to localhost colon 5006 go to stocks and you should see pretty awesome graph all right you should they had some imports they had some things related to function tools and things like light like that least recently used objects and they had some sort of a placeholder for that if you're using Python to version versus three so I sent all that junk out a couple new things we have pretext and we have select we'll talk about those in more depth those are related to widgets that might appear on a model we saw a row before to make a row of objects if you're going to do a layout we now have a new thing called column to do column objects or a column of objects and then we have this cur doc which allows you to interface with the server remember the server is going to produce a document for all the folks who browse to it all right a lot of this is just Python I did go through and I added a slew of comments to each of the functions so you can read that at your leisure I'm not going to go into much of this because this is just pure Python as opposed to bouquet but long story short there's a bunch of folders they all have CSVs or something in them with stock data for like a hundred different stocks we're going to look at stock values for five stocks when I need to select from one of the dropdowns I want to compare how Apple stocks performed in relationship to Yahoo or how Google in performed in relationship to Yahoo and I'm going to have two different dropdowns and if I select Yahoo and one drop-down I don't want to compare Yahoo Yahoo so I want the other drop-down to update in real time and get Yahoo out of there it's going to say oh he's already selected Yahoo let's just show him Apple Google INTC and be our cm let's not show him Yahoo in this second drop-down so Nix is going to take care of that for me [Music] I've got some functions that are going to give me back a data frame if I give it a ticker symbol and it'll have dates and closing prices and if you give it to this function get data if you give it to ticker symbols it's going to load a bunch of data and give you a data frame but it will strip out anything where there's no values the two data sets will get concatenated and the columns get renamed again this is all just pure Python so we're skipping that all right this is bouquet in HTML you have a tag called pre and it allows you to essentially make kind of pre formatted text a lot of people use it to display like courier new style of font or code style of font so this creates a widget that displays blocks of pre formatted text it's all mono spaced formatting so I'm going to make a box that is 500 pixels wide and we're going to set it up with no text to start with but we will update stats later so stats is going to be an object and it'll be updatable all right the Select function it creates a drop-down widget so you can select things you tell it a default value which is in this case Apple and Google it will then run that NYX function to filter Google out of the Apple select drop-down and filter Apple out of the Google drop-down and this will be updatable in real-time so when I change that later on all this stuff will get manipulated in real time all right I'm going to create a data source it's going to be a series of columns so we'll have the date data will have ticker one ticker - will have the return values for ticker one and two so we'll have we'll create a set of columns called source and another set of columns called source static and we'll refer to both of those in our graph I tell it what tools I want there's kind of standard alright but one thing to notice and this this perplexed me for a while I searched the user guide and I couldn't find it this one says X Box select down here it refers to box select and I could not figure out for the life of me what the difference between those two was so again I got into the developer chat on git er and I said hey and like 20 minutes later they're like oh no that's what this means and they sent me a URL and pointed me in the right direction so they did an absolutely fantastic job of kind of helping helping folks in the community alright we'll talk more about what Xbox select does I'll show you so I'm going to create a figure it's going to be a correlations figure this would be kind of a square I set the plot width plot height I tell it the tools in this case I'm just going to use a regular box select and we even seen that multiple times through the day but I've got this preset tools that we'll use later in in two different instances okay all right my correlation it's going to be a circle graph so I'm going to correlate ticker one returns verses ticker two returns and I set colors notice these two right here non-selection alpha and selection alpha when you select something you can choose an alpha value for it and anything that is not selected can have an alpha value as well if I wanted to change the color for things that are not selected I could choose non selection color and selection color so a lot of these kind of colors can be tweaked depending on whether they are selected or not selected so that's a cool function all right I'm going to make another figure this one is going to be one of those kind of classic stock graphs that goes up over time up and down it's going to be 900 pixels wide so it can be very wide but very short and the x-axis type is going to be date/time so bouquet should interpret this as a date times I think Mike was mentioned date times earlier so I said x-axis type is equal to date time and I'm going to set one of my tools to be active there might be multiple drag tools on my toolbar so my active drag tool is going to be xbox select if you have multiple drag tools and you won't want them to show up with that little blue bar next to it to say that it's the the tool that's currently selected that's how you do that I don't show it but there is in the real code another one of these snippets that's for ticker symbol to does the exact same thing we're going to make some glyphs we're going to start off by making a line we're going to make some more glyphs out of circles alright I got a bunch more functions that has various values in the tickers change I want to update stuff on the screen again I'm not going to go into this most of this is just pure Python let's see let's see let's see some good stuff ok whenever a ticker object changes so on change to a ticker object call one of those functions to update some of the values all right let's see that again a lot of this is just Python so we're going to skip that all right how do we lay all this stuff out bouquet has a layouts mechanism you can create a layout and you can tell bouquet to apply that layout to the graph that it's going to produce in this case and I'll blow this code up in a second but in this case I'm going to create a layout object and it's going to be a column and in that I'm going to say I want to have a main row and a series object the main row is going to be a row of things so I'm going to have two things in a row side by side the first object is just going to be a graph it's going to be a glyph but this item is going to be a column of widgets and each of these are individual elements or glyphs to put on there so I can lay rows next two columns inside of rows on top of columns there's all sorts of very intention put here so what we're going to see is a square graph there's going to be a bunch of little manipulators over here that we can tweak I'm going to have a place to show that pre-formatted statistical text and I'm going to have two other graphs down here at the bottom that are going to show the ups and downs so how do we lay all that stuff out and if you're familiar with tools like you know graphical user interface design tools he can turn those kinds of things I started uh Sarah my layout is a column and it'll have a main row and have the series my series is also a column with two objects in it my main row is a row with one of object and the widgets thing and the widgets thing is a column so you just kind of nest to these guys together and when I make my current document I'm going to add a root level object and that's going to be my layout and it will nest all the things within that document I'm going to set a title for it and hopefully magic all right so how do I run the server I'm going to go to my stocks folder I'm going to on the terminal I'm going to type bouquet serve - - show and this little dot that dot says check in this folder for a main dot pie alright if you leave that dot off craziness happens all right so present working directory I'm in the stocks folder bouquet serve tactic show I put the dot all right so let's go look at this first says hey Chalmer we're starting the bouquet server tells me what version tells me what port to look on in case I forget and says when you surf to localhost on that port put a slash stocks and you'll get to where you need to be and then it starts to tell me what's happening and it kind of logs my interactions with it alright see we blow this up a little bit alright so I've got this correlation on the Left I have my two ticker symbols I got my stats which should update in real time and then I've got the little you know little up-and-down graph so let's do some stuff let's go select some things notice is I selected details up here in this graph the other ones at the bottom automatically update as well they are selected the same time if I mentioned we have this thing called an Xbox select what the Xbox select does is it allows me to select data on the graph similar to what I did here but it actually selects in only one direction you'll see when I try and drag and do some selection it'll select everything on the y-axis all the way across so grab and notice it's selecting an entire box so this selects across the x dimension they have a y box select as well if you need to select in a horizontal direction and notice these are completely interactive right I selected up here and this got highlighted I select down here and this gets highlighted so there's interactivity in both directions if I need to select more than one thing I care about the data here I also care about the data over there again you can use your shift button on your your laptop to select more than one thing let's go reset that okay so say I don't care about Apple anymore and notice when I select on this depth this drop down it says Apple int CBR cm and Yahoo Google is nowhere to be found because of that one function so let's change this to Yahoo go down here B or C M oh don't tell me yep so remember earlier I said there was this wonkiness the server app did some very wonky things this is it this is awesome in a horrible horrible way all right normally what is supposed to happen is these tickers are supposed to update in real time and everything's supposed to be good and the stats box is supposed to update and show Apple and B are cm and NTC and stuff and yeah so this is crazy bad all right it's failing me epically all right I was never able to figure out how or why this occurred nope it's not working all right so the it gets wonky sometimes that's legit this morning I ran it everything worked this afternoon I run it and it doesn't and I have yet to figure out all the nuances of why so haven't changed the code throughout the day haven't touched it so and with that we'll move on to the next thing your mileage may vary okay sure sure so this this particular example is is just going to stand one up so you can play with it right when you get into creating an actual web application and the visualization is just some subpart of all of your data right that takes on a whole nother level of complexity there are mechanisms and there is some some right up in the bouquet server documentation to say when you need to build a deployable standalone app right that people are going to be interacting with follow the following kinds of steps you can use templating so that your data can fit into an HTML template he mentioned using Jinja and things like that to create like a django app or whatever so there's mechanisms to embed your server outputs on your app but that's well outside the bounds of this class at this point we don't even get this to work right and we saw them figure out why it is it was crazy wonky like I said it turned things on turn things off do what shouldn't take that long yeah it's gotta cross the Internet yeah but these updates these updates should be nearly instantaneous I mean as soon as I change one of these tickers and say you know select this the other ticker should be instantaneously updated etc it was pretty bizarre I'm cleared out so I've got to get in the the github and identify that this is some sort of a glitch that I don't understand yeah he's working on my machine this morning so okay we'll wait eight hours all right so let's go back to the rest of our conversation sure so I do not know where the problem lies I don't know if it's my browser not talking to my server I don't know if it's my server is not acting nicely so you would have to go through kind of a series of debugging steps and try and figure out where the problem lies I've played with this multiple times and it has occurred at complete random and it has been intermittent some days again without changing the code I haven't touched the code it runs great other days it doesn't and I haven't figured out why yet so and it's the default app right it's it's not code that I created it's their app and on one occasion it works on another occasion doesn't so it's wonky hence the reason it ended up in that section all right so we have like one minute left stuff to do I highly recommend to read the manual there's lots of good stuff in there read the functions themselves using the question mark go into the same people graphs let me pull up one of those and do go to the gallery alright so this gallery has all of these various types of graphs they're crazy good every graph has got code associated with it so you can see how they made a layout with nine graphs on a single figure yada-yada-yada right so go look at each of those things where did where did that dudududu charting your course alright so go look at the sample graphs play with them a lot of the graphs have kind of tips and tricks that you'll come across and allow that's pretty slick you can figure out how effects are generated in the user manuals and I've alluded to this there's lots of sample graphs throughout the user manual that aren't in this sample gallery so as you're reading through the user manual you're going to see cool stuff and go WOW how'd they do that we use the sample graphs a lot they're pretty good and most importantly just make graphs the more time you spend with boquete the more you're going to come across those nuances and you'll start to expand your understanding of it and lastly I've got a number of links for both the examples from the gallery things from the user manual that were particularly useful for me how to set up styling or annotations the types of tools I could use and how to use them how to set up the layouts that I wanted cetera so those are some decent resources and then some additional resources we've got they've got the bouquet manual they have a notebook gallery people have created jupiter' notebooks with graphs in them the github to get into their issues and their bug tracking and stuff you can see conversations about some of the problems you might run across and that can potentially help you the style guide is great for visual attributes and if you want to know more about condor Joop for notebooks there's some links for that as well all right there is a survey that I am told we would hope for you guys to to look into surveymonkey.com I con 190 so surveymonkey.com can't spell Survey Monkey right okay /r PyCon 190 so you go to a survey monkey /r PyCon 190 and there's a survey they're curious about how this went right so please provide feedback survey should not be very long it's only about 10 questions right and they're all pretty straightforward so I'll stick around for a little bit answer any questions for people who are shy do the best I can I appreciate you coming hope you found some use in this grateful for the opportunity to speak to folks so thank you [Applause]

Channel: PyCon 2017

Views: 9,550

Rating: 4.7530866 out of 5

Keywords:

Id: xId9B1BVusA

Length: 204min 11sec (12251 seconds)

Published: Thu May 18 2017