Get it Right in Black & White Episode 10 - Dynamic Charts

all right here we go all right we're live welcome everyone to episode 10 of get it right in black and white dynamic charts so first i'd like to start with some introductions um we have some we have a new person today welcome maxine uh you want to introduce yourself briefly yeah so i'm maxine or max and i worked a bit on um viz hub with curran and that was really fun because i think it's a great product and now i'm a data visualization engineer for mural and in my free time i just try to learn as much as i can about d3 nice i love mural by the way i've started using it it's a really neat tool and uh adil is here hey adele hi karen hi maxine all right so let's start by reviewing the submissions from last time here we go reusable charts so here's one from felipe hilarious um i think it's it's like the data of how many viz hub users there are but with my face in there so nice work felipe um yeah very cool very cool i want to take a just quick look at the code it's still the same scatter plot like template with the you know the methods and everything and it's just been modified internally to show these svg paths pretty cool all right what else have we got something from ej dasbach this is pretty neat i was actually pretty impressed with this one it's a nice actual visualization of covet 19 deaths over time and uh yeah great work adding the title adding the labels for the axes uh vaccination at 30 that's like some additional information pretty cool and great job linking to the source too in the vis itself very cool very cool and this easily could be converted to a line chart just by modifying some of the internals i think a line chart might make more sense for this particular data although it's kind of cool with the dots because you can see the gaps yeah good stuff all right yeah there was some confusion about last week i had to cancel but um all right i think that's it that's all there is so all right great to see some participation [Music] um yeah before i dig into um stuff for today i'm just wondering are there any questions from last week's content adil or maxine that was unclear not from my end i thought it was pretty clear nice yeah it was really very clear um i think returning the method chaining in particular was quite uh uh it was it was just yeah really eye-opening really oh great i'm glad to hear it yeah yeah that the pattern still blows my mind because it's just javascript essentially but you can make these components and you're not tied to any particular framework like react or view or anything i was wondering when um doing the court dot call uh and uh all the method chaining that follows whether there is a uh whether there's a convention uh at all or um a preferred i i guess it's a personal approach but um when uh uh when doing this uh svg.call scatterplot um then all the methods that uh follow um are they is it recommended to do them in any particular order for example to do the data manipulation first followed by the dom anything that touches the dom or is it um is that is i just just it's up to the personal preference yeah that's a great question does the order matter is there any particular patterns no i would say no it's it's like lego blocks you know you can assemble these things in any way that you want and as long as their relationships are correct between like the inputs and the outputs and um the order of like the dependencies uh is correct like first you need to set up the svg then you need to call it but beyond that um no it doesn't really matter it's totally personal preference you know which order you call these things um and even i've seen it quite often where you you construct the the element like right here instead of even having a variable and like that works too it's totally personal preference yeah stylistic yeah that makes sense thank you all right we've got another guest um e um hello there you want to um here let me just pull up the introduction slide are you here yes i'm here eric dosbach here can you hear me yes i can hear you great to meet you yeah nice to meet you um finally got this down figuring out between um youtube and um google meetup so yup yeah it's kind of a kind of a quirky setup but it works yep so just a little about myself a data scientist outside philadelphia trying to upskill my data visualization so i followed the first nine and now looking forward to participating live amazing amazing welcome and i i think i just um presented your work but yep but i would like to welcome you to present it yourself here we go let me find it sure here it is yep yes so a little bit yeah so thank you yeah so the um so appreciate the opportunity the part of what i do and so i work in healthcare and and part of what i do is visualizations of uh these sorts of dynamics and so forth and so i really enjoyed the notion that we had a reusable scatter plot and i think the assignment was to switch out to another data set one and then to start adding axes and so forth and you know i found that framework uh really useful from which to do it i think probably one of the most challenging things i found was just the date scale and figuring that out and just you know trying to get that to line up right i still haven't figured out how to manipulate the months and years and so forth quite right but um because viz hub is so nice and interactive it allowed me to just keep trial and erroring to get it to this point which is really helpful for uh learning nice that's fantastic yeah well so yeah and i guess the other part of the assignment sorry i lost you there you there i lost your sound there eric uh are you still there yeah i'm still here yeah oh you were saying oh yeah i was just saying so i think his sound is down yeah i lost your sound um oh you muted yourself can you hear me now yes okay uh yeah so i was just saying that i guess the other aspect of this was trying to um figure out how to make things item potent oh well yeah that's what i was going to get into today that's what i plan to really touch on today and so um so i was starting to go back to some of your work and react ah yep yeah i mean i would like to address some of your questions around the the day formatting and stuff because it's actually quite simple what you can do to make adjustments to the formatting um so it's a little bit off topic but i'll spend like five minutes on it just to to move that forward because because you were struggling with that so when you oh i gotta fork this to make some edits to have a little more control over the the dates and the formatting there so we've got a scale time here and one thing you can do is control the number of ticks by saying dot ticks and you pass in like five or something that gives you oh it's not loading what happened oh sorry not on the scale not on the scale on the axis yeah that's something that trips me up so when we when we create the axis oh you did that already take six so if we up it till like 16 we get more ticks and so if you want to change the formatting is that what you wanted to do change that formatting yeah i was trying to figure out so um implicitly there was it dropped the years in and then um the months and so forth and i was trying to figure out how how to actually manipulate those sorts of things totally totally so you can use this thing called d3 time format and this old example on blocks.org from zen armstrong 2017 is a tried and true way of figuring out how to format dates with d3 so the way you use it is now it's called time format camel case it used to be time dot format but the api is pretty much the same so you can pass in this weird looking string like you know percent y dash percent m and each of these little codes has a specific meaning like this is the full year this is the the month as zero one but let's see what kind of ticks might be appropriate maybe just the month right like uh yup i think i actually found this site too nice i was i was um somewhere in there i had the month uh aspect oh nice and time format is even there i see yeah nice nice and so when you call axis bottom you can just call dot format oh okay or maybe it's i think it's actually tick format yeah and then you can just pass in a time format of this string which i can't remember i have to look at the example dollar sign b for like jan feb like that yeah then i'll use prettier to format the code and now we get huh that did not seem to do the trick yeah and then you actually to i think manipulate the text to them well that should work that should work let me see if i got any errors no hmm so if i search for d3 axis tick format i hope i get the um the name right yeah axis.tick format i should do the trick [Music] i wonder if it was set somewhere else as well i think you're right i i know i've said it somewhere else i'm trying to remember where yeah because i think if you if you set it on the axis or the scale it might like take precedence or something oh wait a minute there's two oh there there's two axes on the bottom yeah i forgot to clean that out this is when i was going to item oh nice let me see so that might be just not visible and then let's try putting it on this other one let's see we need one more param yep there we go yep there got it boom yeah so that's how it works very nice very nice thank you yeah my pleasure thanks for doing the assignment i'm really happy to see this sort of uh engagement participation yeah i appreciate it thanks all right we've got a bunch more people coming let's see what's going on here we've got um maximiliano hello i think you came last time and we've got ken penn hey ken welcome and we've got uh maribel welcome maribel would anyone like to introduce themselves am i muted oh there you go i can hear you now this is weird um yeah no i'm still i'm new to to um to d3 and um still learning a lot so i have questions i shall listen okay cool well feel free to interrupt me as i go and and ask the questions along the way thank you hey kuran and everyone yes i was here last week how are you everything's fine thanks good good so great to be here again welcome and i see ken joined us can you want to introduce yourself one a little bit uh there was a bit of an echo there so i was kind of holding back uh ken pan um currently who ran on everyone yes i was here yeah you gotta close the youtube sorry it's a little a little funky of a setup good good but if you close out the youtube uh window shouldn't work there we go uh yeah so uh ken pan um have been using d3 for a bunch of years um currently um unemployed and just uh doing this to stay in practice and get tuned up on incorporating d3 with modern javascript nice well i'm glad you could join us ken um ken and i go way back from the d3 meetups in san francisco many fond memories of attending events there meeting people learning things so great you should provide a link to your uh d3 parade entry yesterday that was quite good oh yeah yeah that was fun yeah i hope to actually build something like that as part of this series as we get further along um but yeah yeah it was fun that was fun i i think it was very instructive too and it might be for um for other people where you showed all the the uh wrong steps broken-made art that you had oh yeah persisting to the right kind of like you just helped with the dates just now i thought that you know learning to use it instead of uh just getting frustrated and quit is uh you know how to persist through that is really important i think nice for sure and that's why i'm having this whole thing um you know live with q and a so because i know people get stuck and i want to teach people how to go from stuck to unstuck because that's the key thing yeah all right we've got some other folks joining us um hello felipe how are you and we've also got matt oliver hello matt welcome howdy uh thanks for doing this queren i'm in texas and i'm uh in my day job i'm a product manager and i love building with d3 and love learning new stuff so happy to be here and haven't been able to make the last few but whenever there's time and whenever i can i want to i want to participate so thanks again nice this is amazing wow welcome well this is a full house we've got so many people so let me just dig into this what i wanted to do today was to make our chart that we made last time dynamic meaning what i was thinking of doing is cycling through x and y columns we saw last time that if you change the code to change the function that accesses the data for x then the meaning of the x column changes to be that column and what i want to really touch upon today is like if you want to make such a change interactively without having to go and change the code how do you do that that gets into this concept of idempotent rendering where you have this function that you invoke namely the scatter plot and what you want is that the scatter plot knows how it's supposed to look it has all this configuration set up and what you want to have happen is when you call that function again for the second time or the third time or the fourth time it should just sort of whip the dom into shape and and be like all right dom like you need to now change to represent the new setup the new configuration and with d3 there's all these foot guns um things that you can use to shoot yourself in the foot uh like dot append which doesn't really work well if you invoke the function again and again because it's going to append more and more elements again and again which is not what we want and there's a couple telltale signs of when this is happening that i think i'll get into today and then if there's time after we get the idempotent rendering down if there's time i would like to dig into animated transitions but we may not we may or may not get that far we'll see all right so here is the reusable scatter plot from last time after last week's session i i had like a realization that like oh no it's not actually done because it's not idempotent uh which which sort of defeats the whole purpose of making it reusable and this is a word that you don't usually come across unless you're in like computer science so i just wanted to read the definition of the word i think i think the wikipedia article does it justice idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the results beyond the initial application so the key here is that the result that you get after invoking the function should be exactly the same regardless of the state of affairs before you invoke that function and so that let's let's apply this to our to our code here all right so first of all we need to have some change be happening and so i propose that instead of just calling the scatter plot one time like we're doing here we can call it multiple times in a loop of sorts so let's say every every second or every two seconds we change the value that gets returned from this x value accessor function and that would have the effect of you know changing it from let's say pedal width to sepal width boom see that change that's the kind of that's the change that i want to see without re-running the program so how would we do that there is a construct in javascript called set interval and what set interval does is it accepts a function as the first argument and as the second argument it accepts a number of milliseconds so if i put a thousand here that means every second it's going to run this function whatever it may be so let's just you know console.log here to make sure that this the setup is working then i'll open the console and we can see it's printing out here again and again and again every second and those little numbers mean like that's the number of times that the same thing was printed out again and again and we see it's incrementing every second but if we change this to 100 it's going to increment every 10th of a second so we'll see the number going a lot faster so what i want to do here is let's say every two seconds we're going to change the meaning of x but how do we do that and and this comes back to your question from earlier a deal of like does the order that you do this stuff matter well the order depends on the use case like the scenario where you need to invoke this right now we have to do a little bit of refactoring because there's a bunch of stuff happening in one call but now the task at hand is to tease out what needs to happen once at the beginning versus what needs to happen every time we change x for example we're calling await csv here which will fetch the data and we don't want to fetch the data again every time we we set it up and also the scatter plot is being invoked i mean the constructor of the scatter plot is being invoked within this svg call so we have no way of accessing the instance of that plot to change its configuration so i'm going to have to move this around a little bit what i'm going to do is pull this out into a variable i guess i'll call it plot equals and then we invoke the scatter plot now we have we sort of have a handle on this thing so we can change it after the fact now we can do svg call plot right here to initialize it and we can use the same construct inside of our set interval console.log plot like that so now it is invoking our plot function every two seconds and we're already starting to see some of those telltale signs that that the thing is not item potent namely notice how around these labels it's not quite smooth it's a little um pixelated or like harsh when you look at it me and what's happening is the anti-aliasing is getting messed up because what it is there actually is a bunch of layers of the axis again and again and again and we can see this if we inspect the dom we can see if we zoom out to the right level there are multiple copies of both the x and y axis being added and if we scroll down to the bottom we can see that every two seconds there's another copy of both axes being added uh so this is the problem that we have to set out to fix and this is a very common type of bug when you're working with d3 because the way d3 is structured like you can get something working when you invoke it once and that's pretty straightforward you can use the dot append construct however when you when you have a situation where you need the thing to change over time you have to actually do different stuff you need to make sure that you use the general update pattern of d3 which handles the enter the update and the exit of all of these dom nodes and the problem here is that the parent group for the axis is being just appended over and over so that's what we need to change so let me go back to our um scatter plot although you know let me let me just finish this this thought first what we want to do is change the value of x and we're ready to do that so let's do that first let's change x and then let's go and make our our reusable component idempotent we can just say plot dot x value and instead of d dot pedal width we're going to want to cycle through so we can say d at column and we can say column equals petal width to start and how do we cycle through the various columns i propose we have something called columns which is going to be an array and pedal width will be one of the options another of the options is sepal width and then we can copy those options and add height or no length what was it and let me figure out what that was actually from the data yeah it was length oh yeah length length it was indeed so we can change width to length now we've got these options to choose from for our columns and what we want to do is say column is columns at index let's say i and i is something that i would like to increment every two seconds but once it gets to the top we should it should like cycle back it should circle back so instead of using const i'm going to define this outside of set interval and use let i'll initialize i to 0 and say i plus plus and then console.log i let's see what this does so far it says 1 2 3 4 five and this is where it's getting buggy because um once it gets beyond three it should go back to zero and the way that we can do this is i equals i plus 1 modulo columns dot length minus one and just for well let's see if it runs zero zero oh it's not working just for some context if you haven't seen the modulo operator four modulo two is 0 but 5 modulo 2 is 1. modulo 2 is is a way that you can check if it's even or odd and so what we want to do is make i go from zero to like three and then go back to zero so you know what we could do actually is just we could simply say i plus plus let it increment and then let's console.log i modulo column stat length minus 1. you know it could have been a parenthesis issue let's try like this so we see 1 2 zero one two zero but we have four options um maybe it's just a not off by one kind of a thing let's just modulo by columns.length i think this should give us the right thing now one two three 0. all right so we just need to use this when we access from columns like this so we can just sort of compress this like so and i kind of want it to start at zero so let me just increment i after we do this this stuff here all right now it should be behaving as we expect so this is a dynamic scatter plot the circles are idempotent they're behaving correctly so what it's doing is it's cycling through each of these for x and notice that at a certain point we see a diagonal that's that's when x and y are the same column here's the problem now the axes are getting messed up see the x-axis is just a bunch of copies now let's address that problem of multiple axes but at this point are there any questions about what i've done so far um i i'm [Music] i wanted to ask uh how this concept of item potent relates with the the update pattern in the tree i i i don't know if it's related but but it's like the same principle i guess because the points are dynamically changing but i haven't seen this concept before when we see the update pattern the enter update exit pattern so i'm not sure if how they are related right right and by the way people don't really talk about it they don't use this term very often and the term idempotent rendering i think was used back in the days of jquery when it became like a thing so maybe that's why like it's not a commonly used term it's like a sort of a nerdy computer science term but i think it's the right term to apply here but the relationship between the concept of idempotent rendering and d3's general update pattern is it's the same thing d3's general update pattern makes the rendering item potent and so let's let's take a look at our code for the circles what this does is it uses the new dot join api which does a bunch of stuff internally it creates an enter selection and an update selection and it merges them together and it returns the resulting selection and then we can set the circle the the attributes on the resulting thing so that is an idempotent rendering pattern let me show you a variant of this that is not idempotent if we call dot enter dot append circle and then we set the attributes on those circles what we're going to get is a bunch of overlapping circles oh no i'm that's not right that's not right because it is actually only so when it's when it says select all circle it's picking up the circles that it rendered last time but this is not idempotent because it's not updating it's not updating it's not working right so to make it item potent we use enter dot append circle um well we could use the dot merge thing check it out um circles dot merge sorry this should be circles which is the update selection and then we merge it with the enter selection like this this should be the idepotent version of it see now it's updating properly and this is like super confusing it's sort of a deprecated api to use merge like this it's very low level it helps you out but what this is doing is exactly the same thing as this simplified api where you just call dot join and it and join also handles the exit case so if there were fewer circles oops i must have deleted something data we need yeah so so this this does add important rendering does that answer your question yes yes definitely that now i i get how that concept like is part of the of the of all this structure yeah nice and so the problem is are our axes are not idempotent i mean the axis itself is coded in such a way that it does correctly do idempotent rendering however we are appending a group element on the outside which is the part of this stuff that's not item potent it just appends a new group element on each render which is not right this is the special case of the general update pattern that deals with just a single element and so it's sort of a non it's not the most common form that you see usually we call data and we pass an array of things that actually represents data however if you want to use d3 selections for just a single element we need to just make up some data that has a single element so let's do it selection dot select all um we could try dot g oh we could try g here just to select all group elements however i could tell you right now this is not going to fly because we have multiple group elements one for each axis and we need to disambiguate between them so at this point before i go any further i'm going to i'm going to use a class to disambiguate these select alt dot i'll call it y axis y dash axis because this it's the y axis dot data and this is where we need to pass in an array that has a single element so i could just put the number one or whatever i'm going to use null because i've seen this around in in the d3 source code actually for i think d3 axis uses this so it's it's an array of data that doesn't really have data it's just null it's but the thing is it's a single element so it doesn't really matter what that element is it just matters that it's an array that has a single element because when you use the dot join construct uh like i think i could say join g dot attr i have to give it the class of y axis so that it picks up the one that was added previously this i believe should work and i think we could say g dot y axis just to make it really clear i think that's a valid selector means it means that the the tag name is g and the class is y-axis so the y-axis actually looks good on the side see it's not getting that thick quality to it which indicates to me that it's just rendering once so what happens is every time the function gets executed it's taking a look at what was there from the last time and just whipping it into shape to match what it should be this time and because we're not changing y it doesn't change which is the right thing so now let's apply the same pattern to x this is where we'll actually see it we'll see it work more clearly because the axis won't be totally messed up so i'm just going to copy paste that whole pattern and then change y-axis to x-axis and let's see let's see if that does the trick yeah see that now it's correct yep the axis is not getting all messed up so just to run through what's happening again here we're using this dot join api of d3 which is the modern day you know general update pattern of d3 that handles enter update and exit and we're passing it a data set that just has a single thing a single element which is null but it could be anything it doesn't it really doesn't matter what it is but just the fact that there's one of them is what matters because the the internals of dot join will take that data and say okay i just need to make a single group element so if there's already a group element there that matches this selection then i'm going to use it i'm going to use the old one that was there before i'm not going to make a new one and so that's the beauty of this the first time it runs it will create an a brand new a mint brand new group element but the second time the third time the fourth time it's just going to use the group element that was there before and then it's going to call this axis function which itself implements idempotent rendering properly internally so that makes the whole thing item potent and and working correctly so this is it this is this is the the complete version of the reusable chart now you can use this you know swap out things in in real time and it'll work correctly any questions great i think it's it's great so so if i understand correctly the with the null like uh array that you create it's like a a a unique id for the for the group and that way it recognizes us the same thing when it updates well it's not really an id it's not actually used at all the thing that's serving as an id to differentiate between the different groups is actually the class so let me show you what happens if we get rid of this class if we just say select all g and then select all g for both of these axes let's see what happens see it's only getting one of them the y-axis is not showing up at all and so in terms of the concept of unique id to disambiguate between them the class is really what plays that role the data that array of null is just to signify that there's one thing that we want not two not zero but there's just one thing that's the significance of the of that right right right thank you and just to deconstruct why there's no um y-axis here when this code runs it builds up the y-axis first because select all g the first time there's no group element it'll create one and then after that runs the code that is for the x-axis says select all g boom it's going to select that group element that was used for the y-axis that's what happens that's what happens when you don't use classes it selects whatever group element is there on the page like as a child of this election and so that's why um it's clobbering the y-axis and it's it's using the same group element and it's building it into the x-axis so let's bring back our our selection that uses class yeah so now it says select all elements that have the class y axis if it's the first time there's going to be none so it's going to make a new one give it the class but if it's the second time around it's going to find the one that that this code put there before and use it all right excellent any other questions all right i think uh i think i'll take a five minute break now and um after the five minute break we can we can try adding some animated transitions to our dots let's see how that goes all right scene thank you great thanks you you you all right all right how's everybody doing ready for some uh crazy animated transitions oh yeah let's do this okay so i'm going to leave this reusable scatter plot here as it is because now i feel like it's finished i'm going to fork this because the animated transitions do add quite a bit of complexity so i'll call it animated reusable d3 scatter plot all right in our scatter plot i'm thinking we can focus on the circles as the thing to add animation to and maybe we could potentially animate the axes as well um d3 axes are implemented in such a way that you can animate them which is really cool but first let's get grounded in some documentation or examples so i'm going to search for d3 transitions yeah d3 transition here it is looking for some example code yeah so with transitions like there's a million and one ways you can invoke them um you could even just say dot transition actually let me do that first to see what happens because but i don't think it's the right behavior but just just as an experiment if we just put dot transition right here what happens okay it worked now they're animating see that so that's the simplest way of invoking d3 transitions we just add dot transition and it creates this thing called a d3 transition that kind of looks and feels like a d3 selection it so when you call dot transition it creates a new thing which is an instance of a transition and returns it and so these dot attrs they're being invoked on the transition not the original selection and you can also say dot duration like this and pass in a number of milliseconds like let's say two thousand now the transitions are slower two seconds and you can really see the stuff dance around pretty neat however this is sort of the the lazy approach that ends up in semi-buggy stuff because watch this when this program runs notice how the circles sort of fly in from the corner zoom this is on to me this i mean it's maybe acceptable but like to me this is not right like this is this is kind of buggy you know so i think we need to do the more proper implementation of of transitions which we can get into now from the docs so the thing to do really is to create a transition object that you can use in multiple places because what what this does if you call dot transition it creates a new transition and if we wanted to say transition the axes too it would it would like create a new different transition that has like slightly different timing or something like that and so like if you want everything to be synchronized the best way to do it is to create an instance of d3.transition so that's what i'm going to do now i'll just create it right above our circles i'll use const instead of var and instead of d3 dot transition i will import transition from d3 at the top and i'll get rid of this stuff down here ease linear oh yeah you can specify all sorts of easing functions which is like the the different way that they animate but i think we can just we can just use the defaults for now and i'll we'll keep the duration of two seconds so we can see it clearly now let me find some more examples so we can call dot transition on a selection and pass in the transition that we created so this is the same behavior as before but what i want to focus on now is when the the plot gets initialized for the first time i want to have like some nuanced animated transitions where here's what i want to have happen the circles should appear in their correct locations but they should grow from nothing to something so i would like to to animate the radius of it right i think that that feels to me like the proper way to start off a scatter plot that has animations is to you know do something sensible rather than like the default behavior which is they fly in from the upper left corner which is sort of nonsensical i mean it makes sense in a way technically but like i wouldn't want to wouldn't want to see that you know so for this we have to we have to use dot join in in a different way let me see if i can find i know there's a good example of this somewhere let me see if i can just find it i know it's probably here and observable here it is here it is yeah this is the canonical example by mike bostock author of d3 where it's the general update pattern with animated transitions and these animated transitions are totally um controlled like the code controls exactly all the cases of of these animations and so when when things enter they're green they fall from the sky when they're exit they transition down they become red and transition down and when they update they're black here and they just move from where they were to where they are or where they should be this is the example i would like to follow so here is here's here's what it looks like to do this properly we call dot join and instead of just passing a string the selector we pass in three functions the first function is what to do on enter the second function is what to do on update meaning when there's something there from the last time around and we just need to update it and what to do on exit when something disappears exit we don't need to handle in our case because we don't have a case where things exit i mean that would be the case where like uh when we invoke the scatter plot we change the data so that there's one fewer entry so this case we might not touch upon but we want to touch upon these two for sure so let me i'm just going to copy all this stuff because it serves as a decent template for reference uh kern just a question you went to uh you went to observable rather than the documentation right and uh is that is that the way to look stuff up now i mean go to observable first rather than the documentation well here's the thing like the documentation of d3 and observable are becoming quite um interconnected so for for example if we look at um d3 selection i often do start with the docs in the readme of the github stuff here but the thing is all of this stuff links into observable well notebooks okay yeah and so joining data for example see the selection.join notebook so this stuff this stuff in observable is essentially an extension of the d3 documentation okay i'm just planning on spending a lot of time in the documentation to learn how to do stuff in version six yep yeah the documentation is great but i don't think it contains a complete example i mean this is this is pretty close so this is a place we could start as well but i don't think it contains a full example with the animated transitions so for that we do need to go into observable but yeah my my general approach is to use the docs as much as i can and then if there's something that i can't find in the docs then i'll click through the observable links and import it essentially port the stuff to vanilla javascript by copy pasting little chunks yep thank you yeah my pleasure thanks for the question here it is so yeah i mean if i had my druthers this code snippet should be in the readme but it's not because this deals with how transitions and selections intermingle and work together so instead of using this approach i'm going to delete what we had earlier for the dot join and then i'll paste all this crazy looking stuff but you know really what i want to do is i want to use this as a reference and then code it ourselves so i'll leave that commented out current one practical question how what's the shortcut for multiple commenting out in bishop oh it's well if you're not in vim mode it's control slash control slash yeah and i've been meaning to add something like a list of keyboard shortcuts that would be useful wouldn't it right here in the editor yes it's it's on it's on the to-do list yeah but um yeah control slash comments the line multiple lines super useful thank you the general structure that we need here is a function that takes as input the enter selection and does something with it a function that takes as input the update selection and does something with it and lastly a function that takes as input the exit selection and does something with it enter what we're going to generally do is append something in this case we are appending a circle and on update is where we want to you know change the position and on exit is where we generally want to remove the stuff [Music] i'm just going to keep it there as a good practice but we don't have like a test harness that tests that case yet but anyway what i think we should do on enter is initialize the circles to where they should be so that's where i'm gonna take these lines move them up to here so we are appending a circle a brand new circle the first time it gets invoked and we're setting x and y and what we're left with is now circles that don't move around because we haven't addressed the update selection so let's do the same thing on update i'm just going to paste that logic there so we're setting cx and cy on update as well now we're seeing that the circles change like they should okay so now what i want to do is adopt the transitions the way that we can do this well let's think about first what we want to happen on the enter case when the circles get created what i want to do is transition them from being zero radius to having the radius that they're going to have for the rest of their lives and that is r set to radius here i'm going to move this logic over to here but what i really want to do is initialize r to be zero and that's what we can do here i'll just set r to be zero like this and i wanna i wanna have a transition to the next radius however there are some subtle things to consider here namely this function needs to return the enter selection that's just how this api works it expects the enter selection to be returned so that's why we can't just call dot transition like this because it's going to sort of mess things up internally but then the the last line should be the the radius right i cannot move the radios up and do the transition right or the transition also return returns the enter selection yeah it can be a little confusing so i'm just going to comment out that for now and this is where we need to look to this pattern from from the example from observable it uses dot call and the reason why it uses dot call is that what gets returned from dot call is the enter selection and the enter selection needs to be returned from this function not the transition but this is how we can create a transition derived from the enter selection and then use it to actually do our animations and it takes as input the enter selection and it derives a transition from that and the existence of this this transition is just strictly inside of this callback it doesn't get returned from this this outer function what gets returned is the enter selection because dot call returns the selection that it was called on so this is why you need to use dot call to get at what the transition should do and in here is where we can say attr radius r and radius and all this stuff and i might have like a mismatched paren or something i'm missing comma there it is yeah so check this out if i run this i'm going to make a code change to trigger a run make it full screen so you can see this is the transition that we want yeah i'll run that again so you can see it starts out as little dots that don't really exist to they grow into their final form and actually just to make it super visible let me set the duration to like five seconds check this out they're going to start at nothing get bigger and bigger and bigger and now they're big and if you want to make it smaller after shows then it you're going to exit right yes okay yep we could apply the same pattern on exit um yeah exit is um tricky i think i'll leave that to another day um sure because we don't really have a use case for it at the moment but yes you can control what happens on exit and in the example it's good like make it turn brown and then have it transition to go down and then remove it and the trick with exit and transitions is that you need to call dot remove on the transition because what we want to do is have it wait for the transition to finish and then remove the dom element but anyway let's finish the desired update transition again this this should return the update selection so we need to use the same pattern of using dot call update dot call update and then update dot transition passing in the same transition object that we created earlier and this is where we want to set the cx and cy attributes so now when this runs it should slowly move there we go yeah so now we're having with this kind of a funny a funny effect that's going on because the transition is longer than the time that we're giving it to finish the transition is five seconds but we're changing stuff every two seconds so let me just align on those values two seconds okay this is correct all right and so let's i just want to put a little icing on the cake here um there is a magical thing that you can do with transitions i'm going to make it one second and then do this magical trick which is we can set a dot delay on the transition to be a function that takes as input one row and also the index and we can return the index times let's say 10 or 100 so you can see what the effect of this is oh you can't really see it um i don't know if it actually worked what this should do is make it so that there's a sort of a staggering effect where not all the dots move at the same time um i thought this would work i was doing a few tests another day and i noticed that sometimes the index indexer and the d gets swapped depends of oh really yes i'm not sure why that happened i can check later and let you know but try try to change the ind okay somehow i i i have my doubts but i'll try it yeah no no yeah i don't know maybe maybe we need to call this not here but over here after we call dot transition what if we pass it in over there yeah now it's working this is very cool very cool although it's it's set way too high but if if i set it to like 100 we can see well let's see what happens yeah they all kind of move in a staggered manner let me tweak things a little bit more i'm going to have just a slightly larger delay so instead of two seconds let's make it three seconds and then in the delay let's set this delay to i don't know 50 50 milliseconds to get the desired effect that's still too high let's say like 10 10 milliseconds we can get a really beautiful sort of eye popping effect there it is that's the effect i was going for so the whole thing finishes between the intervals where it changes but it's this beautiful beautiful effect that i love with d3 transitions and setting the delay like this it's got this like voom like this this really almost like a a choreographed effect to it it's really cool very cool nice thank you yeah this is fun so yeah i try to you know spice it up with client work like if you add this to work that you do for other people they're gonna just freak out and love it so it's nice icing on the cake that you can add all right and so lastly i i think we can we can transition on the axes as well so let me get rid of this reference code and just to wrap this up i'm sort of winging it here but if i just say dot transition t before we call the axis all right let me do it on the x so we can see if it works let's see if that works yeah it works boom see that brilliant yeah the way the d3 access is implemented it does it handles transitions correctly which i just it's just fine so i find it so beautiful you can pass and transition and it updates the axis in the animated fashion brilliant alright i think our animation work is complete any uh any questions or thoughts about this um yeah it's actually it's another task um uh the last class uh exercise was to put the title in the axis i did it but i'm not sure i did the right way so i would like to see how you do it sure because in this case you have to update also the the title right exactly or is this the exercise for this class actually it is actually it is this is the perfect exercise okay and the timing is perfect we're getting to the end of the time here um add axis labels updates that are idempotent yes this is a very good exercise that i will leave you all to do and just as a as a little hint you're going to have to use that dot data and pass in an array of of not of one item null or whatever and um so instead of just selection dot append text it would be selection dot select all text dot y axis label you're going to have to use a class to disambiguate between the two things just like we did with the axes and yeah essentially apply the same pattern that we did for the group element that contains our our axes for the text labels and i think this is a great exercise uh because it it actually paves the way to a lot of other things like if you try to add labels to this you're going to quickly find that we don't have enough information we need to add another another accessor to this component which is going to be the x label we're going to have to pass that in too because this passes in a function and from this function we can't tell what the name is of the column so we're going to have to add one additional thing to get it to work so that's why i think it's a good exercise yeah i'm current yeah i was just curious about the arguments that are passed to join and i noticed that there are a couple of lines that are duplicated namely the cx and cy oh and i i recall that you you've brought up merge in the past to help uh remove that duplication and i was just wondering whether there was something equivalent uh for inside the uh yeah before for inside the arguments um yeah i'm i'm thrilled that you brought this up because duplicated logic is is one of those things that i just hate i just hate to see it and so i would like to address it i'd say it's a great call to address that we've got duplicated logic for cx and cy and it's not so bad i mean it's only two lines that are pretty simple but still it's duplicated logic and you may find yourself with complex logic you know 10 lines of complicated stuff that you copy paste between these two places which is not ideal and so [Music] the problem is this logic really needs to be executed at these two different places um in the past like if you if we don't have animated transitions you know like we did before i started adding this stuff it could just go once at the at the end because it when you put it here it acts on the merged enter and update selections but if you do it like that you lose the animated transitions like it's not it's not actually correct anymore so let me go back to the way it was is it possible to me like create a function an external function that updates cx and cy then i call this function in the enter and inside the update is that possible yes exactly you hit the nail on the head that's exactly what i was going to do cool and so let's do it i'll call it um what's it going to do it's going to be positioning the circles so i'm just going to call it position circles and this is going to take as input a selection that contains the circles so i'll just i'll call it circles we could call it anything but it'll take that as input and what it will do is just going to have this side effect of calling dot httr on that selection and it could actually return circles but doesn't really need to um and then to invoke this doesn't need because uh later you will change the r attribute right so it should return the circle or not well it all depends on how we invoke it um if we use dot call then it does not matter what this function returns because dot call is going to return the selection of the circles anyway so it really doesn't matter what this function returns and this is how we can do it we can just say dot call position circles and then in the other place where we have the same logic we can put the same thing dot call position circles like that and it seems to work and on the creation step yeah it seems to work okay yep so yes indeed this is how you can do it this is how you can reduce the duplicated logic across these two places and i think this is the best way to do it because this this logic needs to be executed in both of these places it needs to be the circles need to be positioned before the radius transition happens and they also need to be positioned every time things update and so yeah refactoring it into a common function that's invoked in these two places i think is a good move yeah that's perfect thank you very much uh for for walking through that nice yeah my pleasure and i'm glad you caught that because um when i was planning for this episode that i had this in mind like oh yeah at the end i'm going to refactor it but i totally forgot so thank you for reminding me also uh do you have a you i think you mentioned in a previous lesson request animation frame um yeah versus uh set into a interval do you have a preference for one over the other is is one more does one have benefits of the other well they have different properties request animation frame is actually used internally by d3 transitions and when you invoke a request animation frame and pass a callback that callback will be invoked as soon as the next render cycle happens and it depends on your system setup for example if your if your monitor is 60 frames a second which is most common or 30 frames a second which might happen if you have like a 4k monitor on a computer that's not very powerful and regardless of of your actual hardware request animation frame invokes that function on the next animation frame it's called an animate they call it an animation frame that means when the display updates so let's assume you have a like a standard setup where your refresh rate on your monitor is 60 frames a second that means and that's that's standard like when you move your mouse around on the screen if if you just sit there and up move your mouse around you're going to see that it's not a continuous thing like it actually just appears at different places really fast and every time it appears that's 1 60 of a second which is approximately 16 milliseconds and so the benefit of using request animation frame for things like animation is that it it synchronizes correctly with your display whereas if you use request if you use set interval for animation like if you if you were to use set interval and past 16 milliseconds it would give you it would call your function roughly every animation frame but it might call it twice between animation frames sometimes or it might call it once across two animation frames which would result in a slight almost undetectable visual glitch in the animation and so if you're if you're doing animations then request animation frame is the best thing to do hands down uh and and that's why d3 transitions use it internally for a time it wasn't supported by all browsers and so for a couple years there you had to you had to implement a fallback that used set timeout or set yeah set timeout is the is the like a variant of set interval that just calls the function once um so i remember back in the day i had to do like detection like if request animation frame is there then use it otherwise use set timeout and pass 16 milliseconds or whatever 1000 divided by 60 is and so that would be an approximation however if you have other use cases that are not animation like what we have here um like where you you want to wait for one second or three seconds then set interval is the thing to use actually um yeah set interval is more appropriate when you have longer time scales that you're considering but if you're doing animation then request animation frame is is the thing to use yeah that that makes total sense thank you nice one other question yeah um so go back to scatterplot.jf um and so uh where you are i guess the where did you have the abstraction out for position circles okay yeah there um the question i had in terms of and just sort of style and uh readability i really like the notion of putting position circles there um it just reflects what's actually happening from a uh sort of a coding style would someone do the same thing with the dot call enter that comes a little later right after it it just seems at least to me a little strange but that's probably how javascript works and so forth but could you actually name that as a function that is you know uh more reflective of what's happening just like you did position circles does that make sense yeah um i'm going to select some text do you mean this block here yeah yeah that dot call enter um or this inner one here yeah yeah that inner one yeah just could you name that create a function just like you did dot call position circles in a way that um it reflects what it's actually doing there in other words making the um uh radius bigger or transitioning yeah grow radius exactly for sure yeah i mean it's just a function okay so for sure i mean um you can you can do all sorts of refactorings like this and organize the code and i like doing things like this because in a way it's documentation like variable name is in a sense documentation for what it does exactly that's where i was going with this yeah okay for sure and so i love you know when i can i left this structure code like almost like a book exactly where you learn about okay this grow radius what is that it's some cryptic code that sets the radius but it says grow so that therefore it must be like growing from zero to something and so for sure i would encourage doing this sort of uh slight refactoring to make the code more readable and clear yeah right and it's thank you you're welcome welcome but yeah it's totally a stylistic thing like i personally probably wouldn't do that just because it's setting the radius on the line before uh but how you know it's totally personal preference like this may be more readable for somebody coming at this code for the first time for sure so yeah up to you okay thanks that helps yeah my pleasure but what's going on that the the radios oh the radius is right it's in in the intersects just begin and yeah okay i got it it's just in the enter it's not when it moves okay see if we if we wanted to take this to the extreme we could say initialize radius and pair these together and so instead of saying right here attr zero we could say dot call initialize radius and that's what this would do right here and this actually implicitly returns it which we don't need to do so that's why i'm going to add these curly braces and actually same same with this we don't need to return it so this is like the final form if you were if you were to want to you know factor out things and name them this works as well so it's super clear now that like okay first we're going to position the circles then we're going to initialize the radius and if you read it you realize it's initialized to zero and then we're going to grow the radius from zero to something else using a transition so yeah yeah this code it's definitely more organized more readable in in a certain sense but it's also more code so it's always a balance between how verbose and clear do you want to be versus how minimal and concise and arguably cryptic you want to be yeah there's trade-offs for sure all right maybe i'll take one more if if there is another and then we'll call it wrap it up for the day anyway well karen i have one that's uh on not on this chart but on a for uh a force layout so i don't know if anybody has any other questions on this you know i'll be i'll be working on forced layout in the future so like my inclination is to is to hold off on that until we're dealing with it uh but i'm happy happy to field a question about it actually it's it's mostly i see some places where um you the links are set up as an empty array and then how do you get that data to do that so that instead of me having to do source target for you know a thousand different data points i just do the auto magic javascript and it does it for me yeah there's some there's some magic stuff that happens with d3 force to be sure um i need the force to be with me on this yeah i get it yeah yeah well yeah when you and i think when you call for i don't remember exactly but when you pass data in there's some kind of magic initialization that happens um but i i will be doing forced layouts in in future episodes so maybe hold that idea how long i'm not sure i'm not sure i haven't planned out the full series but i think from here on out i'm debating to go deeper into this to add menus to this so you can interactively select so that's one direction to go another direction to go would be to do a bunch of things where we start from scratch and just implement different visualization techniques like a bar chart a line chart an area chart a pie chart from scratch with just enter and not this general update pattern stuff so those are some of the things that i'm going to be doing in the future any any preferences for for which direction to take it i would say i would say menus are a really good idea if you're uh designing this for a user having them have the power of choice over what what shows on the screen is powerful exactly that's what i was thinking too so like now that we've got it to this point it's a cool technical demonstration but you know we're going to have to add the axis labels and if you were to develop this for an actual use case of someone needing to visualize the data the logical next step would be to make it so that you can choose like you said which yes and i will also appreciate that functionality you know yeah right on right on i like it sweet yeah i agree with ken and everyone yeah sweet yeah yeah let's do it so probably next week i'll add menus to this one and uh yeah that'll uncover some complexities along the way all right i think i'll wrap it up for today so thanks everyone for joining me so much thanks karen thank you yeah this was a lot of fun this was a lot of fun with a lot of people here i love it so um yeah i hope you all can join next week as well and beyond so have a good weekend take care all right bye take care thanks current bye

Channel: Curran Kelleher

Views: 762

Keywords:

Id: U8x862hBk4s

Length: 105min 15sec (6315 seconds)

Published: Sat May 22 2021