D3 Live Coding Stream

testing testing testing 1-2 1-2 testing testing testing testing testing testing one two testing testing testing testing testing testing one two testing testing testing hello hello testing testing testing testing one two so if i change this no one can see my screen testing one two two testing testing testing one two testing testing testing my audio seems to sound like crap testing testing again and again and again testing the sound testing the sound that's way better okay so i'm going to start in about 15 minutes testing testing bye so i am spreading the word about the stream which is going to start in about 13 minutes so okay testing testing testing one two okay wow there's some people in the youtube chat hello so um this is my first experimental live stream and i'd welcome everyone to actually like talk with me and be a part of the stream if you'd like you can ask me questions or whatever so i'm posting this link so you'll end up if you join end up on youtube if you join so just trying to get the hang of this thing you found my visit videos helpful that's great great all right all right you start in about seven minutes all right all right so just sit tight for five minutes testing okay huh huh hi everyone this is an exciting uh experimental live stream hopefully the first of many uh oh there goes my sound on the other machine hang on yeah a lot of this is new to me so forgive me if the setup is not ideal so um what i'm going to do here is hopefully visualize this data uh i'm going to go for about 90 minutes see how far we get this is totally unrehearsed so it may be rough i'm not sure but this started when ian johnson posted this thing only two days ago where it's a set of scripts that pulls the git history for all of the d3 git repositories so just for some context in github d3 is organization and there's the main d3 package which just pulls in all these other packages as dependencies see in the package.json for d3 the dependencies just include all of the sort of sub packages or you know d3 packages so in a sense this d3 package is just an aggregation of these other smaller packages and so in here you can actually see github has this feature of a summary past year of activity so this is the kind of data that i want to visualize number of commits per repository or maybe per contributor to visualize the the contributions over time that's what i'm thinking so this script that ian johnson made two days ago it just reads the package.json of the d3 repository and clones all of those repositories and then this script here runs git log and then formats the output to be a machine readable you know parsable file that has essentially one row per commit and that commit each one has a hash date an author a subject and a body and i looked through the data and the file was quite large so this is my fork of that gist which is just the data pre-processing part and what i did was just included the date and the author because that's really all i think we're interested in the date and the author and then when they get combined the data for all the repositories ends up getting combined and when it does you have a repo field as well so here's what the the data looks like i i put it in this github just so that we can pull from it and use it so we've got a date an author and a repo and we've got a bunch of these so here's what i was thinking of doing with this let me just find the thing here it is the stream graph so this is this is a stream graph visualization that i made a couple years ago about uh trump topics and news from google news data set and this goes back to 2017 and you can see interest in various topics over time so this is aggregated i think by day it looks like so i was thinking of visualizing this d3 git repository history data using this technique so instead of different news stories it would be maybe different d3 repositories that get contributed to over time or different contributors to the github repositories so the question is how do we get from here how do we get from here this data to their this visualization so that's what i'm going to be trying to do today so let's get started creating something i'm going to use vishub for this i'm going to say create a viz i would like to try the new package json template by the way this is a new feature of viz hub where you can specify dependencies in package.json and it hides bundle.js and you can have a simpler index.html it actually pulls it in from a cdn like all the other old visas do but the whole format is much cleaner and you can use it to pull in any libraries that you want to and use es6 import syntax with it so i'm going to fork this one i'm going to rename it to wip work in progress d3 get history viz fork okay so the first thing i'll do is just clear out the readme i was thinking of doing that automatically in visa but because a lot of people never change it but anyway i'll clear out that and i'll say this is an experiment in visualizing and then i can link to this data this is an experiment in visualizing d3 git history so i opened up this google meet and shared the link out so if you want to join this meeting and talk to me and be a part of this stream you're welcome to i'll share it out right here otherwise it'll just be me all alone okay so to get started here the first order of business is to load in the data or actually probably i should just clear out this whole boilerplate as well we don't really need anything and then what i'd like to do next is load in the data i'm going to give myself some space to just focus on the code here and let's um load the data so to do that we we need to import the thing from d3 that we're going to use to load the data which is either d3.csv or d3.json since it's a json file we can we can import json from d3 and then i like to make a variable for the url of the data so that it's not one huge line so here's the gist here's the data file so if i click on raw i'll get this url that is the raw url and then over in our code we can say data url our data url equals i'll just paste that there it's a pretty long line which i don't really like but you know it's whatever and then we can load it in we can invoke the json function passing in that data url and that returns a promise so we can say then some callback that gets invoked with the data and then in here i'll just say console.log data just to see what we've got and then in the console we could see that it worked wow almost 7 000 commits on d3 that's pretty cool okay we've got date we've got author and we've got repo alright step one complete we've loaded the data in order to get from this data into a format that we can use with a stream graph we need to aggregate to get for example the width of these streams for different dates and so the data we have to begin with is individual commit events that happen in a as at a point in time you know this is the day this is the time of day this looks like the time zone interesting yeah mike bostock is on the west coast and uh felipe riviera i hope we're pronouncing that correctly is i believe in france so i think that is the time zone but at any rate what i think we should do is aggregate it by some time interval like week or month perhaps it goes over quite a time range let's see what's the oldest commit 2016 yeah so i think months might be a good aggregation i mean we can always change it we can always decide later to change it but the first step for aggregating would be to parse the dates yeah and then we can use d3 time to figure out you know which month it's in for example if we're going to aggregate by day or week or month we don't really need the time of day and the time zone doesn't really matter i mean it might be like rounded to the the day before or the day after but that's okay we're doing a sort of a coarse grained analysis um and you know now that i think of it we could just chop this string to aggregate by month that might be the dead simple easiest thing to do you know so i think i'll try that so we can break this down into a very small subproblem which is given this string how do we how do we just get the year in the month well we can use dot sub str or sub string you know i can never remember which is like the canonical one i think they both work let me just research that real quick for mdn sub string this one seems legit what about sub str sub str is the substring of the sub string so that's sort of funny okay this one has a thumbs down next to it it's deprecated okay that's really good to know so we should use the long form sub string and the way it works is you give it the start index and the end index so the index start would be 0 and the index end would be i don't know some character so sub string from index 0 to index let's say 5 gives us the first 5 characters but we want to get the month so i think that would be seven yeah see that's what we need so we can just take the substring of zero seven of the date and then we'll get this simplified date string that just gives us the day or no the the month rather the month so let's do that um we can i'm thinking we just iterate over the data and add another field to it perhaps so we can say data.for each d that's just a convention to refer to a single row d dot something what was it again what was that field name date so d dot date we can actually just overwrite that one d dot date dot sub string like that and then we get just the month okay sweet so this code is a little cryptic to come upon so i'll just say um aggregate by month and you know what we ideally i think should be working with javascript dates so while we're at this point let's just parse the date i'm just imagining there's a function called parse date and we can pass in that string and to get that function we can use d3 time format i believe which has the time parsing as well there it is time parse so what we can do is import that um this comment is misplaced i'll put it over here import time parse from d3 and then we can say we can define this variable parse date equals time parse of the specifier that we need and so zan armstrong has this amazing time format example and blocks.org which i find more useful than the documentation just because it has a bunch of concrete examples and this is exactly the format we want month and year so this is the one that we can use so we pass that specifier right here and now if we console.log this data after we do all that stuff we should be able to see those dates oh parse date is not defined oh it's a it's a typo parse data first date okay now it should work okay it worked we get this javascript date object okay sweet sweet so the next step of this would be to aggregate by month so this is actually not aggregating but my month this is more like parsing date by month and then to actually aggregate by month we can use um some new machinery in d3 array it used to be d3 nest but nest is deprecated and replaced by some cool stuff in d3 array i believe it's called group yeah see that d3.group d3.group is a transformation it's one of these methods for transforming arrays and generating new arrays so what d3 group does let's see it groups the specified iterable in an array by the way is an iterable so we can pass in our array into an intern map and this is a very new thing it's essentially a map but it allows states and other non-primitive keys by passing by bypassing the same value zero algorithm okay we're getting into the weeds here but it essentially returns a map here's an example if we have this data we can group them by name by saying d3.group passing in data and passing in a function that we want to group by and this returns a map this actually should be internmap but whatever and the thing is you can pass multiple keys so if you want to group by name and by date this is how you could do it and it gives you a map of maps you know where the keys the names resolve to maps of dates and those dates resolve to arrays that are sort of isolated from the original data so if our aim is to create something like this we do actually need to group by two things one of the things we'll group by is by the month and then the other thing will group by um i think i want to try two different things one of them will be by the repository and then the other one will be by the author and those will give us totally different visualizations that will say different things but by the end of this that's what i want to see is two visualizations with those groupings now we need to take this idea and apply it in our code we want to aggregate by month we can take that code there instead of using d3 dot group i'm going to import group from d3 along with this other stuff and then we could i guess suppose we could reassign data although that would be kind of confusing so i'll make a new variable called grouped data equals group data and then these are just copied from the example so i should actually figure out what the fields that we have are and use those one of those is definitely going to be date because we're assigning it right there so i think this one is okay and then the other one would be let's start by grouping by the repository if we take a look at our data we can see the repository field is called repo so that's what we need to group by over here and um the question arises which which should go first i mean should we group by oops should we group by repo first or by date first and i think that depends on some downstream code namely the the d3 stack api but anyway let's see if this grouping worked so far okay it's a map of 30 things alright see that we've got these various d3 repositories and for each one we've got another map inside of it that's just for each month where there is data notice that this is a map where the key is the date and the value is the array of original commits that happened inside of that date that month and inside of that particular repository okay now we can come at this from the other angle of how to actually code the visualization as a stacked area chart uh that we can then add labels to to make a stream graph for that let me take a look at d3 shape because i know this is where the stacked area stuff is d3 shape lets you create symbols arcs lines and areas yeah this is the kind of thing that i want my question is now what does the input format need to be for area so here's the constructor it takes x y 0 and y 1. and then if you say area of data it generates an area for the given array of data i don't know it's kind of hard to go from this direction and figure out how to use it maybe looking at an example would be better okay here's a stacked area chart example in observable and it's isc licensed let's take a look at how this works we've got the data which is just a flat array where each object has a date and then a bunch of different keys for the various things that get stacked so in our case these keys i guess would be d3 repositories okay this answer is a good question which is what format does the data need to be in to begin with to work with this area generator api just taking a look at the stream here i see um robert is here hello there so good to see you um you're welcome to join the meeting if you'd like to collaborate and actually let me drop the link to the viz as well if anybody wants to follow along and or fork or whatever anyway i'll just keep going so to make the stream graph we need to get from this a map of maps into this a flat array of objects where each object corresponds to a combination of month and d3 repository or no actually sorry that's not what it is we need to convert it to a flat array of objects where each object corresponds to a month and that object has keys for each of the various d3 repositories okay so this means to me that we should aggregate at the top level by month and then we can iterate over those to come up with these objects so back in our code here uh see we're first aggregating by the repository and then by month but i think what we want to do is reverse these so if i just reverse the order to aggregate by date and secondarily by repo and i'll update this comment aggregate by month and repository then we get a map where the keys are dates okay okay this looks good and then the inner maps have keys for each repository and then an array and so what we're going to want to do is for each one of these outer entries create an object that has a repo field and then we use the length of this array we're going to want to count how many commits for this particular combination okay so i think we're well positioned to to take a crack at this i'm going to make a new variable for this data i don't know what to call it exactly area data perhaps or transformed data i don't know transform data and i think the easiest thing to do would be to create an empty array i see somebody join the meeting hello hello there welcome this is my wife everybody cool so feel free to chime in as i go there is a lag in the video it looks like how is it um yeah if you are looking at the stream there will be a lag so just look at maybe close the stream and just look at the meeting because i'm sharing my screen in the meeting too this google me thing this hub is super slick thank you thank you all right let's try this um would you mind just muting yourself because there's some audio bleeding okay great great thank you so here's what i'm thinking we just create this array and then we iterate through our grouped data and as we go we push things onto this array we could do it another way with like reduce or something but i think this one would be the most straightforward to code so what we need to do is iterate over the keys of this grouped data which is a map so i have to just figure out how to do that es6 map iterate the keys if i remember correctly there was just a keys function oh there's a for each function look at that executes the provided function once per each key value pair i don't know if that's what we want because we really just want to iterate over the keys looks like we can just call keys so let me try that console.log groupeddata.keys what do we get it's a map iterator where we have dates okay cool map iterator that's interesting so it's not an array it's a map iterator um to iterate over that i don't know can we use the four of syntax maybe let me try it will this work in i don't know this iterator stuff is kind of new to me iterators and generators for of loop yeah it should be four of right yeah the mighty four of four oh i forgot the var or the let rather let me try let key of group data.keys alright see there we go boom okay this is a good step to get to now what we want to do is build up a row object and then push that onto transform data and what we want to do for each of these rows is make it look like one of these rows from this stacked area chart example because we're eventually going to want to create something just like this it has a date property and then it has keys for each of the things that we want to stack by let's start by adding the date property so instead of key let me just call this date because that's really what it is and then i'll initialize this row with the date and now we can just say console.log transform data to take a look at what we've got here not seem to be running something seems to be broken okay there we go test is back i don't know that was a twilight zone moment so let me try console.log transformed data down here okay all right see we've got a date field and the values are dates okay so far so good so we've got the date field now what we would need to do next is add keys for each of the d3 repositories and we can do that in here by saying you know for each of these things that are inside actually let me get that i'm going to say value which is the value for this key it's going to be grouped data dot get i believe it is date so this will give us the inner map and let's say console.log value yeah see it's maps all of them seem to just have one key that's surprising i thought they would have multiple keys maybe i'm just looking at some of them because i mean for a given month surely there would be commits in multiple repositories this is kind of confusing i'm not sure if i did this right let me take a step back and look at grouped data again before we get into all of this so it's a map where the keys are dates although we've got duplicates in here okay that's what's problematic i think this is the thing that would be solved by internmap as a matter of fact so what's happening is that the dates are not really working as keys in the map so maybe what we should do in fact is not parse them to dates quite yet we can parse the strings to dates later um this is so that the strings could act as keys and then if the same string is encountered the map internal algorithm will know that they are the same it's somehow not picking up on the fact that dates the same date is the same date maybe because they're different objects so you know what i'll just not parse the date quite yet i'll just copy paste that line keep this around for later and then i'll just use this substring now let's see what happens okay look at that there's only 75 entries that looks better that looks more correct okay this is great so for each month we've now actually aggregated by month so there's not duplicates anymore okay fantastic so now in this logic here um yeah dates won't equal says robert um and by the way there was a very very recent change to d3 adopt intern map that addresses this very exact problem that we're facing that was only 20 days ago yeah it groups this specified iterative iterable of values into an intern map and an internmap this is probably actually worth discussing just a little bit they extend the native javascript map and set allowing dates and other non-primitive keys by bypassing the same value zero algorithm which is the same as the oh i thought it was actually equals equals equals but it's not it's something a little different anyway there's a lot of details there but the upshot is it doesn't work with dates unless we upgrade this package yeah this was published as d3 array version 2.11.0 so that actually should be available through unpackage so let's i'm not no i'm torn what to do here deal with the old way or try the new way i wonder if d3 itself the top level package was published recently yeah 6.5.0 17 days ago but i wonder did it upgrade d3 array i think so so let me try that actually six it's a 6.5 6.5.0 17 days ago let me see if this is going to work yeah see the dependencies is set to an older version but if i just upgrade that whoops upgrade that to 6.5.0 and then if i bring back the date parsing let's see what happens now oh my gosh it worked look at that i'm glad i went down that little rabbit hole thanks for bearing with me totally worth it so now you know about internmap and why it's useful because you otherwise you couldn't use dates as keys but now you can look at that it's working with actual dates as keys because i brought back the parse date okay this is actually what i want to be doing it just feels better to be dealing with dates from the get-go now what we can do is get back to our original task of building up this row object so let's again take a look at what we need to get to we need to get to a row object that has a date property which we have now and then it has keys for in our case the various d3 repositories and the values for those keys are going to be in our case the count of commits or the length of the commit array the way that we can do this is that we can use this value which is again a map let's just inspect for a moment this value this is what i would have expected to see yeah multiple keys so what we want is a key on the row object for each of these keys on the map so we can again use the for of construct to iterate through and this is going to be let repo of value.keys this will iterate over each of the repositories and then in here we can take a look at which repo it is and then value dot get repo should give us the array of commits for each of these repos and it does and what we're most interested in is the length of this array so we can say dot length to essentially use the count aggregation operator if you were to think of it in terms of sql or whatever so this gives us the number of commits and now we're in position to add these keys to the row so row at the key repo can be value.getrepo.length and i think this should do the trick let's take a look at our transformed data after doing all this we got 75 entries one for each month okay looking good and then we've got keys for each of the repos but it's not actually for each of the repos i mean some of them are missing i'm not sure if that's okay or not in terms of how d3 area works and we may not have data for each and every month as well and that's another thing where i'm not sure if that's okay or not see here's here's one for august 2015 and then the next one is for october 2015 so we don't have any entry here for september 2015. so we may need to go through and backfill zeros but i'm not sure maybe it'll just work with d3 area but so i think i'll try it with d3 area and if it breaks i think the solution would be to backfill each and every month and the combinations of each and every repo with zeros even though that's not directly in the data we may need to do that but i'll try without it as a first pass so let's again study this isc licensed stacked area chart example in observable i want to know how it works so it uses d3.stack dot keys data.columns.slice1 which is all of these columns without the date so we need to come up with somehow the list of all repos and then use that to pass it into stack.keys yeah the stack transformation is the first step that we need before we start getting into the rendering logic so let's try to make that happen and we could even do it right here i'll call it stacked data stack and i just pasted that from the example we need to import stack from d3 and then for the keys we need to give the repos repos so i'll imagine that there's a variable called a repos and then i'll fill it in and the example code uses data but what we need to use is actually transformed data because that's the data that has the row structure that we need that matches the example okay so let's figure out what repos should be repos um we are grouping by date and repo but what we really want to do is just get the listing of all the unique repositories we could do that a number of ways i mean we could use an ordinal scale we could use a javascript set javascript map or we could do something very similar to what we're doing here just do another grouping that just groups by repo and not date and i'll call this repo grouped data it may not be the most efficient way of doing it but it's the least cognitive overhead because we already have code like that so the surface area of the apis we're using here is minimized which makes the code you know easier to to read later so let me just console that log repo grouped data and see what we see it's a map it's a map where the keys are the repos and the values are the array of commits for each of those repos and now that i think of it this would be kind of cool to do as a bar chart you know what i mean which d3 repositories have the most commits that would be kind of interesting but i digress so we can get the list of repos by just calling dot keys on this repo grouped data got dot keys and then we get a map iterator but um i i think that will work uh it's not an array but let's just see if that works i think it will work because d3 has recently been upgraded to work with iterators in most of the parts of the api that used to just accept arrays so now i can just assign repos to be repo groupeddata.keys and now with this stack invocation should work and let's take a look just make some space let's take a look at stacked data by console.loggingit console.log stacked data gives us an array of arrays okay that's cool it didn't break completely i do not fully understand this data structure oh but we do have some not a numbers yeah that's a that's going to be a problem for sure for sure so i think what we really need to do here is make sure that we have all of the repositories represented on all of the rows oh yeah somebody said keys.2 array that could work but i think the the d3 api does accept iterable so it's not not required so the main problem is that's not a number and i think it's coming from the fact that not all rows have all repositories represented what we can do is to address that instead of iterating over the keys that are present for just that particular month meaning the repositories that had commit commits for that particular month we can iterate here over all repositories because we have that now repos so let repo of repos and then now sometimes this will come up empty so what we can do is um we can say commits equals value.getrepo and commits may very well be null or undefined because not all months have commits for all repos so what we can do here is say uh set the value to be commits like are there commits if there are then we can just say commits.length otherwise if there are no commits for this repository in that one particular month we should just use the value 0 to represent the fact that there were zero commits that month so now we have filled in all the values for all repositories for all months for which we have data but i think something has gone terribly wrong because we're getting empty array for stacked data let me check out what these row objects look like console.log row it's not working like it's not it's just not working uh so maybe this let of doesn't work for well i don't know maybe that's that not working in this inner loop is it even working like going over these repos let's check it does seem to be did i get the logic wrong for commits let's see what are these commits okay a lot of it's undefined but that's okay hmm so i want to see the combination of repo and commits so let me just console.log repo and commits should be getting a lot of these uh but what's happening here is take a look at that it's just working for the first one and then the subsequent ones it's not working and it's it's strange because it seems to just be going over the repositories once but it should be going over all of the repositories for each month so i don't know it's twilight zone moment here let me just comment out this code and make sure that we are going over all of the dates that we should be going over here we are we are okay we got a date for each month and then the value is for each of those months different yeah okay it's still coming up empty let me console.log transformed data see it's only working for the first one i have no idea why why is it just working for the first one and not the other ones this is so strange uh to be honest i'm kind of new to this four of construct so maybe there's something weird going on with that with the control flow no let me try repose.for each my beloved functional programming way of doing it let's see if that works oh repos.4 h is not a function because it's not an array it's an iterable oh it's an iterable not an array hey so whoever that was that suggested earlier oh now you're saying uh four x in array metal guitar covers thank you um sure i'll try that yeah because the iterable i think you can iterate over it once and then it's like spent it's done so let's try for in that also didn't work all right so i think what i would like to do is try array.from so we're not dealing with any of this iterable stuff that's confusing what now it looks totally different um we've got indices yeah so that's what the let in the for in actually iterates over the indices of the array which is not what we want so but now let's try for of over an array and so we've got it there for the first one my question is for the other ones is it there yes it is yes it is okay done we did that step got it okay so we learned something about iterables you can only iterate over them once but if you use array.from uh or that to array you have an array that you can iterate over multiple times not just once using for of okay cool good stuff good stuff okay so now we pretty much have the data structure that we need to pass into d3.stack the one thing that kind of bugs me is that we don't necessarily have rows for each and every month but let's just make the visualization and see how that plays out it might be just fine so now we can invoke this stack and then console.log stacked data and this looks better we don't have any naughty numbers anymore cool all right i think we're done with the data transformation stuff now we can get on with the visualization of it again i'm going to consult this observable thing which is straightforwar you know relatively straightforward d3 code where it's just depending a path for each element of series and how is series defined series is defined as uh well this is data what about series okay it's stacked data so what is series here i'm referring to as stacked data and we're going to need this area generator and we're going to need these scales pretty much the same thing uh it's a max of maxes i'm just sort of surveying what's here but yeah let's let's build this up sort of from first principles there's a bunch of logic here that is sort of cluttering of this file so i think what i'll do is just before we move on i'll make a new file called transform data dot js and then export a constant transform data equals data in and then i'm just going to drop all that transformation logic into here to split it out from our main module so i'll cut that code and then paste it over here into transform data and then return stacked data and then to use this in index.js we can import it import transform data from dot slash transform data and then here we can say const stacked data equals transform data data and it should be doing all the same stuff that it was doing before oh parse date is not defined sure yeah we got to import that stuff in that other module now we don't need json but we do need all this other stuff and then we also need parse date i'm going to move this to that other module parse date okay now it's working great so now we've got a simple simple index.js and now we can you know i think i'll make another function called render stacked data and now we can say render is a function of stacked data and this is where we can do all of our fancy d3 stuff so we're going to need an x scale and a y scale i notice there's some stuff in the chat where is this recent viz let me just show you if you go to my profile page and then sort by most recent should be here and i'll drop it in the chat as well all right so now that we've got this stacked data we are in position to render it with d3 and we're going to need an x scale and a y scale i'm going to make this a i think i'll start by making it like a pretty traditional stacked area chart and so we're going to need an x scale and a y scale the x scale is going to be time so let's start with x scale it's going to be a um i'm just realizing we don't need these imports the x scale is going to be a time scale i believe we can use scale time for this which we can import from d3 and then we got to set the domain and the range the range is going to go from zero to inner width and inner width i'm going to use the margin convention here so this is sort of a bunch of code that i sort of just know how it's going to all work out so width minus margin dot left minus margin dot right and we can define our margin to be an object that has top i'll start it all off at 20 right 20 bottom 20 20. and we're gonna need inner height as well so let me just say inner height it's going to be height minus margin dot top minus margin dot bottom and width and height i like to just derive that from the running program so we can use window dot inner height or sorry this is with inner width and then we can say height is window.inner height so now let's think about the domain of the x scale it's going to take as input well you know it would be useful to have the data before we stack it actually so why don't i just move that or you know we can do is just return stacked data as well as transformed data or just data because what i'm thinking is we just really want to iterate all over all the data and and figure out what the min and max date are so i think i'll return data and stacked data so over in here we can just pass the output of transformed data into render and then in here we can destructure that to data and stacked data and so for the domain we can say it's the extent which gives us the min and the max over the data along the x value and this is just something that i like to use so that the code is general const x value is this is going to return the date so what this should do is figure out the min and max date so let's figure out let's just see if it worked we can say xscale.domain and if you invoke it like that without any arguments it should return the value that was set okay it's saying extent is not defined that's okay extent we have to import from d3 okay that seems to have worked see it goes from 2013 to 21 2021 okay great now let's just take a look at the range make sure that's okay zero to 920 that looks that looks good that looks good because i know the width is 960 and then the margin left is 20 so 960 minus 40 is 920 okay great great great and now let's take a look at the y scale this is going to be a linear scale and this is where i'm going to consult with the example because this is where some funky stuff goes on regarding figuring out the max see it's a max of max's and i'm pretty sure this is the logic that we need to use as well so i'm just going to copy paste that logic and that's going to be used to set the domain so we could say the domain is going to go from 0 to this max of max's and instead of series we're calling the same thing stacked data and instead of using d3.max i'm just going to use max which we can import from d3 with this es6 import syntax yeah and this this just works with the structure that's output from d3 stack i believe the value at index 1 is the max of that particular slice so we're taking the maximum over all the various slices so let's take a look and see if this is working okay we need to import scale linear from d3 and now we can take a look at the domain to see if it worked console.log yskill.domain okay that looks probably like it's correct 778 would i guess be the max number of commits that we would see for all the repos combined which is what the stacking does for a particular month so that's kind of interesting information right there for a particular month meaning the the busiest month of d3 saw 778 commits in that one month that's pretty impressive okay now let's consider the range the range of the y scale is going to go from zero to inner height but actually since y is flipped the y coordinate is flipped in in computer graphics it's going to go from inner height to zero and i'm just using prettier here to autoformat the code so let's just take a look at the the range see if it worked out right okay that looks decent okay now we can construct a d3 area generator the area generator is d3.area but i'm just going to use area and import that from d3 and again we can refer to the example the area just looks like this it defines x y zero and y one we can just use all these wholesale because our data adheres to the same structure meaning there's a field called date and d dot data is created by the d3 stack function and so is this array of two values so we can just use this i'll just copy paste that wholesale area.x and y y and then we can get into creating the svg stuff which again can look quite similar to this example except this uh this seems to be some d3 you know some observable specific stuff because it's return yeah it's returning the svg node but uh in vanilla javascript we can just you know append an svg element so i'll say const svg equals i'm going to just append it to the body so we can say d3.select body dot append svg and select we're going to have to import from d3 so now we we've got this svg the first thing you always need to do with svg is set the width and the height because the default is some weird set of numbers so they set the width oops w id set the width attribute to width the variable that we have from earlier which is window dot inner width and we can do the same for height height is height so now if we run this program and inspect it we can see that indeed uh oh we should have an svg element there but we don't don't know why oh something is wrong d3.area.x is not a function oh oh you know um pretty sure we need to invoke that as a function because this is the area constructor that we imported from d3 so we have to invoke it as a function to create a new instance of this area generator and then use the method chaining yeah that's what it was so now if we inspect the element we have an svg there but we get these unfortunate scroll bars which is something that is a recurring problem we can address it with some css on the body we can say margin is zero but we still get a vertical scroll bar so we can say i researched this one time by the way it's something something very weird about cascading margins or something so i think this is the best way to deal with it overflow to be hidden that will just get rid of our scroll bars okay that looks good no more scroll bars all right now that we've got this svg we can implement the margin convention by appending a group element and then trans translating it by the left and the top margin so we append a group element we set the transform attribute to be a string template literal that says translate and i'm going to put these placeholders here and we want to translate in the x direction by margin dot left and in the y direction by margin dot top i'll use prettier okay now this should give us a group element which i'll refer to as g this is a d3 selection of this newly created group element and now we can say g dot and this is where we go in and create a bunch of path elements so we can use the d3 data join pattern and say g.select all path dot data and we can pass in stacked data here dot enter dot append path and i'm coding this in such a way that it's only going to run once if we wanted to run this multiple times we'd need to use the general update pattern but that could be a later refactoring as needed so once we append these path elements we can set the d attribute of each path which is an svg thing it's a domain specific language that can be used to define the coordinates of the path for that we can just invoke this area generator with the data or the data at a certain index i don't quite remember i'm going to consult the example so the attr d is just sent to area which is kind of interesting uh what this ends up being is it's going to be invoked for each element of the series which is what we want okay so this should actually work does it run do we get some paths we do but they're all empty i'm not sure what's going on oh that that's probably what's going on x is not defined where is x being used oh oh yeah well okay fine in the observable example x was the x scale but in our code it's specifically x scale so that's what the problem is x scale and y scale just a naming convention all right look at that got something to show up as night falls i have to adjust my green screen settings here oops apologies oh no hmm oh forget it all right we've got something to show up this is pretty cool i'm i kind of like these broken states so i'm going to actually fork this i'm just going to leave this the way it is because it's kind of cool so what is going on here no idea honestly yeah i don't know one thing we could do is try making the different paths different colors so let me try that set the fill to be just just random colors i'm just going to create some random colors here math.random times 255. this is an old trick i like to use just to make random colors okay that's kind of cool too isn't it it looks like there's some some problem with like the way that it wraps around oh hey quincy larson nice looks like a makeshift skateboard ramp right on right on yeah this is kind of cool too i'm going to fork from here to to keep this broken state around that's a artifact of history here and every time you've refreshed by the way it's different colors it's kind of cool see if i run it again boom different colors the total d3 broken made art situation but why would this be i mean honestly i don't know someone's calling me oh no oh i have to i have to go i have to take that well this is this is fun made it this far so i will stream another time thanks you thanks everyone for tuning in hopefully i'll stream again uh same time next week and we can wrap this up and make a really cool stream graph um but this has been a fun experience a fun experiment in knowing um yeah one thing it's like i think i can get more done than i actually can in 90 minutes so it's a good learning experience but anyway thanks all for joining i will do this again i'll let you know take care bye

Channel: Curran Kelleher

Views: 823

Keywords:

Id: v76alPtul2c

Length: 109min 46sec (6586 seconds)

Published: Thu Feb 11 2021