Datavis 2020 Episode 56 - Organization Graph Diagram

hello curran here let's dig into a fun project about graph drawing with nodes and edges and d3 force someone came to me and said i want something like this and this is the data that it should show this work is for a foundation and their goal is to show their vast network so the foundation is connected to the arts web and within the arts web we've got community vision silicon valley creates and within each of these they just want to have you know 20 or so smaller unidentified nodes just to demonstrate the volume of unnamed arts entities associated with this node so for example arts web maybe would end up as one of these big hubs and then community vision would end up as one of these smaller nodes connected to the hub and then their vision is that these smaller hubs would have a bunch of tiny tiny nodes that don't even have names so this is the task before us i'm thinking you know i'll do it in vishub and just use probably d3 oh and by the way the deliverable what they're looking for is an image at the end so no need for advanced interaction or anything like that so i'm going to go to vishub data visualization platform and then i'm going to sign in with github entered my username and password and i'm in so i think what i'll do is go back to the home page and look at the most forked and start from this hello html example i'll go in here click fork this viz and i'll start by opening the editor and setting the title to be organization network graph i know for sure we're going to need d3 so let me just pull in d3 from unpackage which is my go to source all right we're at version 6.2 nice so i'll make a script tag and set the source equal to that url that pulls in the d3 library from unpackage i noticed the description is out of date so i'll just uh clear out the readme okay i know it's going to be a d3 force layout so let's check out d3 force the package that provides this functionality and let's see if we can get something like this up and running maybe with like two nodes to start this looks like the opening move right here so i'll copy that go back to the code and then i'll start building our javascript i'll create a file called index dot js and in this file i'll paste that but in vishub we can import for simulation from d3 so we don't need d3 dot and let's say import force simulation and by the way control space will auto complete from d3 and this is a variable the documentation used var but let's upgrade it to the more modern const and we pass nodes into force simulation so let's see what should nodes be the api reference says alright d3.4 simulation creates a new simulation with the specified array of nodes so if i click nodes okay each node must be an object what i'm really looking for is an example of the data structure here we go here's a good example nodes and links let me just copy this so it creates three nodes alice bob and carol and then the links have source and target so this represents the connections between alice and bob and the reason why zero maps to alice is because alice is the first one in this array and the index of this entry is zero so zero for alice one for bob two for carol all right let's use this as our starting point i'm just going to drop that in here and change var to const for just style all right so now we've got nodes defined passing that into force simulation but what about links we're not even using links how do we get the links in there and keep in mind our goal for right now is just to get something to show up with the force simulation and then after that we can add our particular data but let's get links associated with this simulation how do we do that it looks like the constructor just takes nodes as input but if i just search for links ah look at this oh yes what we need to do actually is add a force to the simulation like this and you know what all of these simulations look good okay so this is i remembering now that force simulation is it doesn't really have any forces in it and you have to add these forces yourself this was an api change sometime in d3 so we need to add the charge force which i'm pretty sure spreads things out it sort of repels things from each other and then the links force pulls things together that have links between them and then the center force just pulls things towards the center of the screen so it doesn't go you know wandering off so let's just grab this whole thing and drop it in here and remove this unnecessary line that we already have at this point i'll run prettier just to get nice indentation and formatting okay so instead of d3 dot we can import force mini body from d3 and also force link like this and also force center i'll run prettier again and we gotta remove this d3 dot okay now this should conceivably work but we need to somehow display the result of this and we can do that with d3 d32 it says here the simulator starts automatically use simulation.on to listen for tick events as the simulation runs so a tick event is when all the positions change a little bit and this corresponds to animation frames where we can re-render the graph and get that cool bouncy kind of animation so let's try simulation.on i think we can just say simulation.on and pass in tick as the type and then pass a callback function that gets invoked on each tick so let's try that here simulation dot on tick and then pass in a function and then i'll just say console.log tick okay now we've got something that should actually run but i don't think we've actually wired up this stuff yet so in viz hub it uses roll up to take index.js and parse this import syntax and then generate bundle.js which is vanilla javascript that we need to ultimately include in our index dot html so here's what i'll do instead of this script right here i'll say this script oh that should be inside the body this script has a source equal bundle dot js and we can get rid of this hello html we can also get rid of the style on the body for the text which we don't need and let's see what happens does this run if i open up the dev tools check that out we got 300 ticks okay our goal is to render this somehow and i think the go to tool of choice is svg and we could create the svg programmatically or we could just drop it here in html and give it an id so that we can query for it i'll just call it container and here we can set the width to be 960 which is the default width in vis-hub and the height to be 500 which is the default height in this hub inherited from blocks that work run prettier okay so to get something oh look at that we got some scroll bars this is always an issue for this we can get rid of the default margin that's there on all html pages by in the css saying margin is zero and just to make sure that the scroll bars don't show up we can say overflow hidden and by the way i'm hitting shift enter to run this that's why it's not automatically running okay so now that we've got this container in our javascript here we can get access to that container and for the variable name i'll just call it svg and for this we can use d3.select and then pass it that id but since select works with selector strings like css we need to add this hash to make it catch that id and since we can use es6 imports i'm not going to say d3.select i'm just going to import select from d3 along with all this other stuff and run prettier to get the formatting all right still nothing's going to happen but you know let's continue writing this code such that our nodes will be displayed so we've got nodes here which we want to create a circle i'm thinking for each of these nodes and you know the nodes are not going to change over time so we can just set up that d3 selection once and then just update the x and y inside of the tick event so here's what i'll do i'll say svg dot and here comes the general update pattern select all circle and there are none at first dot data nodes so there's our data join and we can say dot enter to get the enter selection dot append circle that will append a circle for each of our nodes but to see it we should specify some attributes so we can say uh i'm thinking we need to specify the radius and the x y position so the first attribute will be r the radius and let's just set it to some fixed value for now just say 30. and at this point something should show up let's take a look alright we've got a little circle whose radius is 30 pixels okay great now um well actually there's three circles in one spot so what we want to do is get this as a selection so i'll say const circles equals this thing now prettier will put it on multiple lines so now what we can do is inside of this simulation tick thing we can say circles.attr we're going to set the x and y so cx is going to be a function that takes as input one of the nodes that we pass in here as d and then it's going to return d dot x and where does d dot x come from because it's not on these objects here we didn't specify x the forces actually define x and y on each node so they will be populated when the simulation starts all right so cx is d dot x and let's do the same thing for y i'll just make another line and say cy is d dot y and let's see if this runs okay it's working look at that those things moved around a little bit but somehow it's not understanding our the size of our display here let's see what we can do about that i think this is one of our problems for center defaults to 0 0 but we could pass in x and y so let's try that that should at least get our stuff into the center so if we pass in for center to be well the center x and y let's try defining center x and center y as new variables up here i'll just say const center x equals well what should it equal i mean it should be the width over 2 and width we can get from the svg so actually let's move this svg up here so we can access it because recall that we had set the width and height already on our svg see width is 960 height is 500 and we can use that as our source of truth for the dimensions so we can actually get out the attribute by passing only one argument we could say svg.attr width and that's going to come out as a string so let's just make it a number by adding unity plus and we can do the same for height so we've got width and height and now we can define center y to be height divided by two so now if we run this boom okay we get some activity in the center this is promising towards the end of all this we can get into tweaking the forces and whatnot but for now i think i'll just make the circles a bit smaller so we can see that they're actually in different places okay so now we need to add the links make those links show up here so i think i'll just use the same pattern as this for the links and i'll just call it lines and we can use svg line elements and the data will be links which we do have to find up here and each link is source and target as indices into the nodes array so we can create a line element for each and every link and then here it gets tricky we can set x 1 and by the way here's an example of a well-formed line element it has x1 y1 x2 and y2 and you have to specify the stroke for it to show up so i'm just going to copy this as a reference paste it right in here so our x1 is going to take as input d one of these links and you know now that i think about it it's a bit confusing to use d everywhere maybe let's buck the trend and and just use node for nodes and link for links so i'll replace the d down there with node and here i'll use link so where should the x1 and y1 be for this link well we need to use the source of the link so yeah let's say the line starts at the source and ends at the target so we can say link.source and link.source is an index of the array that is nodes and each node has x so we can say dot x right here so what this does is it takes the link gets the source which is an index into the nodes array so that's why we would use the square brackets to access the node at that index and then once we have the node at that index we can just get dot x on that node just like we do down here so now that we've got that we can do the same for y1 and just change that to y and now that we've got this whole thing we can do a variant of it for x2 and y2 that uses target instead of source and i think that should work um last but not least we need to set the stroke i'll set it to be black for now now we don't need this reference all right let's see if this works oh it does not work cannot read property x of undefined oh i think i might be prematurely trying to set x and y because this is outside of the simulation tick we only need to really set these inside of the simulation tick so i'll move that to be in here and we can say lines dot attr all this stuff now it runs and we're still getting this error cannot read property x of undefined so let's go back to basics here and try to debug this so my theory is this could be coming up as undefined why would that be i thought link.source was the index and indeed it is i mean this is zero so nodes at index zero should work um let's just do some debugging and console.log nodes at zero i mean that should work okay so that worked it gave us this this thing here but we're still getting cannot read property x of undefined that's weird is it coming from circles what it is coming from this wait a minute this is what i like to call a twilight zone moment all right let's bring this back see if it breaks yeah that's still breaking okay let's try to just debug this one here we're setting up our lines uh selecting all lines passing in links as the data so let me just try right here running this okay that reproduces the error let's break this into a function with multiple lines so that we can do debugging in here so let me bring that console.log statement into here i know that this one this line is causing the error so let's start from square one console.log link and see if this works and i believe that's the only console.log we've got going okay this is good uh what about link.source oh look at this the d3 library has actually replaced source with the node object oh my gosh i didn't realize it would do that okay so it has actually mutated our objects here so after passing it into the simulation it's not zero we can't expect zero we're going to get this node object interesting okay so we should just be able to say link.source.x i'm just like this okay that looks good so with this newfound knowledge we can go back and fix our broken code so no need to look up these nodes i'm going to use my vim super powers to select this line here all these lines at once and get rid of that and then get rid of that boom hey look at that it's working fantastic all right this is huge this is a great place to be you know what i think i'll just um this is a good checkpoint so i'm going to leave this viz as it is and i'll say force directed graph starter because this by itself is pretty cool okay so far we've got a force directed starter now let's bring in this data and try to achieve our original goal with this crazy graph viz i'm going to start by forking this force directed graph starter and then renaming this to be organization graph diagram let's revisit the original goal the original goal is to create something kind of like this based on this data but the problem is this is not exactly data in the sense that you would think of it you know it's not a json file it's not a csv file so our first order of business probably would be to create structured data from this using code so what would the structure of this data look like well we've got these main nodes that are bigger and the things that are within these main nodes in the outline should be connected to these main nodes with edges so these are essentially children of this parent so in a sense we're creating a graph data structure from a tree data structure that is represented roughly in this outline here let's start with something small let's start by making the arts web node connected to the community vision node and also the arts web node will be connected to the silicon valley crates node so let's take this text arts web and let's instantiate this in our code and you know i think this whole thing about generating the data is going to be a whole other can of worms and this file is already around 50 lines so let me just go ahead and and before we even get into it i'll create a new a new javascript module just for the data stuff i'll call it data dot js i guess and pasting that text here arts web and let's go to index.js and identify the data elements of it which is this here nodes and links so i'm imagining that we just move these definitions over into data.js and then we can export these so export nodes export links and then in index.js we can import nodes and links from dot slash data now this should still be working and indeed it is now we can focus just on the data itself okay so we've got arts web we want that to be the name of one of our nodes so let me just take that text and replace alice with that text now this node here will connect arts web to bob so now let's bring in some of those other aspects in addition to the arts web aspect we've got community vision so we can turn bob into community vision and so now this thing represents the connection between arts web and community vision i'll run prettier to get this on multiple lines and lastly we've got carol we can make silicon valley creates into this node here replacing carol with silicon valley creates okay now we've changed the symbolic meaning of this stuff in theory at least but the structure is not right what it should be is arts web to community vision this one is good but this one should be arts web to silicon valley creates so arts web let's just look at the outline again arts web is the parent the main node so artswab is connected to this one and also connected to this one to make that a reality we just need to change the source on this one here from one to zero now this represents a connection from arts web to silicon valley creates but you know looking at the the output you wouldn't know this so i'm thinking now might be a good time to figure out how to display text on these nodes that'll help us as we as we drive this forward for debugging purposes so let's let's stop and do that in index.js is where we can create text elements for these nodes at this juncture there's a number of ways we could go about it but i'll just i'll just favor this simplest possible thing which is just to have a bunch of text elements that are on top of these circles one thing we could do if we wanted is like make group elements and then the group elements have the circles and the text which we might want in the end but let's just do something simple now which is instead of circles this can be text select all text elements append the text and then i believe the way to set the text content is to say dot text which is different than dot attr and then pass in node dot what was it called what was it called i think it was called id yeah node.id node.id this will just establish the text elements but they won't be in the right spot unless we update them along with the circles in the simulation tick so let's copy the logic for the circles and apply that to the text yeah text is our variable name and text elements don't accept cx that's particular for circles i don't know if it's like circle x or center x it could be center x but text accepts x and y so let's see if that shows up any text and indeed it does look at that all right we're really getting somewhere okay so from here we can go so many directions um one thing i kind of want to do is make the arts web bigger because the size variance is something that we're going to have to deal with throughout this process of generating the data so let's just start now but what would the size be i mean we could attach something to nodes let's call it size but um you know what what would the size really be um it could be like the radius of the circle in pixels yeah let's try that uh let's say it's gonna be 50 pixels now if we want to um instantiate this in the graphics we need to go to the place where we're defining the radius of the circles and make this a function of the node so node is the input and we can say node dot size um this will break if there's no size defined so let's just make sure we've got size defined for all of these nodes these will all be smaller let's say 10 and let's see if this runs okay i think it's working but the problem is it's all sort of bunched up in here we need to get some more spacing around it how can we do this with the d3 forces maybe we can tweak uh many body yeah d3 dot force many body dot strength and theta i don't know what exactly these mean but it says here for strength a positive value causes nodes to attract each other while a negative value causes nodes to repel each other similar to an electrostatic charge that's cool so the default is a function that returns minus 30. uh let's try tweaking that over in index.html we've got our simulation and here we're defining the many body force and right in here i think we can say dot strength and i wonder can we pass in a number or does it need to be a function i would guess we can pass in a number okay this runs and it does the same thing as before but if we make it like 90 uh does it do anything oh if we make it a thousand look at that it's spreading things apart a little bit more i don't know if these values are like legit but you know it does it does what we want it to so i'm noticing a couple things i want to correct right now which is um like the text we can't read black on black and the text is not centered the text should be centered with respect to the nodes so let's just work on that a little bit how do we go about centering the text i remember it is some attribute on svg and i always need to google this one if we google svg text align yeah we get text anchor and alignment baseline here we go text anchor should be set to middle so let's do that attr text anchor should be set to middle and if i use prettier it will replace the double quotes with single quotes so if we run this all right look at that the labels are centered horizontally at least what about vertically they should they're a little bit too high let's try to center them vertically as well and i think we can use alignment baseline for that yeah let's just set alignment baseline to be middle like like this example so what it should look like is something like this it's almost exactly the same as this other line but just its alignment baseline instead of text anchor so this should do the trick let's see all right looks decent looks decent but we still can't see the black on black let's just fix that real quick one thing we can do there is set the the color of the circle i think i'll just make them gray for now we can iterate later and the way we do that is i'm pretty sure we set the fill to be gray like that gray is a named css color that's why this works okay and we've got this this looks like a pretty decent starting point now let's go back and continue working on the data what are some of the next steps here it's going to get more complex coming back to this outline we've got 20 smaller unidentified nodes and we've also got the rest of this structure there's a fork in the road should we try to deal with this 20 smaller identified nodes or should we get all of these named nodes in there hmm my gut feeling is that we can just we should focus on getting maybe this section totally finished and once we do that it can serve as a blueprint for the others so therefore let's let's focus on introducing these 20 smaller unidentified nodes the number 20 we can take from this document and somewhere in here we're going to use this number 20. so far we've been hard coding this stuff and there's no algorithmic element to this but i think we'd be better off with a more algorithmic approach for example like we shouldn't need to think about or type these indices because they're going to be essentially automatically generated so let's let's start down that road the first thing i'm going to do is make nodes be just an array to start that way we can push things onto nodes with nodes.push and we can pass in these objects and i'll do the same for these others okay now we're pushing these and it still runs what i'm thinking is that as we push these nodes we can have some more programmatic notion of a main node versus a child node versus maybe we'll call it a leaf node those 20 anonymous nodes what i'm really getting at here is we need to find a way to automate adding these links at the correct time so maybe i'll introduce a function called um add main node and maybe another function add child node and what we would want to do is invoke add main node and pass in this object here and then inside of add main node is where we would do the nodes.push and it would accept as input this node this object here would become node and then that would get pushed onto there one thing that is characteristic of main nodes is the large size so rather than like i mean once we have multiple main nodes we don't want to duplicate the size all those places so let me just move that into this logic here we can say node.size equals 50. and this even can be a variable um a constant rather i'll call it main node size and this naming convention is um i like to use it for constants like things that are just defined in one place that never change so main node size is 50 that's the size that the main nodes get before they get pushed onto the nodes array so now if we run this it still does exactly the same thing okay great next let's think about adding the child nodes both of these are children of this main node so let's say add child node and pass in this child node here and then again for this other child node here and the nodes.push will come inside of add child node and i'll make this argument called child node which we push and again we can define the size of this inside this function so we don't need to duplicate it so i'll get rid of size here get rid of size here and then the size on at child node should be child node size which we can define up here to be i don't know 20 whatever it was i don't remember but once they're all in this one place then we can tweak them later easily okay so we're adding these child nodes but the oh there's some cleanup i gotta do but the links are still defined here but in any case this should still run let's just double check oh we got an error oh i forgot to update the name here to child node okay it does not seem to run let's see what's going on node is not defined oh it's nodes.push node for add main node but for add child node i called this child node and i forgot to uh update it over here okay now it runs so i called this child node for a reason because that is as opposed to the parent node so here's what i'm thinking instead of hard coding these links like this we build it up programmatically so what i'm going to do is move this definition up to the top so that we can have access to it and i'm just going to comment these out because these are what we want to add programmatically and we can add these programmatically in the function at child node so what we want to do is push that onto links so links dot push this object with source and target and the source will be the parent node and that the target will be the child node and i'm pretty sure with the d3 library you can actually just put the node objects as source and target no need to use the index although we could but let's try it like that if we say the target is the child node and the source is the parent node now we need to know what the parent node should be i'm thinking we can take this as the first argument but where do we get parent node well we can pass it in here as the first argument parent node and we can just define parent node right here parent node is this the return value from add main node and we can pass it in here as well but the problem is add main node does not return anything i don't know we could make it return the node or i don't know we can just factor this out and say okay the parent node is this and then we add main node parent node this is in a sense simpler yeah we can just do it like this so now if we run it it still works it does the same thing all right sweet okay now we are in a position to add these twenty smaller unidentified notes now how should we approach this we already have this function add child node and what we want to do is essentially add more children to this node which is already a child of of its parent and the first step towards doing that would be extracting this so that we can reference it multiple times i'll make a new variable here called child node that can be this one and then now what we want to do is add 20 anonymous children to this child note this line here adds this child to its parent but let's try adding a child to the child node so the first argument here would be the child node and the second argument would be some anonymous node so i'm going to copy this node definition but i'll i'll just make the id um anonymous and let's see what happens all right look at that looks like a solar system diagram or something all right this is pretty cool this is doing the right thing so what we want is a bunch of these anonymous nodes connected to community vision oh i'm just noticing i've got this junk we can clean up all right so to add more of these we can just uh copy paste this line a bunch of times and we've got more and more and more of these anonymous children and it's working look at that okay we're getting somewhere one problem i see with this is that we want these anonymous nodes to be smaller and we don't even want any text to show up here so how could we do that um maybe if we just you know give an empty string as the text that's probably the easiest way to accomplish the no text and there we have it no text okay that's good and we want them just to be a little smaller the problem there is this line of code here it's setting the size of the child node to be child node size uh but what we want to do is make the size something else i think i'll introduce another one of these constants for leaf node size and that that's going to be tiny like i don't know five and what we need to do now is is use that somehow when we invoke add child node we should um maybe we can pass in the size i mean what if we had another argument where i could just pass in leaf node size like that yeah that could work in which case we would need to modify this function to accept another argument called size and we still want these other invocations to function correctly um so we could make the default value of this child node size and then we can use use size here like that so if size is not specified it'll get child node size so this invocation will work because it doesn't have that argument but if we do pass in a different value like leaf node size that will get assigned to size here and then used here as the size of the node so at this point there should be a single tiny leaf node let's see if it worked indeed it did see there it is now let's make all of these leaf node size but you know at this point i want this to be um more programmatic we shouldn't have to be copy pasting lines like this so let's use an old school for loop and i'm always cautious in viz hub because sometimes the for loops crash things with the auto run so i'm just going to type it up in a comment for let i equals zero i is less than 20 i plus plus so this should run this chunk of code 20 times and that chunk of code that we want to run 20 times is this chunk of code right here so now i can get rid of these lines and this should work all right look at that that looks pretty darn cool all right where should we go from here i'm noticing that all of these nodes that are connected to each other are pushed away from each other with the same force but the force really should be different i think depending on what type of node they are i mean the child node should be pushed away from the the main node like it is here but all of these anonymous nodes i mean they could be closer to this child node um let's see how could we do that somehow maybe the links could contain some information about um how strongly they should be repelled from one another maybe here when we define the force link force we may be able to specify some function which is unique to each link that determines the strength let's see we had force link is what it was called here's the documentation apparently there's something about distance and strength parameters for each link distance sounds interesting ah look at this if distance is specified so the default distance is 30. strength is a whole other beast and i don't think we want to touch this yeah i think distance is what we need here and it can be specified as a number or a function and that function is invoked for each link okay great so over here we can specify the distance like this and that distance is a function of the link and let's just return 30 for now that's the default let's see if we change it to 20 what happens um not much maybe maybe our charge strength is too high that just seems too high from the beginning if we make the charge strength 500 and then the link distance to be like 50 okay if you make it 10 things are closer together we're getting somewhere but i want it to be different for each and every link so let's just say link dot distance and then when we generate our data we need to attach a distance property to each and every link that we push so we've got add child node with source and target and and here on this object is where we need another property called distance and this can be um perhaps similarly to size we could pass in something or maybe it could be a function of size maybe let's have distance be its own thing that gets passed in and just like size it can have some default like default distance and default distance we can define up here as 30 sounds decent that's what the d3 default is and now when we add the child node for each of these anonymous nodes we can pass in maybe let's call it leaf no distance and we can define leaf node distance up here to be let's say 10. okay this is doing something i think maybe um let's just experiment if we set it to be larger then they do go farther away see that so this whole scheme is working and if we set it to be smaller they do go closer but let's say if i put this to zero this is as close as they're gonna get and i think that's because the many body force is pushing them apart so we can go back and tweak the strength of the mini body force which is 500 here and you know i kind of want to be able to tweak this from the same spot as we're tweaking everything else so let me just do that i'm just going to make this a variable called many body strength and the value of many body strength can be defined right here as -500 and many body strength we can import from dot slash data along with nodes and links and over in data.js we can define it and we can also export it to make it visible okay this is fantastic now we can tweak everything in one spot so if we make many body strength like 50. look at that those anonymous nodes are right on top of each other maybe we can increase the default distance to be like 100 hey look at that it's getting closer leave no distance i mean we can push those a little further apart so maybe 20 yeah we got some variation now in that and let's see default distance let's try 200 i don't know maybe 150. okay we're getting somewhere we're getting somewhere this is looking pretty okay um the readability is a bit off but that can be a final pass i think what i'd like to focus on next is getting this structure working for all of our outline yeah so we're still in the middle of this arts web after that we can move on to these but we still need to add the 20 smaller unidentified nodes to this silicon valley creates child node so let's do that in here we are adding 20 child nodes and we want to do this again for this child note here so maybe what we could do is just copy this exact same pattern and just change community vision to silicon valley creates so let's try that i'll just copy this entire thing paste it here and then change community vision here to silicon valley creates now we've got this block and this block essentially copy pasted and let's see if that works ooh child node has already been declared yeah that's the thing about const if you use const you're not allowed to reassign so we can make that not break by using let instead and then down here we don't need to use light again because here it's defined it's assigned and then here it's assigned it's reassigned it's reassigned and that's what let lets you do okay this is working hooray now it is a little bit of a bad code smell in my opinion to have such copy-pasted logic so let me go back to using const and what i'm thinking is we can encapsulate this into a function so that we can just call this function again and again for each and every child node i'll comment out this for now and this one i will put it inside of a function i'll call this function um maybe assemble child node yeah so we've got this function and inside the body of this function we can put all of this logic just like that and use prettier to format it and the only thing that's going to be different is the name here this id so what we can do is just accept that as an argument um and this here can be simplified with es6 syntax is the same as this here and now we can invoke this function assemble child node and pass in this string here community vision so now if we run it we only get community vision but now that this is set up like this we can just invoke that same function again and pass in silicon valley creates and now we get both of these here like this okay sweet that's exactly where i wanted to end up now we don't have this sickening duplicated logic and now we can go all out tackling the rest of the outline okay now we are done with this part let's do now this part okay so we're introducing another main node let's start there because everything else is going to be attached to that it's called social impact commons [Music] what can we do for that we need to invoke add main node again and this is where i think we want to essentially copy this pattern i'm just going to paste that text social impact commons and then i'll copy paste this this chunk and change arts web to social impact commons and when we run it we get that issue about const so this is where i think i would like to use let here and just a pattern i like to do with let is just declare it once so that it's clear that it can be reassigned and then assign it in multiple places like this so now if we run this um there was another parent node you see it sort of it sort of ends up moving off to the side because it's not connected to anything let me just tweak uh tweak these constants to make sure that it's in view i think we can do that by increasing many body strength so if we make that again like minus 500 oh that's that's pushing it away so if we make it small and -30 things are not pushed away as much okay that's what i'm after i don't know minus 10. okay now artsweb is there on the screen at least we can see that it's there and actually at this juncture i'm wondering should there be an edge between these two main nodes let's refer back to our original example in this example that we're trying to emulate the main nodes are all connected to one another so now that we've got two main nodes let's just add that connection that will solve the problem that that thing is floating off all right so the way we can do this is we want to add a link between these between these two parent nodes this is where it's kind of problematic that we're using the same name for both so you know i think i'll just i'll just rename this to be arts whip i think this is fine because there's not going to be that many main notes we can have a variable for each now we can go back to using const arts web and then pass that in here and now we can have another const social impact commons and add that there but i think we might be referring to parent node down the line yeah here here we're using add child node parent node but the parent node is going to be different over time so i think what we want to do is accept parent node as an argument here and then when we invoke this assemble child node we should pass in arts web as the parent yeah i think this is the way to go okay now this is still running correctly and now we are in a position to connect these two main nodes together with links and this is the pattern that we need to follow for adding the links so i'm just going to copy paste that and the source will be arts web and the target will be social impact commons and distance needs to be something let's just make distance default distance for now okay and that worked okay great see that we've got arts web and social impact commons connected now we are in a position to add children to social impact commons so let's do that i feel like this this is sort of cluttering things up i'm going to move this definition to up here with all our just to group it with all our function definitions so that we can just focus on the data itself down here so we've got arts web and within arts web we've got these child nodes so i think i'll just move this to group it up there with with this because these are all sort of associated together and then now that we introduce another main node that's where we can add this link and we can follow the same exact pattern of adding child nodes and in this case the parent will be instead of arts web it'll be social impact commons and these are not the right headings so i'll just get rid of those and then we can refer back to our document to get the children of social impact commons and this is going to be theater bay area so i'll use that as the first one the second one will be east side arts alliance and there's going to be another one called local color so i can just um note that down here copy paste this line and use local color right here so now if we run this check it out it worked social impact commons now has these three children sweet at this point we have got these two main nodes totally done now we just have to add two more looks like we're about halfway done with the data side now let's focus on this one community arts stabilization trust this is a main node just like arts web and social impact commons are main nodes so we can follow the same pattern as this block so i'm just going to note down the name of it and then i'll copy paste this entire block but even before i copy paste i see this as duplicated logic so let me just make this into a function and this is a function that connects main nodes so i'm imagining i want to have some invocation here like connect main nodes and just be able to pass in arts web and social impact commons and source will be source target will be target in this new function that we're going to define called connect main nodes so in this function body is going to be all this logic and it's going to take as input source and target and again due to es6 this can be replaced with just this this should work i'll just move this up to where all of our other functions are defined to remove this clutter from the data oriented section let's just see if this still runs yeah looks good now this is a nice compact chunk that we can just copy paste and the name of it will be community arts stabilization trust instead of social impact commons and that can be the variable name too as camel case that's that's huge so i'm just gonna rename it to c a as t because we're gonna have to put this in here over and over and over okay great now these headings are out of date so i'm just gonna delete these for now and we can consult the document for what these should be okay we've got a bunch we've got counter pulse put that in there luggage store gallery put that in there and i know we're going to have more than three so i'm going to preemptively lay out some space we can just paste things next performing arts workshop and all of these have 20 smaller unidentified nodes so i'll just paste that there and the next one 447 ministry that one only has five smaller unidentified nodes so i'll paste that here and uh this this has five child notes so i'm just thinking how do we handle this it's a deviation from the normal pattern let's just add another argument to this function assemble child node so let's pause here and make that work assemble child node can take as input a third argument which is the number of um anonymous leaf nodes so let's let's check out assemble child node we can add a third argument here i'll call it uh the number of leaves so like num leaves like that and by default this can be 20 but if not you know we use the value passed in like five and then instead of having 20 hard-coded here we can use num leaves so let's see if this worked we should see one of these with five but it looks like we need to go back again and tweak our settings because some stuff is going off the screen maybe let's even make the main notes a bit smaller like 40. um child nodes even can be smaller like uh 15. and the default distance i maybe we can make it smaller like 100 perhaps or maybe even smaller like 30. leaf node distance is okay oh i think what it is is that not all of the main nodes are connected to one another see we're missing a connection between social impact commons and community arts stabilization trust i think that's why they're sort of going off the screen let's fix that real quick we're connecting the main nodes uh for each of these but i think i want to group all of these together at the end because we need to essentially have all pairs connected so i'm going to copy paste these and we need to get the connection between between social impact commons and cast now let's check it out okay this is looking better everything is connected but it's still like going off the screen oh one thing we could try is to just set the height of the viz um yeah maybe let's try making it a square so if the height is 960 um we can update our code to use a height of 960 here okay look at that now it's it's a square and everything is inside of that square okay what i really was going about here is checking that 447 ministry actually only has five anonymous child nodes and indeed it does indeed it does great great great so we can keep going with the addition of data so where were we adding the data we were at 447 ministry let's keep going in our document okay after this one there's only one more keeping space oakland with 20 smaller unidentified nodes so let's put that in here and we don't need to pass in any second argument because 20 is the default and that should do it okay so far so good we've completed this entire section and now let's do this other last section ambitious amb tious that's going to be another main node so again we want to have another copy of this entire block and the name of it is going to be ambitious and we can rename this variable to be ambitious and replace that here all of these and you know what i want to do is just have an empty place to paste into these children okay what are the children of this main node we've got east bay permanent real estate collective that'll be an awfully long label uh i might just i might just use the abbreviation here yeah just put the abbreviation that has 20 unidentified nodes here's another one sustainable economies law center again uh this is long so i think i'll just use this abbreviation paste that in here um and you know what these don't have any 20 smaller unidentified nodes in this document um i wonder if that wasn't just an oversight or they don't have a volume of entities but i'll just assume that was intentional and assume that you know these all should not have any child leaf nodes so we already have this additional argument to pass in what i mean if we just pass in zero that should work and that's the pattern for the rest of these so i will create this empty vessel to just paste these other labels the runway project paste that there common future paste that there freelancers union paste that there us federation of worker cooperatives paste that there and get rid of our one dangling placeholder and if we run this it works but this whole thing is disconnected from everything else i think we just need to make sure that we add the connections between ambitious and all the other main nodes so how would we do that we just have connect main nodes from ambitious to everything else so i'll just copy paste that a bunch of times and so we've got cast we need this other one social impact commons and also arts web so now this looks more or less correct i think all right so far so good i mean this is looking decent and it has all of our data in it so you know from here on out i believe it's just going to be tweaking these various parameters and then we should definitely do a styling pass and color too i mean maybe we should use color let's take a look at the original example yeah the original example uses color which is related to the data so maybe we should do that next focus on that next but anyway this is substantial progress for now with one last push we should be able to have something presentable that we can deliver all right this is what we've got so far but it's still a far cry from this example which we kind of want to shoot for this aesthetic we have a lot to do like tweaking the parameters of the force um changing the sizes a little bit working on the text wrapping but i think the first thing that would make sense to focus on is the colors notice how each set of nodes has colors based on its main node which we're calling it and this is related to the data i mean we're gonna have to come up with these colors in the data generation part of it so yeah let's let's go ahead and try to add the colors in the code the colors well we can maybe modify this add main node function to generate a color for each main node as it gets added but first we're going to need a source of colors there are all sorts of color palette generators out there i think somebody once recommended paluton to me looks like a pretty neat tool but how many main nodes do we have anyway how many colors do we need it looks like we have four main notes arts web social impact commons cast and ambitious so we only need four colors that's great because palaton only gives us a max of four main colors it looks like and we can drag this stuff around to sort of tweak the aesthetic so i'm just gonna you know tweak this stuff until something kind of pops out i mean that that looks decent let's try these four colors i think we can export this yeah this is cool it's nice that there's light and dark variations you know we could try these out here is what we get when we generate the text export so let's take this as our input and the color definition is a whole other thing so i think i'll make a new file called colors.js and then i'll paste all that in here and i like having all these different shades available because we can pick the shade and tweak it later so let's get this into a structure that we can use we're going to want to export const colors equals i'm thinking it'll be an array of arrays so it'll be an array and then i'm going to use my vim superpowers to essentially crop out the parts that we don't need and yeah so i'll just delete that i'll also delete the rest of this and now this is pretty close to an array of arrays we just have some extra new lines random text we need to surround these with quotes to make them valid strings so we can append quotes to the end somehow but anyway this will be the first array this will be the next this will be the next after that and this will be the last i close it out let's see if macros work in here looks like they do so i'll end quote put a comma go to the next line and then execute that macro so we can pretty much automate the editing of this text and then we can whip it into shape with prettier boom there we go beautiful beautiful colors and i think i'll just drop a note in here you know generated with paluton i'll just grab this link paste it in there credit where credit is due okay now that we've got these colors in our data generation logic that's where we can use this stuff in this module let's import colors from dot slash colors and what we want to do is associate the color to each of these main notes so i'm thinking we introduce another field alongside size called color and this can be colors at some index that's the main color and then within each of those entries there's an a sub array with the different shades and i think the shade we can use for now is the middle one so i'll just hard code two for that but the question remains what is i i should be incremented each time we call this so we can say i plus plus right here and we can just say let i equal 0 to start with out here and just to check if this is working let's do console.log node.color so what this should do it's going to start i at 0 and then the first time add main node is called the value of i is going to be 0 and the way the plus plus operator works is that this expression here returns the current value of i and then after returning that value then it mutates i to be i plus 1. so the overall effect is the first time admin node gets called this is going to evaluate to colors at index 0 and then it's going to get the mid-range shade from that and then assign that to be node.color and then the second time add main note gets called this expression here will get the second color meaning i will be 1 at that point so it's going to get colors at index 1 get the middle shade and so on so let's see if this works if i run this it appears to have worked because we get these four different colors all right sweet all right now that we know that's working we can get rid of that debug statement and then leverage node.color when we actually render those those nodes as circles so in index.js let's find where we're rendering those circles and then when we set the fill let's make this a function that takes us input the node and it returns node.color and if there is no node.color we can use gray i like to use the or operator like this because if node.color is undefined it's falsy it's a false ish kind of a value that triggers this or to return the next one so this means everything except the main nodes should be gray let's see if it worked all right look at that it worked our main nodes have some color fantastic okay this is what we've got so far i'm looking at just spending you know as little time as i can to to get something presentable from here so we can you know deliver something uh one thing i want to do is make these colors propagate to all the nodes and then do a stylistic pass and tweak all the parameters of the force layout to get it to look decent so let's start with looking at these colors in add main node we are assigning node.color when we add the main node so what we can do is when we add a child node we can just carry over that color from the parent node which we do have access to so let's try doing something like this parentnode.color is what we want to use as child node dot color so we can just say child node.color equals parentnode.color and voila look at that the colors have been updated it would be nice to make the links also have the same color right now the black is pretty harsh and it would be nice and smooth uh if it was the same color as the nodes so let's try that all right so in add child node we're assigning the color of the child node and we're also adding this link we can just take parent node.color and assign that to a new property on the links we can call color so color is parentnode.color then format that with prettier now we can access color on each and every link object so in our rendering logic we can take advantage of that here we're rendering the lines and we're assigning the stroke to be black instead if we assign the stroke to be a function that takes as input a link and uses link dot color or if there is no link.color we can use black this should do it all right look at that the links are the same color as the nodes but the links are on top of the text that's just an ordering issue i think we can fix that fairly easily we just need to be sure to add the text after we add the lines see right now we're adding the text and then we're adding the lines but if i just swap the order of this i'll just cut that text and paste it to be after we add the lines okay that problem is solved now see the text is on top i kind of want to color these black lines that connect the main nodes let's do that here in connect main nodes we can just assign a color to these links and let's just arbitrarily say that we're going to use the source color okay cool now they are different colors at this point we can start tweaking one thing that i notice is that the black on this sort of dark color is not that readable i think we can go up one shade make it a little bit lighter and recall that in our colors we're using the shade at index two so if we just change this two to one we get nice light colors okay that looks better i think another thing that pops out is we're using the default font which i think is like times new roman or something we can use a slightly nicer font by specifying the font family to be sans serif okay that looks a little better the problems now are you know issues of overlapping text and the layout is not quite right let's see what we can do by tweaking our various parameters i'm noticing we have this default distance but what i'm really looking to do with this tweaking is to make the main nodes farther apart from one another but not impact how far apart the child nodes are from the main nodes so i think i would like to introduce another constant here like main node distance and then use that when we add the connections between the main nodes yeah when we connect main nodes instead of default distance this should be main node distance now if i increase main node distance they should be a little more spread out one thing i'm noticing is that all of these nodes seem to be bunched up and i think these are the ones that don't have any uh anonymous leaf notes maybe we can just add a couple anonymous leaf nodes for those like maybe three for each just to make them spread out and you know i kind of wish that i could just drag these around to position them to you know avoid the overlaps that would actually get us to something presentable luckily there's a module that we can use that does exactly this d3 drag here's how we can use it instantiate a new instance of d3.drag to do this we can import drag from d3 and then introduce a new variable which is a new instance of drag let's call it drag interaction just so that we don't conflict with the variable name drag and here's how we can use it we can select the nodes and then use dot call and pass in our instance of d3 drag then we can add some event listeners to handle the actual interactions in our case we're calling it drag interaction and we want it to be on the circles so right here we can just say dot call and pass in drag interaction now after we construct our drag interaction instance we can say dot on start so here's drag interaction uh we can say dot on start and this can be a function that we define right here and i'll just say console.log start just to see if we're here and now if i click on this it works but what happened the first time as i clicked on the text accidentally we can solve that issue by adding some css to the text to make it not selectable we can set pointer events to be none i think that should do it no that didn't quite work maybe that needs to be style okay that worked now i can no longer click on the text oh another thing i notice is that the lines are inside the circle but they should not be this is another thing we can fix just by changing the ordering so that the circles come after the lines okay there we go so let's get back to implementing this drag interaction it turns out that when we use drag.on we can pass in start drag or end and what i'm thinking is we can just use drag this is after every move and in our drag handler we just set the x y position let's try that so instead of start this will be drag and i believe drag takes as input some arguments let me just see what these are i think there are two arguments and i'm just going to console.log these arguments to see what they are it appears that the first argument is a drag event and the second argument is our node object this is great because we can just set x and y on our node object so b is node a is the event so what if we just say node dot x equals well we want to get the x from the event and i don't quite remember what that looked like so let's console.log the event and take a look this is what the drag event looks like it's got x and y right here so we can in fact say node.x equals event.x and also node.y equals event.y now we should be able to drag it and look at that we can totally drag it around the problem though is that after i release it i can't drag it around anymore the whole simulation just freezes this is where we need to interact with the simulation because what happens is the simulation cools down and stops when there's no more motion so we need to essentially restart the simulation or maybe we need to set alpha target or alpha i'm not sure exactly what way to do it but it says this method can be used to reheat the simulation during interaction such as when dragging a node so that's perfect for us so let's try that we want to call simulation dot something but um i kind of like to define things before we reference them so i'm just going to move this to be defined after simulation so let's try it simulation.restart on each drag we restart the simulation it still freezes after dragging so maybe we need to say simulation.alpha equals well whatever this whatever it started out as let's see what is simulation.alpha okay it defaults to one so let's try setting it to one and then calling simulation.restart every time we drag all right this seems to be working sweet so now we can untangle some of this stuff oh wait a minute wait a minute something is wrong i don't know why it cooled down and stopped like this i don't know maybe we need to use alpha target instead let's try that alpha target oh wait a minute wait a minute maybe this needs to be called as a function maybe that i just missed that let's see does this work okay i think this is actually working yeah not bad but you know after i drag it it sort of gets sucked back into the simulation what i really want to do is have it just be in a fixed spot after i drag it it turns out to fix a node in a given position you can specify fx and fy let's try that instead of specifying x and y let's specify fx and fy now if i drag these things around and let go it remains fixed to the place where i dragged it so let me just you know separate out some of these things this one's going too far and this is great this is great this is really shaping up let's bring in that one it's getting too far away and you know we can make it more more of a landscape kind of a thing so it doesn't go so high vertically oh that one's going too close there we go there's our first deliverable so what i can do now is just take a screenshot of this and send it over and say look here's what we've got as a first pass

Channel: Curran Kelleher

Views: 1,860

Keywords:

Id: y7DxbW9nwmo

Length: 96min 18sec (5778 seconds)

Published: Mon Dec 28 2020