Designing Data Visualizations with Noah Iliinsky

great thank you all for coming here today we'll probably be seeing more people coming in as the hour progresses we're very excited to have no Alinsky here to come speak to us at LinkedIn we care a lot about data visualization and making beautiful impactful designs and so it's great to have an expert in the field of interaction design and interaction of visualizing a data visualization speaking with us know as an author of a Reilly author of data visualizations and an editor and contributor to beautiful beautiful visualizations and so you can check out those books and hope you will enjoy the talk again we're very excited to have him here the talk will be about an hour long we'll have a Q&A session and then a small break and then we'll have a continuation of the talk with two extra segments that Noah will go into and then he'll also be available for discussions after the the talk ends for anyone who is interested know as a consultant in this field and so please join me in welcoming here no thank you hi so I think and speak and write a lot about data visualization which is a field I sort of stumbled into accidentally in grad school as it happens and and accidentally wrote a master's thesis on the sort of thing several years ago and I've been thinking about it quite a lot so starting with just a little bit of background why visualization other than the fact that they're fun visualizations are fun or cool or whatever there's an example this is a set called ants coms quartet it's put together ants commas a statistician and intentionally crafted this this set to show that data is is deceptive if you if you don't visualize it that it's difficult to tell what's in a set of data so the properties of this data set three out of the four series of data here the X these are the same in all of them the y-values the average are the same standard deviation is the same if you were to look at this on a spreadsheet or look at this with statistical analysis you would find these to be very similar datasets and then you graph them and you get very very different looking graphs from each data set very distinct and so visualization gives you some access into this data it makes makes dataset accessible but it it lets you see what's actually happening within the data and that's very powerful it turns out that our eyes and our brains have very sophisticated software built into them for things like pattern recognition for things like detecting when there are pattern violations and on a variety of factors so there's in terms of position in terms of SKU color size blur shape and a number of other properties these are all things that we are they're called pre-attentive properties we can we can detect these very quickly when something is different when something is out of position and if you leverage these well you can design things where you can get a lot of information into somebody's brain very easily or very quickly and if you do not leverage these well you end up in a situation where becomes difficult you're actually you're design works against you in terms of communicating that information to people so fundamentally the brain how we see as actually we're doing not just pattern matching the pattern violation detection when something appears to be different when something changes we're doing a dif visual diff and seeing when something is different and so when you when you're working with visualizations what you're doing is you're putting trends on the screen you're putting patterns on the on the screen or on the page for people to look at and what people are going to detect what they're going to see is they're going to see these trends but they're also going to see places where you have things like gaps then we have outliers and that's where the interesting stuff is right because if the trend is consistent it's it's predictable there's no surprises there but the gaps and the outliers are what you're looking for where where the noteworthy material tends to be so here's an example I I ride my bike i buy some bike things online this is from a little bike shop in Walnut Creek one of my favorite online bike stores called Rivendell and they've actually since changed their website so this is no longer a real-time example but it's a small enough shop that they don't have of sorting to have kind of broad categories in this case is their page for things like tires and tubes and pumps and patches and the sort order is reverse chronological which is to say the newest thing that's in the catalog is at the top of the page and that's fine if you're only looking for a small number looking at a small number of parts so if you're looking at rear derailleurs they only sell four different of them and so it's fairly easy to go through and look at what the different options are but if you're looking at something like tires they sell them for three different wheel sizes they sell I think 19 different tires some tires are available for multiple the wheel sizes some of the tires are only available for one wheel size some of these are skinny little tyres for racing and going quickly some of them are big knobby tires for mountain biking some of them for touring and there's no way to sort and filter these so you have to do see really look at each example click on through the page read the size read the features of it it was just intolerable the data scientists me couldn't handle it so I built this instead and very quickly it lets you determine what it is which tires are of interest so there's there's a couple of factors going on here the horizontal axis is the width of the tire the vertical axis is sort of this intentionally ambiguously named or unnamed axis that goes from more burly to more svelte which is sort of a toughness wait speed analogue and then the three colors the black and the white and the grey circles indicate the rim sizes and only one of those is actually going to fit on your bicycle so you can very quickly filter out the colors of the dots that are not the tires you're looking for and that gives you an option to very quickly look in a smaller portion of this graph and you know click on the dot or click on the tire name and very quickly go to the page and look at the three or four tires there might be interesting to you rather than searching serially through all 19 tires that are available for sale so that long-winded example is just to say with visualization it takes seconds to isolate and to look at three or the four maybe five tires that are interesting to you whereas if you present something in a list or in some other format the work is not done for you if you put this up in a graph the work is done for you the thinking has been done for you and you can very easily and much more efficiently get to the information that is actionable which is to say in this case buying a tire visualization gives access to huge amounts of data if you have not seen hands Rosling any the Gapminder talks TED talks and some other talks that are on YouTube go check it out he's very good and he he talks he's a Swedish I believe researcher who works on things like global health and population and the sort of thing and so he he does these very long-term visualizations where you're looking at four or five decades worth of data across the whole world and you're looking at things like how GDP has changed over five decades on a on a per continent basis or a per region basis and how that's affected things like infant mortality so there's a lot of data when you're looking at tens of tens of regions or hundreds of countries when you're looking at decade's worth of data in these couple of factors and and he he does these beautiful graphs these beautiful little animated graphs in such a way that you can actually see what's going on and if somebody were to present you a spreadsheet of these of this volume of data we extremely difficult to see what the trends are to see where the patterns are to see what the outliers are in this case I can remember exactly what this is but it's worth it's worth looking up his talks not just because you're injured in public health but because of the volume of data that he can present compellingly in a very short a short period of time and with not so many pixels as you would need to present you know huge pages of spreadsheets and he does that by by sort of wrapping everything in a narrative in a story and this phrase telling stories with data visual storytelling the word stories has come up a lot in the last year associated with data and we don't really think of data and stories going together it's coming up a lot with visualization particularly and the reason for that going back to the Rosling is the story makes the data relevant it it tells you what to look at it tells you what to look for it tells you why this is interesting or useful what to pay attention to if you just give someone a spreadsheet and say here's some data you say thanks I've got some already I know what I don't really have a good place in my decor for this data right but if you say here's some data I've highlighted the parts that are interesting let me tell you what this means you're much more able to absorb it and actually make some use of it so this the focus on stories is is kind of taking it to the next stage the first stage is analyzing it and understanding it and then and then this next phase is being able to share that knowledge because if all you can do is analyze it for your own use and you don't have any way to distribute it effectively you're very limited you don't scale right as an individual you've got to be able to put it out in the world in the way that other people can consume it usefully incidentally I do I do enjoy an interactive sort of a thing so if anybody has questions or if I don't go into enough detail about something or if I speak too quickly raise your hand yell and we will back up and take the first question hands Rosling are OSL ing and he's on YouTube he's given a couple of TED Talks totally worth checking a few out alright so just briefly concepts and definitions visualizations infographics these terms are not actually well defined yet there's still some flux and some debate within within the visualization community about what these terms mean the definitions I'm going to give you are the definitions that are used by people whose work I respect and I think I think they're pretty good definitions so fundamentally it comes down to the process of how you get to the data and sort of the density I'm sorry the process is how you get to the visualization and the density of the data that you're representing so something like a data visualization in this case represented by tableau generated in tableau this was generated by software you if you wanted to change the data if you wanted to update it you can very easily make a little tweak and have the software redraw the whole thing it's not particularly rich in terms of the aesthetics no individual had to go through there and draw every line or plot every point by hand or with the mouse in Illustrator or something like that it's algorithmically laid out and that's a useful thing because it means it does scale infographics are going to be manually drawn when I say manually this is probably an adobe illustrator not necessarily with a pen and pencil but somebody literally had to place every pixel on the screen and this does not scale if you need to change the data if you want to use a different data said if you want to update it with the next two years data somebody has to go in and manually make those changes and so the trade-off here the value of doing this by hand is that you can actually do it in a very rich way you can pay a lot of attention to the aesthetics and in some situations that's useful but you're limited to tens or maybe hundreds of data points you can't do thousands or millions or you know hundreds and millions of data points and a lot of the work that we do deals with large volumes of data so we're going to be probably more on the data visualization end of the spectrum and less on the infographic end of the spectrum so this this briefly sort of shows the trade off here on the end of my right to your left is is the data visit the high volume data visualization end of things where you've had a lot of data and not very much a static treatment and on the other end on your right side my left side you've got a lot of aesthetic treatment that can be done but you're very much limited by data and that in the middle kind of doesn't really count there's no there's no good use case where you're spending a whole lot of effort on on many many many data points just a really scale so the horizontal axis there is how much how much effort is required manually in terms of drawing the vertical axis is volume of either data or volume of beauty of aesthetic the rigorous people in the audience will be thinking combined roxies like that you can't put beauty and data on the same axis so the units on the beauty there are millions which is to say the amount of Beauty required to launch one ship classics majors in the audience are laughing now okay different use cases for your data exploration or analysis and then explanation or presentation right this is this is actually kind of a fundamental concern because it's whether whether you know what you're working with whether you're presenting it to someone else or whether you're still in an exploratory mode this is a little bit more of an exploratory an analytical visualization we don't actually know what the story is so this is a curated data set it's it's some football information I don't have a lot to say about football but some people think it's really interesting and this is a this is an interface that allows you to sort of pivot through these different properties and see some different things about the different teams and and their relative performance it's not a fixed story it's not the story of how the team that won the Super Bowl got there because they were clearly in the top two or three in both offense and defense for the year it's not that story it's here's some data you can explore it so it's not it's not something that's ideally suited for telling a very specific story rather it's something where you can give the data to an audience and say here's some data play with it enjoy it make of it what you will so that's much more the exploration the analytical end of things when we want to go to explanation or presentation there's a story to be told we know what the answer is and we want to present this to somebody else so this this happened this week of the iPhone total revenues the iPhone more than all of Microsoft that's an interesting story this is not a sophisticated data visualization but it's very compelling right because there's a story in it because it's a very compelling story and so we don't have to do a lot of fancy treatment with the graph I'm not sure what else you would do it's only two numbers but it's it's pretty interesting so this is not spreadsheets this is not quarterly reports this is not revenues and margins it's just look where we're at today by how things have changed it'd be interesting to put you know all of Apple's revenues versus all of Microsoft's revenues ten years ago you could do that there's a lot you could do with that but again these are all very much in the in the presentation you're explaining something you're presenting an answer to some people rather than saying here's some data enjoy some data player that yourself finally education and persuasion these are very different but they won't tell you that if they're trying to persuade you they'll say we're trying to educate you so here's a graph sorry an infographic so I would call this this is much more on the education end of the spectrum there's not necessarily a political bias here there's not necessarily a point of view that's trying to be put forward other than hey look Bernie man's kind of interesting but there's not a particular bias or political agenda that's going into this contrast this is a diagram of the House Democrats proposed health care plan I'd remember exactly which version if this is what passed or not and you think wow that's pretty miserable and then it's very interesting if you look right down here the sources Joint Economic Committee Republican staff this is not a document that is meant to educate this is a document that's meant to persuade and when that when that switch happens from trying to educate to trying to persuade really different design choices get made really really different design choices you can spend a half an hour and I have talking about all the very intentional design choices a huge amount of effort that went in to making this look like it does and the way that it looks is really painful really difficult really obscure there's a whole lot going on I mean the consumers are all on one side and my doctors like completely on the other side of the diagram here and like the whole federal government's in between every design choice made in this was made in such a way as to make this whole proposition seem overwhelming and so it's not about education it's not about providing knowledge to people it's about advancing a particular point of view and that's a choice that that an editor or a designer can make and there are situations where that may or may not be valid but understand that that's an intentional choice and that's very different from a piece where the information is just being made available so that was just a little bit of definitions let's talk about how to do this how to design it's really simple like everything else in life all you have to do succeed is make good choices in this case I can tell you how though fundamentally what it comes down to is making intentional choices because what happens is when you're when you're designing something and you use all the same defaults you're not intentional about the choices you just sort of arbitrarily picking some things let's use blue it's the company color let's do a timeline we always do a timeline you miss out on the opportunity for some depth of consideration and you miss out on the opportunity to reveal things the particular interested in a data set you miss out on the opportunity to customize what it is you're presenting to the audience that you're presenting it to and all these things matter quite a bit so there's three main inputs that are going to guide these choices these intentional choices that you're going to make as you are as you're designing your data visualization the data itself is going to have some things to say about how it should be represented the audience that you're presenting to the reader or the customer or your boss or your team absolutely is going to influence what it is that you're going to represent because they're all going to have different means different biases different vocabularies different time available to them to read what it is you're giving them and finally you as the designer are going to have a reason a motivation for putting something together for building a tool for building a visualization and that's going to obviously influence what it is that you're going to communicate so we're going to talk about these inputs first of all let's talk about you this really is all about you to begin with why are you here what are your goals of this visualization it might be that you're just exploring the data and that's fine perfectly legitimate but as soon as you go to communicate it with somebody else there's probably something in it that you want to show them because if there's nothing interesting in the data there's no reason to show somebody else right but as soon as you say oh I'm going to show take this down the hall to the person in the next cube the person over there and I'm going to share this with them there's usually a thing you want to communicate there's something of interest there you have to understand what that is what you want to get out of this interaction you're going to have with them if you don't know what it is that you're trying to communicate you have a very difficult time being successful in that communication that's pretty fundamental you have to know what it is you're trying to achieve if you want to be able to get there so the type of information product that you're creating is going to vary a little bit depending on the focus if if you are if you are doing the purely I'm going to present knowledge to somebody else it's really not about me as an editor it's not about my point of view or my bias so much I really just want to convey data to somebody else that's going to be this leg here on the left between the reader or your customer or your audience and the data you're providing basically an informative product if you do have a personal bias a point of view something that you want to advance a political agenda whatever it is and it's more about it's more about conveying that point of view maybe with using the data to support your point of view that's a little different that's going to be this persuasive arm the right leg of the triangle here where it's you and your relationship with the reader which is the motivating relationship finally if you kind of don't care about your reader or your audience or your customer and you're just doing cool things with the data that's the foundation of this triangle here we call that art and that's not a bad thing but be really clear that when you are when you are creating art or when you're choosing to exclude your customer or exclude your reader you're not expecting them to get the same thing out of it as maybe you put into it you're not expecting a specific message to be conveyed to them you're just having fun and there's some beautiful beautiful data art in the world I'm not saying this is a bad thing at all but I'm saying it's a different pursuit then either putting together knowledge that you want to convey to somebody or putting together an opinion or a point of view that you want to convey to somebody so your audience let's talk about them they have needs you need to satisfy those needs in fact if you don't satisfy those needs you are going to fail it doesn't matter how brilliant you are how magical the encoding is you choose are how great the layout is if your audience looks at it and says I don't get it it may or may not mean they're stupid and that's valid they can be stupid but you still failed if I'm up here and I've got this brilliant talk and I'm giving this brilliant talk to an audience and I'm speaking a language that none of you speak I'm not a very successful speaker regardless of how good my content is there we go so your success it's defined by the success of your customers people don't want a relationship with your brand they're not curious about your process necessarily they want to get their work done and if you can provide tools for your customer to get their work done that's going to make you look good that's pretty fundamental if you can't satisfy their needs if you can't help your audience be successful then you're the one who's failing and if they're doing it wrong if they are unable to achieve success of the tool that you're giving them you can blame them or you can fix the tool so see the earlier slide about iPhone and Microsoft so how do you make your users happy you actually have to understand who they are odds are very very good that they are not like you and that's a very difficult thing and that's been a very difficult thing to understand in technology and design for all all of eternity or at least all the 20th century where the people who have the tools and the insight to create new products to create technologies that are going to be used out in the world are not the people who are using them by and large there was a a famous sort of cranky exchange about two two-and-a-half years ago I think there's a CEO of 37signals so 37signals makes some of you know this stuff Basecamp and backpack and whatnot they make online tools for software development to manage your projects and this sort of thing and the CEO posted this sort of Randi blog post about I don't know why people care about user research and user experience like it's it seems all very unnecessary all you have to do is just design good things and then people use them and be happy and and really come on guys how hard is it and and and the entire user experience community kind of got their panties in a twist and wrote angry blog posts back saying you fool your blinds to the fact that you're in this privileged position where your customers are just like you you're a software developer who is writing tools for other software developers use of course you know what they're going to like you know what their priorities are you know what their biases are and and that's a very special position but most people are not designing tools for people who are just like them if you ever use two microwave oven if you've ever remember VCRs and tried to set the date on a VCR or a digital clock or anything it was designed by somebody and probably members in this audience oh yeah VCRs no problem I got that a canoe of my eyes closed but but most people in the world are not going to be successful in those domains and it's not because they're not smart it's because they're just not interested in spending those brain cells to learn how to program VCRs capably they have other things to do so if you want to be successful in regards to these audiences understanding them is very powerful and that includes political identity it include includes languages they speak the jargon that they use it includes how much time they have to understand what it is you're trying to present to them and includes whatever bias they bring to this conversation that you're having they're their sort of context they're their frame of use all of this matters enormous Lee will talk a little bit more about interpretations later on so just for this audience we can think of here's a variety of different audience members customers for your data or your product or your information all these audience members a data scientist a developer someone in marketing a potential investor remember the general public if you're talking about the company or the product that you're working on all these different members of the audience are going to have different parts of it that they're injured in right the investor probably cares about what you're like you know five-year plan is in terms of being profitable and remember of the general public wants to know how it's going to like help them be better data scientists might ask you know which statistical models were using to I mean everybody has a very different interest and you can give them all the same report or the same conversation about what it is you're doing but if you want to be successful you're going to tailor those communications to the different people you're talking to protip this also works well for humans the conversations you have with your mother with your partner with your children you're going to use different vocabulary with your boss it works better when you actually customize a communication for who that's for finally the third leg of the stool that's going to influence your design is the data itself the data has properties and the properties of the data are hugely going to influence how you want to represent that those data visually so going back to my lovely bike tire example here it's got a number of interesting properties it's got the wheel size so we measure with numbers but it's actually categorical right there's only a small number of different size wheels you can get for a bicycle so even if they are measured in numbers I can't just arbitrarily pick any number out of the whole set of numbers and say I want to bike wheel in that size there's three or four on the market that are relevant so that's that's a categorical slice of data the width of the tire on the other hand is pretty much continuous numerical anywhere from like nineteen to probably up into the 65 millimeter range or something right and if I want to go out and buy a 33 millimeter tire or a 26 millimeter tire or a forty seven millimeter tire somebody's going to make one about that wide so that's more or less continuous numeric data we're going to treat that very differently from the categorical data of how big is the wheel price probably also more or less continuous from you know 15 bucks for the cheapest tire you can buy up to probably I don't know how much expensive bike tires are more than 75 I'm sure so again that's continuous you can probably pick a sort of a price range and find some tires in that range and then the tire is going to have a couple more properties whether it's got some anti puncture treatments whether it's foldable and you can stick in your backpack or it's got to stay in the big size so these are more sort of binary properties and we could break them down a little bit but there's a variety of different flavors of data and you're going to use different properties to encode them ideally if you want to be successful which you do so we've had all this conversation about why we're here what our goals are we've understood who our audience is and what their needs are and what we have to do to support them so that we look good we've understood what our data is and how that's going to work and so now we can actually start designing I haven't talked at all about data formats yet putting anything visually on the page we're still all this has been data collection mode so we're gonna start by looking at our statement of goals so show the sales figures that's kind of okay it's not a great a great goal it's a little ambiguous doesn't give us much to work with is the problem with this so we're going to we're going to be to that and move to something a little more detailed so a goal like this show the sales figures per product per region for the last twelve quarters this is very specific and in fact what this is is a miniature specification for what it is that we're going to draw it tells us what data we want to include it tells us what relationships we want to have represented among that data it implicitly is telling us which data we're going to exclude because that data is not mentioned because maybe we've got a lot more data than is listed here it provides some boundaries around how much we want to use right so this is saying we're only going to look at twelve quarters we don't want to look at all of history we're only looking at this at the regional level maybe not the zip code level or the state level so we can exclude some information there and this is all very powerful both what you're including and excluding the problem with with that we get we get sucked into we get seduced into very often is we want to show all the data we've all been taught show our work show everything and it's cool if you've got more data right you've done you've done more interesting things you've got more to show the problem is more data is indistinguishable from noise if it's not the data your audience cares about and the more points you put on the page that aren't the ones they care about the longer it's going to take for them to find the ones that they do care about so reducing the amount of data that you represent is a really powerful technique to get to clarity sometimes slicing it and doing three different visualizations that are all related but different can be much more powerful than trying to lump all of that into one right you can have three clear graphs or one money one so a good clear statement of goals will help you make these choices filter what you're going to include and how you're going to represent the relationships in that visualization define the desired knowledge before you pick a structure this is kind of a fundamental again fundamental design skill whether you're writing software building a house whatever it is designing a data visualization you need to know what it is you're building before you pick the shape of it and here's an example so this is a series of donut graphs which are basically pie graphs but they're missing the part in the middle where you can compare angles and so all you're stuck with is a little arc lengths are on the outside as individual donuts go these actually are pretty okay because they they're showing a relatively small number of segments which is fine we're not looking for a ton of precision and and the differences in size are sufficient that it's pretty easy on any one donut to see what the rank is so this is different usage shares for email clients desktop mobile web mail so if we're only looking at one it's kind of okay and they've got the numbers there anyway so I can just look at the numbers if I really care for that level of precision so that's fine but it's not just showing us one little segment it's trying to show us how things change over time and in showing us how things change over time this is a miserable a miserable visualization because it's very difficult to compare those are clinks across many circles so if I say is the arc length in the May 2011 or the September 2010 donut larger well the easiest thing is just to go read the numbers off the side right and at that point forget the visualization you myzel just put the stuff in the spreadsheet and the problem here is not that these are numbers that are impossible to show changing over time instead this is a format that isn't about change over time it's about comparing slices of a whole one discrete whole and instead we're looking at seven of them what they wanted was the maybe aesthetically less interesting but much more practical line graph which is a great way to show data changing over time and now if I say you want to compare show me the month that had the highest usage of the webmail clients well it's right there it's obvious it takes no time at all to find that there's a very little comparison that needs to be done and that's because position that height is a property that's we're very good we're very very good at detecting and seeing subtle differences in its wired into the brain the software is there from birth it's not something we have to learn it's not something you can teach or unteach someone it's just there and so this is a format that leverages that it's not causing us to attempt to mentally transpose and compare little arc lengths which we don't have built in software to detect and so we've got a sort of brute force those this is built in and so by leveraging it you get a much more compelling a much more effective information product although it may not be as aesthetic and I think that was the issue with with what had been done in the previous example is they were told make something interesting and aesthetic rather than make something useful those are very different goals and you end up with three different products finally the topic of appropriate encodings this comes back to the data flavors I'm not going to go through these again but just a reminder the data itself is going to have different properties and this is very much like a database spec right you need to know what your different data is that you're building your containers for in this case you need to know what the data is that you're using visual properties to represent so is this can people more or less read this you have to be able to see all the details that I put the URL huge at the bottom here your intended to click on that URL or go to that's my blog and there's a PDF of this graph that's one page that you're supposed to print out and like tack up in your cube so this is your shortcut guide to selecting good visual encoding so the URL is complex diagrams comm slash properties I'm not the first person to draw this table or a table like this this one's one that I drew but there's been many versions of this historically the goal of this table is to you match up a data type in a visual property that represents that data type well and so what we've got is a couple of columns that discuss the properties of of the visual encoding two main factors of it whether it is naturally ordered in our brains and how many useful versions how many useful increments or useful variations of it can we actually perceive and work with and that combination is going to map very nicely to a data type so just to pick a couple off the top here position replacement isn't ordered in the brain yes absolutely and again this is not something that's learned it's not something that you can teach or unlearn absolute placement and things related to that like height and length deeply deeply hardwired into the brain and we can detect very very small differences if you put a page layout in front of somebody and they'll say how come that picture is two pixels further to the right than all the rest of them like people will see these tiny little increments and the resolution the difference that we can detect is very small and the number of useful variations is like how big is your screen and how high-rez is it right you can have a very high number of useful useful small incremental differences on a big enough screen with position moving down the chart a little bit something like angle yes angles are naturally ordered in the brain we can rank them but we're not as good at detecting fine differences right so on a clock five minute five minute differences no problem but one to two minute differences if the hands are just sort of arbitrarily floating out in space we might have a little bit of difficulty either comparing which ones which ones greater or or what the difference is so that one has maybe a medium number of useful values as opposed to position which has a much higher number of useful values the lower half of the graph sorry I'd lower half the table is visual properties that do not tend to be ordered in the brain so things like shape things like pattern and texture things like lines being solid versus - to be versus being dotted these are very good for things like category and relational encodings but not good for quantity or for ordinal encodings because they're not ranked in the brain naturally there's not a natural sequence or if you give someone a dashed line and a dotted line and say which one comes first that mapping doesn't exist we can we can make it up we can pretend but that's something has to be taught and has to be it's not something you can depend on every audience member just knowing so we'll come back to this there's one more example here that I'm going to put put you to think about for a little while and that's color so we're not talking about brightness we're not talking about saturation we're talking about color cycling through the rainbow and the thing that trips everybody up is color is not ordered and we'll talk about that in a little more depth but I'm going to prime your intellectual pumps with that right now so to map visual properties on to data properties is a little bit of a tricky thing and that's what the table is there to help you decode and and it was summed up so beautifully last summer by Moritz Stefan who's brilliant and and does the same sort of work this is this visual encoding and data visualization and he said position is everything and color is difficult I was like oh man like that's that's like half a visual encoding in six words so I've been using this slide encoding him since then because it's such a beautiful summation and we're going to unpack both of these because they're both really important and really powerful to understand well so position is everything if you look back at the chart which I've now communally moved off of you see that that position spatial placement can be used for quantitative things we can tell if something is twice as long or twice as far along a path as something else we can use it for ordered things ranked for second third we can use it for relational things things that are clumped or grouped together I categories like there's lots of things you can do a spatial placement it turns out that placement is more or less the most powerful visual property because you can use it to encode any kind of data effectively it Maps well to all of them that's why position is everything it's very powerful using it well is incredibly incredibly rich and useful so is anybody use Hipmunk ok so like everyone but the six of you who have raised your hands if you're planning airplane flights and you're not using Hipmunk you're causing yourself a lot of pain that you don't have to hit monk is like the best thing in the world and I do not say that lightly their interface is magic they are really really good at leveraging the visual design to convey knowledge so it's pretty simple it's like okay there's some flights and they they take off and they land and they're color-coded by airline and there's some prices and it's it's not actually exactly sort by price we'll talk about that in a second you'll notice that all these flights there's no numbers associated with the flights like there's a timeline across the top different time zones east and west coast because that's the I've got to Seattle to New York flights and those two time zones here but the flights are not listed there's no information about when they take off when they land how long the flight is you don't need that you can see which one's longer you can see which ones have a stopover which ones take off first which ones land first like it's all very clear so here's how much information is spatially encoded with no digits displayed about these flights now when you mouse over flight when you click on a flight you can get all the details that it's 847 that it takes off and that it's a two-hour and 16 min a flight the details exist they're there when you dig into it but they don't need to present that at the first level at the first level you care about well I don't want to get to the airport before a flight that you know is going to leave at 10 o'clock in the morning and so you can just mentally slice off those flights you can also see actually the fees they've got these there's a little black arrows at the top corners here and those are those basically act like theater curtains and as you pull those narrower it just chops off any flights that it touches so you can very quickly focus but but fundamentally what they've done is they've done great things with the placement with that horizontal placement the flights so you can very quickly and easily compare the duration whether it out there's a stopover time of day all the sort of thing in fact they've done such a good job with that that they've got room now to use their vertical access for something else entirely they don't have to sort for example by time of day vertically instead they have an access they call agony and that's a combination of time of day length of trip price number of stopovers all these things and it's a really really useful sorting actually and they have the luxury of adding this extra value because they've done such a good job with the rest of the encoding that they've got this whole extra access to use right there very cleverly leveraging that that vertical placement for something other than like time because it's already been well done so position well we'll talk about AXYZ quite a bit later no we'll talk about actually quite a bit now so actually Tsar great they give you information for free because all the shapes all the all the data points in your gallery are going to inherit properties from the axes so I just do this with the United States map and then I gave a talk in Canada and so I switched up the mapping and now it's kind of fun to have a Canadian map so first of all if I'm looking at these various shapes I could have lots of notes or text or something in them saying this one's more to the north and this one's kind of in the central and this one's in the south and more no I don't have to label them at all in that way the axes tell me that if it's up in that corner it's on the northerly end of that one axis and on the westerly end of that other axis and lo and behold I don't have to label that shape or that province with those properties it works the other way around too if I know that I'm looking for something in the east I can ignore the whole Western two-thirds of that map right I don't have to look at those the axes tell me this is where the thing I'm looking for exists and so you get information sort of both both both about the property sorry you get information about the properties of the object and also tells you which objects you're going to look for that have those properties so really powerful it breaks my heart every time I see something like a timeline that's got a really well-defined one strong axis and then the vertical axis is like we kind of don't care it just doesn't matter that there's an opportunity to convey knowledge that is being thrown away that's being wasted because somebody didn't take the time to actually use that vertical axis for something useful lack of axes is a problem perhaps you've all seen like social network graphs they look like this because there's not strong axes defined and I have a whole separate mini lecture that's going to come after the break with me ranting about how social network graphs are usually not very useful and not very interesting and some ideas about how that could be fixed I thought maybe that'd be good for this this group one more thing to remember axes are negotiable right so this is the London Underground map and it doesn't actually look like London I mean it kind of does but not really and it turns out that these tubes are not all straight lines if you get on the subway and you're going around corners stuff these corners are not represented well on the graph and that Subway's definitely you're not making these like 90-degree right angle turns but when you're on the subway you're on the car now and it pulls into a station what are your choices hmm stay on the train or get off at the station there's your only choices right you're on a one-dimensional track and so your choices basically exit now or maybe some other time at that level this map is fine this is a logical relationship of the subway system and yeah it's tweaked to to more or less represent London also because you need to know when you're near the zoo or the palace of the gardens or something but where hey you know where you walk when you get out of the station to get to the zoo entrance that's a different map this map is about the subway system and and the power the innovation of this map in the 20s 30s on embarrassin I know this hairy backs innovation because he was an electrical engineer and he was used to laying things out on circuit boards in these straight lines at 90 and 45 degree angles that abstraction that he brought to the map was very powerful because it led to a cleaner easier to understand map a map that was better suited to this goal of navigating the subway system which is a different goal than navigating London so point is actually czar negotiable if the axis that you have are not working for you you can tweak them you have to tell people that that's what you're doing but you can tweak them and make them more well suited to what you have to display part two of Maritza's two rules for encoding color is difficult so at first glance you say oh this map is very simple it makes perfect sense the Alps are much hotter than the rest of Europe turns out that's not the case so this is done all the time this is an elevation map using rainbow spectrum of colors but you'll see this on medical imaging right you'll see this on you know density of the rock underground or they're going to drill the new tunnel for that like this this is the sort of this artificial rainbow and Co encoding is used all kinds of places and the problem is most of the time I'm not going to say hundred percent of time but almost all the time it's it's a very difficult encoding to interpret it's a very misleading encoding and the problem here is that because color is not ordered in the brain you cannot depend on people understanding which color is greater or more important or more interesting than the other colors so I understand that it is a physical property in the world got the physics degree I understand the wavelength changes but that's not how we see it you can't ask somebody to say okay blue and orange which one comes first there's not a right answer there that you can depend on in this case if you say you know green and teal someone can kind of look in the you know look at the scale and try to figure it out and interpret that but it's not something that's well ordered in the mind it's not built in it's not built into the software that we use this has also got kind of a weird skew of where the rainbow starts in it goes from red to purple and there's just some things that are very difficult about this the other thing that they've done wrong about this by the way is that vertical axis the high altitude is at the bottom of the scale and the low altitude is at the top of the scale and that's like you want your mappings to sort of approach approximating reality where you can that's a situation where they could have made it match reality a little more closely so colors not ordered we've ranted about that so here's a much better way to map things that are ordered things that are sequenced instead of changing through the rainbow spectrum you change either saturation or you change intensity and what that gets you is an ordinal ranking and this is built into the brain this you can depend on someone will say yes that is definitely darker than the other one now I can't tell you how much it's not something that we have quantitative analysis with I can't say that's twice as bright as that we're not very good at that but we're very very good at saying this is darker this is lighter this is more saturated this is less saturated so this is an elevation map done almost right except they again similarly have the the vertical axis inverted excuse me my throat's a little sore up and talking for two and half days okay here's a a permissible perhaps the only permissible use of rainbow as an encoding this is it this is a temperature map and I say this is permissible for a couple of reasons one of which is we have really really strong cultural conventions around blue being cold around red being warm and green being kind of Pleasant and in the middle and because those conventions exist because they're very strong and because this rainbow spectrum is cycled through to the point where the end points are it's sort of a rational point that matches up you notice there's no real purple in this right that's left out because we don't like what temperature is purple we don't have that so that's left out of this and so this is a scale that actually Maps relatively well to what our cultural conventions around temperature are the other thing that they did right on this map is that they've got the higher numbers or higher up on the scale and the lower numbers are lower on the scale and they get a gold star for that the other map the precipitation map pardon me is is actually pretty well done in terms of where there's a lot of precipitation up in Seattle where I live it's darker and whether it's very little down through the southwest that bright green is almost entirely absent the wonderful aw I would say with this map is that sort of slate background is near enough in some ways to the green that it kind of is difficult to tell when the green is faded completely away and I think a background color that was white or some other color where there's a higher contrast would would reveal a little better when there was no color at all but that one's actually pretty good and that's typically what you want to do when you're looking at intensity or something whether it's a heat map or a you know you're accumulating something as you do it with with a gradient along a single a single color axis color is also very meaningful there's an example that I use sometimes of a coursework options for a it was like an urban design urban planning coursework at a university in Australia and all of the courses are this sort of pale blue or pale pink color and everyone's first thought is like other different classes for different genders like what's that about and then they realize that like the one classes are Co shared with one other department in the other class like it's something that has nothing to do with gender whatsoever and I don't know why they chose pink and blue but unfortunately what that hat what happens is there's an implication there and people recognize those colors and say I know what it means when we put these colors together and then they try to interpret what they're looking at through that lens and then they realize they're wrong and they have to unlearn something and you've just wasted all this time and all this processing power of your potential customer causing them to think they understand realize that understand be confused and then actually try to go back and learn what it is they're they're trying to understand so you want to avoid the situation you want to make things as easy for your audience as you can and the problem with that when it comes to color is that there are so many potential associations with with single colors with permutations of colors when they're put together that you can color can have a lot of meaning if you if that you don't mean to so be careful walking through the Irish neighborhood on st. Patrick's Day wearing an orange shirt right if you if you're traveling in the Middle East green and white means very something very different for green and blue black and white and grey for for morality right like there's all these these things that that have meaning to some people in some situations that do not have meaning to everybody in all situations and so if you either depend on there being a meaning in that combination of colors or you depend on there not being a meaning in that combination of colors you're going to get tripped up somewhere one example here was I heard of recently junior developer and they said you're gonna you're gonna build the dashboard for our system monitor so the dev goes off and comes back a while later and brings up the dashboard and says here it is and then brings up the dashboard and everything is red and the managers like is the dashboard broken our systems down like red that's bad right and the devas what red means good luck right everything's good total cultural mismatch you can't depend on that color being interpreted in a consistent way so be careful code is difficult it's really pretty though people want to use color but it's difficult to get right use defaults defaults are really powerful and the reason that we have defaults and the reason that defaults are powerful is because it means you don't have to teach a new language it means you're using a language that your audience or your customer already knows about and they can make some assumptions and if you using your defaults well that those assumptions are substantiated by what you're using and you end up with a shortcut an intellectual shortcut in terms of getting data into their mind because they understand the mental model you're dealing with so here's a quick one this is a map from the New York Times of electoral college results from a 2008 election and it's a it's a perfectly good map there's nothing wrong with this map it's really good if you want to navigate it's really good if you're talking about real estate or surface area it's kind of a lousy representation of influence in the electoral college though because what you end up with is you end up with these these large states Montana Wyoming North Dakota South Dakota Idaho where basically nobody lives like these are states for those five states have the minimum number of electoral college representatives because they're very very sparsely populated you can't have fewer representatives and they have on the other end of the country you have this tiny little peanut here kind of wedged in between New York and Pennsylvania it's too small to have a label that's the state of New Jersey by itself it has 15 electoral votes the surface area ratio is something like 55 to 1 for those 5 states versus New Jersey but they're almost on par right 15 to 16 votes in terms of electoral college and it turns out as a default map this is a lousy map for conveying electoral influence but it's a very familiar map like it's a map what's wrong with it it's it's and and so the New York Times did something very clever they made an alternate version and you can click that little that little label down off the coast of South Carolina there and you get this map and this is a map where each state is not represented by its geographical size but is built out of squares that represent the number of electoral college vote that that state has and so this is not a map of geography it's not a map to navigate it's a map of electoral influence and what happens is those enormous states like Montana that very few votes become these tiny tiny little you know two and three and four pixel wide boxes and New Jersey which was that tiny little peanut before is actually a pretty large powerhouse state here on the East Coast and now that comparison of 16 to 15 electoral votes becomes much more accurately represented and in fact when you're looking at the results of the election this actually shows more blue than red whereas if you're looking at the results of the election based on real estate based on surface area it's much more difficult to tell where the ratio of red and blue is and in fact I suspect that there's still quite a bit more red on this map than blue even though it was a pretty strong Democratic victory in 2008 so so the lesson here is use defaults unless you've got a better answer right unless you've got something that conveys what's interesting or useful about your data in a way that's more compelling than the default format can reveal yes not talk about the red and blue that's fine I'm talking about the default in this case use of a geographically accurate map rather than an electoral map so if you use Excel for graphing anything all the defaults are wrong like I'm not joking the default colors are wrong the labels are wrong the fact that it's in 3d is wrong like the fonts like everything about the defaults in Excel are wrong because they've never put any care into it they've never thought about how to make this actually generate a more useful information product they've made them 3d now which is wrong but they haven't actually made them more useful and so whether it's the software default whether it's using a map because we're doing geo data as the default I gave a talk yesterday and went not to use maps at a Geo conference because there's a lot of times people say we've got geo put it on a map there might be more interesting relationships that don't actually include that map right so it's really again this goes back to understanding what you're trying to convey who it's for and and what are the relationships in the data that are most interesting and maybe geography is not one of them right incidentally I've got a handful of books here that are gonna be going to people to ask good questions yes yes yeah great question the question is you have a rule of thumb for the trade offs and in terms of using a default versus using something innovative innovation is expensive intellectually right if you're going to innovate you have to invent a new language and you have to teach that language and then you can once you've thought of this new language invented it and taught it's your audience then you can start talking about your content if you have a default in place you've already got that shared understanding and you can talk about your content immediately so there's there's a higher barrier to success if you're using something innovative and so again the question is what how much payoff is there and I don't have a standard rule of thumb for that in that case absolutely that map benefits by being a permutation of an existing thing that we're very familiar with and by being literally shown like like on the screen you click the one button it shows the one you click the other button and it morphs back and forth so that when there's a low cost relatively because it's in a context where it's very easy for us to understand what that new map means but there definitely are situations where an innovative representation does not necessarily provide more value or the cost is too high and you know that depends a lot on your audience right if you're if you are bringing some new knowledge to fighter pilots or brain surgeons and you know they have to learn it it's okay to make it more difficult because once they learn it they'll become expert users and it may be something they use for their whole career if it's someone who's like flipping through pages of a magazine and you've got about four seconds for them to say yeah I get it or I this doesn't make sense flip the page it's a very different challenge right and so again the level of attention that your audience has to expend on understanding matters so the payoff may be very very high but there may be a higher barrier to them understanding as well so the answer is it depends I understand that's unsatisfying so we're almost wrapping up the first half of talk here we'll take some questions a little break and then we'll dive into the second half so just here's the quick summary there's some design strategies for successful visualizations limit the detail you include limited to the stuff that actually matters right use position for your most important relationships try to axes think about what if this wasn't a time-series what if this wasn't geography what if we what if we only show we show it from most important to least important we don't care what the time was we don't care where it took place that kind of thing play with your accents do some different things yeah is this deck going to be available where people can be able to download this yeah I'm happy to supply it just I see people like madly scribbling these things downs like we'll give you the PDF you don't have to write this all down so consider defaults and and and I use the word consider I don't say use default I say consider because sometimes defaults are going to be the right answer and sometimes they're not use color for the categories colors are great for separate categories but don't assume that people are going to be able get rankings out of them excuse me and use that table for successfully encoding other data properties so here's the table again just a very very quick overview of just a small handful there's many many many data visualization tools out there right now d3 is a JavaScript library that's excellent written by Mike Bostock who who wrote of his who's involved he came out of the Stanford visualization group and was involved in I think previews and flair came out of that group so very very smart guy d3 is a very flexible very capable very well supported JavaScript library that's getting a ton of traction and it's it's this sort of in this magical sweet spot where provides excellent structure and infrastructure for you to graph what you want and then gives you infinite flexibility with how to represent it it's really nice processing is a little less structured and sort of a little more freeform in what it is able to represent easily it's been described as a sketch pad for data and so some really beautiful data art that you may have seen in the world like the Aaron Copland's flight path Maps just showing all different flights flying from all the airport's across 24 hours was done in processing and it's a really good language for sort of a little more creative a little more flexible work with data if you're doing any kind of statistical analysis are as an open source stats package and weekly hat Headly Wickham has written a plug-in called ggplot2 for doing graphing and visual presentation as an our library and people seem to think that's that's really good my understanding I've not used it my understanding is that it generates stuff that is that is very good but it doesn't look it's not like presentation quality and then you want to slip it into another tool and tweak it and polish up to look a little bit so those are all more or less code based free tools open source tableau is a commercial tool it's Windows only desktop tool there is a free version of it called tableau public which is somewhat limited in that your data is all public and they store it for you you don't have anything privately locally but tableau is a really good tool like if you ever wished Excel was more capable and had better defaults and was a little bit less frustrating tableau is a really good tool it has its own moments of frustration but it's really really nice from my home town in Seattle and it's the it's the analytics and data visualization tool that a lot of people using for for dashboards for interactive infographics in journalistic pieces you'll see this in a friend of mine sent a number of visualizations that have been picked up by the Washington Post in terms of campaign contributions or any number of things it's got maps plugins it's really great tool so that's the end of the first half so let's take ten minutes let's take ten minutes and come back yeah yes let's do that all right so don't leave yet we'll take questions next segments I've got a little bit of a boil downside of series of prompts which are the questions you want to ask yourself as you're as you're going through design process so we can just run through those really quickly if we had more of a workshop kind of thing I'd actually have you like take out pen and paper and do that so we've got a little a little run-through of those I've got a sort of a little mini lecture doesn't slides or so on social networks and social network visualization which I haven't seen done very well yet ever for the most part and finally another stack of just a whole bunch of visualizations and I mean I can talk about this stuff all night till my voice gives out but we'll use for questions now there'll be time for questions later so let's do a brief round of questions and then have a couple minute break and then resume if you want to come back question go ahead oh yeah if you want to walk over the orc is I can repeat it into this very sensitive mic that's attached to me the data right yep one leg of the triangle yes I'm going to actually walk all the way through to that slide because it actually is going to be easier to have this conversation that one so the question was there's these three legs the designer of the reader and the data that influenced what it is that you're creating and two of these are human one of these is not and is there necessarily bias is there necessarily persuasion in a visualization the answer is partially right because anything that human designs is gonna have some choices that were made some editorial choices and that's necessarily going to affect the outcome now whether those choices are intended to add bias right so whether it's this this right hand leg here where it's less about the data and kind of more about you talking directly to the audience that's a more biased or a more more persuasive angle on things the other side of it where it's really about the reader and their connection to the data it's really on that informative side and you're doing as much as you can to avoid bias so you're you're using graph best practices you're showing the zero on your y-axis you're citing the source of your data you're not intending to mislead and trying to adhere to that it's more about the data and it's less about your personal opinion or your representation so yes anything made by humans going to have some bias and yes there are ways that you can minimize that and it's a good question it's a tough question it's a question that comes up with particularly in the political season with what is the source of that data how much of that is truth and how much of that is spin anytime somebody tells you they're not spinning it they probably are right it's complicated but absolutely efforts can be made to minimize that if your goal is not to be biased question here could you flip to the Canada map for a second so I have two questions so one is um I really like that plot I think those colors were selected they're very vibrant I didn't pick those colors for the record okay well whomever it was that picked those colors I think they did a good job and so I'd like to know what are good rules of thumb to try and understand what the right palette is to color a plot like that such that it does appear very vibrant like that another question I have is how do you also do this while taking into account that some people are colorblind and they might not be able to see a yes a green and a red yes to each other in differentiate between them both great questions the best tool for picking a color palette is a website called color Brewer - and I should put that on one of these slides and color it's magic you say I need I need five different colors and I need them to be divergent on one axis you know going up and down from a zero point or I need them to be separate in color space or it actually was designed exactly for this for mapping and you can pick the palette of blues or the palette of greens or the the blue orange accent like you can just pick so the answer is go to that website and do it there is also a standard palette of 12 colors that are very separate from each other in color space and are also very separate in terms of naming so you don't run into questions where you say it's the light blue and they say do mean the teal or do you mean the sky blue or the robin's egg blue right that list is as in this book and as in other places online and basically colors that have been chosen because they are distinct enough that most people can can differentiate them visually pretty easily code blindness is absolutely a thing particularly with red-green which is a pairing that people want to use all the time there are ways that you can choose shades of red and green so that they are a little bit less likely to collide I don't know if it's happening down here but where I live the the green traffic lights are being replaced with ones that are a little more blue they've got a little bit of turquoise of a hue going on and you don't think about it but when you stop and look they're a little more blue and apparently that's separate enough from the red that people who are red green can can see that that's got some some blue in it so I'm pretty sure that color blue color Brewer has palette choices that you can say is colorblind safe I know that tableau has a colorblind safe palette and there's also mean there's other ones there there are like red yellow color blinds and other ones you have to watch out for but those are even smaller fractions red green I think's about 7% of the population so use good tools use standard references is the answer and then do you think um textures at all could be useful to fulfill the same function if you have to have something in black and white yeah absolutely textures are a pretty good analog for color in terms of they're there you can map like texture density to intensity but if you but if you're not really trying for that it's pretty it's it's pretty good to be able to use separate textures if you're going to be like mimeo graphing something right or black and white photocopying color is not going to work texture is good for that for sure and you'll see on the on the visualization properties table they're actually very similar and what they're good for not too not too many occasions these days where we have to design for black and white but yeah textures are really good one for that actually and you'll see like all the old in newspapers normal text books it's you know this is the styling with the dots and this is the little left cross hashes and this little right crossed hashes and this is the vertical stripes and that's it's a very old technique it's been done for a long time I'll have two questions the first one is a complete question about the slide about desktop mobile and the tab and tablet can go to the light yes this one suppose I have like 20 devices like I want different between different devices like iPhone like a ipad samsung tablet so I have 20 of them should I also salams them together at 20 curves because then it looks very confusing it's a better way of visualizing it that's the concrete question a modern question is how to visualize high dimensional data which data high dimensional data meaning this basically is actually one dimensional data you know if you have like the Thai exempt we have different features like the with the the categorical with standard I don't remember the other I think where this puncture yeah and foldability normal to visualize basically it's for exploration perspectives because I think the first part of the talk and maybe focus on presentation yeah just some comments on visualizing height image data for exploration purposes sure sort of answer your first question in terms of how would you if you have a mobile phone if you have an iPad that's kind of arbitrary how you choose to slice that like this iPhone mobile or whatever or is the iPad you get to decide you're the designer so you get to pick that definition for the second half second second question high dimensional data that's a really good question that's a hard problem and the strategies I usually take are you can get three or four pretty easily onto one graph if you're doing things like an X and a Y and maybe something like a size and then a color for category a lot of bubble graphs you'll see that and that's and that's not too difficult to get beyond that usually I say slice the data right so you might have several graphs that are related but not the same so they might share an x-axis and then you have different different categories of the data or or different slices so that it's it's spread out in space so you're not trying to look at all of those properties all at once and you might see you might see some different trends on one graph that is not apparent on another graph because they're sliced different ways that's tricky and it depends the choice of how to slice it is going to depend a lot on what is the data that you have and so my advice when when slicing it is if there are categories or other ways that I know are very discrete very separate from each other and I can slice along those lines then you can sort of see them separately and you might see that each of them has a trend within it but they may not have correlations with the other you might also want to slice it in two dimensions so that you can see the trends this way in that way so across these categories and across those categories just to make sure you cover your basis high one just as a follow-up for the color question color Brewer actually has a really nice are package so if you're already nice a our package yeah our color Brewer okay so it sort of just does stuff magically for you which is nice my question is you talked about sort of the tension between exploration and explanation yes when you're designing visualization and so it's clear you know when you're doing this sort of professionally that it's a life cycle of creating the particular visualization right like you don't just get a data set and produce it and there's some kind of iteration and there's a point at which you sort of transform between the exploration step and the explanation step when you find that a thing that you want to show so yeah I guess part of that must be an art but if you can talk about some of your process that goes around when you when you're doing that what so to briefly summarize for the recording how do you how do you transition from exploration or the analysis phase to the presentation phase right and I guess there's I guess there's either a predefined definition if somebody says go find for me what the product that lost the most money was and so you go and you play with the data and then you come with an answer you say aha here it is and you bring that back to show it so you've got a sort of a predefined boundary where you know what the success condition is and then you go find it then you're and you win or the other end of the spectrum where you're looking and you're looking at you're looking and you see something interesting expected or not and you say that is noteworthy enough that I want to share it it's remarkable right it's worth working about and and I'm going to take that piece that thing that I found now I might want to dig deeper and see what the correlations and the causes are but once you once you've found that that interesting a little a little bit of gold in your sand then you've got a story to tell you've got something to share or at least you've got a punchline at the end of it right you may not have the whole backstory yet but it's whether or not that you had that coming in or whether not it was something you discovered in both situations you have something you want to share you have something relevant that's actually worth saying hey I need to take some of your time I want to show you this thing and then you're going to craft that information deliverable that is that is separate from all the noise of the analysis that you can do it Thanks yeah hi I'm the bigger data machine learning field AHA all the visualization do you deal with graph dB Maggie be helped a lot or you really don't need a troublemaker I am not familiar with Graf DB I'm sorry it's not a tool I know so I don't have I don't have opinions about it because a lot of people in right now especially in big data is a very hot chatter deal with personal result persistent with Graf TV mm-hmm but does they go back like a deal with the high dimensional data consistent with graph DB still with a bigger capacity a lot of other issues so yeah I'm sorry I haven't used it so I don't have so when you deal with bigger data what can tool do you use so for persistent Oh what school finishing ah d3 is pretty good for visualizing large data sets it's got pretty good performance it's JavaScript based some of the other tools tableau for example is working on maybe they already have this ability to point to a Hadoop data set so you can sort of take arbitrarily large data and work with it there I don't know enough about the real big data representation tools I mostly have worked with stuff that's little bit smaller scale so thanks yeah enjoin a question here in the front you been waiting I'm sorry so the question is one potential lesson from the conversation about colors being difficult is should I be suspicious of legends that are that are trying to do more than factually represent magnitude and is that justified I think the best thing that you can that you can leverage is some critical analysis and look at what's being represented and look at how it's being represented and how it's being marketed how it's being promoted as this is showing you a thing and see if it makes sense to you if the level of granularity if they have citing their sources that kind of thing I'm trying to think of a good example of one or the other case and I'm not going with anything off the top of my head but if you've got examples in mind of legends or whatever I mean certainly there's any number of things that you can encode whether its category whether it's magnitude whether it's relationship right and those are all valid things to encode and that would things to put into a legend I guess I would be suspicious when you see color particularly used for quantitative things or ranked things and there's going to be some conventions blue ribbon red ribbon white ribbon or gold medal silver medal bronze medal right but those are those are categorical conventions that's utterly about the colors themselves yeah when we're here then in the mic it's a good question the question is how do you represent things like date and a calendar the short version is maybe dates aren't the way to organize it long answer is finding at the breaker afterwards and like get at a pan and we'll do that next question here in your example for Hipmunk you provided that how data visualization is really helping in presenting a lot of data elements in in a particular beautiful graph in here so is there a thumb rule for the number of data elements that should be presented I mean is there a maximum data element said that should be present in a particular graph that you have or visualization you're doing and we should not exceed that because in many cases like sales example which you gave the executives wants to look for a set of data elements so is there a limit or a thumb rule which you can help us with yeah so the question is there a limit to the number of data elements or data dimensions that you should represent three and four is very common six and seven can be done well though it takes some effort but I've definitely seen them done well I naively did some in grad school I had one of my grad school students do a very nice design so I'm not going to say that there's a limit you should never do more than four or never do more than six it gets increasingly challenging to add another dimension of data because you've probably already used a lot of your visual encodings and so to introduce another dimension of data you have to pick another encoding that is going to be not not conflicting with you've already got so you might not want to use color a second time for example but you could maybe if you know so I think with care you could you could do more than that I'm trying to think of examples I don't think I can think of one more than about seven I would probably stop there and and again it gets a little more challenging but it can totally be done yeah good question there are there are business cards on the back table here for this book designing data visualizations and my other O'Reilly book which is called beautiful visualizations this book is the process this book is kind of how to do it and it follows the arc of the talk pretty closely the other book is like 400 beautiful color pages of examples of all different it's 20 chapters in by 20 different teams or individuals case studies of different visualizations that they've done there's a discount code on the back of the cards it's the same code on both cards so you don't need both cards the code will work for both books you have to buy it directly from O'Reilly but if you want the books that's a way to get them cheap so that's on the back table and I've got some up here so and there's at least one copy of each book floating around LinkedIn somewhere and a few copies of the books now in the audience it's not too late by the way there's still two more books for good questions alright so this is a like I said a little rant visualizations of social networks mostly because we're LinkedIn and it's a topic that I've been ranting about for a long time and it's nice to have a audience that I can rant about this particular topic too so this is not just social networks this is any kind of networks so networks of people or other things networks obviously made of nodes and links networks of people the nodes are just people sort of like so on the green so social networks how to do it right know your data this is remarkably absent I think from a lot of the social network representations that I've seen where you end up with a map and like okay so people know some people but I know that they know each other right like we all went to school together of course they know each other this is not very interesting this is kind of the lowest common denominator visualization and when there's more interesting data available and we know a lot about the people that are in our datasets typically when there's more interesting data available it's sort of unfortunate that that's not being leveraged to build anything interesting so there's all kinds of engine things anybody here at Facebook don't have to be shy so so about four years ago when I signed up for Facebook I would say I know Bob and it would say how do you know him and I'd say we met through a friend and it would say which friend I'd say my friend Jane and it would say when was that and you'd say we went to school together we had a class together we dated oh you still together did you break up but we broke up are you still in good terms not like it had this whole questionnaire for every person you add it now I'm sure that data isn't in a backup somewhere it doesn't prompt you like that anymore but like that's a really rich data set and the visualizations that you could make and of course everybody at some point or other has either seen or attempted to draw like the network of their friends or the characters in the TV show who dated or whatever there's a couple of those actually in this deck that's a really interesting data set and so who knows who is I find not very interesting it's minimal like that's that's the least you can show but all these other questions of what kind of relationship do they have what groups where the geographical affinities that are different from the logical affinities who spans groups that they don't they don't necessarily belong to like if there's somebody who knows somebody in in another group all kinds of instinct questions that could be asked and answered visually that tends to not be and I think there's great potential there for people to to do interesting work with that and again the bottom line what are the trends gaps and outliers which is to say what are the patterns which is fine and what are the pattern violations which is interesting that we want to be able to see so start with knowing your data and what's there and what's possible second part once you know that tell a story this isn't interesting this is a and ruin this is from a chapter in beautiful visualization he's talking about Senate voting patterns and the connections I forget exactly what the threshold is but there's a connection between two senators when they vote similarly on some some fraction of the other Senate votes and you can see that for one hundred and second Senate there was two large partisan clumps there was a few outliers like Jesse Helms who are very very far in this case to the right and there's some very interesting people in the middle Arlen Specter richard shelby bob packwood from oregon who was a remarkably consistent republican consistent in his moral stance which is to say he was a republican he was against abortion he was also against the death penalty and also against the vietnam war so very moderate you know in a way that we don't tend to see anymore and so positions like that put him in the middle between these parties so this is not just which senators on which party or were they from but this tells you something about how they are behaving and then this is the continuing the continuation every two years as the new senators come in and it shows the increasing partisanship in some years and then the diminishing diminishing partisanship in some other years of the senate as a body so this is a very interesting story that's being told and you can you can correlate i don't remember exactly which these years are i apologized I think 108 Senate which you can see probably the most partisan v's was the year that Bill Clinton was in office and there was the whole Lewinsky scandal and all that and it was incredibly partisan time I think this is the Newt Gingrich young revolutionaries kind of a year and then things of course drifted to be a little less partisan over the next couple Senate so fascinating stories that can be told here this is not a ton of pixels this is not a ton of deep math to get there but it's really interesting when you talk about what was going on and put this in a context of what was happening over the course of history at that period of time so good stories available if you present them well and then finally encode usefully and then here's the funny examples I don't know what the show is people seem to know what this is some of them but I saw this and said this is a layout there's choices that have been made here that are optimized for aesthetics but not optimized to actually show what's going on so first of all there's a couple of different kinds of relationships here but these are mostly sort of the interpersonal dynamic and so that's fine but the layout makes it very to see where something interesting is happening so I abstracted these a little bit so the upper half is basically the reproduction of what we saw the lower half I've sorted the relationship types where that whatever that red stripe is relationship is now on the top and the non red relationships are on the bottom and so we can group those a little bit if we want to look at one or the other kind of interaction these characters are having but I still find that this is not a very useful not a very useful way of representing these things so I took one more stab at the layout and this is what I came up with and some very interesting things emerge when you can look at it like this now you can very quickly say which characters are more heavily involved you can you can probably pick which characters the most central characters in the show because they're most highly connected right the characters that have connections to every single other or almost every single other character are the more important characters than the ones that are only connected to two or three other characters instead of two four or five other characters great potential they're sort of lost or the original layout here's another encoding fail although the layout is not particularly useful as well and I point that out because there's some longer lines and some crossings that I just don't think need to be there you could just put these in proximity and save yourself some ink but the other issue with this is we're using color to encode all different kinds of relationships right some of these are family some of these are whether or not they have a romantic relationship some of these are whether they were married which is sort of a legal thing which may or may not have any other kind of romantic implication or not and and so there's color use for all of them but we've got other properties and lines you could have dotted lines you could have dashed lines you could have thick and thin lines you can have lines that are weighty versus straight you can have arrows on the ends of lines if you have something like a crush or unrequited love and in fact if you look at a standard family trees there's a whole language in Jena grams there's a whole language of visually representing on lines things like divorces things like who's alive and who's dead full siblings versus half siblings all sort of the the Ginetta relationships and then there's a whole nother layering of that in terms of the emotional relationships whether it is it is close or as distant whether it is positive or negative whether there's been abuse of some kind or other all those visual representations exist there is a language although it's not a very commonly used one and again this is sort of this this this failing here is this lowest common denominator where I can't even tell I can't even say color means this was a romantic thing and a dotted or dashed line means family right I have to go back and look at the at the at the legends to understand what every one of these means and there's there's nothing that the encoding itself tells me about the flavor of data it is used to represent so we're sort of overloading color by using it to represent many different dimensions of data rather than simply one here's a very different layout this from a website called they rule which is a website for exploring political influence of individuals and corporations and the flow of money and you can't see I apologize that table that's the board right people who sit on the board that middle one is JP Morgan Chase this is a few years old now there are some board members around that table they sit on the next outer ring of boards and then there are other shared board members and and what this is showing is that there's a very small network a very intimate network of a relatively small number of individuals who sit on the boards of a very high number very influential companies and that's so if you have the fortune 500 you don't have 500 unique non-overlapping boards of directors you have a very small number of individuals relative to this number of companies and these very small number of individuals actually wield a disproportionate amount of political power so that's what the graph is showing and it could have done just something like a force directed graph where this was just sort of allowed to become one of those knots kind of like if you look at just one of the Senate parties where there was no real meaning to the spatial placement this doesn't really have an XY axis but it has a radial axis that makes quite a bit of difference where the center is this this JPMorgan Chase Board and then the further you get from that you're further from that particular locus of influence but but it gives some nice arrangement so that it's easy to understand what where some of these players fall in the and scheme of things so I just thought that was a nice use of encoding to sort of tell the story and a little bit more clarity question I'm sorry for the win yes intentional layout for the win there they they're improving the quality of their presentation by having some intentionality and how they've chosen to represent this so if you're designing social network graphs or graphs of any other kind of network use axes man make your placement count it's really powerful use it to actually convey some knowledge rather than just doing an arbitrary force directed graph and letting it go with that so here's the process it's time to visualize something you've got a database you've got a dump you've got a blank sheet of paper in front of you what do you do how do you start this is a hard process I wrote this book because I don't know how to do it so this is this is the process that you go through to actually get something onto the page like there's nothing in the world more intimidating than a blank sheet of paper right or a blank screen and like I find my creativity doesn't sort of just originate spontaneously I have to go through a little bit of process to get things flowing so this is the process so the question is what do you want to show but that's not really specific enough really the question is what questions are you trying to answer and that's pretty good because that points you to some kind of data that you can actually supply but really what you're talking about here is what actions or decisions are you trying to enable and that's going to lead you to questions and that's going to lead you to what you want to show and so when you're looking at data and saying well which one of these do I put on the page that's a hard question but if you're saying what decision am I trying to make with this data where to invest money which food processor to buy which bike tire these kind of thing that will help you by informing you by telling you which data matters and which ones to put on the page who's consuming the data and what are their needs these are fundamental if the person who's consuming the data is not you you have to think about them a little bit you have to do some research potentially to understand what it is that they're going to need for them to be happy so what are their priorities what are their biases what are their limitations limitations again is like how much resource do they have how much time do they have to look at these what don't you know about them this is sort of a hard meta question but there may be things you you know very specifically I actually don't know what desktop OS they're running that might make a big difference I don't know whether they're on mobile or they're on desktop that might make a big difference in how I present this data so these are things that if you're aware you don't have a good answer to you can either go find that or you can you know design something that works in either situation what data dimensions do you have to play with so this is things like the tire size whether it's puncture resistant price all those kind of things and this gets back again like I said a little bit towards something that looks a lot like a database spec you've got the different fields and the different properties of the fields so what types of data do I have once I've listed the data that I have what kind of data is it is it categorical is it ordinal is it quantitative is it relational and then location location is complicated right because location could sort of be ordinal it could be ranked or not ranked it's probably categorical at least it could be positional if you're plotting an XY so location is complicated but it's another flavor of data what are the key relationships probably I say probably because often the first instinct is a good instinct but it's not right and so you think you know what's interesting or you think you know it's important and you go in and you're going to answer this question and you get there and there's nothing interesting there at all and you on the way sort of saw this one other thing that caught your attention and so you start to go chase that down and the relationship that you thought was interesting is really boring but this other thing was emergent and you didn't know about it beforehand so the key relationships are the ones you think are probably going to be key but you can't be sure once you've identified what you think the key relationships are what's the data that you need to represent those from the set right because you've got a lot of data and you're going to pick a smaller fraction of it to represent visually what's required for that and then so here's an example statement right so the relationship between a and B potentially expanding to C across X or across X&Y from M to n so you've got some data you've got some boundaries you've got some relationships and which what are you actually going to do with that data right what what is the data points that are actually called for in that spec so this spec needs a and B or maybe C and need some X it needs M to n so that's told us this is the fraction this is the subset of all the data in the world and I'm going to look at today for this project right now we can get rid of the other data or at least set it aside for the moment so now we're getting into actually putting stuff on the page how can you use position to reveal the key relationships so I gave a talk yesterday at the where conference which is all about maps and geo about when not to use maps and I gave that talk because it's really just like it's easy to just push the graph button in Excel or make a pie graph it's really easy to say we've got geo data what's plotted on the map and I put up a couple of really interesting examples of of location data or data that had location as an aspect to it where location was not the most important thing it was not the most interesting thing and if we'd use two dimensions of map data two dimensions of spatial placement to put the data on a map that's those two dimensions that I can't use for other things and instead there was an example from New York Times that has timeline and then a vertical grouping is a category and that was really powerful and really interesting and there was a color code for the continent that was fine we didn't need to like zoom into the city where something took place that was less interesting than the place in history and this category so understanding what the key relationship is thinking about ways you can place it and is there a good default format or not for this kind of relationship if there's a good default format consider it don't have to buy it put a 30-year mortgage on it but rent it maybe see if you like it try some other things to list some different combinations of axes they might give you different kind of insights list some that you're pretty sure are not going to work and plot them out anyway just for fun just to see what you get because by definition the data or the relationship that you don't know exists you're not going to be able to plan a format that it's going to reveal right you're going to stumble onto it accidentally so list some different things some versions once you've decided more or less what your axes are going to be what your spatial placement is going to represent then you want to do things like pick the other varieties of encoding that you're going to use for the other data factors so these are maybe not your most important dimensions not your key relationship but some related ones that you'd like to then put on the page and pick good encodings for that again using something like that table see how it looks maybe it's terrible maybe you maybe use size when you sort of use shape or you use shape when you should use color mix it up try something different I've been doing this a long time and my first instinct is usually not right anyway it's close right but but the data in my mind is not the data on the page and I put it on the page and and it looks different than I had imagined it it's not as satisfying this yeah it's not actually gonna work the way I wanted I'm going to mix it up a little bit it's it's way more valuable than I thought or it's not nearly as variable as I thought let's do this differently so at the end of the day iterate a lot show some people ask for advice take it down the hall to someone who knows nothing about your data and say what do you think of this is this makes sense and like that's like the super budget usability testing right just get opinions from somebody who doesn't already have an opinion about it and see what they think of what you've got that's all I got if there's more questions or thoughts or comments we can dive into that or for heaven's sakes it's after 6:00 you can go home and that's fine too but there's still two books to give away if there's more good questions thank you question yeah aha is there a good way of going about testing that right so you mentioned iterate and go yeah halt and ask you know somebody hasn't seen it but you'd think that the people who made those went through to some of those steps themselves so is there a good prescriptive way to go about testing visualization you've created so the question is is there a good way to go about testing the visualizations honestly when you say I think they probably did that I think they probably didn't for a lot of them I think unfortunately a lot of the bad examples which is to say most of what you see out in the world they're not designed very intentionally to be a useful information product they're either through the design to be visually interesting which is different than being useful like those last two ones I showed a very pop culture right there's not a lot of depth of consideration and depth of data rigor that goes into those they're fun and that's fine like it's its fully legitimately fun and I picked those as examples because because they can be improved fairly easily but also because they're kind of interesting and people are willing to engage with them a little bit more so so I think a lot of them are not tested honestly and I think even a little bit of testing is useful but in terms of a metric like like three people right get three people who don't know anything about it or don't know much about it and if two of them say the same thing wait what's this about I don't understand this part then you go back to the drawing table if they're sufficiently representative of your audience right if you're going to if you're you know going to show a performance of different car sales and you're only going to show it to people who sell automobiles and you take it to the butcher shop they might not get it right that's fine if they are not your target audience and you're very clear on that but if your target audience is generalizable but not the people in the cube next to you trying to get someone who's more or less in that domain if you can is is going to be useful there's I mean there's industries dedicated to running those tests finding those people etc for you that's a sometimes expensive and painful process but you can you can do it cheaply with a little bit of selectivity and yeah I mean I think I think it's absolutely valuable it's absolutely does not need to take a lot of effort and I think most people don't do that because they look at it and say oh it makes perfect sense I get it Bob who sits in the cube next to me all day every day he's been looking over my shoulder for two years gets it it makes perfect sense and you know it's one more thing to do right it's more effort to expend so it's not done enough I think I have a is this a question I guess in your sort of appreciate of the choir here I would assume I'm sorry I can't quite hear you I see you're sort of preaching to the choir here Otis young we're all data peep I hope so um in terms of the market for analytics it seems like there's still a big hurdle to overcome like there's a lot of companies that are developing analytics based products yeah um but I haven't seen any of those really hit the rocket you know hit the sort of proverbial rocket ship in large part probably because they don't do this well enough yeah what do you see is sort of the market for analytics based products in the next couple years I I see it really booming so there's a Riley has a conference called strata that's held twice a year it's in the spring down here just happened in February and it's in the fall in New York and that conference is like one year old and it is booming right thousands of people show up and it's a it's a conference about big data and about what you can do with it and I think there's sort of been this this need this unrealized potential because we're accumulating data really quickly right really quickly more and more data than we've ever had before orders of magnitude literally over over small numbers of years and you the data is worthless unless you can analyze unless you can get into it so I think the market is only going to increase I think tools related to managing data but absolutely tools for understanding data I think that markets going to increase dramatically there are some companies that have been around a little while tableau and and Spotfire and some others that are sufficiently well established that they're going to do really well I think I think there's still lots of room for new tools ie either niche tools so there's company outside of Seattle called tech bot that's a Boeing spin-off that does visualization just for aerospace but but I think there's a lot of room both for for good general tools since I don't think that really has been satisfied yet and absolutely for specialized tools if there's an industry or a particular angle that there's a room for anyone who's got data is going to need to figure out what to do with it or it's worthless no more questions anybody here Oh would be so cool if we were actually taught this yes the question is is any good data visualization education happening at the K through 12 level and my suspicion is not at all like I don't think it's I don't think it's happening well at any level and I think visualization as as a field I mean it's been incredibly gaining popularity over the last couple years when I started doing this work in 2004 you know there was like three blogs and like one or two companies that were kind of doing anything related to it I mean a little more than that but it was not much and every year the number of people who are interested the number of blogs the number of tools number of examples you see in the world is increasing radically any sort of actually broadly applied education around this I think would be awesome but I haven't seen it yet but I think it I think there's potential there because the more of these that people see and the more accessible tools there are the more people want to try their hand at it I mean it's it's one of the many outgrowths of having a desktop computing power right so and and there's a lot of pop culture accessibility in election year all the graphs of that the World Cup soccer was it I forget what year it was there was great incredible complex visualizations of like which of the 64 teams was playing which other team in which stadium on which day in which neighborhood like really movie wasn't 64 teams but like there was these very complex diagrams going on and and and it was fun and people who wouldn't think of themselves as being numbers people but who were sports fans got really into it right and so you see that you think oh I could do that or I want to do that and so the demand is increasing certainly trickling down to K through 12 I would love that like that would make me really happy I think it's a few years off still but ah none of the stuff I've talked about is really hard it's just unfamiliar it's an unfamiliar way of thinking but you know that was an hour and you guys have got a ton of information so there's good potential I think somebody has to prioritize it though which is harder anymore hey one more I was just wondering what do you think is the most challenging hurdle to overcome in better analyzing and visualizing data and how do you overcome it great question do most challenging hurdle in analyzing and visualizing data do you mean in terms of personally when you've got the blank page in front of you or do you mean in the industry oh I'm in personally personally when you vet the blank page in front of you I think I think the hardest part that that is often the most overlooked and the least well treated is that contextualization in terms of the audience because that's a way of thinking that that that we're not taught very well and the encoding stuff I don't want to say it's an afterthought its core it's important but but it's very heuristic like that's a solved problem more or less we know how to do that well and there's been books and there's been research in the cognitive psychology that's going back decades like we know that people are very sensitive to things like length and less sensitive to things like curved paths in terms of what we can you know how much visual discretion we have like we've known all those answers for a long time some of the tools like tableau do a really good job of good defaults of those that's a solve problem the hard part is is the squishy part the human part of really understanding who is this for how do you satisfy them effectively what are their needs and that is not addressed well and that's a very hard thing to build a heuristic into your tool for like that's that's still that's still in the human domain and my particular perspective is because I accidentally went to graduate school I intentionally went to graduate school I intentionally through graduate school for user experience which is this study I wanted to design interfaces I wanted better solutions that serve people well the accidental part was getting into data visualization so I already brought this awareness of who is my audience who is the customer that I need to satisfy with this informational product I brought that with me when I when I started doing this realization work most of the visualization work that I've seen there's sort of the one end of the spectrum of the pop culture graphic design II stuff and the other end is like stats Big Data computer graphics and it's really rich interesting deep intellectual work but it's also separated from the utility that it is that it is attempting to satisfy sometimes right it's all about tools and not about solutions and I think that's a cultural gap that that is closing and I think there's a lot of factors that that's going to bring that I think the biggest one honestly has been Apple Apple success over the last decade has made people sit up and listen to design and listen to the fact that paying attention to your audience matters because for decades before that you say look at Microsoft they hate their customers and look how rich they are look at IBM they've never done anything nice for anybody and and nobody ever got fired you're buying IBM they're absolutely sure things successful right and that doesn't fly anymore now you actually have to provide a better experience and so people are waking up to that people are I mean look at Hipmunk it's beautiful right like I use kayak for years and it's a great tool and Hipmunk is hands-down way better it doesn't as many airlines it's not it's not as comprehensive it's absolutely a better tool it's absolutely a better experience because it's so much more satisfying to use and people are starting to wake up business and technology is starting to wake up to the fact that your customer matters and you have to make them happy and that is a good way to get ahead so I think that's more of a cultural limitation than than any kind of a technological challenge and it's changing but it'll take a while still we'll get there and can you also touch on if people don't have a question I was also curious about the the other half about the industry like what challenges I'm sorry other half about the other half of the question about the data what challenges the data visualization industry is facing yeah and what you see for the coming year so I guess I sort of got both of those conflated a little bit in my answer that that both on an individual basis and as a as a as an industry designing for an audience is more challenging than getting the technology right most places also don't have the technology right for the most part Excel being the example or these other tools that are either have poor defaults or a sort of blind to that end usage right the points are on the page we're done right and the answer is no you shouldn't you can put points on the page what's put the right points on the right page with the right colors in the right layout and so that's the algorithmic encoding layer and then who's it for right is the other layer so I think both on an individual basis being aware of it and and culturally industrially having tools that support that is is that is the next big things thank you thanks whoa let's be done it's late the to the talk let's say check back tomorrow down the LinkedIn event and I'll post the link to where you can download the slide decks for the talk and the instructions for the video streaming in case you want to forward it to your colleagues and with that lets thank know again

Channel: LinkedInTechTalks

Views: 76,883

Rating: 4.9455042 out of 5

Keywords: Noah, Iliinsky, Noah Iliinsky, Data Visualizations

Id: R-oiKt7bUU8

Length: 109min 2sec (6542 seconds)

Published: Thu Apr 05 2012

Reddit Comments

[u/Hamspankin]

Thank you for posting! Great talk