Creative Coding in p5.js - GitHub Satellite 2020

ah [Music] hi my name is Cassie and today I'm gonna be talking about the basics of creative coding by doing a little live coding and p5.js so if you've just seen some of the past talks and you're feeling really inspired or kind of like intimidated like well I don't even know how I could possibly ever do this and I think this talk is for you but first I want to introduce myself hi I'm Cassie I am a technologist and artist and educator so that means that I do a few different things one thing I do is I work on the software for creative tools so this is a picture of a graphical programming language called max/msp it's a good musical programming language it's managed by a company named cycling 74 that I've worked with for a few years I also teach here is a picture of me teaching a max workshop this past summer down to some high school students and I also teach at a grad program at NYU called ATP or interactive telecommunications program and talked creative coding there for about two years I also have my own creative practice this is a picture of me performing in a musical project called lullabies for AI which was about writing songs to teach our AI children how to love and exist in the world but really what I want to talk about today is p5.js something is near and dear to my heart p5 is an open-source JavaScript library for creative coding the focus on making coding accessible and inclusive and it's managed by this foundation called processing foundation is created by a person named Laura mccarthy about seven years ago and so you would use p5 to make interactive audio visual art so that means stuff like you know you can draw shapes you can do you can alter like videos and images you can do data visualizations have some quick examples just so you can kind of see stuff made of p5 here's an example here's an example like a video shader you can also make like an interactive movie poster just like this is you can do a lots lots and lots stuff p5 was really inspiring to me because my background is in engineering and when I started when I first started learning how to code I was writing like command line applications in Java and like the whole thing was like really intimidating and scary to me and I felt like after I like spent all this time learning I like didn't actually know how to make anything and from ep5 was really inspiring that like I could know very little and be able to make a lot so I loved it so much that I started my own open-source project which was an in-browser code editor for p5 so the idea here is that like you really removed the barrier to entry for writing code or writing creative code and you just like have the code on left preview on the right could just like open this website and get started so this is what we're going to use to do some live coding so let's so I'm just gonna open the web editor and this is like the sketch that you get when you first open it I'm so I'm just gonna hit play and we can see what we get and we just got this like kind of boring gray square but let's like break down what we've done here or what what it gives us so first important line we have here is create canvas and this is creating like our drawing a surface where we're gonna draw all of our visual elements it just takes like a width and a height as arguments so I could like make the speaker and then we'd have something bigger right and then we have a background and this is just like painting a whole big color like one color over the whole canvas and this argument that it takes is a color and this color can be like you can specify in a lot of different ways so if we just have a number then that's just a grayscale value right so if we put in 0 it's gonna be black if we put in 255 it's gonna be white I'm if you're familiar with hexadecimal colors that 255 is white if you're used to like using colors on the web like that you can also make it a red green blue value right so man if I just specify a red component and then if I put in you know I could put in a blue component and then it's gonna be like paint color but yeah colors are just numbers and in fact like to our computer everything is numbers eight images text video everything is just numbers and that's really one of the like exciting parts of creative coding is that we can like take these different numbers and like map them and interpret them and use them in different ways one other thing I want to note is that there are these two functions set up and draw and these are being called behind the scenes by five piece by the p5 library and so setup is being called one time and draw is actually being called 60 times a second right it's just like constantly being executed over and over again it's what computers are good at and this is what allows us to make animations and and that kind of stuff so now what I want to do is I want to add like an interactive element here and I just want to add like a really simple one so we can kind of understand it and then we can like keep building on this and this is how you might if you're just like starting working on it creative coding project how you might how you might just start so what I want to do is I want to draw a circle at the mouse position so the way I do that is they use circle function yeah that's that's right by p5 and then I can pass it takes like a X&Y position and a size so I'm just gonna put in the mouse x position this is a variable that's provided by p5 it just specified every go draw loop every time the draw loop gets run it gets updated I'm just gonna put in a size and we can like see oh you see I have this in the wrong order background is getting drawn over it put this in the right order and now we can see our Sierra Circle right and if I move it around I have the circle following me and you're like okay Cassidy was so kind boring there with me so now I think it gets slightly more interesting when we move background up to the setup because then we have them were no longer like painting over the canvas every frame and then we're able to like see this trail of me moving the mouse and you know actually it's actually like a bad it's like a bad drawing up and you kind of move the circle around faster and slower I mean you can like swing your eyes you could like see this like three dimensional tubes filter or something it's kind of cool but let's keep making this more interesting what I do its I want to add some color to this and I want to have the color be affected by the mouse position right I'm kind of doing like a one-to-one mapping here or I'm just mapping the mouse position to like the position of the circle we can get a little more more fancy so let's change the color at the circle where that we do that is we use function called fill just fill color I'm gonna put in Mouse X for the red value and let's gonna put in maximum for the green and blue let's see what that looks like so as I move the circle across the canvas we see that all the way on the Left where the mouse x position is zero it's like it we can see just the the green and blue components and then as I move it to the right it's white because all of those colors some together is white but one interesting thing to notice about this is it actually changes to white like halfway through the canvas and that's not quite what I want right like I want this to be like a nice gradient all across and the reason that this is happening is because these two values the position of the mouse and colors have different ranges right the mouse position is a value between zero and the width of the canvas and colors are values between 0 and 255 so we need to need to take the mouse position and map it to the right range and the way that we do that is we use a function called map I'm saying what sort of map and I couldn't think of it Red Dog equals Mac and so map takes five arguments one of which is the input and then the input range and then the output range so the minimum maximum and the minimum and maximum so this is going to be a value between 0 and rick p5 also just gives us width of the canvas py that's very nice to us gives us the things that we want and now as we move oh I have to actually use this variable rather than just define it and then it's we move the mouse across you can see that now we have like a nice gradient and we can like enjoy it even more maybe I'll remove the stroke around it ooh and then we could like kind of draw around we got like a nice gradient so that's fun but now I want to like keep playing with this I want to like use a different shape here um I happen to like using line and I want to slide the line takes like a start and end position and I'm just gonna like offset the X the X the start act I'm gonna make it like you know you'll see what I mean level with the expert a little bit offset to the left and right so I'm gonna be Mouse X minus 50 now Y takes X Y X Y mouse X plus 50 Mouse Y cool and we're not gonna see anything because I removed the stroke and lines are an empty shape they have no fill they only have strokes I actually just need to like get rid of my circle and then I need to specify a stroke color if I just call stroke with a color then I got a color now if I start drawing here yeah I get like a line but I'm not really I'm still not liking this variation in color I want to add a little bit more and so I'm going to have the mouse Y control the green green position so then I'm going to change this to green brown make this mouse why make this type trait because Y is the value between 0 and height Y positive is actually down takes some getting used to it's actually really common and like every everything digital I don't know did a lot of creative coding frameworks okay cool so now I'm getting more more color variation that I like it's kind of like a vapor wave the colors and I could like play around with these color ranges and like see what I like I'm happy with this color range so I'm gonna keep going it's kind of nice as I like move my lying around I can see how like they overlap and make these like textural shapes I'm liking this but now what I want to do is I want to add some rotation to this line to like have a little bit more to see like what other kind of shapes arise there and so there's a Pew 5 function called rotate and it takes one argument that's an angle and it takes by fault radians which maybe is scary but but unfortunately in creative coding you have to do a lot of I feel like you have to do a lot of maths like I need to work arranging visual stuff on the screen it's like you have to do math and my students are always like oh no I thought we were avoiding math by going to art school I'm like hahaha you have to learn trigonometry we have to remember trigger object okay well as I'm like are you trying to use this rotate it's like not really working how I want like I wanted it to rotate the shape but like as I have my mouse over here it's like now it's position as well as you don't really understand what's happening while I'm pretending that I don't understand something I do understand something I have an example here to show you so it's like offset in this weird way and so what's happening is so here's an example where I'm rotating a rectangle at the origin I've actually translated the origin here to the centre of the screen and so we'll see if the second square which is the blue one I've rotated it PI over four like the same amount 45 degrees and we'll see that it rotates in place right when you rotate rotate something at the urge and it just rotates the shape but if you move the rectangle out from the axis so here I'm gonna I'm setting the x and y position so the red one is the unrotated one that's connected with the black axes and the blue one is the rotator one you see how like it's XY position gets moved and so that's what's happening in our inner sketch now I don't want that to happen I wanted to get rotated in place so then what I need to do is translate the axes to the mouse X and mouse Y position now six that's why right and then what I need to do is then draw the line back at the origin so then I get rid of the zero makes this zero get rid of you get rid of you oh I got put a zero here actually so cool what did I do wrong what did I do wrong what's happening cute Oh negative 52 50 I was like the line had nose sighs okay cool so now I have a rotated line this is fun but it's kind of just like the same thing that we were doing but it's like tilted yeah but now I want to now that I'm like able to rotate it how I want I want to like have the rotation change with the mouse X and last by position so let's create a new variable called rotation when you use our trusty friend map again what do you mass Y which I know is a value between 0 and height and let's map it to a value between 0 and pi pi is also just provided by p5 so now I can get some squiggly shapes as I move my mouse and that's fun I could try making this like 2 pi I'm like honestly if I'm like playing around with mapping it's something like this like I don't I'm not like thinking about this too hard I just kind of like put in a random number and just like see what it looks like it's really nice that like you can just write code you know and then just like immediately see what it looks like and be like oh that looks good that looks bad and that's just really common in creative coding okay like how this kind of had this like san shape comics like a leaf or something cool so now I could also try playing around with like the line size I want the line to be breaker 300 and then I kind of get like a totally different um different effect so now like I don't like that every time I do this I have to like move my mouse around to test it I just want to hit play and I had the computer do the work for me and so that's a really nice thing about I'm using computers and you're when you're trying to make visual art is that you can just hit play a bunch of times and like see what it generates and pick the ones that you pick the ones that you like it's like you're laying the computer do the hard work doing what it's good at and you can also have it like have some randomness so every time you hit play it does something different and then pick the good ones and so the way that I want this like the line position to move is I have this like bouncing ball example and I want it to like just bounce around the canvas like this and rotate around and we'll kind of see what that looks like and because I already have this example like I'm not like you know it's all this is doing is really just like it has some x and y position and it's just updating the speed every it's updating the position with the speed and then when it hits the wall it's just reversing the X speed or the Y speed it's pretty it's pretty simple and really like I don't want to think about how to do this again like I'm unlike I know those code works so I'm just going to go back to my example and paste it only thing to do is merge my setups so I don't have to set up functions the one that comes second is the one that's gonna override okay so now I have this and if I hit play they're just gonna be the things I haven't connected it at all I also need my code that's in my draw loop that's doing all the updating so it's gonna copy that go to my draw loop and now so this is my code that's updating the position and this is the code that's making it bounce off the walls and in the other example I had the circle that had a diameter but my line has no diameter I just wanted to directly hit the wall and so in this case D is zero so I can just get rid of these these or set it to zero where it makes sense okay so now if I hit play it's still not working because I haven't actually connected it so what I need to do is change this to X check every time I have mouse X or Mouse Y got to change it to Mouse X or change it to X or Y whoo okay so now I can just like let it hit play and just like see what happens that's kind of fun I really like how it looks if this speed it's a little bit slower or like that let's see how that looks yeah and you have it slower you get these more a patterns that I really like because the the line is then closer together yeah this is fun this feels like this feels like math textbook energy would be like multivariate oculus third edition or something I don't know cool so I hope this has given you an example of how you might just like start from something really really simple and like iterate on an idea and just like play it around if I hit play a bunch of times I could get totally different things oh this is cool like how it's just spinning in the center whoo I could like take a screenshot there and be like this is my art piece you can really generate cool ideas but now I want to show you how to take like different types of inputs and how you might just like take some code that maybe you don't know and just like put like just sort of like plug different things together and also work with libraries that you maybe don't know so what I want to do is I have this example that is in one of the p5 examples and like where'd I put it and what it is is that it's really simple synthesizers so like when I just press the mouse it like plays a sound and just changing the mouse position changes the amplitude and frequency I'm like this is great I want it and then what I want to do is I want this synthesizer to be controlled by my face what I want to happen it's like when I open my mouth I want to play a note so how do I do that well luckily there's this JavaScript library called CLM tracker that I know is really easy to use and really great doing face tracking so I'm like cool I can kind of look at this documentation I see it gives me a bunch of points for like where my mouth and eyes are and stuff I'm just going to need to figure out like the distance my mouth opens right song I need to like measure the distance between 60 and 57 and I have some example code and so here I have open the CDN so I can just like copy it to my sketch I can then put it in my HTML yeah so this on the right is actually a hole it's like a whole web page just running in an iframe here so yeah that's how I add new files to my sketch and if I hit play nothing's gonna happen well actually I need to get my face input right like I'm not getting my video in here so what I need to do create function your variable called capture I'm actually gonna change I want my the sizer to be a sawtooth instead I think it's more annoying and I like that so I'm gonna do capture equals create capture this is another function probably by p5 that allows us to get video input and then I'm gonna do capture got size 400 225 so now we can see there's my face and double because you can also see my face over this thing and then what I want to do is you can see that it's like not on the canvas and that's because this actually creates a totally different video element on the screen it's like it's like a literally a video tag I'm doing this for triangle HTML element so I want to draw it to the canvas and so let's first set my canvas to the same size and then I'm gonna do use the image function this draws an image to the p5 canvas at some position all I have to do is pass the image or video capture 0 0 and then we see now I have two of me and I can like hide the other one capture dot kind so now now if I click on my face I have the synthesizer here I don't want any of this text here okay so now I have my face and then I need to get the points in my face and I know I'm gonna need to like somehow plug it into this amplitude and frequency right um so now what I want to do is go to my face tracking thing and I want to just get the starter code so I'm just gonna copy this it's the initialization code then I have my tracker I want to make this a global variable I like can't even let a var exist in the world I'm like a bar is evil it must not exist so I initialize it emit and then I have a I need to connect my video input this was just the video input from the example so then I need to put capture a perfect captured got me LT because P 5 has its own wrapper for Dom elements okay so now I have this and I'm still not gonna see anything's I'm not actually getting my face points so that I need to go here and this is the line code I need this is the friend that I need I'm going to come back here go into my draw loop and have called this line and then what I want to do is I want to iterate through all of the positions it gives me like however my positions like 60 something and draw like little circles on all them so we can like visualize those points so I'm going to need to do I have to put this in it and this is just a thing that I know from running through this a bunch of times is that positions doesn't get defined and maybe at least you have to just oh sorry you have to do it like this because I want to do positions dot for each and it was like people ignoring the errors like positions is not fine but I'm gonna save you I'm gonna save you that okay so yeah I have my for each loop and then I want to draw a circle each point so first I need to set my fill color I'm just gonna make red then I need to set I'm gonna be no stroke and then I'm gonna draw a circle at the point so I'm gonna do circle positions at zero positions at one so the way this positions array works is that it's an array of two-dimensional arrays where 0 as X and 1 is y so I'm just gonna be circle 0 0 5 and be real tiny of course let's see oh if I think I spelled it wrong positions Wow I made up a new variable ok cool so now let's see if that working a has my face points so that's fun so now I need two unlit allure this three now I need to figure out what my mouse distance is and so the way that I do that is I need points 60 and 57 I need to like find the distance between them and I can't just subtract the Y components I need to like do some vector math like if my face is tilted but luckily I can just let p5 do all the vector math for me and I don't have to think about it so what I need to do is create two variables mouth top and mouth bottom this is going to be create vector it's gonna be positioned at 60 0 at 61 it's bigger so you can see it and then I'm gonna do the same thing but mouth bottom but it's gonna be 57 57 then I just need to get the vector difference and magnitude of that difference to get the distance of the mouth so I'm gonna do Const mouth fist it's gonna be now top dot subtract now bottom got neg and this is all p5 functions this is all p5 vector stuff so now what I want to do is like I need to figure out what the range is for my mouth and this is God screenshot that so what I need to do is just like see what range of values I get and so this is this is really common when you're trying to work with some kind of input for in your creative coding projects you've got to like see what your range looks like and usually notice like print it to the console mouth yeah it's because Mouse dist is not defined now you okay so it seems like it's a number from like 3 to 17 I can't talk well I'm - I'm trying that out okay so now we have our values Oh so I need to make this global and to do let or more global announced bist don't need to print it out anymore and then I just need to connect it so let's connect it to the amplitudes and then I can just do mouth now fist which I know it's a value what did I say it was between like 3 and 17 I'll make it 4 to 17 I'm gonna make this value between 0 and 0.5 in case it's like really loud okay okay Mona truth [Music] nice and I have this like kind of stupid mount synthesizer but it literally took me like 10 minutes to make but it was pretty exciting and this you know I could make this much more exciting and like make the sound elements more exciting I could use like a better base tracker this is just the easiest one to use but I could also maybe connect it to the frequency to good try let's try that we could do just do the same thing Mouse fist which is a number between 4 and 17 [Music] so yeah that's your mouth synthesizer project cool I hope you've enjoyed learning how to make visual art and a weird mouth synthesizer um cool so now I want to go just sort of have some closing remarks here well I've sort of summarized my points about creative coding in the slide that you probably can't read if you want to like screenshot it and take it read it later but really I've tried to cover just like some of the basic stuff you would do in creative coding and a lot of these principles apply to any creative coding environment whether it be something like open frameworks or Maximus P or Sonic PI or shaders or whatever all all applies I want to encourage you to learn p5 which I love so much this is a screenshot from a YouTube channel called the coding tram just talked by a person named Dan Schiffman who's on the on the board of frosting foundation and makes really amazing toriel's but there's tons of other ways to learn like you can read there's examples on the p5 website so many books I also encourage you to join the p5 community which just really inspires me every day this is a picture from the p5 contributors conference which happened this past august at carnegie mellon see here's just a really amazing opportunity to meet in person these are a lot of people who I like talked to you over the internet over github a lot as I'm working on the web editor it's really awesome to see everyone in person I also encourage you to contribute if you have wanted to contribute to open source tools this is a great place to start it's a really friendly and open community and there's so many different ways to contribute you can contribute code but contributing to open source is not just writing code you can do you can do documentation you can teach you can create you can also just donate money if you want to donate money to processing foundation and keep this project going p5 and the web editor highly recommend classy I just want to say thank you so much for watching if you want to follow me on Twitter or github these are my handles and I hope you have learned something or loved watching me frantically livestream are frantically laughs good thank you so much for watching and I hope you enjoy the rest of the conference

Channel: GitHub

Views: 19,861

Keywords: git, github, github universe, github satellite, collaboration, programming, version control, open source, software development, octocat

Id: sRLWIAPaiRI

Length: 30min 53sec (1853 seconds)

Published: Thu May 07 2020