Okay, we're ready for the next stage. So now that we've kind of looked at the... the pixel array and understand how the pixel array mapped to the actual pixels in a window the next thing I want to do is think about this particular scenario. So here is our Processing... our Processing window. Now what if we have an image, and the image I have in the example that I'll work with is of a frog. So what if I have an image frog.jpeg, which I am not gonna attempt to draw a frog, has a frog in that image. Let's say I want to look at every single pixel of that frog and copy that pixel into the window. Why might I do that? Well for the one thing we know the image function frog is a function that takes an image and draws it to a given x y. So essentially what we're doing here is learning how to write the guts of this function. How we have an image, we can look at every single pixel, take every single pixel value and put those pixel values into our Processing window. The reason why we might do this without the image function is we want something else to happen right here. In the moment of taking this pixel and transfering it to the screen. Maybe we want to make that pixel, brighter or darker or turning to black and white or... insert you're much more creative and interesting idea for image processing right there. So... this is... this... if we can understand this algorithm of looking at the pixels, putting them into the window and doing something to manipulate the colors of those pixels in between... then we can start building a Photoshop and you know from scratch. I don't know what I'm saying exactly. But, you know what I'm saying. Okay, so this is the idea. Let's look at how we might do this. Ah! But before we do I want to mention something. So this is the way that I'm going to build our first example of looking at image processing however I'm going to suggest that there might be a slightly... a way that's... it might actually be a little bit better in the end for a variety of reasons. So let's just say our Processing window is here. This is our Processing window, and we have our image of our frog. What if I had another image? A blank image... another PImage object and here I actually process all the pixels from the frog into this image object through our algorithm pixel by pixel by pixel and then here I just use the image function to draw this new blank image into the window. The reason why this is useful to do is this is going to work really nicely for demonstration purposes, but as soon as the window is a slightly different size than our source image, we're going to have some problems. So this way if I am creating an empty PImage I'm manipulating the pixels of this frog. Putting it into this empty PImage and then putting that PImage onto the screen. Then my window it kind of retains its purity it could be any size I could be drawing stuff on it and I don't have to worry so much about having some type of conflict between setting the pixels and drawing stuff. So this is a nice way of working as well in terms of image Processing. I should mention that we also could just to give a third option here.... If this is my frog and this is my Processing window I could just like manipulate the pixels of the frog itself onto itself, and then draw it to the window. The reason why this is not the best strategy in all cases is once I've manipulated the pixels of the frog image to draw it the window I've lost my original frog image. So this scenario saves this frog image. It never changes it. If I'm manipulating the pixels, the new pixels are going to a separate image and that separate image is going to the window but again let's start with this simpler case of just reading the pixels of the frog and putting those pixels in the window and let's take a look at how that might work. Ok, so I'm back over here, and if I run this particular sketch... here's my frog that I just grabbed from the internet. And I'm loading the image and drawing the image and there it is in the screen. Now notice here the size. Why is my size of this window 858 by 536? That's because this image of the frog I downloaded those are the dimensions of that image and to make this example work really nicely without by accident running into a problem... you know with all this math of multiplying by width... If the width of the image and the width of the window are different.... our math has to be a little bit more sophisticated to account for that. Which is an important thing to learn in practice I would say maybe try that as an exercise. First get this idea of image processing to work with an image and window of the same dimensions, then see what happens when you change the dimensions. Although, this again is going to be a good scenario for that. Doesn't matter what size the window is because these will always be the same exact image. Okay, so let's come back over here, and think about how to do this. Now... I have from our previous example in the previous video that code commented out, so if I... Comment this back out, in and run this. I now have, oh, I have that weird... dark center white.... white edges, result. Right? Because I'm setting every, the pic, every pixel according to some arbitrary algorithm that's coloring the pixels according their distance from the center. But I don't need that here. I can get rid of this line of code and what do I actually want here? I want image dot pixels loc. Look at this single, ok it's not image, I'm sorry it's frog. Look at this line of code. What I'm saying is: "Go to that frog image, give me the pixel from that location, take that pixel and put it into the screen at the same location. For every single pixel go and get the pixel for the frog put it on the screen." Now one thing I missed is... you see how I'm loading the pixels for the window because I'm going to mess with those pixels? I probably should also alert Processing to the fact that I want to look at the frogs pixels as well. So I'm going to say frog.loadPixels(). The truth of the matter is I've a feeling if I didn't add that in this would still work. Loading and updating pixels is sort of a funny thing like, sometimes you need to, sometimes you don't. I would say: "Why not just do it all the time?" Then you don't have to worry about whether you needed to or not. Okay. Look at this. There's our frog. I just... Just to prove that this is really doing that I'm going to comment out this line of code which was drawing the entire frog image to the window. There it is. There's our frog. We have now written the guts of that image function. We have now written the code that says look at every pixel from an image and put it all, everyone of those pixels in the window. We haven't really done anything yet interesting cuz we're just verbatim taking that exact color and putting it exactly in the window. Let's look at what we might do to kind of mess with that. So one problem here is that, you know, I could start to do something weird like... Well, what if I just said like divide by 2? That's kind of a strange result right? The thing about, that's kind of cool actually, a sort of surprising result. The.. The reason why this is giving us such a strange, unintentional, confusing, difficult to predict result is remember the actual color is just this conglomerate of the RGB values into one big integer. So the actual number is somewhat irrelevant. What we need to do is actually look at the red, green and blue components individually. So what I think might be a good strategy here would be to say give me... the red part of the pixel, the green part, the blue part, and then... make a new color RGB. Okay, so I added one little bit here. What did I do? I've got my frog again it's exactly what it was before. But instead of saying that exact pixel put it there I said: "Hey, give me the red, give me the green, give me the blue, make a new color, RGB and put it there. Now I at least understand what's really going on. I now have access to the red, green, blue components of every single color. What am I choosing to do with that information? Absolutely nothing. I'm just gonna get verbatim on the screen, but I could it could be really mischievious here, and I could say: "Aha, let me swap the green and the red values." And now I've kind of done something strange to the color of this frog. So you can start to see that once I've done this I now have the ability to start doing anything I want to those pixels. I could say like you know, the blue value times 2. Which is making it much bluer. So this is really... now... now we're here in the world of image processing. We are looking at every single pixel.... For every single pixel we were grabbing the RGB components, and we are manipulating those values and drawing a new image. This is essentially what an image filter does, you know... Sell your Instagram now for 2 billion dollars or whatever and send me a thank you note in the mail or whatever. Um, so... This appetizer is in many ways a good place to stop because here you now could kind of recreate this algorithm and your assignment, your exercise would be like come up with something a little bit more interesting to do, but let's... let's... before we get Before, before I leave you with that kind of exercise, Let's think about some creative possibilities that or that go beyond just kind of changing the colors around. One thing is, the pixels are spatially oriented. We know they're x y values so we can do something different to each pixel. Right now we're doing the same operation to every single pixel. We could say like for example. What if I... You know if x is greater than 200 do this... otherwise if X otherwise do, you know, put the pixels back to normal and we can see, you know, I am only have only doing this weird algorithm to these pixels and not to these pixels. So you can start to do whatever you can kind of imagine in here, based on the X&Y values. Another idea that I might have for you is to go back to that distance idea What if I said... okay, give me the distance between the... oh sorry, the center of the window... and the xy of that pixel and now whoops, pixels loc equals a color. Why don't I say... use that distance to and... to add it, to each one of these colors. Now I can start to see well everything just got really really bright. So one thing we can notice here. Is that how do we affect the brightness of a pixel? The brightness of a pixel is affected by the magnitude of its color value. So we're using distance here but perhaps an easier demonstration would be let me just add mouseX to every single pixel color you can see that over here I'm brightening, brightening, brightening, brightening the image, of course once all those values get to 255 I have a completely white image. Another way of doing this would be to multiply each color. So what if I multiply each color times you know 1.1 I'm making each color 10% brighter. So, you know, if you're not familiar with the map function of Processing, the map function says given a value of certain range and map it to another range. So what if I have a factor? And I map the distance, the distances, you know somewhere between like 0 and 200 or something like that.... And why don't I map that to between 0 and 2? And why don't I multiply each... each component of the color by that factor? So what I've done here now when I run this... oops sorry the distance.... I'm saying, has a range between 0 and 200 map that range to a range between 0 and 2. And if you notice as I... the pixels are now darker in the center and brighter to the, towards the edge and really I think it might be more interesting if they're brighter towards the center and darker towards the edge. Look I kind of highlighted the center of this image. And why am I hard coding in the center when I could actually just move the mouse around? Now I have this kind of like flashlight effect. So I'm manipulating the brightness of every single pixel according to its distance from the mouse. Again I kind of quickly just like mocked up this demonstration the point is, it doesn't matter... What you do here. The point... What matters is I'm getting the RGB and I'm making a new color. So for you.... Come up with your own idea of how you might manipulate the image. Look, just, just, I should probably finish this video... But just to do one more demonstration. An interesting... I'm going to comment this out, so I have access to it later if I'm saving these and putting them somewhere. Something we might know about an image is the brightness of a pixel, so let's say I just get the brightness... of the frog's pixel at a given location. A threshold filter... Something we could do is we can say let's redraw this image... But I'm only going to only draw a white or black. It doesn't matter. I'm not going to draw any kind of interesting color. I'm not even going to draw a grayscale color. I'm only ever going to say... Make every pixel black. Not every... Make a pixel black or make a pixel white. Now, what if.... I make all the pixels white if the brightness of the frog pixel is greater than 100 but black if it's less than 100. Let's run this... and you can see... I sort of see the frog in there. Pixels are white or black and you can see what's the threshold? The threshold is 100. If a pixel is brighter than 100, it's white and, you know, maybe what I should do here is kind of use the mouse to control this threshold and we can start to see like I can now kind of recreate this frog image and my threshold with the mouse over here seems kind of like a nice one. For I'm seeing the frog as white and the background is black. So... the reason why I point this out to you is, this is starting to kind of walk into this area of computer vision, that we're going to look at later. If I wanted to find the frog in this image... Applying a threshold would allow me to sort of separate out the image into only white or black pixels... And maybe I could then get the average location of all the white pixels And that's about where the frog is. So you could see how like visually it's kind of almost easier to recognize it's actually not easy to recognize visually for us because we're a human being and all the colors are very meaningful to us but to a computer algorithm now it's very easy to recognize a white pixel versus a black pixel. So perhaps this idea of different types of image processing filters might be useful later as we begin to try to recognize or analyze or get, gain some sort of aspect of the... knowledge about the image according to its pixels. So these are some sample ideas of things you can do. I could just set every pixel to black or white, I could set every pixel according to the distance. I would say here is now an opportunity for you to do a couple things. One... try to invent your own image processing algorithm or recreate one that you know about. Another thing that you might consider... I don't know if I'm going to have time to put this in the videos right now or I'll come back and do it later is.... If you could make your own image by saying createImage... this is the function you could look this up in the Processing reference that makes an empty image. See if you could redo this example from drawing... and I have this, I'll try to link to it below that... from instead of drawing your pixels from the image to the window draw your pixels from an image to a blank image and then draw that blank image to the window, and then you could have that blank image rotate, grow, shrink... There's lots of possibilities for what you could do from there, but I'll try to fill in a video that covers that at some point. So maybe by the time you're watching it'll actually just be the next video. Okay. Thanks very much, and this one is finished now.
