Coding Challenge #112: 3D Rendering with Rotation and Projection

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
Hello, welcome to a coding challenge! And I can summarize this coding challenge very quickly in one sentence! This is a spinning cube. It is a three dimensional cube that you are seeing on your two dimensional screen. I am able to create this three dimensional cube because I am using the P3D renderer, processing, Java based, creative coding, development, platform thing. I can make 3D stuff using the P3D rendering engine and then functions like box, and rotate X, and rotate Z, and that is how I get this! But how does this really work? How does the processing even make this happen?! I am going to do the most basic, simple thing. I am going, I said one sentence and I'm like 12 sentences, I'm like 200 hundred sentences in if I'm being perfectly honest. I am going to change this from P3D to P2D. I'm going to eliminate all of the 3D rendering capabilities. I'm not going to allow myself to use any of them, and yet I'm going to recreate exactly the same digitalization. I'm going to do matrix rotations, projections, those sorts of things. So this is not going to be comprehensive, like how to build a 3D engine with ray casting, ray tracing, and lighting, and shadow, and sorts of complicated stuff. I'm going to just do the basics, just to create this 3D illusion in 2D really because, because if I can use the P3D, if I can create the 3D illusion and render it into 2D space, what if I create a 4D shape and then render it into P3D space, right? If I can project a 3D object into 2D space, there's no reason why I couldn't project a 4D object into 3D space. and then I dunno, I dunno, my computer's going to melt, and I'm never going to make any videos again. But let's just stick with this! Okay so let's begin coding. So I am going to take out, I'm going to hit save, and I'm going to take out all of this 3D stuff. I'm going to leave translate, stroke, strokeWeight, noFill. I'm not going to be able to use these anymore. I'm not going to be able to use these anymore. I'm going to leave all this, and I'm in P2D. Actually, I'm going to take out P2D The default renderer is also a 2D engine, so I don't need to actually say P2D. Now, I should also mention that I, in this particular video, I'm making use of a few functions I've previously written. I actually might have a video. I definitely did it on a livestream, so either way in the video description I'm going to link to where you can watch me code all these functions. But what these functions are for are for certain matrix operations that I'm going to do. Don't worry, I will talk about what they are when I use them. So the first thing that I want to do is let me just get some points on the screen. Hmm, how am I going to do this? Let me make a, so I'm going to make pretty heavy use of the PVector object, and I'm going to say, I'm going to make an array of PVectors that just has four points in it. And I'm going to say points index zero is a PVector at, and I'm just going to build a little square. So I'm going to say the first one is like at -50, -50, and then second, third, fourth, is at 50, -50, then 50, 50, then -50, 50. These should be the four points of a square. So, what I want to do here now is just say for every vector in points, just draw a point at v.x, v.y. So now, oh but these are. The whole point of me doing is I can't do this, right? Okay, so these are actually going to be 3D. The point of this is they're going to be in 3D. Bear with me for a second. All will come to pass. Those are going to be 3D points, but right now I haven't done anything 3D yet. Let's make the strokeWeight quite a bit bigger so I can see them. There we go. So those are my four points. So, first, what I need to do, and actually hmm. I'm not going to worry about this part right now. Okay, so, what do I need to do make a 3D engine? Okay, here's the thing. I'm going to search for some props. Okay, I found some props. I'm actually going to turn this light off too. I'm going to try to do a demonstration in the dark. This is a 3D object. It is the book A Million Random Digits with 100000 Normal Deviates. The way that a 3D object is rendered on a 2D dimension screen is through something called a projection. That projection you could think of is basically like a shadow. So if I have this book here, and I shine this cellphone flashlight, you can see that there's a projection onto the wall. It looks like a square. If I rotate this it rotates. If I rotate around the Y axis, well actually I'll have to do this. If I rotated around the Y axis, you can see it starts to become thinner. I can rotate around the X axis, it does this. So, you know, so this is the idea. I need to create this same exact I need to create that same exact idea but in code. And the way that that's done is with something called a projection matrix. So, the point, the 3D point, is actually something like this. It's a matrix that has X, Y, and Z in it. This is actually three rows by one column. This is something that I've gone over in other matrix math videos. Here's kind of a summary of it. A projection matrix, if I'm projecting into 2D, right? If I'm projecting this 3D point into a new 2D point, what I need is a projection matrix. Now here's a really simple projection matrix. This projection matrix has a one in the X spot, a one in the Y spot. X, Y, Z, X, Y, Z, and it has both zeros in this sort of Z column here of this projection matrix. What this is actually always going to do is give a two dimensional matrix, not two dimensional, sorry, a two column matrix with just the X and Y. This projection matrix just eliminates the Z. And actually, this will get us pretty far. In fact, this exact matrix is what's used for like orthographic projection. There's all different, stereographic, and all different kinds of projection matrices. This one is going to work just fine for us. So, let's put this in our code. Okay, so the first thing that I need to do is I need to create that projection matrix, and this is actually an easy thing. Now of course if I were doing this in more sort of robust way I'd probably have like a matrix math library. I just have a few matrix math functions that are in this tab that I wrote in a previous video. So, I'm going to say make a projection matrix, and I'm going to just make it as a two dimensional array. So one, zero, zero, comma, zero, one, zero. So this is my representation of a two row by three column matrix. Two row by three column matrix, that is this right here represented as a two dimensional array. Now, what I'm going to do here is I'm going to say for every point, for every 3D point here, I'm going to move this over a little bit. So what I want to do is, okay, so what I want to do here for every single 3D point, right? And this was sort of pointless that I did before. what I want to do is I want to create a PVector. I'm going to call it the projected2d vector, and what I'm going to do is I'm going to say matrix multiply the projection times v. So this is what I want to do to project a 3D point into 2D I need a projection matrix which I just made, and then I need to matrix multiply the projection matrix and the vector. Now, how the math for that work involves this times this, plus this times this, plus this times this, goes here, and this times this, plus times this, plus this times this. I'll refer you to matrixmultiplication.xyz. That website has a nice visualization, and I've made other videos about how matrix multiplication works, which I've also linked to. I'm just going to assume that matrix multiplication works. And you could see how this makes sense. This one leaves us only with the X. This one leave us only with the Y. So now I'm going to then say point projected2d.x, projected2d.Y. I probably could come up with a better name than that. And now if I run this, same exact thing, right? Because in this case it's as if the point light and the place that I'm projecting onto, and the thing, they're all kind of in exactly the same spot. So there's no skewing, there's no skewing of the sort of prospective. So, if I were to move that light around, and change the projection matrix, different things could happen. But I'm not going to worry about that right now. What I want to do now is see if I can apply some rotation. Okay, so how do I apply rotation? Well, I know that I could, I could just say like rotate by angle, right? If I say rotate by angle, processing is going to do that transformation for me. But what if I want to rotate X, rotate around the X axis? I can't do that. I'm getting some sort of error somewhere or nothing. I mean it should say not available with this renderer. Oh yeah, rotateX can only be used with a renderer that supports P3D. I need to write my own rotateX function! And guess what I need for that! Rotation matrix! I think the ding is better after. Guess what I need for that! Rotation matrix! Oh yeah, that was great. That was exactly what I meant to do. Okay, so now I am going to create a rotation matrix. Now how does that work? Luckily for me, I already have this Wikipedia page opened up. Look at this. A rotation matrix is a matrix that is used to perform a rotate in Euclidean space, and this kind of makes sense, right? Look at this rotation matrix. Have you ever done one of things where you do polar to Cartesian coordinate transformation, and you take the angle, and you keep increasing the angle, and then you convert that to X and Y points, and you have this like spiraling thing? I've made videos where I do that. Basically, to rotate in two dimensions, this is all I need. So just for fun, let's first just do this. I'm going to say float, I'm going to make another, I'm going to call this rotation, and I'm going to make another matrix, and I'm going to do exactly this. It's going to be two by two, and it's going to be cosine theta negative sine theta. And I'm calling it angle, cosine angle, negative sine angle. And then it's going to be, what was it? Sine angle, cosine angle, is that right? So this is, I didn't get this right. There's, this needs a bracket. I dunno what I'm doing here. And this needs a bracket, right? It's a two dimensional matrix. There we go, nope? Semicolon, there we go! So this is my rotation matrix, this is my rotation matrix. So what now, what if I were to say, no, I'm going to take out this rotateX. Not using the native rotate function, and I'm going to say PVector rotated equals matmul, rotation times projected2d. And now I'm going to draw the point at rotated. So I am projecting, V is a 3D vector. I'm projecting it into 2D, and then I'm rotating it, just a 2D rotation. And if we run this, okay, what's wrong? Columns of A must match rows of B. Oh, oh boy, I made a problem. Okay, something horrible happened. I got this out, I got columns of A must match rows of B. I made a mistake in my matrix multiplication thing. Here's the thing, I'm trying to be all clever to show you in 2D first. The truth of the matter is, PVectors, they're 3D. Even if I'm drawing just the X and Y, there's always that Z component. So I actually need include this extra column with zeros in it to do this demonstration. So now you can see that's me doing 2D rotation. And actually, I'm actually do 3D rotation, right? Because really what this should be is this. This is, what I would call, XY rotation. I am rotating around the Z axis by, you know, using the X and Y axes, right? Look at this. Now I'm going to say, but if I kept scrolling down this Wikipedia page, we would see this. We would find this right here. This is the one that I'm doing. Rotation Z, right? Oh, I need a one there, of course, not a zero there, very important. I don't want to lose the Z point. I want to rotate the X and Y axes around the Z axis, but I don't want to lose the Z point, so I need a one there. In the 2D case, oh okay, so I need this here. So this I'm going to call rotationZ, and now I'm going to make some other ones. Let's call them rotationX. And the rotationX is Y and Z. So I need to have my rotation values in the Y and Z spots. So hard to do this. I should just copy it from somewhere but this will work. And I think this is right. So this should be, right, you can see this is, am I spacing this off? Help me! This was rotationZ, this is rotationX, and let's do one more. Let's do rotationY. So I need to have zero, one, zero in the middle. The last one will be sine of angle, zero, cosine of angle, and the first one will be cosine of angle, negative sine of angle, with a zero in the middle. Okay, so this should be, this should all line up nicely. This is rotationY. Do I have any syntax errors? RotationZ, rotationX, rotationY. Okay, now,so here's the thing, I'm going to do something terrible. What if I just put, like, if I put rotationZ here, oh I have an error. Oh, I'm missing a comma here. Thank you. So I think I got it everywhere else. Right, that's rotationZ. What if I try instead rotationX? Oh, weird that that's working. I guess, oh! You know why it's working? It's working by accident really, because my Z points, so but I've done something incorrect, but you can see this looks as if those two points, and I want my angle to move up a little bit faster. You can imagine this as a plane almost spinning around the X axis. But here's the thing, I should be doing, the projection should happen at the last moment, right? I should really be rotating first, and then I should project the rotated vector. And so here, so sorry, projection. For it to be correct, it's got to be in this order, because what I need to do is first rotate and then, it worked by accident because my Z values are zero, but now let's do something. Let's make eight points. Okay, I'm doing this in a highly manual way. Let's now have all these be at -50. I wonder if I want to put them, I think this'll be fine, -50, and then now these all at 50, right? I basically, if you think about it, a point has zero dimensions, a line has two dimensions with two points as the bounding box, a plane, a square, has two dimensions with four lines bounding it, and then a cube is in three dimensions with six planes, but really I can kind of make the cube just with eight points, which are the edges, all those connections. So let me just run this for a second. Oh boy, this works too fast. I didn't want this to work so fast. So now we can see, let me just go back to rotate Z. And look at this. That looks, now why do I not see eight points? Because I'm using orthographic projection as if the light is right there, there's no kind of perspective. I'm not moving the light source away to create this kind of more 3D perspective look. I can't see, when the points are right in front of each other, the shadow projects on exactly the same spot. But, if I were to do rotate Y, now you can see. You can kind of imagine that being a cube rotating. And here's now where I can get fancy. I can now say rotated equals matmul rotationX times rotated. So I'm just going to do this a couple times. I'm just going to keeping rotating by all the different, so I'm going to do this as separate operations. Now there's a way I could probably combine all those, but I'm just going to do all those matrix rotations, and here we go. Now we have my cube spinning along all axes. Now it doesn't look like a cube. Time out, in all of my like fussing with this, I left an extra zero here. This rotationX should be one, zero, zero, zero, cosine, negative sine, zero, sine, cosine. Alright, so hopefully that doesn't change much. Eh, still looks right. So that probably was just being ignored. I was ignoring it but I don't want that in my code. So now we can we've got this rotation. The truth of the matter is I really want to draw this as cube. Oh, I want to draw this as cube! Let me think about this. There's got to be, perhaps I could manually connect all the points I want to connect. Also, this is not the same size as what I started with. Alright, let me do something that's going to help us in the long run. What I actually want to do is instead of using 50 all in here, I'm going to change every instance of 50 to just the number one. So I'm going to kind of have normalized the cube, actually I kind of want to change it to .5. Instead of one I'm going to change it to, because I want the sides, the lines, to be a length one. So, whoops, replace, do I have, oh I have ones in other places. Bad idea! Hold on, I'm going to change all the instances of 50 to .5. Okay, so now if I were to run this, you know, I don't see anything here, but very easily what I can do is, at the last moment, I can scale things up. And again, I could do this with some kind of, I'm just going to do multiply each vector by 100, and then I have this. And if I multiply it by 200, then I have this, which more closely matches what I started with. The angle of rotations are different, but that's no big deal. Okay, I wanted to do that because I also would like to show you perspective, and I think if all my coordinates are normalized, that's going to make it a little easier to do. But before I show you perspective, let me connect all the edges. I is less than 12, i++. Right, what am I connecting? I'm connecting, let me think about this, I'm connecting, like if I say connect, what did I call the points? Let me just, zero and zero. Let me write a function. Let me write a function called connect, and it gets two PVectors, and it just draws a line. I'm going to say strokeWeight, and it draws a line between one and the other, stroke So if I were to say connect, oh and it doesn't get two PVectors, it gets two integers, and PVector a1 equals, I'm going to call this i and j. PVector a = points[I], and PVector b = points[j]. This is like, so I should have a line between the first two points. How come I don't see that. Oh, wait, wait, yeah, point O. Oh I need to connect, okay, so I'm going to make this PVector projected equals, oh I'm going to make an array that has eight spots in it. And then, instead of drawing this here, I'm going to say projected, so now I really need to, oh this is fine, int index = 0, I could just use now a for loop but index++, projected[index] = projected2d. So I'm going to put everything in an array, and then I'm going to say connect zero, zero, in projected. And then this will be an array of points. And so now this should, where's that line? Let me just make sure. Now if I say for PVector V in projected, point, I really didn't want to draw the edges. This is why. I wanted to just leave it at that, leave it a viewer challenge to draw the edges. Projected2d, projected[index] = projected2d, index++, PVector v. Where's all that strokeWeight stuff? This stuff should be here. Where am I drawing all the points? So, okay, there are the points again. Now, I want to connect zero and one. Zero and one. There we go! Took me forever! So now I'm connecting zero and one, then I want to connect zero and two. No, zero, then I want to connect one and zero? That's the same thing. Connect one and one? No, no, no, one and two? This is good trial and error. This is fun! Then I'm going to connect two and three. I just want to figure out if there's an algorithm for this. And right, because this is the plane. This is the first plane. Three and four. Ah I see, this makes sense. Whoop, whoop, whoops. No, three back to zero. That's the first plane. And then the next plane is the same thing but at the end, which is like four to five, five to six. Oh, no, three to four, four to, right? Because didn't I put the next plane, would be zero, one, two, three, four. No, no, no, no. No, no, no, no. Yes, four to five, five to six, six to seven, and seven back to four. Okay, so that's that. Now all I need to do is connect, oh, this is easy! So there's definitely an algorithm for this, 'cause now I want to connect zero to four, one to five, right? Two to six. Hopefully the algorithm is revealing itself to you. And seven to eight. No, no, no, sorry, three to seven. There we go! There's my cube! I have done it! Now I should really refactor this, and I plan to do that, but right now I think I need a little bit of a break. Alright, so, actually no I don't. I got to do it right now. I'm sorry, this video's going to be really long. I don't care! I equals zero, i is less than four, i++. So what I'm going to do, right? There's four sides. I am going to connect i with i plus one, modulus four, then I also want projected, then I also want to connect i plus four, and i plus one modulus four, plus four. Let's do that. And then I just want to connect i with i plus four. I think that's it. That's three little blocks. Woo-hoo! I'm so glad that actually worked! Okay, so this is connecting the edges. This is going to really help when we have to do the, I hope, when we have to do the hypercube, connecting the edges. Okay, I should really go! I only have 10 more minutes in real life time while I'm recording this. But let's actually add perspective! Let's add perspective. How do you add perspective? So the way that you add perspective. Perspective projection. Basically, we're going to add a variable which indicates the distance, remember my ridiculous experiment earlier like about four and a half hours ago, if you are actually watching this video all the way to the end, where I had this light source and I was shining the light source, and you were seeing the shadow here, that projection? I need a variable which is the distance away from the object, the surface, that light source is. That's how I get perspective projection, right? So, to create this idea of perspective, right? I want to move the light source, or move the camera further away. There's a lot of different ways to do this, and highly sophisticated methods, but basically I need to scale the X, Y positions by the depth, right? Objects in this mirror appear closer than they really are. Like, things that are further away appear closer in, things that are closer appear further out. So to do that, this projection matrix can't just always be constants of one and one in the X and Y position. So I'm going to move this, and I'm going to before I project, right? Before I project the final rotated matrix, I need to dynamically calculate this projection matrix according to the Z value itself. So, one thing that I could do is I could just say float z = rotated.z, and I could like just try like dividing, dividing by it. I'm sort of scaling it according to that Z. Let's see what that does. Eh, something maybe. It looks all weird and crazy. That's not right. Well, a nice formula for doing this is actually saying one divided by, this is now the distance basically, the camera distance, the light source distance, minus rotated.z. So if I put that in here now, and I were to say float distance = 2, for example, now you can see this looks much more like perspective projection. So this is no longer orthographic, it's more like perspective. You know, again, I'm not saying that I've comprehensively solved every possible projection, 3D way, with focal length and all sorts of field of view stuff. I'm just doing something kind of basic. This is probably similar to like weak project maybe, I think. But you can see here that if I change this distance to one, now this object appears much closer. It's more distorted. If I were to change this to 10, wow, it's so far away but you can see there it is. Those points are little bit too big. Now one thing I'm not doing is changing like the strokeWeight to make it seem further or smaller, but you can see I'm just altering the perspective. Okay, we've done it! Where's my train whistle?! I need the train whistle! No train whistle for me. I'm going to reward myself with a piece of space melon. I have made this coding challenge. So, what can you do? First of all, if you want to challenge to yourself, just go make this in four dimensions now. But that's what I'm going to do next. Now that I have successfully drawn a 3D cube, and just to prove the point, I am in the P2D renderer. The P2D renderer using my own matrix math, and my own rotation matrices. You know, you don't need to do this. I would probably just go back and always use P3D, but this should let me unlock the fourth, the fifth, the sixth, the seventh dimension. You know, try making different shapes other than cubes. Try, you know, you can do weird stuff, like I'd probably get something like super weird if I just, like let's make the distance two again, and let's change this negative to positive just for rotationY. And like look what I've got going now, like I could really start to like distort the world by like just messing around with these numbers. Like what if I say like five times cosine of angle down here in rotationY? Boy, that's probably a terrible idea. So what kind of crazy, weird, distorted, rotation perspective things can you do? Can you actually do this with vertices so you apply fill? Can you think about color? Make some sort of weird, pretend, 3D world that doesn't follow our own sense of three dimensions by playing with this code, and share that with me on the comments or on Twitter, and also on thecodingtrain.com, there'll be a link to the webpage where you can submit user-made things.
Info
Channel: The Coding Train
Views: 118,348
Rating: undefined out of 5
Keywords: programming, daniel shiffman, tutorial, coding, the coding train, nature of code, processing, matrix math, matrix multiplication, math 3d rendering, math 3d, projection map, projection matrix, processing 3d, orthographic, orthographic projection, orthographic drawing, orthographic view, 3d perspective, rotation matrix
Id: p4Iz0XJY-Qk
Channel Id: undefined
Length: 33min 12sec (1992 seconds)
Published: Tue Aug 21 2018
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.