COS 302: Vector Basics

in this video we're going to talk informally a little bit about vectors now vectors are something that you probably already have some intuition for from like high school physics class right there are often things that we sort of write with uh say you know like little arrows above them like that or like that and typically we think of these as being directed line segments maybe we have some two-dimensional space and we have a line with a little arrow to represent a direction and a magnitude now these are things that we think quite a lot about as representing say forces or velocities of various kinds so maybe a typical kind of thing would be where we have like an inclined plane right and maybe there's a block and the block is subject to some kind of gravity and maybe it slides down with some kind of velocity and maybe there's some frictional force that's holding it back now these are useful intuitive ideas for reasoning about two and three dimensions and thinking about physical systems but the kind of the key property that we care about here is going to be the idea that we can add them together and scale them and those properties are going to turn out to generalize to lots of other kinds of objects that we'll still call vectors and these kinds of vectors form the basis of many different kinds of things that we do in machine learning they're kind of a core object for reasoning about high-dimensional spaces and how we can combine things and featurize things and so on so just to remind you how this works we can think of things sort of geometrically and we can also think of things algebraically the geometric view is really useful and it's very helpful for thinking about things in two and three dimensions but it doesn't necessarily generalize to higher dimensions but the algebraic view does so just to remind you if i have some vector say x and i want to add it to some vector y then of course what i do is i just put them head to tail right geometrically and this gives me a new vector that i'll call x plus y similarly if i want to scale something maybe i want to scale it by some by some quantity let's call it gamma then if i have an x hat that's some vector and then i multiply it by some gamma then i get some new thing that might be shorter or longer but is pointing in the same direction the way these things work out algebraically is pretty straightforward and it's exactly what you'd expect which is to say that if we have some if we have some components right so we would typically say that maybe x is equal to some amount in the y i hat in the i hat direction plus some b in the j hat direction and then y is equal to say some c in the i hat direction and some d in the j hat direction then we can do the same operations but we kind of do them by manipulating these quantities right so now x hat sorry x the vector x plus y is equal to a plus c i hat and then we add that to the b and d j hat and similarly if i want to multiply x by some constant gamma then that's just going to wind up being inside the each component so i'll wind up with some gamma times a i hat plus gamma times b j hat right so i and j are sort of our designations for these unit vectors that give the a and the b and the c and d kind of different meanings now it turns out that the geometric view of vectors as directed line segments is a little bit limiting so what we want to do part of what we want to do in this class is go beyond directed line segments so what does that mean well what we want to do is think about different kinds of objects that belong to some set and for which we can take two objects of that set add them together and get another object in that set that is to say we want to have vectors be able to add and we also want them to still be able to scale so we want to think broadly about other kinds of objects in which it's possible to add them together and it's also possible to scale them and so there are three ingredients generally that we're going to look for when we talk about other kinds of vectors so these ingredients r we need a set we need addition and we also are going to need multiplication by a scalar and kinds of things that have those properties we're going to treat those as vectors and these will be interesting algebraic objects that won't necessarily translate to simple kinds of directed line segments now the the easiest way to go is to directly generalize the high school physics type vectors that we that we just looked at and instead of just thinking about them as being say pairs or triplets of numbers for two and three dimensions to think about them as being arbitrary tuples of some size n so one natural thing to do is to think about lists of numbers as being a vector space so we could think about some kind of vector let's call it x and i'm going to in general in this class write this right vectors with a single line beneath them we can say that a vector x is an x1 and an x2 and then out to some xn right so just a list of real numbers each of these x's say x i is a real number and we'll write that by saying it's a member of the set of real numbers this double struck r is the set of real numbers so then if we want to talk about tuples of n real numbers we could talk about this x this vector x as living in r n it's a little bit hard to think about this in n dimensions because we may not have much intuition for directed line segments in more than three dimensions but algebraically this still makes sense and we can still do the same kind of thing we would do in two or three dimensions so if we have some x that's like that and then a y that's defined similarly then we could say that something like x plus y is equal to the set of tuples that is going to be x1 plus y1 and then x2 plus y2 and so on also algebraically we need to be able to scale things and so just as before if i wanted to talk about some gamma where gamma is a real number and i wanted to multiply that by some vector x then exactly as in our sort of intuitive directed line segment view then we get gamma times x1 gamma times x2 and so on now this feels of course like a direct generalization of the kinds of objects that you're already used to and that's good because we spend a lot of time thinking about rn but it's worth realizing that vectors can be more complicated objects so let me give you one example of how we can generalize our interpretation of what a vector is let's think about functions in particular let's think about cubic polynomials so let's imagine that we have some function let's call it a z and we're going to talk about the coefficients of az as being let's say a zero plus a one z plus a two z squared plus a three z cubed now let's imagine that we have a second function that's defined almost exactly the same way let's call this bz and this is going to be b0 plus b1z plus b2 z squared plus b3 z cubed right and we can immediately see that adding these things together adds the coefficients exactly as we'd expect that is to say that if we think about a z as a vector object plus b z right then what are we going to get we're going to get something that looks like a0 plus b0 plus a1 plus b1 z right plus a squared plus sorry plus a2 plus b2 z squared plus finally a 3 plus b3 z cubed so that's cool so we have a way to take two of these kinds of objects and add them together and then we get another object of the same type so we can think of our set z here as being the set of cubic polynomials similarly we can scale these things and they operate in the way that we'd expect right so we could take our gamma that we've been using and we could talk about gamma times az right and then as before we get gamma a0 plus gamma a1z and so on and it behaves exactly as we'd expect and so from a sort of computational point of view this means that we can treat these these coefficients as points in r4 right there's four numbers that matter here but we can interpret these vectors and interpret the sort of algebra over these vectors as manipulating these these functions with a more complicated interpretation and where things get really cool is that it turns out that you can generalize this to situations that have an unbounded number of coefficients in fact we could think of this as being the arbitrary set of polynomials in which all of the higher order terms are all just i'll just have zero coefficients and all of this math would still push through essentially right because they're all zeros we would just be adding those all of those zeros together and they would stay zero now we're not going to talk about these kinds of infinite dimensional vector spaces in this course but in future courses about machine learning especially kind of more advanced things there's really interesting infinite dimensional objects that it turns out that you can manipulate with finite amounts of computation and if you've ever heard of something like a kernel or a support vector machine that's often the kind of thing that that we're talking about i'd like to give you two more examples that might help you think a little bit more broadly about what a vector is the first is just to think of a color as being a vector you're used to the idea there being rgb colors for red green and blue and you can think of those three numbers as being the components of a color vector we can scale any given color and make it lighter or darker and of course we can add them and blend together different colors here i'm showing a few random scaled sums of the red green and blue vectors sounds give us another example of something that we can think of as a vector we can add together and scale sound waveforms to make a single piece of music from individual instruments here's a simple example that i recorded i've pulled this into python collab as a numpy vector and we can actually listen to it here [Music] from the computer's point of view this really is just a big list of numbers and so if we want we could plot the waveform like i'm doing here this is actually a sum of eight different instruments and eight different waveforms i'm plotting each of them separately here and of course we can listen to each of these separately [Music] [Music] [Music] if we think of these as being vectors then we can scale and add these different eight instruments in order to make new vectors with new compositions so let's weight them with random numbers and then add them up and see what we get [Music] [Music] the point of this isn't that you would actually make music this way so much as to just sort of illustrate the idea that vectors can take different kinds of forms and have different kinds of meanings beyond just directed line segments

Channel: Intelligent Systems Lab

Views: 437

Rating: 5 out of 5

Keywords:

Id: NzSsjZVe7q4

Length: 15min 2sec (902 seconds)

Published: Tue Jan 26 2021