Abstract vector spaces | Chapter 16, Essence of linear algebra

I'd like to revisit a deceptively simple question that I asked in the very first video of this series what are vectors is a 2-dimensional vector for example fundamentally an arrow on a flat plane that we can describe with coordinates for convenience or is it fundamentally that pair of real numbers which is just nicely visualized as an arrow on a flat plane or are both of these just manifestations of something deeper on the one hand defining vectors as primarily being a list of numbers feels clear-cut and unambiguous it makes things like four dimensional vectors or 100 dimensional vectors sound like real concrete ideas that you can actually work with when otherwise an idea like four dimensions is just a vague geometric notion that's difficult to describe without waving your hands a bit but on the other hand a common sensation for those who actually work with linear algebra especially as you get more fluent with changing your basis is that you're dealing with a space that exists independently from the coordinates that you give it and the coordinates are actually somewhat arbitrary depending on what you happen to choose as your basis vectors core topics in linear algebra like determinants and eigenvectors seem indifferent to your choice of coordinate systems the determinant tells you how much of transformation scales areas and eigenvectors are the ones that stay on their own span during a transformation but both of these properties are inherently spatial and you can freely change your coordinate system without changing the underlying values of either one but if vectors are not fundamentally lists of real numbers and if their underlying essence is something more spatial that just begs the question of what mathematicians mean when they use a word like space or spatial to build up to where this is going I'd actually like to spend the bulk of this video talking about something which is neither an arrow nor a list of numbers but also has vector ish qualities functions you see there's a sense in which functions are actually just another type of vector in the same way that you can add two vectors together there's also a sensible notion for adding two functions F and G to get a new function f plus G it's one of those things where you kind of already know what it's going to be but actually phrasing it as a mouthful the output of this new function at any given input like negative four is the sum of the outputs of F and G when you evaluate them each at that same input negative four or more generally the value of the sum function at any given input X is the sum of the values f of X plus G of X this is pretty similar to adding vectors coordinate by coordinate it's just that there are in a sense infinitely many coordinates to deal with similarly there's a sensible notion for scaling a function by a real number just scale all of the outputs by that number and again this is analogous to scaling a vector coordinate by coordinate it just feels like there's infinitely many coordinates now given that the only thing vectors can really do is get added together or scaled it feels like we should be able to take the same useful constructs and problem solving techniques of linear algebra that were originally thought about in the context of arrows in space and apply them to functions as well for example there is a perfectly reasonable notion of a linear transformation for functions something that takes in one function and turns it into another one familiar example comes from calculus the derivative it's something which transforms one function into another function sometimes in this context you'll hear these called operators instead of transformations but the meaning is the same a natural question you might want to ask is what it means for a transformation of functions to be linear the formal definition of linearity is relatively abstract and symbolically driven compared to the way that I first talked about it in chapter three of this series but the reward of abstractness is that we'll get something general enough to apply two functions as well as arrows a transformation is linear if it satisfies two properties commonly called additive 'ti and scaling additive 'ti means that if you add two vectors V and W then apply a transformation to their sum you get the same result as if you added the transformed versions of V and W the scaling property is that when you scale a vector V by some number then apply the transformation you get the same ultimate vector as if you scaled the transformed version of V by that same amount the way you'll often hear this described is that linear transformations preserve the operations of vector addition and scalar multiplication the idea of grid lines remaining parallel and evenly spaced that I've talked about in past videos is really just an illustration of what these two properties mean in the specific case of points in 2d space one of the most important consequences of these properties which makes matrix vector multiplication possible is that a linear transformation is completely described by where it takes the basis vectors since any vector can be expressed by scaling and adding the basis vectors in some way finding the transformed version of a vector comes down to scaling and adding the transformed versions of the basis vectors in that same way as you'll see in just a moment this is as true for functions as it is for arrows for example calculus students are always using the fact that the derivative is additive and has the scaling property even if they haven't heard it phrased that way if you add two functions then take the derivative it's the same as first taking the derivative of each one separately then adding the result similarly if you scale a function then take the derivative it's the same as first taking the derivative then scaling the result to really drill in the parallel let's see what it might look like to describe the derivative with a matrix this will be a little tricky since function spaces have a tendency to be infinite dimensional but I think this exercise is actually quite satisfying let's limit ourself to polynomials things like x squared plus 3x plus 5 or 4x to the seventh minus 5x squared each of the polynomials in our space will only have finitely many terms but the full space is going to include polynomials with arbitrarily large degree the first thing we need to do is give coordinates to this space which requires choosing a basis since polynomials are already written down as the sum of scaled powers of the variable X it's pretty natural to just choose pure powers of X as the basis function in other words our first basis function will be the constant function be 0 of x equals 1 the second basis function will be b1 of x equals x then b2 of x equals x squared then b3 of x equals x cubed and so on the role that these basis functions serve will be similar to the roles of I hat J hat and k hat in the world of vectors as arrows since our polynomials can have arbitrarily large degree this set of basis functions is infinite but that's okay it just means that when we treat our polynomials as vectors they're going to have infinitely many coordinates a polynomial like x squared plus 3x plus 5 for example would be described with the coordinates 5 3 1 then infinitely many zeros you'd read this as saying that it's 5 times the first basis function plus 3 times that second basis function plus 1 times the third basis function and then none of the other basis functions should be added from that point on the polynomial 4x to the seventh minus 5x squared would have the coordinates 0 0 negative 5 0 0 0 0 4 then an infinite string of zeros in general since every individual polynomial has only finitely many terms its coordinates will be some finite string of numbers with an infinite tail of zeros in this coordinate system the derivative is described with an infinite matrix that's mostly full of zeros but which has the positive integers counting down on this offset diagonal I'll talk about how you could find this matrix in just a moment but the best way to get a feel for it is to just watch it in action take the coordinates representing the polynomial X cubed plus 5x squared plus 4x plus 5 then put those coordinates on the right of the matrix the only term that contributes to the first coordinate of the result is one times four which means the constant term and the result will be four this corresponds to the fact that the derivative of 4x is the constant for the only term contributing to the second coordinate of the matrix vector product is two times five which means the coefficient in front of X in the derivative is 10 that one corresponds to the derivative of 5x squared similarly the third coordinate in the matrix vector product comes down to taking 3 times 1 this one corresponds to the derivative of x cubed being 3x squared and after that it'll be nothing but zeros what makes this possible is that the derivative is linear and for those of you who like to pause and ponder you could construct this matrix by taking the derivative of each basis function and putting the coordinates of the results in each column you so surprisingly matrix-vector multiplication and taking a derivative which at first seemed like completely different animals are both just really members of the same family in fact most of the concepts I've talked about in this series with respect to vectors as arrows in space things like the dot product or eigenvectors have direct analogs in the world of functions though sometimes they go by different names things like inner product or eigenfunction so back to the question of what is a vector the point I want to make here is that there are lots of vector ish things in math as long as you're dealing with a set of objects where there's a reasonable notion of scaling and adding whether that's a set of arrows and space lists of numbers functions or whatever other crazy thing you choose to define all of the tools developed in linear algebra regarding vectors linear transformations and all that stuff should be able to apply take a moment to imagine yourself right now as a mathematician developing the theory of linear algebra you want all of the definitions and discoveries of your work to apply to all of the vectors things in full generality not just to one specific case these sets of vectors things like arrows or lists of numbers or functions are called vector spaces and what you as the mathematician might want to do is say hey everyone I don't want to have to think about all the different types of crazy vector spaces that you all might come up with so what you do is establish a list of rules the vector addition and scaling have to abide by these rules are called axioms and in the modern theory of linear algebra there are eight axioms that any vector space must satisfy if all of the theory and constructs that we've discovered are going to apply I'll leave them on the screen here for anyone who wants to pause and ponder but basically it's just a checklist to make sure that the notions of vector addition and scalar multiplication do the things that you'd expect them to do these axioms are not so much fundamental rules of nature as they are an interface between you the mathematician discovering results and other people who might want to apply those results to new sorts of vector spaces if for example someone to find some crazy type of vector space like the set of all pie creatures with some definition of adding and scaling pie creatures these axioms are like a checklist of things that they need to verify about their definitions before they can start applying the results of linear algebra and you as the mathematician never have to think about all the possible crazy vector spaces people might define you just have to prove your results in terms of these axioms so anyone whose definitions satisfy those axioms can happily apply your results even if you never thought about their situation as a consequence you tend to phase all of your results pretty abstractly which is to say only in terms of these axioms rather than centering on a specific type of vector like arrows in space or functions for example this is why just about every textbook you'll find will define linear transformations in terms of additive 'ti and scaling rather than talking about grid lines remaining parallel and evenly spaced even though the latter is more intuitive and at least in my view more helpful for first-time learners even if it is specific to one situation so the mathematicians answer to what our vectors is to just ignore the question in the modern theory the form that vectors take doesn't really matter Aeros lists of numbers functions pie creatures really it can be anything so long as there's some notion of adding and scaling vectors that follows these rules it's like asking what the number three really is whenever it comes up concretely it's in the context of some triplet of things but in math it's treated as an abstraction for all possible Triplets of things and lets you reason about all possible triplets using a single idea same goes with vectors which have many embodiments but math abstracts them all into a single intangible notion of a vector space but as anyone watching this series knows I think it's better to begin reasoning about vectors in a concrete visualizable setting like 2d space with arrows rooted at the origin but as you learn more linear algebra know that these tools apply much more generally and that this is the underlying reason why textbooks and lectures tend to be phrased well abstractly so with that folks I think I'll call it an into this essence of linear algebra series if you've watched and understood the videos I really do believe that you have a solid foundation in the underlying intuitions of linear algebra this is not the same thing as learning the full topic of course that's something that can only really come from working through problems but the learning you do moving forward could be substantially more efficient if you have all the right intuitions in place so have fun applying those intuitions and best of luck with your future learning [Music]

Channel: 3Blue1Brown

Views: 896,014

Rating: 4.9821653 out of 5

Keywords: blue, vector space, 3brown1blue, math, 3 brown 1 blue, one, abstract vector space, three, 3b1b, brown, linear algebra, three brown one blue

Id: TgKwz5Ikpc8

Length: 16min 46sec (1006 seconds)

Published: Sat Sep 24 2016

Reddit Comments

[u/[deleted]]

This series is the gold standard of educational math videos.

[u/StormStooper]

The "rules" of vector spaces seemed to relate in a weird sense to group theory and the "rules" of groups, fields, and rings. Can anyone help me see this connection more concretely?

[u/lucasvb]

Such an awesome series! Thanks for the great work, /u/3blue1brown!

As for the message at the end, like I like say, Ceci n'est pas un vecteur!

[u/BittyTang]

He didn't say this explicitly, but it's worth noting that ANY (finite dimensional) "abstract" vector space (using the field of real numbers) is isomorphic to Euclidean space (Rⁿ ). So you can come up with any whacky vector space and describe linear transformations on it with a matrix of real numbers.

[u/r4and0muser9482]

I guess word2vec could be a cool example of one such "abstract vector space"?

[u/seanziewonzie]

I hope I get to TA an Intro to Linear Algebra course soon because I'm going to push this series heavily.

[u/PJBthefirst]

This was a beautiful final video for the series. It all makes so much sense now!

[u/NervousBlackRabbit]

As much as I love linear algebra now, I remember my first year linear algebra course defining a vector space in the 2nd or 3rd lecture. It seemed like a joke... entirely unmotivated.

When I finally understood it's purpose (taking the common properties of euclidean vectors and functions into a unified abstract framework), it was like a revelation, but that didn't come until long after the course was already done.

I'm glad this video explains things the right way, starting with the concrete and building to the abstract.

[u/mullemeckmannen]

really liked this video, good insperation for someone 3 weeks in to solving boring systems of equations