This right here is what we're going to build to, this video: A certain animated approach to thinking about a super-important idea from math: The Fourier transform. For anyone unfamiliar with what that is, my #1 goal here is just for the video to be an introduction to that topic. But even for those of you who are already familiar with it, I still think that there's something fun and enriching about seeing what all of its components actually look like. The central example, to start, is gonna be the classic one: Decomposing frequencies from sound. But after that, I also really wanna show a glimpse of how this idea extends well beyond sound and frequency, and to many seemingly disparate areas of math, and even physics. Really, it is crazy just how ubiquitous this idea is. Let's dive in. This sound right here is a pure A. 440 beats per second. Meaning, if you were to measure the air pressure right next to your headphones, or your speaker, as a function of time, it would oscillate up and down around its usual equilibrium, in this wave. making 440 oscillations each second. A lower-pitched note, like a D, has the same structure, just fewer beats per second. And when both of them are played at once, what do you think the resulting pressure vs. time graph looks like? Well, at any point in time, this pressure difference is gonna be the sum of what it would be for each of those notes individually. Which, let's face it, is kind of a complicated thing to think about. At some points, the peaks match up with each other, resulting in a really high pressure. At other points, they tend to cancel out. And all in all, what you get is a wave-ish pressure vs. time graph, that is not a pure sine wave; it's something more complicated. And as you add in other notes, the wave gets more and more complicated. But right now, all it is is a combination of four pure frequencies. So it seems...needlessly complicated, given the low amount of information put into it. A microphone recording any sound just picks up on the air pressure at many different points in time. It only "sees" the final sum. So our central question is gonna be how you can take a signal like this, and decompose it into the pure frequencies that make it up. Pretty interesting, right? Adding up those signals really mixes them all together. So pulling them back apart...feels akin to unmixing multiple paint colors that have all been stirred up together. The general strategy is gonna be to build for ourselves a mathematical machine that treats signals with a given frequency... ..differently from how it treats other signals. To start, consider simply taking a pure signal say, with a lowly three beats per second, so that we can plot it easily. And let's limit ourselves to looking at a finite portion of this graph. In this case, the portion between zero seconds, and 4.5 seconds. The key idea, is gonna be to take this graph, and sort of wrap it up around a circle. Concretely, here's what I mean by that. Imagine a little rotating vector where each point in time its length is equal to the height of our graph for that time. So, high points of the graph correspond to a greater disance from the origin, and low points end up closer to the origin. And right now, I'm drawing it in such a way that moving forward two seconds in time corresponds to a single rotation around the circle. Our little vector drawing this wound up graph is rotating at half a cycle per second. So, this is important.
There are two different frequencies at play here: There's the frequency of our signal, which goes up and down, three times per second. And then, separately, there's the frequency with which we're wrapping the graph around the circle. Which, at the moment, is half of a rotation per second. But we can adjust that second frequency however we want. Maybe we want to wrap it around faster... ..or maybe we go and wrap it around slower. And that choice of winding frequency determines what the wound up graph looks like. Some of the diagrams that come out of this can be pretty complicated; although, they are very pretty. But it's important to keep in mind that all that's happening here is that we're wrapping the signal around a circle. The vertical lines that I'm drawing up top, by the way, are just a way to keep track of the distance on the original graph that corresponds to a full rotation around the circle. So, lines spaced out by 1.5 seconds would mean it takes 1.5 seconds to make one full revolution. And at this point, we might have some sort of vague sense that something special will happen when the winding frequency matches the frequency of our signal: three beats per second. All the high points on the graph happen on the right side of the circle And all of the low points happen on the left. But how precisely can we take advantage of that in our attempt to build a frequency-unmixing machine? Well, imagine this graph is having some kind of mass to it, like a metal wire. This little dot is going to represent the center of mass of that wire. As we change the frequency, and the graph winds up differently, that center of mass kind of wobbles around a bit. And for most of the winding frequencies, the peaks and valleys are all spaced out around the circle in such a way that the center of mass stays pretty close to the origin. But! When the winding frequency is the same as the frequency of our signal, in this case, three cycles per second, all of the peaks are on the right, and all of the valleys are on the left.. ..so the center of mass is unusually far to the right. Here, to capture this, let's draw some kind of plot that keeps track of where that center of mass is for each winding frequency. Of course, the center of mass is a two-dimensional thing, and requires two coordinates to fully keep track of, but for the moment, let's only keep track of the x coordinate. So, for a frequency of 0, when everything is bunched up on the right, this x coordinate is relatively high. And then, as you increase that winding frequency, and the graph balances out around the circle, the x coordinate of that center of mass goes closer to 0, and it just kind of wobbles around a bit. But then, at three beats per second, there's a spike as everything lines up to the right. This right here is the central construct, so let's sum up what we have so far: We have that original intensity vs. time graph, and then we have the wound up version of that in some two-dimensional plane, and then, as a third thing, we have a plot for how the winding frequency influences the center of mass of that graph. And by the way, let's look back at those really low frequencies near 0. This big spike around 0 in our new frequency plot just corresponds to the fact that the whole cosine wave is shifted up. If I had chosen a signal oscillates around 0, dipping into negative values, then, as we play around with various winding frequences, this plot of the winding frequencies vs. center of mass would only have a spike at the value of three. But, negative values are a little bit weird and messy to think about especially for a first example, so let's just continue thinking in terms of the shifted-up graph. I just want you to understand that that spike around 0 only corresponds to the shift. Our main focus, as far as frequency decomposition is concerned, is that bump at three. This whole plot is what I'll call the "Almost Fourier Transform" of the original signal. There's a couple small distinctions between this and the actual Fourier transform, which I'll get to in a couple minutes, but already, you might be able to see how this machine lets us pick out the frequency of a signal. Just to play around with it a little bit more, take a different pure signal, let's say with a lower frequency of two beats per second, and do the same thing. Wind it around a circle, imagine different potential winding frequencies, and as you do that keep track of where the center of mass of that graph is, and then plot the x coordinate of that center of mass as you adjust the winding frequency. Just like before, we get a spike when the winding frequency is the same as the signal frequency, which in this case, is when it equals two cycles per second. But the real key point, the thing that makes this machine so delightful, is how it enables us to take a signal consisting of multiple frequencies, and pick out what they are. Imagine taking the two signals we just looked at: The wave with three beats per second, and the wave with two beats per second, and add them up. Like I said earlier, what you get is no longer a nice, pure cosine wave; it's something a little more complicated. But imagine throwing this into our winding-frequency machine... ..it is certainly the case that as you wrap this thing around, it looks a lot more complicated; you have this chaos (1) and chaos (2) and chaos (3) and chaos (4) and then WOOP! Things seem to line up really nicely at two cycles per second, and as you continue on it's more chaos (5) and more chaos (6) more chaos (7) chaos (8), chaos (9), chaos (10), WOOP! Things nicely align again at three cycles per second. And, like I said before, the wound up graph can look kind of busy and complicated, but all it is is the relatively simple idea of wrapping the graph around a circle. It's just a more complicated graph, and a pretty quick winding frequency. Now what's going on here with the two different spikes, is that if you were to take two signals, and then apply this Almost-Fourier transform to each of them individually, and then add up the results, what you get is the same as if you first added up the signals, and then applied this Almost-Fourier transorm. And the attentive viewers among you might wanna pause and ponder, and... ..convince yourself that what I just said is actually true. It's a pretty good test to verify for yourself that it's clear what exactly is being measured inside this winding machine. Now this property makes things really useful to us, because the transform of a pure frequency is close to 0 everywhere except for a spike around that frequency. So when you add together two pure frequencies, the transform graph just has these little peaks above the frequencies that went into it. So this little mathematical machine does exactly what we wanted. It pulls out the original frequencies from their jumbled up sums, unmixing the mixed bucket of paint. And before continuing into the full math that describes this operation, let's just get a quick glimpse of one context where this thing is useful: Sound editing. Let's say that you have some recording, and it's got an annoying high pitch that you'd like to filter out. Well, at first, your signal is coming in as a function of various intensities over time. Different voltages given to your speaker from one millisecond to the next. But we want to think of this in terms of frequencies, so, when you take the Fourier transform of that signal, the annoying high pitch is going to show up just as a spike at some high frequency. Filtering that out, by just smushing the spike down, what you'd be looking at is the Fourier transform of a sound that's just like your recording, only without that high frequency. Luckily, there's a notion of an inverse Fourier transform that tells you which signal would have produced this as its Fourier transform. I'll be talking about inverse much more fully in the next video, but long story short, applying the Fourier transform to the Fourier transform gives you back something close to the original function. Mm, kind of... this is... ..a little bit of a lie, but it's in the direction of the truth. And most of the reason that it's a lie is that I still have yet to tell you what the actual Fourier Transform is, since it's a little more complex than this x-coordinate-of-the-center-of-mass idea. First off, bringing back this wound up graph, and looking at its center of mass, the x coordinate is really only half the story, right? I mean, this thing is in two dimensions, it's got a y coordinate as well. And, as is typical in math, whenever you're dealing with something two-dimensional, it's elegant to think of it as the complex plane, where this center of mass is gonna be a complex number, that has both a real and an imaginary part. And the reason for talking in terms of complex numbers, rather than just saying, "It has two coordinates," is that complex numbers lend themselves to really nice descriptions of things that have to do with winding, and rotation. For example: Euler's formula famously tells us that if you take e to some number times i, you're gonna land on the point that you get if you were to walk that number of units around a circle with radius 1, counter-clockwise starting on the right. So, imagine you wanted to describe rotating at a rate of one cycle per second. One thing that you could do is take the expression "e^2π*i*t," where t is the amount of time that has passed. Since, for a circle with radius 1, 2π describes the full length of its circumference. And... this is a little bit dizzying to look at, so maybe you wanna describe a different frequency... ..something lower and more reasonable... ..and for that, you would just multiply that time t in the exponent by the frequency, f. For example, if f was one tenth, then this vector makes one full turn every ten seconds, since the time t has to increase all the way to ten before the full exponent looks like 2πi. I have another video giving some intuition on why this is the behavior of e^x for imaginary inputs, if you're curious 😉, but for right now, we're just gonna take it as a given. Now why am I telling you this you this, you might ask. Well, it gives us a really nice way to write down the idea of winding up the graph into a single, tight little formula. First off, the convention in the context of Fourier transforms is to think about rotating in the clockwise direction, so let's go ahead and throw a negative sign up into that exponent. Now, take some function describing a signal intensity vs. time, like this pure cosine wave we had before, and call it g(t). If you multiply this exponential expression times g(t), it means that the rotating complex number is getting scaled up and down according to the value of this function. So you can think of this little rotating vector with its changing length as drawing the wound up graph. So think about it, this is awesome. This really small expression is a super-elegant way to encapsulate the whole idea of winding a graph around a circle with a variable frequency f. And remember, that thing we want to do with this wound up graph is to track its center of mass. So think about what formula is going to capture that. Well, to approximate it at least, you might sample a whole bunch of times from the original signal, see where those points end up on the wound up graph, and then just take an average. That is, add them all together, as complex numbers, and then divide by the number of points that you've sampled. This will become more accurate if you sample more points which are closer together. And in the limit, rather than looking at the sum of a whole bunch of points divided by the number of points, you take an integral of this function, divided by the size of the time interval that we're looking at. Now the idea of integrating a complex-valued function might seem weird, and to anyone who's shaky with calculus, maybe even intimidating, but the underlying meaning here really doesn't require any calculus knowledge. The whole expression is just the center of mass of the wound up graph. So... Great! Step-by-step, we have built up this kind of complicated, but, let's face it, surprisingly small expression for the whole winding machine idea that I talked about. And now, there is only one final distinction to point out between this and the actual, honest-to-goodness Fourier transform. Namely, just don't divide out by the time interval. The Fourier transform is just the integral part of this. What that means is that instead of looking at the center of mass, you would scale it up by some amount. If the portion of the original graph you were using spanned three seconds, you would multiply the center of mass by three. If it was spanning six seconds, you would multiply the center of mass by six. Physically, this has the effect that when a certain frequency persists for a long time, then the magnitude of the Fourier transform at that frequency is scaled up more and more. For example, what we're looking at right here is how when you have a pure frequency of two beats per second, and you wind it around the graph at two cycles per second, the center of mass stays in the same spot, right? It's just tracing out the same shape. But the longer that signal persists, the larger the value of the Fourier transform, at that frequency. For other frequencies, though, even if you just increase it by a bit, this is cancelled out by the fact that for longer time intervals you're giving the wound up graph more of a chance to balance itself around the circle. That is...a lot of different moving parts, so let's step back and summarize what we have so far. The Fourier transform of an intensity vs. time function, like g(t), is a new function, which doesn't have time as an input, but instead takes in a frequency, what I've been calling "the winding frequency." In terms of notation, by the way, the common convention is to call this new function "g-hat," with a little circumflex on top of it. Now the output of this function is a complex number, some point in the 2D plane, that corresponds to the strength of a given frequency in the original signal. The plot that I've been graphing for the Fourier transform, is just the real component of that output, the x-coordinate But you could also graph the imaginary component separately, if you wanted a fuller description. And all of this is being encapsulated inside that formula that we built up. And out of context, you can imagine how seeing this formula would seem sort of daunting. But if you understand how exponentials correspond to rotation... ..how multiplying that by the function g(t) means drawing a wound up version of the graph, and how an integral of a complex-valued function can be interpreted in terms of a center-of-mass idea, you can see how this whole thing carries with it a very rich, intuitive meaning. And, by the way, one quick small note before we can call this wrapped up. Even though in practice, with things like sound editing, you'll be integrating over a finite time interval, the theory of Fourier transforms is often phrased where the bounds of this integral are -∞ and ∞. Concretely, what that means is that you consider this expression for all possible finite time intervals, and you just ask, "What is its limit as that time interval grows to ∞?" And...man, oh man, there is so much more to say! So much, I don't wanna call it done here. This transform extends to corners of math well beyond the idea of extracting frequencies from signal. So, the next video I put out is gonna go through a couple of these, and that's really where things start getting interesting. So, stay subscribed for when that comes out, or an alternate option is to just binge a couple 3blue1brown videos so that the YouTube recommender is more inclined to show you new things that come out... ..really, the choice is yours! And to close things off, I have something pretty fun: A mathematical puzzler from this video's sponsor, Jane Street, who's looking to recruit more technical talent. So, let's say that you have a closed, bounded convex set C sitting in 3D space, and then let B be the boundary of that space, the surface of your complex blob. Now imagine taking every possible pair of points on that surface, and adding them up, doing a vector sum. Let's name this set of all possible sums D. Your task is to prove that D is also a convex set. So, Jane Street is a quantitative trading firm, and if you're the kind of person who enjoys math and solving puzzles like this, the team there really values intellectual curiosity. So, they might be interested in hiring you. And they're looking both for full-time employees and interns. For my part, I can say that some people I've interacted with there just seem to love math, and sharing math, and when they're hiring they look less at a background in finance than they do at how you think, how you learn, and how you solve problems, hence the sponsorship of a 3blue1brown video. If you want the answer to that puzzler, or to learn more about what they do, or to apply for open positions, go to janestreet.com/3b1b/
The world needs more 3Blue1Browns
I posted this in the youtube comments, but here's a live demo of what 3B1B talks about, wrapping a signal around a circle and summing it up. It's in presentation form so click through it and wait (its quite CPU intensive)
This is very different from what I learned.
This is just a nitpick, but wouldn't it have been "more accurate" to visualize the displacement as the norm of the complex number, instead of just considering the real part?
Wow, that was fantastic. I had heard about this channel, but I am definitely going to binge it now for the cool visuals and intuitive explanations.
Anyone know hows to visualize it in desmos?
This was deeply interesting to me on a number of levels. For totally separate reasons, I have worked a decent amount with the (fast) fourier transform and the rose function, which I used in an app to create shapes exactly like the ones shown in this video. I had looked for a long time for an explanation like this and could never find one. Thanks for posting
I took and passed a course about Fourier transforms for my degree 10 years ago and held on to the textbook because i still didn't understand just wtf I had learned. I think I get it now. Lol
How(more like why) do dirac delta "functions" come into play with FTs of sines and cosines (I know they do when you solve those integrals, but there must be some intuition)?