Quoting Steven Strogatz, “Since Newton,
mankind has come to realize that the laws of physics are always expressed in the language
of differential equations.” Of course, this language is spoken well beyond the boundaries
of physics as well, and being able to speak it and read it adds a new color to how you
view the world around you. In the next few videos, I want to give a sort
of tour of this topic. To aim is to give a big picture view of what this part of math
is all about, while at the same time being happy to dig into the details of specific
examples as they come along. I’ll be assuming you know the basics of
calculus, like what derivatives and integrals are, and in later videos we’ll need some
basic linear algebra, but not much beyond that. Differential equations arise whenever it’s
easier to describe change than absolute amounts. It’s easier to say why population sizes
grow or shrink than it is to describe why the have the particular values they do at
some point in time; It may be easier to describe why your love for someone is changing than
why it happens to be where it is now. In physics, more specifically Newtonian mechanics, motion
is often described in terms of force. Force determines acceleration, which is a statement
about change. These equations come in two flavors; Ordinary
differential equations, or ODEs, involving functions with a single input, often thought
of as time, and Partial differential equations, or PDEs, dealing with functions that have
multiple inputs. Partial derivatives are something we’ll look at more closely in the next video;
you often think of them involving a whole continuum of values changing with time, like
the temperature of every point in a solid body, or the velocity of a fluid at every
point in space. Ordinary differential equations, our focus for now, involve only a finite collection
of values changing with time. It doesn’t have to be time, per se, your
one independent variable could be something else, but things changing with time are the
prototypical and most common examples of differential equations.
Physics (simple) Physics offers a nice playground for us here,
with simple examples to start with, and no shortage of intricacy and nuance as we delve
deeper. As a nice warmup, consider the trajectory
of something you throw in the air. The force of gravity near the surface of the earth causes
things to accelerate downward at 9.8 m/s per second. Now unpack what that really means:
If you look at some object free from other forces, and record its velocity every second,
these vectors will accrue an additional downward component of 9.8 m/s every second. We call
this constant 9.8 “g”. This gives an example of a differential equation,
albeit a relatively simple one. Focus on the y-coordinate, as a function of time. It’s
derivative gives the vertical component of velocity, whose derivative in turn gives the
vertical component of acceleration. For compactness, let’s write this first derivative as y-dot,
and the second derivative as y-double-dot. Our equation is simply y-double-dot = -g.
This is one where you can solve by integrating, which is essentially working backwards. First,
what is velocity, what function has -g as a derivative? Well, -g*t. Or rather, -g*t
+ (the initial velocity). Notice that you have this degree of freedom which is determined
by an initial condition. Now what function has this as a derivative? -(½)g*t^2 + v_0
* t. Or, rather, add in a constant based on whatever the initial position is. Things get more interesting when the forces
acting on a body depend on where that body is. For example, studying the motion of planets,
stars and moons, gravity can no longer be considered a constant. Given two bodies, the
pull on one is in the direction of the other, with a strength inversely proportional to
the square of the distance between them. As always, the rate of change of position
is velocity, but now the rate of change of velocity is some function of position. The
dance between these mutually-interacting variables is mirrored in the dance between the mutually-interacting
bodies which they describe. So often in differential equations, the puzzles
you face involve finding a function whose derivative and/or higher order derivatives
are defined in terms of itself. In physics, it’s most common to work with
second order differential equations, which means the highest derivative you find in the
expression here is a second derivative. Higher order differential equations would be ones
with third derivatives, fourth derivatives and so on; puzzles with more intricate clues. The sensation here is one of solving an infinite
continuous jigsaw puzzle. In a sense you have to find infinitely many numbers, one for each
point in time, constrained by a very specific way that these values intertwine with their
own rate of change, and the rate of change of that rate of change. I want you to take some time digging in to
a deceptively simple example: A pendulum. How does this angle theta that it makes with
the vertical change as a function of time. This is often given as an example in introductory
physics classes of harmonic motion, meaning it oscillates like a sine wave. More specifically,
one with a period of 2pi * L/g, where L is the length of the pendulum, and g is gravity. However, these formulas are actually lies.
Or, rather, approximations which only work in the realm of small angles. If you measured
an actual pendulum, you’d find that when you pull it out farther, the period is longer
than what that high-school physics formulas would suggest. And when you pull it really
far out, the value of theta vs. time doesn’t even look like a sine wave anymore. First thing’s first, let’s set up the
differential equation. We’ll measure its position as a distance x along this arc. If
the angle theta we care about is measured in radians, we can write x and L*theta, where
L is the length of the pendulum. As usual, gravity pulls down with acceleration
g, but because the pendulum constrains the motion of this mass, we have to look at the
component of this acceleration in the direction of motion. A little geometry exercise for
you is to show that this little angle here is the same as our theta. So the component
of gravity in the direction of motion, opposite this angle, will be -g*sin(theta). Here we’re considering theta to be positive
when the pendulum is swung to the right, and negative when it’s swung to the left, and
this negative sign in the acceleration indicates that it’s always pointed in the opposite
direction from displacement. So the second derivative of x, the acceleration, is -g*sin(theta).
Since x is L*theta, that means the second derivative of theta is -(g/L) * sin(theta).
To be somewhat more realistic, let’s add in a term to account for air resistance, which
perhaps we model as being proportional to the velocity. We write this as -mu * theta-dot,
where -mu is some constant determining how quickly the pendulum loses energy. This is a particularly juicy differential
equation. Not easy to solve, but not so hard that we can’t reasonably get some meaningful
understanding of it. At first you might think that this sine function
relates to the sine wave pattern for the pendulum. Ironically, though, what you'll eventually
find is that the opposite is true. The presence of the sine in this equation is precisely
why the real pendulum doesn't oscillate with the sine wave pattern. If that sounds odd, consider the fact that
here, the sine function takes theta as an input, but the approximate solution has the
value theta itself oscillating as a sine wave. Clearly something fishy is afoot. One thing I like about this example is that
even though it’s comparatively simple, it exposes an important truth about differential
equations that you need to be grapple with: They’re really freaking hard to solve. In this case, if we remove the damping term,
we can just barely write down an analytic solution, but it’s hilariously complicated,
involving all these functions you’re probably never heard of written in terms of integrals
and weird inverse integral problems. Presumably, the reason for finding a solution
is to then be able to make computations, and to build an understanding for whatever dynamics
your studying. In a case like this, those questions have just been punted off to figuring
out how to compute and understand these new functions. And more often, like if we add back this dampening
term, there is not a known way to write down an exact solution analytically. Well, for
any hard problem you could just define a new function to be the answer to that problem.
Heck, even name it after yourself if you want. But again, that’s pointless unless it leads
you to being able to compute and understand the answer. So instead, in studying differential equations,
we often do a sort of short-circuit and skip the actual solution part, and go straight
to building understanding and making computations from the equations alone. Let me walk through
what that might look like with the Pendulum. Phase space
What do you hold in your head, or what visualization could you get some software to pull up for
you, to understand the many possible ways a pendulum governed by these laws might evolve
depending on its starting conditions? You might be tempted to try imagining the
graph of theta(t), and somehow interpreting how its position, slope, and curvature all
inter-relate. However, what will turn out to be both easier and more general is to start
by visualizing all possible states of the system in a 2d plane. The state of the pendulum can be fully described
by two numbers, the angle, and the angular velocity. You can freely change these two
values without necessarily changing the other, but the acceleration is purely a function
of these two values. So each point of this 2d plane fully describes the pendulum at a
given moment. You might think of these as all possible initial conditions of the pendulum.
If you know this initial angle and angular velocity, that’s enough to predict how the
system will evolve as time moves forward. If you haven’t worked with them, these sorts
of diagrams can take a little getting used to. What you’re looking at now, this inward
spiral, is a fairly typical trajectory for our pendulum, so take a moment to think carefully
about what’s being represented. Notice how at the start, as theta decreases, theta-dot
gets more negative, which makes sense because the pendulum moves faster in the leftward
direction as it approaches the bottom. Keep in mind, even though the velocity vector on
this pendulum is pointed to the left, the value of that velocity is being represented
by the vertical component of our space. It’s important to remind yourself that this state
space is abstract, and distinct from the physical space where the pendulum lives and moves. Since we’re modeling it as losing some energy
to air resistance, this trajectory spirals inward, meaning the peak velocity and displacement
each go down by a bit with each swing. Our point is, in a sense, attracted to the origin
where theta and theta-dot both equal 0. With this space, we can visualize a differential
equation as a vector field. Here, let me show you what I mean. The pendulum state is this vector, [theta,
theta-dot]. Maybe you think of it as an arrow, maybe as a point; what matters is that it
has two coordinates, each a function of time. Taking the derivative of that vector gives
you its rate of change; the direction and speed that it will tend to move in this diagram.
That derivative is a new vector, [theta-dot, theta-double-dot], which we visualize as being
attached to the relevant point in this space. Take a moment to interpret what this is saying. The first component for this rate-of-change
vector is theta-dot, so the higher up we are on the digram, the more the point tends to
move to the right, and the lower we are, the more it tends to move to the left. The vertical
component is theta-double-dot, which our differential equation lets us rewrite entirely in terms
of theta and theta-dot. In other words, the first derivative of our state vector is some
function of that vector itself. Doing the same at all points of this space
will show how the state tends to change from any position, artificially scaling down the
vectors when we draw them to prevent clutter, but using color to loosely indicate magnitude. Notice that we’ve effectively broken up
a single second order equation into a system of two first order equations. You might even
give theta-dot a different name to emphasize that we’re thinking of two separate values,
intertwined via this mutual effect they have on one and other’s rate of change. This
is a common trick in the study of differential equations, instead of thinking about higher
order changes of a single value, we often prefer to think of the first derivative of
vector values. In this form, we have a nice visual way to
think about what solving our equation means: As our system evolves from some initial state,
our point in this space will move along some trajectory in such a way that at every moment,
the velocity of that point matches the vector from this vector field. Keep in mind, this
velocity is not the same thing as the physical velocity of our pendulum. It’s a more abstract
rate of change encoding the changes in both theta and theta-dot. You might find it fun to pause for a moment
and think through what exactly some of these trajectory lines say about possible ways the
pendulum evolves for different starting conditions. For example, in regions where theta-dot is
quite high, the vectors guide the point to travel to the right quite a ways before settling
down into an inward spiral. This corresponds to a pendulum with a high initial velocity,
fully rotating around several times before settling down into a decaying back and forth. Having a little more fun, when I tweak this
air resistance term mu, say increasing it, you can immediately see how this will result
in trajectories that spiral inward faster, which is to say the pendulum slows down faster.
Imagine you saw the equations out of context, not knowing they described a pendulum; it’s
not obvious just-looking at them that increasing the value of mu means the system tends towards
some attracting state faster, so getting some software to draw these vector fields for you
can be a great way to gain an intuition for how they behave. What’s wonderful is that any system of ordinary
differential equations can be described by a vector field like this, so it’s a very
general way to get a feel for them. Usually, though, they have many more dimensions.
For example, consider the famous three-body problem, which is to predict how three masses
in 3d space will evolve if they act on each other with gravity, and you know their initial
positions and velocities. Each mass has three coordinates describing
its position and three more describing its momentum, so the system has 18 degrees of
freedom, and hence an 18-dimensional space of possible states. It’s a bizarre thought,
isn’t it? A single point meandering through and 18-dimensional space we cannot visualize,
obediently taking steps through time based on whatever vector it happens to be sitting
on from moment to moment, completely encoding the positions and momenta of 3 masses in ordinary,
physical, 3d space. (In practice, by the way, you can reduce this
number of dimension by taking advantage of the symmetries in your setup, but the point
of more degrees of freedom resulting in a higher-dimensional state space remains the
same). In math, we often call a space like this a
“phase space”. You’ll hear me use the term broadly for spaces encoding all kinds
of states for changing systems, but you should know that in the context of physics, especially
Hamiltonian mechanics, the term is often reserved for a special case. Namely, a space whose
axes represent position and momentum. So a physicist would agree that the 18-dimension
space describing the 3-body problem is a phase space, but they might ask that we make a couple
of modifications to our pendulum set up for it to properly deserve the term. For those
of you who watched the block collision videos, the planes we worked with there would happily
be called phase spaces by math folk, though a physicist might prefer other terminology.
Just know that the specific meaning may depend on your context. It may seem like a simple idea, depending
on how well indoctrinated you are to modern ways of thinking about math, but it’s worth
keeping in mind that it took humanity quite a while to really embrace thinking of dynamics
spatially like this, especially when the dimensions get very large. In his book Chaos, James Gleick
describes phase space as “one of the most powerful inventions of modern science.” One reason it’s powerful is that you can
ask questions not just about a single initial state, but a whole spectrum of initial states.
The collection of all possible trajectories is reminiscent of a moving fluid, so we call
it phase flow. To take one example of why phase flow is a
fruitful formulation, the origin of our space corresponds to the pendulum standing still;
and so does this point over here, representing when the pendulum is balanced upright. These
are called fixed points of the system, and one natural question to ask is whether they
are stable. That is, will tiny nudges to the system result in a state that tends back towards
the stable point or away from it. Physical intuition for the pendulum makes the answer
here obvious, but how would you think about stability just by looking at the equations,
say if they arose from some completely different and less intuitive context? We’ll go over how to compute the answer
to a question like this in following videos, and the intuition for the relevant computations
are guided heavily by the thought of looking at a small region in this space around the
fixed point and asking about whether the flow contracts or expands its points. Speaking of attraction and stability, let’s
take a brief sidestep to talk about love. The Strogatz quote I referenced earlier comes
from a whimsical column in the New York Times on mathematical models of love, an example
well worth pilfering to illustrate that we’re not just talking about physics. Imagine you’ve been flirting with someone,
but there’s been some frustrating inconsistency to how mutual the affections seem. And perhaps
during a moment when you turn your attention towards physics to keep your mind off this
romantic turmoil, mulling over your broken up pendulum equations, you suddenly understand
the on-again-off-again dynamics of your flirtation. You’ve noticed that your own affections
tend to increase when your companion seems interested in you, but decrease when they
seem colder. That is, the rate of change for your love is proportional to their feelings
for you. But this sweetheart of yours is precisely
the opposite: Strangely attracted to you when you seem uninterested, but turned off once
you seem too keen. The phase space for these equations looks
very similar to the center part of your pendulum diagram. The two of you will go back and forth
between affection and repulsion in an endless cycle. A metaphor of pendulum swings in your
feelings would not just be apt, but mathematically verified. In fact, if your partner’s feelings
were further slowed when they feel themselves too in love, let’s say out of a fear of
being made vulnerable, we’d have a term matching the friction of your pendulum, and
you two would be destined to an inward spiral towards mutual ambivalence. I hear wedding
bells already. The point is that two very different-seeming
laws of dynamics, one from physics initially involving a single variable, and another from...er...chemistry
with two variables, actually have a very similar structure, easier to recognize when looking
at their phase spaces. Most notably, even though the equations are different, for example
there’s no sine in your companion’s equation, the phase space exposes an underlying similarity
nevertheless. In other words, you’re not just studying
a pendulum right now, the tactics you develop to study one case have a tendency to transfer
to many others. Okay, so phase diagrams are a nice way to
build understanding, but what about actually computing the answer to our equation? Well,
one way to do this is to essentially simulate what the world will do, but using finite time
steps instead of the infinitesimals and limits defining calculus. The basic idea is that if you’re at some
point on this phase diagram, take a step based on whatever vector your sitting on for some
small time step, delta-t. Specifically, take a step of delta-T times that vector. Remember,
in drawing this vector field, the magnitude of each vector has been artificially scaled
down to prevent clutter. Do this repeatedly, and your final location will be an approximation
of theta(t), where t is the sum of all your time steps. If you think about what’s being shown right
now, and what that would imply for the pendulum’s movement, you’d probably agree it’s grossly
inaccurate. But that’s just because the timestep delta-t of 0.5 is way too big. If
we turn it down, say to 0.01, you can get a much more accurate approximation, it just
takes many more repeated steps is all. In this case, computing theta(10) requires a
thousand little steps. Luckily, we live in a world with computers, so repeating a simple
task 1,000 times is as simple as articulating that task with a programming language. In fact, let’s write a little python program
that computes theta(t) for us. It will make use of the differential equation, which returns
the second derivative of theta as a function of theta and theta-dot. You start by defining
two variables, theta and theta-dot, in terms of some initial values. In this case I’ll
choose pi / 3, which is 60-degrees, and 0 for the angular velocity. Next, write a loop which corresponds to many
little time steps between 0 and 10, each of size delta-t, which I’m setting to be 0.01
here. In each step of the loop, increase theta by theta-dot times delta-t, and increase theta-dot
by theta-double-dot times delta-t, where theta-double-dot can be computed based on the differential
equation. After all these little steps, simple return the value of theta. This is called solving the differential equation
numerically. Numerical methods can get way more sophisticated and intricate to better
balance the tradeoff between accuracy and efficiency, but this loop gives the basic
idea. So even though it sucks that we can’t always
find exact solutions, there are still meaningful ways to study differential equations in the
face of this inability. In the following videos, we will look at several
methods for finding exact solutions when it’s possible. But one theme I’d like to focus
is on is how these exact solutions can also help us study the more general unsolvable
cases. But it gets worse. Just as there is a limit
to how far exact analytic solutions can get us, one of the great fields to have emerged
in the last century, chaos theory, has exposed that there are further limits on how well
we can use these systems for prediction, with or without exact solutions. Specifically,
we know that for some systems, small variations to the initial conditions, say the kind due
to necessarily imperfect measurements, result in wildly different trajectories. We’ve
even built some good understanding for why this happens. The three body problem, for
example, is known to have seeds of chaos within it. So looking back at that quote from earlier,
it seems almost cruel of the universe to fill its language with riddles that we either can’t
solve, or where we know that any solution would be useless for long-term prediction
anyway. It is cruel, but then again, that should be reassuring. It gives some hope that
the complexity we see in the world can be studied somewhere in the math, and that it’s
not hidden away in some mismatch between model and reality.
Grant is fantastic. The pacing, the visuals, the explanations are all top-notch. I never took an explicit different equations class, but this kind of makes me want to.
oh my god thank god i'm taking diff eq in the fall
I like these kind of videos, but I feel like too many people think this can substitute books or exercises.
These videos are so great. I've learned this topic before YouTube was around, solved hundreds of problems but never got this intuition. Wonderful visuals.
[removed]
"IT'S REALLY FREAKING HARD TO SOLVE"
I’ve failed Diff EQ three times. It is the only remaining math class I need for my B.S. in Physics. Feel free to snicker.
I’m 38 and went back to school 6 years ago to get said physics degree. (Previous B.A. in Theater from back in the day.) Hit a wall in 2016 with 6 physics classes and one math class to go.
Still raises my blood pressure when I hear the words “Differential Equations”
Starting college next year and diffeq will be my first class, can't wait.
His videos really remind my why I find the maths so interesting. It’s easy to lose sight of this when you spend hours trying to solve a set of problems.