Hello, everyone! I'm James Murphy. And welcome to mCoding, where we try to get a little
better at programming each episode so that we can ultimately
determine whether or not we're in the Matrix. In today's episode, what are Python's `future` imports. While they look like normal
import statements, they're anything but. This video is sponsored by me. Thank you to me for sponsoring myself. Did you know that I'm available for Python and C++ consulting,
contracting, training, and interview prep services? Again, let's give a round of
applause to me for sponsoring myself. This is the future module. It's a real module.
But it doesn't really do anything. It defines some constants and has
a class that represents some metadata. And then it just has a bunch
of metadata about the future features. But, beyond just keeping track of this metadata, the code in this file doesn't
implement any of those features. What actually happens is Python looks for
these future imports whenever it compiles a module. and then sets a specific feature flag depending
on which future imports you used. For instance, if we from future import `annotations`, then the `CO_FUTURE_ANNOTATIONS`
flag will be set in the compiler. Let's take a look at the C code
that this actually affects during compilation. I know compared to Python, this might look very verbose. But I promise you don't need
to know any C in order to understand this. This is the function that gets called to
convert an annotation in an argument into bytecode. The important part is right here. We have a set of features
represented by a bunch of bit flags, And we do a bitwise AND with
the feature flag that we're interested in. In this case, `CO_FUTURE_ANNOTATIONS`. If the feature flag is set, we use this visit macro,
which just expands to calling this function. The bytecode operation this
corresponds to is then loading a new constant. The constant that we load is
the annotation expression as Unicode, i.e., treat the annotation as a string. Let's see that in action. Here, we have a `Node` class
with an annotation for its `data` member. We can access the annotations at runtime like this. Notice that what gets printed out
is that the annotation has the string 'int' in it. But if we comment out the future import, then we see the annotation
contains the actual `int` class object. So, a future import is doing
something really fundamental. It's completely changing the
way that Python interprets your source code. It can literally cause Python
to generate different bytecode. And therefore completely
change the meaning of your program. So, what are all of these future imports? And why are they used instead of just
telling a user to upgrade to the next version? After all, if I have a new feature
like the match statement from Python 3.10, it's easy to tell people, "If you want to use the match
statement, you need to upgrade to 3.10." But if you were already using
annotations in a previous version, upgrading and changing the behavior can break old code. We don't want to go around just
breaking people's old code for no reason. We need a really good reason if
we're going to do something like that. This is where Python might differ
from some languages like C and C++. In C++, backwards compatibility is
a much stronger priority than in Python. A broken, incomplete, or not well-thought-out
feature might very well stay in the language forever. In Python, you get about
three to five years to upgrade. And the way that you opt into
an upgrade early is by using a future import. So, why do we need a future import for annotations? What's wrong with just
sticking the class object in there? Well, if an annotation is just
treated like a normal expression, that implies that the annotation should
be defined before the thing that it's annotating. This makes it impossible to define a
recursive data structure like a linked list. When I try to annotate
the `next` pointer in the linked list, I'm in the process of defining the `Node` class. So the `Node` class is not
defined yet, and hence I get a name error. The annoying solution to this
is to put quotes around your annotations. And then hope that your editor
and your type checker understand. As we already saw, the future solution to this is to
just automatically always treat annotations as strings. Very few programs actually
care to access type hints at runtime. But those that do can use `typing.get_type_hints`
to evaluate the strings back into class objects or whatever. Eventually, the behavior of the future
import will just become the default behavior. But do keep in mind that with future
imports, the behavior is not always completely settled. When you opt in early to a feature, it's possible
that that will change before the feature actually comes out. In fact, with annotations specifically, there's a decent chance this behavior will just be
replaced by some kind of lazy computing of the annotations. That's definitely something to watch out for. But I don't really worry about it. And in fact, I use this particular
future import in almost every file that I define. So, what are all the rest of the future imports? When in doubt, of course, go to the source. The good news, since these are
breaking changes after all, is that there are only two that even apply to Python 3. This one is just an easter egg. As you can see, it's set to appear in Python 4. All the rest are in Python 2. `unicode_literals`: allowed you to create
bytes literals like this using the `b` and then quotes. `print_function`: introduced the `print` function. `with_statement`: introduced the `with` statement. `absolute_import`: changed and clarified
the way that absolute and relative imports work. `division`: made the single
slash always mean float division and introduced the double slash to mean int division. `generators`: introduced generators. And `nested_scopes`: allowed
the ability to define nested scopes And guaranteed that they
would be computed at compile time. In particular, this allowed for
lambdas or inner functions to access variables that were defined inside
another enclosing function scope. And that's all the future imports
except for one, which is `generator_stop`. `generator_stop` was introduced because
of the following behavior. We want the generator to yield one, two, three, four. Pretend one, two, three, and four are
just stand-ins for some complex operations. Maybe two and three are
particularly complicated to compute. So we factor it out into a subgenerator. But we made a mistake in our subgenerator. We're only yielding one thing
instead of two things before we're done. However, we have two unguarded calls to `next`. The first call works as normal and returns two. Then, for the second call, we hit
the `return`, which raises a `StopIteration`. So what's effectively happening is a `StopIteration`
is getting raised at this point in the generator. We're not inside a loop. We made a call to `next`, and we
got an exception, which is propagating up. But nothing's shown to the user. That's because we're in a generator, and that
`StopIteration` gets raised all the way up to this `for` loop. Normally, when a `for` loop receives a
`StopIteration`, that means we're just done iterating. So, the program prints out one, two, and then stops. Instead of raising the exception
and showing the user an error message. We silently loop over
less data than we intended to. So, I really don't want `StopIteration` in
generators to cause `for` loops to just terminate. If I wanted a `for` loop to just terminate, I can always just return from the function, which will end up raising a`StopIteration` in the `for` loop. But having an unhandled `StopIteration`
propagate outside a generator is almost always a bug. So, `from future import generator_stop`, and now any `StopIteration` that's raised
out of a generator gets turned into a runtime error. No more silent errors - we get a big, beautiful
error message telling us exactly where things went wrong. The future import allowed us, starting
in Python 3.5, to opt into this new behavior. But hopefully, you or your company are
now using a version of Python that's at least 3.7. It's fine to leave the future import in.
It doesn't hurt anything. But as of 3.7, you can delete
it, and that's now the default behavior. So realistically, the only one that you really
need to worry about is `from future import annotations`. And a quick tip that you might not have realized: Since annotations are now just strings, I can actually start using typing features
from way later versions. For instance, this bar notation for the
union of types was introduced in Python 3.10. But it's working just fine in 3.7. This works completely fine as long as you're not
trying to actually evaluate those annotations at runtime. If you're doing that,
then you just need to upgrade. Actually, if you dig into the C source code,
there's one more future import you might be interested in. Anyway, that's all I've got. And finally, I want to give
a huge shoutout to Neil Rashania. Thank you to Neil for subscribing
at the factorial level on Patreon. I really appreciate the support. Of course, thank you to
the rest of my patrons as well. Don't forget to subscribe, leave a comment. And as always, slap that
like button an odd number of times. See you next time!
