[MUSIC PLAYING] BRIAN YU: OK. Welcome back, everyone, to web
programming with Python and JavaScript. And now, at this point, we've seen
a number of different techniques and tools that we can use in
order to design web applications-- HTML and CSS, to describe how it is that
our pages look; a programming language, like Python, using a
framework like Django, in order to listen for requests, process
them and provide some sort of response. And then more recently, we took a look
at JavaScript, another programming language that we can use in
particular on the client side, running inside of the user's web
browser, in order to make our web pages even more interactive and user-friendly. Now what we'll transition
to today is taking a look at some of software's best
practices, some tools and techniques that developers actually use when
they're working on web applications, especially as those web
applications start to grow larger. In particular, we'll start
by discussing testing, this idea of verifying that our code is
correct, and then transition to CI/CD, short for Continuous Integration
and Continuous Delivery, some other best practices
that are used in making sure that the work that software
developers are working on can be tested and deployed
readily and very quickly. So we'll begin the
conversation with testing. And testing is really about this
idea of verifying and making sure that the code that software
developers are writing are, in fact, correct, to make sure
that the functions work the way they're supposed to, that the
web pages behave the way that they're supposed to. And ideally, we'd like some way to be
able to efficiently and effectively test our code over time, and as
our programs grow more complicated, to allow our tests is to
make sure that our program is behaving the way that we want it to. So we'll go ahead and start
simple and consider the basic way that we might take a function, for
example, written in Python and test and verify to make sure that it
works the way we would expect it to. And to do so, we can start with a
command in Python known as assert. And what assert does in Python
is it asserts or just states that something should be true. And if that something is
not true, then the assert is going to throw an
exception, some sort of error, so that whoever is running the
program or running the command knows that something went wrong. And this can be a very basic way that
we can leverage Python's abilities to test a function and verify
that that function behaves the way we would want it to. So let's go ahead and try a simple
example of writing a Python function and then trying to test to make
sure that that function works the way we would want it to. So I'll go ahead and create a new file. I'll call it assert.py. And let me define a new Python
function, for example, that is going to take an integer and square it. Just want to take a number
and return it square. So I'm going to define a function called
square that takes as an input a number, like x. And I want to return x times x. It's a fairly straightforward function. But I would like to now
verify that the function works the way I would expect it to. Now there are a number of
ways that you could do this. One would be just like let's print out
what the square of 10 is, for example, and just see what that's equal to. And then you could run a program,
something like python assert.py and just say, all right,
the answer is 100. And I could say to myself, OK,
that's what I would expect it to be. But I now have to do the mental
math of squaring the number 10, making sure that the answer comes
out to be the value that I expect. It would be nice if I could
automate this process. Well, one thing I could do is print
out, does the square of 10 equal 100? I know that I want the
square of 10 to equal 100, so I could just print out that
value, print out, does square of 10 equal the number 100? I go ahead and run the program again. And this time, what I get
is, true, for example, because those two things
are equal to each other. And if, on the other hand, I had tried
to check for something that wasn't true, like does square of 10 equal 101? You run the program, and, OK. Now it's going to be false. So this is nothing new,
nothing we haven't seen before. But now what I can do
is, instead of this, I can just say, let me assert that
the square of 10 is equal to 100. Here I am just asserting that this
expression, that the square root of 10 is equal to 100 is going to be true. And now I can run the program. And what you'll notice
is nothing happens. No output, nothing at all because
when an assert statement runs and the expression that it's
checking turns out to be true-- that the square of 10 does equal 100-- it effectively ignores
that statement altogether, just continues on to the next thing,
no output, no side effect of any sort. And this is helpful
because it just means that if I want to assert
that something is true, I can assert it and then just
continue writing my code. And it's as if I hadn't
written that assert statement at all so long as the thing that
I am asserting is actually true. But if there were a bug
in my code, for example, some sort of mistake, where
instead of returning x times x, imagine that I
accidentally said, return x plus x to calculate the
square instead, something that would be a bug in this case. Well, then when I try to run python
assert.py, what I'm going to get is an exception. And the type of exception that I get is
something known as an assertion error. And I can see that here. There is an assertion error. And then I see the reason why
the assertion error happened. And the assertion error happened
on line 4, which is this line here, where I said, I would like to assert
that the square of 10 is equal to 100. So one way we can
imagine testing our code is just by including a number of
these different assert statements. If I want to verify
that my code is correct, I can write various
different assert statements. And for a function that's fairly simple,
like the square function, probably not too many tests that I
would need to write. But you can imagine for
more complex functions that have multiple different
conditional branches, being able to assert that no matter which
conditional branch the program chooses to follow that the
code will actually be correct can be a valuable thing
to be able to say. And this can be helpful too when
in working on a larger project, you want to deal with
the problem of bugs that might appear inside of a project. And this gets at the idea
of test-driven development, developing while keeping this
notion of testing in mind. And one of the best practices
would be if ever you're working on a program of your own and
you encounter some bug in the program, you'll first want to fix the bug. But then you'll want to write a test
that verifies that the new behavior is working as expected. And once you've written
these tests, these tests can start to grow over time. And as you continue
working on your project, you can always run those existing set
of tests to make sure that nothing-- no new changes that you make
to the program down the line, no future features that you
add or changes you might make-- are going to break anything
that was there before. And this is especially valuable, as
programs start to get more complex and testing everything
by hand would start to become a very tedious process,
to be able to just automate the process of just run a whole
bunch of tests on all of the things that I know that I would
like the program to do and making sure they work as expected. That can be quite helpful. So assert then is one
basic way of just saying that I would like for
this statement to be true. And if it's not true, go
ahead and throw an exception. And using Python, we know
we also have the ability to catch those exceptions in
order to make sure that we're able to handle those appropriately. So we can display a nice error message
for example, if we wanted to do so. But now let's go ahead and try
and write a more complex function, something more complex than just
taking a number and squaring it, somewhere where there's more
room for various different cases that I might want to test and
more room where I, the programmer, might make a mistake, for example. So let's imagine writing a new file--
and then I'm going to call prime.py-- where here, I'll go ahead and
say that I would like prime.py to implement a function called is_prime. And what the is_prime function should
do is check to see if a number is prime or not. The prime number only has
factors of 1 and itself. And I would like to write a
function that verifies that fact. And so how might I go about doing that? Well, if n is less than 2,
then it is definitely not prime because we say 0 and 1
are not going to be prime. And we'll only deal with
numbers that are 0 or greater. And we'll deal with that for now. But let's start then with other
numbers, numbers that are 2 or greater. Well, what do I want to do? I really want to check each
of the possible factors. Like if I want to check whether
or not 100 is prime or not, then I want to loop over all of
the possible numbers that could be factors of 100, like 2, 3, 4, 5, 6. And when I get to a number
like 2 or a number like 5 that do go into 100 cleanly, well, then
I'll know that the number is not prime. So I could say for i in range from 2
all the way up through n, for example, let me go ahead and say, if n mod
i equals 0, then return false. So what am I saying here? I'm saying go ahead and start at 2. Go up through but not including n. So if I'm checking to see
if 10 is prime, for example, I'm going to check for i
is 2, 3, 4, 5, 6, 7, 8, 9. And for each of those numbers, check
if n, my input to this function, mod i, the factor that I would
like to check, is equal to 0. This mod operator, this
percent, if you don't recall, gives us the remainder when you
divide one number by another. And so if n mod i equals 0, that means
the remainder when you divide n by i equals 0-- meaning i goes into n
cleanly with no remainder. And that means that it's not prime
because it does have a factor. Whatever i is is going
to be that factor. And if I get to the
end of this for loop, then I can go ahead and
just say, return true. If we weren't able to find a
factor for the number other than 1 and the number itself, well, then,
we can go ahead and say that true, this number is going to be prime. And so this, for example,
could be a function that checks to see if a number is prime. But if I'm trying to
optimize, I'm trying to make my function
more efficient, I might realize that you really don't need
to check every number from 2 all the way up to the number and itself. I could really just check up to
the square root of that number, for example. That for a number like 25,
I want to check 2, 3, 4, 5 because 5 squared is going to be 25. But after 5, I don't need to check
any more numbers beyond that. That after you get to a number, after a
number-- the square root of that number is multiplied by itself,
there's never going to be a case where a number bigger
than that could be a factor that I won't have already known about. So just thinking about things
a little bit mathematically, we might be able to make
some sort of optimizations where instead of going from
2 all the way up through n, I might go up to the square root of n. And I'll go ahead and import math to
be able to use math dot square root. And I'll convert that
number to an integer just in case the square root doesn't
already happen to be an integer. So I think this works. I've at least talked
myself into thinking that this is a function that might be
able to check if a number is prime. So what could I do if I
wanted to verify this? Well, I could write
some assert statements. Another thing I could do is
just use the Python interpreter. I could say, all right. Let me go ahead and type python. And I'm in the Python interpreter. And I can say, from prime, go
ahead and import is_prime. prime is the name of that file. is_prime is the function in that
file that I would like to test. And let's just try,
all right, is_prime(5)? That's a prime number. Hopefully, it'll say
true, that it's prime. All right, it does. Let's try is_prime(10)? See if that works. All right. is_prime(10) is false
because 10 is not prime. That's good. That seems to be working as well. Let's try is_prime(99)? That's not prime because 3 is a
multiple of that, for example. All right, false, so that's good. This seems to be working. And I could, in the
interpreter, test this function to make sure that it works the
way that I would want it to work. But let's now see some other ways
that I might go about testing it. Well, one way is that I could
write a file like tests0.py. And what tests0.py is going to
do-- instead of using assert, I'm just going to do our Boolean
checks like we were doing before. I'm going to import
the is_prime function. And I've defined a new
function called test_prime, which is going to serve the role
of testing to make sure that when you square some number or when you
check to see if some number n is prime, that you get some expected value, where
that expected value is either true for it is prime or false
for it's not prime. What, then, is this function doing? Well, the function is checking. We're calling the is_prime
function on this number n and seeing whether or not it is equal
to the expected value that we get, where we expect it to
be either true or false. And if we run is_prime on n and
it is not equal to what we expect, well, then, we print out,
OK, there is an error. We expected some value true or false. But it turned out not to be the case. And so now that I have this test_prime
function, well, I can say, all right. Let me go back into
the Python interpreter. From tests0 import test_prime. And now I can say, all
right, let me test_prime. Make sure that 5 is prime. So I'm passing in with my
first input the number n, the number I would like to check. I want to check if 5 is prime. And the second input I provide is what I
expect it to be-- either true or false. And here, nothing happens,
which is a good thing. If there were an error, it would
have printed something out. And the fact that I see nothing printed
out means that everything was OK. If I test_prime now and say
something like, all right, make sure 10 is not prime-- make sure that 10 when you pass it into
is_prime is going to give us false. Again, nothing happens. Seems to be working just fine. Let me now try-- I can try more examples. Maybe I try test_prime 25. I want to make sure that 25 is not
prime because 25 is not a prime number. All right. We get some sort of error. There's an error on is_prime(25) where
I expected the output to be false. But for some reason, it looks
like is_prime returns something other than false. It probably returned true. And so might indicate some
sort of bug in my program, that somehow I don't think that
25 should be a prime number. But my program thinks
that 25 is a prime number. And that error can be a clue
to me as to how to do this. But ultimately, especially as programs
start to grow longer, especially as I start to add more
and more functions, testing each of those functions by
hand is going to start to get tedious. So one thing I could
do is write a script to be able to run all these
tests for me automatically. And so here what I have a tests0.sh,
.sh being like a shell script, some script that I can just
run inside my terminal. And what this is doing is it's running
Python 3, for Python version 3, dash c, which means I'm just
going to give it a command. And it is going to run that command. And so I can just run these. And each of these lines does what? From tests0, it imports
my test_prime function, that function that is
going to test to make sure that the prime function produces the
output that I would expect it to. And each time I'm testing
a different number, making sure that 1 is not prime,
making sure that 2 is prime, 8 is not prime, so on and so forth. And I can just write a
whole bunch of these tests. And then rather than have to run each
test one at a time, what I can do is I can just run tests0.sh. I can just say that I would
like to run ./tests0.sh. And, all right. I see that I get two errors. I get an error on is_prime(8),
where I expected it to not be prime. But for some reason,
it seems to be prime. And then again, here,
exception on is_prime(25), where I expected it to not be prime. But for some reason, my program
thinks that it is prime. So a very helpful way for me to
know immediately that there's some sort of error
that is going on here. But ultimately, rather than have me
have to write all this framework for how to go about testing my
code on my own, there exists libraries that
can help us with this. And one of the most popular in Python
is a library known as unittest. And what unittest as a
library designed to do is it is going to allow
us to very quickly write tests that are able to check whether
something is equal to something else. And then unittest is built
in with an automated test runner that will run all of the
tests for me and verify the output. And unittest gets built in to a lot
of other libraries within Python. We'll see how we'll soon be
able to apply this sort of idea to our Django applications as well. But let's now translate these tests
that we have written ourselves just by writing a function like test
whether the prime number is what we expect it to be and now translate
it to using this Python unittest library instead. And so just to get a sense
for what this looks like, I'll now go ahead and open up tests1.py,
where here first thing I'm doing is I'm importing unittest, which
we get for free with Python. I'm also importing the function
that I would like to test. And now I'm defining a class which
will contain all of my tests. This is a class that inherits from or
derives from unittest.TestCase, which means that this is
going to be a class that is going to define a whole bunch of
functions, each of which is something that I would like to test. And so, for example, in
this very first test, this is a test that checks to
make sure that 1 is not prime. And so the way I do that is by calling
self-- this testing object itself. It happens to have a method or function
built into it called .assertFalse. There's an equivalent .assertTrue. But I would like to .assertFalse. And what would I like
to assert that is false? is_prime(1). So whatever is_prime(1)
is, that should be false. And I would like to just
assert that it is false. Likewise, for the number 2 now, I want
to check that the number 2 is prime. And the way I do that is
by calling self.assertTrue. I would like to assert that when I run
the is_prime function on the number 2, the output that I get is going to
be a true value-- self.assertTrue. And I can translate each
of the rest of my tests into one of these self.assertTrues
or self.assertFalse. And then I say that if you
go ahead and run the program, go ahead and call unittest.main, which
will run all of these unit tests. So now, when I run python
tests1.py, here's what I get. I get some nice output,
where up at the top, I see dots every time a
test succeeded and a letter F for a test that happened to fail. It says that it ran six tests. And down in the bottom, I see
that there were two failures. So it's immediately going to
tell me exactly what failed. And it'll give me some
rationale, some reason for why it is that those tests failed as well. So we can see, all right. Here is one test. Here is another test. This test that failed is the test
that checked that 25 is not prime. And this sentence here
is what I supplied inside of what was known as a Python docstring
inside of those triple quotation marks, underneath the declaration
of the function. Those triple quotation marks,
otherwise known as a docstring, serve a number of purposes. They can serve as just a comment for
describing what it is the function does. But they're a special
comment insofar as someone who's looking at the function
can access that docstring that's usually used for documentation for
what it is that the function is doing. And they can use it inside
of other places as well. And so what unittest is
doing is for every function, it uses that docstring as a description
of what the test is testing for, so that if a test fails, then I
can see exactly what the name is of the test that failed and what it was
tested, where the description describes what was happening. Now in this case, where
it's just one function and I'm testing a whole
bunch of different numbers, it doesn't seem all that useful. But again, if you imagine
projects that start to get more complex, being
able to know immediately when you run your tests
which parts of the program or which parts of your web
application are working the way they are expected to can
actually be quite helpful. So test 25, that was the function
that triggered an assertion failure in this case. And the line that caused it was
self.assertFalse(is_prime(25)). And the reason that it
failed is because True, which apparently was the output
of this function, is not false. And I expected it to be false instead. And so multiple, different ways
of trying to run our tests. This happens to be
one quite popular one. But this now tells me that I
should go back and try and fix my is_prime function. I can go back into prime.py
and say, all right. I would like to figure
out why this went wrong. And if you look at this enough and
maybe give it a little bit of testing, you might see that I
have a slight off by 1 error, that I probably need to check
one additional number than I actually am because in checking whether or
not 25 is prime or not, for example, I might need to go up to
and including the number 5 to know that 5 is a factor of 25. But before, I was going up to the number
5, but I wasn't including the number 5. So I also need to just
check one more number. And now to verify that this is
right, I could just manually test the function myself. Or I could just run these tests
again, run python tests1.py. And this time, all these dots
mean all these tests succeeded. We ran six tests, and everything was OK. No failures. And so this can be a helpful
way for me to know immediately that things seem to be working OK. So the takeaways from here are
that these tests can definitely help as you begin to write
new changes to your program, especially as you begin
to optimize functions. You might make a function
more efficient but then run your tests to make sure that
in making these improvements, you haven't broken anything. You haven't changed any behavior,
that the way the program was supposed to behave and now
it doesn't behave that way, you are able to verify with much
more confidence that that is true. But of course, that
only works if your tests have good coverage of all the things
that you would want the function to do, and you've covered appropriately
all the various different cases for how the function should behave
because only if the tests are comprehensive, will they
actually be useful to you in indicating that the change that you
made isn't going to break anything. And only then can you actually feel
confident in those changes themselves. So now let's take this idea
of using unittest to be able to write these tests that
verify that a function works and apply it to something
like a web application-- like a web application
written in Django that we would like to now use in order
to be able to test to make sure that various, different functions
inside of our Django web application work as well. So what I'm going to do is actually
take a look at the airline program that we wrote back when we
were first talking about Django and first talking about storing
data inside of databases. And I'm going to open
up models.py where you see that I've made one addition
to our definition of a flight. And recall from before,
when we first introduced this idea of defining a model
inside of our application for a flight inside of an airline,
we gave that model three properties. It had an origin and a destination,
where both origin and destination referenced an airport object,
where an airport object was an object we defined separately. But a flight has an origin
airport and a destination airport. And in addition to that, every flight
has a duration, some number of minutes long that that flight is going to last. And I might like to have
some way to validate, to verify that a flight is a valid
flight, that there isn't some error somewhere in how the data was
entered into the database. I would like to just generally
make sure that given a flight, I can check to make sure
it's a valid flight. And what does it mean
for a flight to be valid? Well, in general, given
these particular fields, I'll say there are two things that need
to be true for a flight to be valid. The origin and the destination
need to be different. That's condition number one. It wouldn't make sense to have a
flight whose origin and destination are the same airport. And condition number two,
the duration of the flight needs to be greater than 0 minutes. If ever the duration is 0
or the duration is negative, that probably indicates to me that there
was some sort of mistake in data entry or some problem that
happened with how it is that these flights were configured. So I want to make sure that
the duration is greater than 0. And those then are my two conditions
for what makes a valid flight. And I've, in fact,
written a function here called is_valid_flight that
just works on this flight class that simply checks given a flight,
make sure that it is, in fact, valid. And the way it's doing that is by
checking for these two conditions that I've just described. It's checking to make sure that the
origin is not equal to the destination. It's checking to make sure
that the duration of the flight is greater than or equal to 0. And maybe I should change that
to greater than to make sure it's entirely positive. But this then is my
definition for what it means for something to be a valid flight. And what I'd like to do now is
test these various, different parts of my application. I have this is_valid_flight
function inside a flight that I might like to test as well. But we also have all of
these other properties that I would like to
test, these relationships, that a flight has an
origin and a destination. We have passengers that can
be associated with flights. So there's lots of
relationships between my data that I would like to test
and verify to make sure they work the way we would expect it to. So to do that, whenever we
create an application in Django, like this flight's application here,
we were also given this tests.py file. And we haven't yet used the
tests.py file for anything. But what it's supposed to
be used for is for writing these sorts of tests, testing that
verifies that our application behaves the way that we want it to behave. So let's go ahead now
and open up tests.py and see what happens to be in here. What we can do is we can
define a subclass of TestCase, which behaves very similar to unittest
and is based on that same idea. I'll define a new class
called FlightTestCase that will just define all
of the tests that I would like to run on my flight's application. And so things to know
about this is that first, I might need to do some
initial setup in order to make sure that there's some data that
I can actually work with and test with. And what Django will do when
I go ahead and run these unit tests is that it will create an
entirely separate database for me just for testing purposes,
that we have one database that contains all the information that
actually pertains to the flights that are actually there on my web server. But we might also like to just
test things with some dummy flights and some dummy airports just to
make sure that things are working. And then once we're
confident things are working, then we can deploy our
web application to let actual users begin to use whatever
new features we've added to the web application, for example. So inside of this database, I might
need to do some initial setup. And I can do so by defining a setup
function inside of my test case class. This is a special function. And Django knows that when
it's running these tests, it should first do any of the
setup steps that we need to do. And so how are we doing this? Well, what we're doing
is inside of the setup, we're going to just add some
sample data into the test database. Again, this won't touch the
database that users actually see and actually interact with. This is just our test
one for testing purposes. And we'll start by going ahead
and creating some airports, so Airport.objects.create,
and then specifying what the values for these fields should be. We'll just have an airport
whose code is AAA for city A and an airport whose code is BBB for
city B. Just a dummy airport names. They're not real airports but
just used for testing purposes. And I'll save those airport objects
inside of these values, a1 and a2. And beneath that, what I'm going
to do next is go ahead and create some flights where I create
using Flight.objects.create three different flights,
one that goes from a1 to a2 with the duration of 100 minutes; one
from a1 to a1 with a duration of 200 minutes; one from a1 to a2 with a
duration of negative 100 minutes. So I have a whole bunch of these
flights now that I would like to test, that I would like to make sure work
in some predetermined or expected way. And so now if I scroll down, we
can see that I have a whole bunch of these various different tests. Here is one test that just
tests the departures count. So every airport has
access to a field called departures, which ideally
should be how many flights are departing from that airport. And I'd like to make sure
that departures_count works the way I expect it to. So here, I go ahead and get
the airport whose code is AAA. And now, using unittest-like syntax,
I would like to say self.assertEqual. So assertTrue verifies
if something is true. assertFalse verifies
if something is false. assertEqual verifies that two
numbers are equal to each other. And here, I'd like to verify
that a.departures.count-- if I take airport a and
count how many flights are departing from that
airport, that that should be 3, so just verifying that works. And then after that, if
this test passes, then I can be confident that elsewhere
in my program, if I take an airport and call that
airport.departures.count, I can feel pretty confident
that that is going to work the way I would expect it to. I can do the same thing for
arrivals, get the airport, and assert that
a.arrivals.count, that that is going to be equal to the number
1 if there's only one flight that arrives at airport a1, for example. So that tests these relationships. And I can also now test the
is_valid_flight function as well, that here I get my two
airports, a1 and a2. This is the one whose code is AAA. This is the one who's code is BBB. I'll go ahead and get the flight whose
origin is a1, whose destination is a2, whose duration is 100. And let me just assertTrue
that this flight is going to be a valid flight
because this fight is valid. The origin is different
from the destination. Its duration is some
positive number of minutes. And so I should feel
pretty confident that this is going to be a valid flight that I
can verify by calling self.assertTrue. I can do the same thing for
testing for an invalid flight, testing for an invalid flight
because the destination is bad. I can get the flight, airport
a1, and get the flight whose origin and destination are both a1. And now let me self.assertFalse,
say that this should not be a valid flight because the origin
and the destination are the same. What's the other way a
flight can be invalid? Well, a flight can be invalid
because of its duration. So I could say something like, go
ahead and get airports a1 and a2. And get me the flight whose
origin is a1, destination is a2, but the duration is
negative 100 minutes. That was one of the flights as well. And, well, that should
not be a valid flight. So I'll say self.assertFalse
is_valid_flight because when I call
is_valid_flight on that flight, it shouldn't be valid because the
duration makes it an invalid flight. So here now I've defined
a whole bunch of tests. And there are more done below that we'll
take a look at in a moment as well. But I've defined a whole bunch of these
flights now or a bunch of these tests. And now I'd like to run them. And the way that I can run tests in
Django is via a manage.py command. manage.py has a whole bunch of
different commands that we can run. We've seen makemigrations
and migrate and runserver. But one of them as well
is if I go into airline0, I can say python manage.py test that's
just going to run all of my tests. And we're right. It seems that we ran 10 tests. But two of them failed. So let's go ahead and see,
why did those two tests fail? Well, the way to read this is that we
get this heading anytime a test failed. And so we failed the
test_invalid_flight_destination function. And we failed the
test_invalid_flight_duration function. And docstrings could have
helped me to know what it is that these tests are exactly doing. But it seems that true is not false. I wanted to assert that this should
not be a valid flight, that it should be false. But for some reason, these
appear to be valid flights. So something seems wrong
with is_valid_flight where it's returning true when
it should be returning false. And so this then gives me
a place to start looking. I can say, all right. Let me go to is_valid_flight and
make sure that function is correct. So I'll go back into models.py. I'll take another look
at is_valid_flight. Maybe I'll think
through the logic again. All right. I wanted to check that self.origin
is not self.destination. I wanted to check that the duration
is greater than or equal to 0. I could change this to greater than. But I don't think that's the issue
because my duration was negative. And so that already
should have been invalid. But the other thing I might
realize looking at this now is that, OK, the logical connectives
that I have used was not the right one. I want to check that for
it to be a valid flight, it needs to satisfy
both of the conditions. The origin and the destination
need to be different. And the duration of the flight needs
to be greater than 0, for example. And here, I've used
"or" instead of "and." So I can just change it. All right. And hopefully, that will fix things. And to verify as much, I can
rerun python manage.py test. Go ahead, and press Return. It's going to check things. I ran 10 tests. Everything is OK. And all right, it seems that now
I have passed all of these tests. I notice here at the top,
it created a test database. So it just created a
test database for me in order to do all this testing work. And then it destroyed that test
database at the end as well. So none of my tests-- I'm adding data, removing data. It's not going to touch
any of the actual data. Inside of my database
for the web application, Django will take care of the process
of keeping all of that separate for me by first calling that setup function to
make sure that my new test database has everything that it needs to. So, all right. We now have the ability,
by using unittest, to be able to test various,
different functions inside of our web application. We first saw that we could test a
function like the is_prime function, just a Python function that we wrote. But we can also test things
like functions on our model, like checking to make sure
that a flight is valid, checking to make sure that we can
take a flight and access all of the-- or take an airport, and access all of
its arrivals and all of its departures. But I'd like to do more than that,
especially for a web application. I'd like to check that
particular web pages work the way that I want them to work. And so to do that,
Django lets us simulate trying to make requests and get
responses to a web application. And so let's go ahead and look at
some of these other tests as well. Here what we have is a
function called test_index. And what test_index is
going to do is it's just going to test my default flights page
to make sure that it works correctly. So we start by creating a
client, some client that's going to be interacting
request and response style. Then I'm going to call
client dot get slash flights. That is the route that gets me the
index page for all the flights. And I'm saving that inside of
a variable called response. Whatever response I get back
from trying to get that page, I would like to save inside of
this variable called response. And now, I can have multiple assert
statements inside of the same test if I would like to. Sometimes you might
want to separate them. But here, I want to check
that the index page works. And what that means
is a couple of things. It means that response.status_code,
well, that should be equal to 200. 200, again, meaning, OK. I want to make sure that
whatever response I get back, that that is going to be a 200. And if there was some sort of error,
like a 404 because the page wasn't found or a 500 because of
some internal server error, I would like to know about that. So let me first just assert that the
status code should be equal to 200. But then Django also lets me
access the context for a response. And what is the context? Well, recall, again, in Django, when
we rendered a template, for example, we called return render
then provided the request and what page we were going to render. But we could also provide some
context, some Python dictionary, describing all of the values that we
wanted to pass in to that template. And Django's testing framework
gives us access to that context so that we can test to
make sure that it contains what we would expect it to contain. And on the index page
for all my flights, I would expect that to contain
a listing of all of the flights. And we created three sample flights
inside of this test database. So I should be able to assert
that these two things are equal. response.context flights,
that gets me whatever was passed in as flights in the context. Dot count, well, that
better be 3 because I want to make sure that there are
exactly 3 results that come back when I look at the context
and access whatever it happens to be inside of that flight's key. There are other tests I can run as well. So in this case, I've gone ahead
and gotten a particular flight, the flight who, in this case, had an
origin of a1 and a destination of a1, that it was not a valid flight. But we'll go ahead and get it anyway
because it exists in the database. And now I can get slash
flights slash that flight's ID because on my flight's
page, I would like to be able to go to slash flights
slash 1 to get at flight number 1, and go to slash flights slash
2 get at flight number 2. So if I take some valid ID,
some ID of an actual flight f and go to slash flights slash
dot id, well, that should work. It should have a status code of 200. Meanwhile, though, if I
test an invalid flight page, this is a Django command that will
get me the maximum value for the ID. This id__max gets me the biggest
possible ID out of all of the flights that happen to exist
inside of my database. If I go ahead and try and get
slash flights slash max_id plus 1-- so a number that is 1 greater
than any of the flights that were already inside of my database--
well, that shouldn't work. There shouldn't be a page for
a flight that doesn't exist. So here then, I can assertEqual
that the status code of what comes back, that that is
equal to 404 because I would expect that page to return a 404. And finally, I can also check
various different contexts for things about the passenger page. So in this case, I've added some
sample passengers to the database. So inside of my test, I can
manipulate the database as well, adding data into the
database and checking to make sure that when you count up the
number of passengers on a flight page, that that is going to be like
the number 1, for example. So a number of different
tests that we can then write in order to verify various
different parts of our web application. I would like to verify not
only that our database works the way we would expect the database
to work in terms of how functions on our models work, in terms of
relationships between those models like the relationship between
a flight and an airport, but we can also simulate a GET request,
simulating the request to a page and verify that the status
code that comes back is what we would expect it to be;
verify that the contents of the page contain the right contents; verify
that the context that was passed into that template is correct as well. And then all of that
can then be verified by saying something
like python manage.py test then go ahead and running that. And we see that in this case, all
of the tests happened to pass, which means that everything
seems to be OK, at least for now. So this then, again,
very helpful as our web programs start to get more complex,
as we have multiple different models, multiple different routes,
the ability to test, to make sure that if we change
things in one part of the program, that it doesn't break
things in another part. That can be quite helpful too. What we haven't yet
been able to test though is any interaction that's happening
exclusively in the browser. But I've been able to test a lot of
things that are happening server-side. And recall that Django is all about
working on writing this web server-- a Python application that acts as a web
server that listens for requests that come in from users, processes those
using these various different views and models, and then provides
some sort of response back. And we can test the contents
of that response, things like, does the status code match
what we would expect it to? Does the context match
what we would expect it to? But there are some times where
we would like to really simulate like a user clicking on buttons and
trying things on a page and making sure that page behaves we would like
it-- how we would like it to as well even if we're not using
Django, or even if we're just dealing with the front end. So let's create a sample
JavaScript web page that we might like to test now
by using these sorts of ideas-- the ability to automate
the process of testing various, different parts of
our application to verify that they do, in fact, work correctly. What I'm going to do now is we'll
get out of the airline directory. And I'm going to create a new
file that I'll call counter.html. And recall before, we created a
counter application using just JavaScript where what the
counter application did is it let me click a button,
like an increase or a count button that just incremented a number. It went from 0 to 1 to 2
to 3 to 4 and so forth. I'm going to do the same thing here. We'll add a little more complexity
though and give myself both an increase button to add a number
and a decrease button to decrease the number by 1 as well. So I go ahead and start with our
usual DOCTYPE html and our html tag. I'll give this page a title of Counter. And now, inside of
the body of this page, I will go ahead and start with
a big heading that just says 0. And then beneath that,
I'll create two buttons. I'll have one button which
will be the plus symbol and one button which will be the minus symbol. So now no JavaScript yet. This won't actually work. But I can go ahead and go
into open counter.html. And I can see that I now have 0, and
I have a plus and a minus button, although those Plus and Minus buttons
don't actually do anything right now. So let's go ahead and
make them do something. Let's give this button
an ID called increase so that I can reference it later, and
give this button an ID called decrease, again, so that I can reference
it inside of my JavaScript. And now, in the head
section of my web page, I add a script tag where I want
to start running some JavaScript once the page is done loading. And to do that, you'll recall, I
can say document.addEventListener c to say, go ahead and run this
function once the DOM is loaded, once all the contents of this page have
loaded the way I would expect it to. And what am I going to do? Well, I first need a variable,
something like let counter equal 0. And then I can say, all right
document.querySelector increase. Get me the element whose ID is increase. That's that plus button. And when you are clicked, let's
go ahead and add an event handler in the form of this callback function,
this function that will be called when the increase button is clicked. And what would I like to do? Well, I'd like to increase the counter. Go ahead and say counter++. And then I'm going to update this
h1 that currently contains 0. document.querySelector h1. That gets me the H1 element. I'll go ahead and update
its inner.HTML, and go ahead and set that to whatever the
value of counter happens to be. And then I'll do the same
thing for the decrease button. document.querySelector,
get me the element whose ID is decrease-- that's the
button that will be the minus button. And when you are clicked, go ahead
and run this callback function, which will do the same thing,
except we'll first do counter-- decrease the value of the counter by
1, and then get me the H1 element, set its innerHTML equal to the counter. I think this should work. And I could verify that by
opening up counter.html. I'll refresh the page. And I can test these buttons. Test the plus button. All right. That seems to work,
increases the value by 1. I can test the minus button, make sure
that that decreases the value by 1. That all seems to work just fine. But, of course, this requires me
having to interact with this page. I have to open up the page. I have to click on these buttons. And it's not the type of
thing where I'm simulating like a GET request or a POST request. There is no server that I'm sending
a request to and getting back a response of 1, 2, 3, 4 from. This is all happening in the browser. And so what I might like
to have the ability to do is some sort of browser testing. And there are a number of different
frameworks for doing this. One of the most popular is Selenium. And what this is going
to allow me to do is I can define a test file that
using unittest or a similar library can effectively simulate a web browser-- can simulate a web browser and
simulate a user interacting with that web browser using something
that we call a WebDriver that's going to allow me to,
using code, control what it is that the browser is doing and
how it is that the user is interacting with this program. So how is this going to work? Well, I'm going to go
ahead, and just as a test, let me open up the Python interpreter. And let me import from
tests import star. And that's going to give me access
to a couple of different things. But the first thing you'll
notice that it's doing is because I'm using this WebDriver,
it's going to give me a web browser. And here I'm using Chrome. But you could use another browser. But notice it here, Chrome
is telling me Chrome is being controlled by automated test
software, that Chrome has the ability to allow me, using
automated test software, using Python code, control what
it is the web browser is doing. And so what I can do here is
the first thing I need to do is tell Chrome to open up my web page. And it turns out that
in order to do so, I need to get that page's URI or Uniform
Resource Identifier, just some string that will identify that page. And I've defined a
function called file_uri that gets me the URI of a
particular file in this directory. So I'm going to say I
want open up counter.html. And I need to get it's URI. But now what I can say
is driver.get(uri), meaning tell this web
driver, this Python-- part of the Python program that is
controlling the web browser that I would like to get this web page as
if the user had gone to the web page and pressed Return
after typing in the URL. So I say driver.get(uri). I go ahead and press that. And what you'll notice on
the right-hand side here is that Chrome has loaded this page. I am effectively controlling this web
browser window using my Python program. I said driver.get(uri), meaning go
ahead and open up the counter.html page. And then, inside of this test
window, Chrome has opened that up. And inside my Python program
now, using this web driver, I have the ability to see the
same things that the user sees when they open the page. So what does a user see
when they open the page? Well, they see, for example,
the title of the page. So I could say driver.title and see
that, all right, the title of this page is Counter. That was, in fact,
the title of the page. But I could verify that
inside my Python program that by checking that driver.title,
by taking my WebDriver, getting the title of the current
page that it's looking at, making sure that that is Counter. And likewise, if I looked at
driver.page_source, press Return, what I see there in string format-- so it's a little bit messy-- is the content, the HTML
content of this page. And you'll notice things
like DomContentLoaded, here are my onclick handlers. Here's my h1 that says 0. It's very messy because it's being
represented as a Python string. And these backslash n's refer to new
lines where there's a line break. But this is the content. And this is really all the browser gets. The browser takes this
information and just knows how to render it in some nice,
more graphical representation that's easier for a user to look at
and, therefore, be able to understand. But this is fundamentally all that my
web browser is actually getting back when it tries to load a web page. So what can I do from here? Well, I would like to simulate a user's
behavior on this page that assures I can get a page and see the title. But I want to simulate like clicking
on the plus button, for example. So in order to do that,
first thing I need to do is actually get the Plus button. And to do that, I could say something
like driver.find_element_by_id. There are a number of ways that I
could try and find an HTML element. But I would like to find
an HTML element by its ID. And I know that the Plus
button-- the Increase button-- has an ID of increase, for example. And so if I find element by ID, let me
find the element whose ID is increase. And, all right. It seems here, that I've gotten some web
element object back from my web driver. And I'll go ahead and save that
inside of a variable called increase. So what I now have is a
variable called increase that represents the Increase button that
my WebDriver has found on the web page. It's effectively the same
thing as you, the human, going through the page looking
for the Increase button. The WebDriver is doing the
same thing, except instead of looking for the button
based on what it looks like, it looks for the button based on its ID. And so this, again,
another reason why it's helpful to give your HTML elements IDs. In case you ever need to be
able to find that element, it's very useful to be able to
reference that element by its name. But now that I have a button,
I can simulate user interaction with that button. I can say something like
increase.click to say I would like to take the Increase
button and simulate a user clicking on that button in order to see whatever
it is that the user would get back when they click on that button. So increase.click, I press Return. And what you'll notice happens is
that the number increases-- increases from 0 to 1. It's as if I, the user, had
actually clicked on the Plus button, except all I did was say increase.click
to say, go ahead and press the Increase button
and let the browser do what it would normally do in response. And what it would do in response
is get that JavaScript event handler, that onclick
handler, and run that callback function that increases the value
of counter and updates the h1. So I can say increase.click to simulate
increasing the value of that variable. But this is just a
function call, which means I can include it in any other
Python constructs that I want, that if I want to repeat something
like 25 times, for example, and press the button 25 times,
I can say for i in range 25, go ahead and click the Increase button. And now, very quickly, 25 times, it's
going to click the Increase button. And I'm going to see the result
of all of that interaction. So I can simulate user interaction
just by using the Python interpreter. Likewise, if instead of
increasing, I wanted to decrease, well, then, I'm going
to do the same thing. I'm going to say decrease equals
driver.find_element_by_id. Let me get the decrease element,
the element whose ID is decrease. And now say, decrease.click. And that will simulate me
pressing the Decrease button. Press it again. And the more I press it, every time,
it's just going to decrease by 1. And if I want to decrease
it all the way back to 0, well, then, I'll just do it 20 times. For i in range 20, go
ahead and decrease.click. And that's going to go ahead and
reduce the count all the way back to 0 by simulating the user
pressing a button 20 times. And what you'll notice is that
it happened remarkably fast. Like I can simulate 100
presses of the Increase button by saying for i in range
100, increase.click. And very quickly, you'll see
that number 100 times go ahead and go up to 100 faster than a
human could ever have clicked that Plus button over and over again. And so these tests
cannot only be automated, but they can be much faster than any
human could ever be in order to test this behavior. So how then can we incorporate
this idea into actual tests that we write, into like a
unit testing framework that allows me to define all of these
various different functions that test different parts of
my web application's behavior? Well, to do that, let's go ahead
and take another look at tests.py. Inside of tests.py, here, again,
is that file_uri function, where that function has the sole purpose
of taking a file and getting its URI, and we need the URI to
be able to open it up. Then we go ahead and get
the Chrome WebDriver, which is going to be what's
going to allow us to run and simulate interaction with Chrome. And in order to get
Chrome's WebDriver, you do have to get ChromeDriver separately. It is separate from
Google Chrome itself. But Google does make it available. And other web browsers make
equivalent web drivers available as well if you'd like to test how
things would work in other browsers because different browsers
might behave differently. And it might be useful to be able to
test to make sure that not only does everything work in Google
Chrome, but it's also going to work in other
browsers that you might expect users to be working with as well. Here, then, I've defined a class that,
again, inherits from unittest.TestCase that is going to define all of the tests
that I would like to run on this web page, that here I have a function called
test_title that's going to go ahead and first get counter.html. It's going to open up that page. And then just assertEqual, let's
make sure the title of the page is actually Counter. That's what I would expect it to be. So I can write a test in order
to test for that case as well. Here, I test the Increase button
by finding the element whose ID is increase and
clicking on that button to simulate a user pressing
the Plus button in order to increase the value of the counter. And then, what do I want to check? Well, I want to check that when
you find element by tag name, h1, and so find_element_by_tag_name,
similar to find_element_by_id, except instead of finding
something by its ID, it's going to look at what's the tag. And there is only one
element that is in h1. And so here I'm saying, go
ahead and get me the H1 element and access its text
property, meaning whatever it is that is contained
inside of those two H1 tags, I would expect that to be the number 1. And I would assert that
this is equal to 1. And likewise, I can do the same
thing for the Decrease button-- finding the element whose ID is
decrease, clicking on that button, and then asserting equal,
find me the H1 element and make sure that
the contents of it are equal to the number
negative 1, for instance. And this final test, just test things
multiple times, that three times I'm going to press the Increase
button and make sure that after I press the Increase button
three times, when I check the h1, check what's inside of its text, that
the answer should, in fact, be 3. So now I should be able
to go ahead and test this code by running python tests.py. And what that is going to do is
it's going to open up a web browser. And what you're going to see, very
quickly flashed across my screen, were all of those tests. We tested the increase by 1. We tested decreased by 1. And then we tested like increase
3 times after we had checked to make sure the title was correct. And then we can see here is the output. We ran four tests in
this amount of time, and everything turned out to be OK. None of the tests failed. But if one of the tests
had failed, well, then, we would see a different output. So let's imagine, for example, that I
had had a bug in my decrease function, for example, where the decrease
function wasn't actually working. What would that bug look like? Maybe I forgot to say
counter minus minus. Or maybe, perhaps more likely, what
might have happened is I wanted to-- I already had written
the increase function, and I decided to very quickly
add the decrease function, and I thought I just-- like Copy/Paste, like copy
the increase event handler. The decrease event handler
is basically the same thing except I need to query
for decrease instead. And maybe I just did that
and forgot to change plus plus to minus minus, a
bug that might happen if you're not too careful about how
you copy and paste code from one place to another. Now when I run these tests, python
tests.py, we'll see the simulation. A whole bunch get simulated. And when I go back and check
the output of my tests, see what actually
happened, I see that we do seem to have an assertion error here. The assertion fail was on
the test decrease function. And it happened when I tried to assert
that what was inside of the H1 element was negative 1 because 1
is not equal to negative 1. So this is the value of this
assertion error as well. And this is helpful, an
advantage over just assert. Assert just tells you there
is an assertion error. But here, in unittest,
we actually get to see if I asserted that two things are equal,
it tells me what both of those things are. It tells me the actual
output of h1's text. It was 1. But what I expected it
to be was negative 1. So it tells me exactly
what the discrepancy is. I know that for some reason,
it was 1 instead of negative 1. And that can be a clue
to me, a hint to me, as to how I can go about
trying to solve this problem. And I can solve the problem by going
into my decrease event handler, seeing that, all right, this
was increasing the counter instead of decreasing it. Change plus plus to minus
minus, and now rerun my tests and see all of the test simulated
inside of my Chrome Driver. And we ran four tests. And this time, everything was OK. So all of my tests appear
to have passed this time. So those, then, are some possibilities
for being able to test our code, especially taking advantage of
unittest, this library that we can use in Python in order to
make various types of assertions about what we would like to be
true or false about our code. And unittest contains a
number of helpful methods for being able to perform
these sorts of assertions. Some of them are here. So we can say things like I would
like to assert that two things are equal to each other, which we've seen. There's a counterpart to that,
assertNotEqual for making sure the two things are not equal to one another. assertTrue and False,
we've seen as well. There are others as well though,
things like assertIn or assertNotIn, if I would like to assert, for example,
that some element is in some list, for example, or that some
element is not in some list. There are other assert methods
as well that we can use in order to verify that a part of our program
or a part of our web application does, in fact, behave the
way we would want to behave. And we can integrate this
type of idea into a number of different types of testing. We saw integrating it into Django
itself, using Django as unit testing in order to verify that our database
works the way we expected it to, and that our views works the way
that we expected them to and provided the right context back to the user after
the user makes a request to our web application. And there are also
applications of unit testing, whether using the framework or
not, to browser-based testing, when I want to test inside
of the user's web browser. Does it actually work when a
user clicks on this button, that the JavaScript behaves the
way that I would expect it to. And I don't need to especially use
JavaScript in order to do those tests. I didn't write those tests
using Python, using unittest, to be able to say, click on
the button that has this ID and verify that the result that we get
back is what we would expect it to be. So that then was testing. And now we'll go ahead
and take a look at CI/CD-- Continuous Integration
and Continuous Delivery which refer to two best practices
in the software development world that has to do with how it
is that code is written, especially by groups or teams of
people; how it all works together, and how that code is eventually
delivered and deployed to users who are using
those applications. So CI, which refers to
Continuous Integration, involves frequent merges to a
main branch of some repository, like a Git repository, and
then automatically running unit tests when that code is pushed. So what does that generally mean? Well, in the past, you might imagine
that if multiple people are working on some project at the same time
and multiple people are each working on different features or
different parts of that project, then after everyone's done
working on those features and we're ready to ship some
new version of a web application or ship some new version of a
software product, well, then, everyone's going to have to take
all these various different features and combined them all
together at the end and figure out how to then try
and deliver that program to users. And this has a tendency
to cause problems, especially if people have been
working on different big changes all simultaneously. They might not all be
compatible with one another. There might be conflicts between
the various different changes that have been made. So waiting until everyone
is done working on a feature to merge them all back
together and then deliver it is not necessarily the
best practice, which is why increasingly, many
more teams are beginning to adopt a system of
continuous integration, that there is one
repository somewhere online that's keeping the official
version of the code. Everyone works on their own version of
the code, maybe on their own branch, for example. But very frequently,
all of these changes are merged back together
into the same branch to make sure that these incremental
changes can be happening such that it's less likely that there's two
really divergent paths that the program has gone under, and as
a result, it's much more difficult to merge those
two paths back together. In addition to frequently
merging to its own main branch, another key idea of
continuous integration is this idea of automated unit
testing, where unit testing, again, refers to this idea
of on our program, we run a big series of tests that verify
each little part of our program to make sure that the web application
behaves the way it is supposed to. And unit tests generally refer to
testing particular small components of our program, making sure that
each component works as expected. There are also bigger scale tests--
tests like integration tests that make sure that the entire
pathway from user request and response, that everything along
a certain pipeline works as well. But there are various
different types of testing. And the important thing is
making sure that anytime some new change is merged into
the main branch or someone wants to merge their changes
into the main branch, that these tests are run to
make sure that nobody ever makes a change to one
part of a program that breaks some other part of the program. And in a large enough
code base, it's going to be impossible for
any one person to know exactly what the effect
of one particular change is going to be on every
other part of the program. There are going to be unforeseen
consequences that the one programmer may or may not know about. And so the advantage of
unit testing, assuming they're comprehensive and cover all
of these various different components of the program, is that any
time someone makes a change and attempts to merge that
change into the main branch according to the practice of
continuous integration, the fact that it doesn't pass a test,
we'll know about that immediately. And as a result, that
programmer can go back and try to fix it as opposed to
waiting until everything is done, merging everything
together, and then running the tests, realizing
something doesn't work, and then being unsure of where to begin. We don't know where the bug is, which
change happened to cause the bug. If everything is merged
more incrementally, it's easier to spot those
bugs, assuming there's good coverage of tests
to make sure that we're accounting for these various
different possibilities. So continuous integration
refers to that idea-- frequently and more incrementally
updating the main branch and making sure that the
tests are, in fact, passing. And it's closely tied to a related
idea of continuous delivery, which is about the process of how it
is that the software is actually released to users, how the web
application actually gets deployed. And there are a couple
of models you might go about thinking with regards to
how it is that some program or web application gets deployed. You might imagine that the
release cycle might be quite long, and the people spend months working
on various different features on some software development team. And after they're happy
with all the new changes, they've released some new
version of the web application. But especially, with
web applications that are undergoing constant
change, that have lots of users, that are moving very
quickly, one thing that's quite popular is this notion of continuous
delivery, which refers to having shorter release schedules. Instead of immediately
releasing something at the end of some long cycle,
you can in shorter cycles make releases every
day, every week, or so in order to say that let's just go ahead
and incrementally make those changes. Whatever new changes happen to
have merged to the main branch, let's go ahead and release those as
opposed to waiting much longer in order to perform those releases. And that, again, lends
itself to certain benefits, the benefit of being able
to just incrementally make changes, such as
something goes wrong, you know more immediately
what went wrong as opposed to making a lot of changes at once,
where if something goes wrong, it's not necessarily
clear what went wrong. And it also allows new features to
get out to users much more quickly. So especially in a competitive market
where many different web applications are competing with one another,
being able to take a new feature and release it very quickly
can be quite helpful. So continuous delivery is all about
that idea of short release cycles. Rather than wait a long time for
a new version to be released, release versions incrementally
as new features begin to come in. It's closely related to the idea
of Continuous Deployment, which CD will sometimes also represent. Continuous deployment is similar
in spirit to continuous delivery. But the deployments
happen automatically. So rather than a human
having to say, all right, we've made a couple of changes. Let's go ahead and deploy those changes. In continuous deployment, any
time these changes are made, the pipeline of deploying
the application to users will automatically take place as well,
just removing one thing for humans to have to think about and
allowing for these deployments to happen even more quickly as well. So the question then
is, what tools can allow us to make continuous integration
and continuous delivery a little bit easier? What techniques can we
use in order to do so? And there are a number of different
continuous integration tools. But one of them produced by GitHub more
recently is known as GitHub Actions. And what GitHub Actions allows us to
do is to create these workflows where we can say that anytime, for example,
someone pushes to a Git repository, I would like for certain
steps to take place, certain steps that might be checking to
make sure that the code is styled well. That if a company has some style guide
that it expects all of its programmers to adhere to when working
on a particular product, you could have a GitHub Action such that
anytime someone pushes to a repository, you have an action that
automatically checks that code against the
style guide to make sure that it is well-styled, well-commented,
documented, and so forth. You might also, for instance,
have a GitHub action that tests our code to make sure that
anytime anyone pushes code to a GitHub repository, we
automatically run whatever tests we would like to run
on that particular code base. And GitHub Actions can allow us to do
that as well by defining some workflow to be able to do so. And so that's what we'll take
a look at in just a moment, using GitHub Actions to automate
the process of running tests so that the human-- though it would be a
good thing for the programmer when they're done writing their code to test
their code and make sure it works-- we can enforce that by making sure that
every time anyone pushes to a GitHub repository, we'll
automatically run some GitHub action that is going to take care
of the process of running tests on that program. And we'll know immediately
as via an email that GitHub might send to you to say
that this particular test failed. And you'll know every time
you push to that repository. So how do these
workflows get structured? What is the syntax of them? Well they use a particular
type of syntax known as YAML, which is some language,
a configuration language, that can be used in order to describe-- often described for configuration of
various different tools and software. GitHub Actions happens to use it. Other technologies use it as well. And YAML is a file format that
structures its data sort of like this, in terms of key value
pairs, much in the same way that a JSON object or a
Python dictionary might, where we'll have the
name of a key followed by a colon followed by its value-- name of a key, followed by a
colon, followed by a value. And the value doesn't necessarily
need to be just a single value. It could be a sequence of values,
like a list of values, for example. And those are generated this
way, by like a hyphen indicating a list-- item 1, item 2, item 3. And in addition to just
having single values and lists of items, these ideas-- these lists, these
sequences, these values-- can be nested within
one another, that you might have one key that
leads to another set of keys that are associated with
values that leads to other sets of keys associated with values as well. Much in the same way,
that a JSON object, like a representation
of keys and values, can also have nested JSON
objects within a JSON object. Likewise, too, we can have
nested key value pairs as the value for a particular key too. So we'll take a look at an
example of what that actually looks like in the context of
creating some GitHub workflow that will run some get GitHub Actions. So what will that look like? Let's go back into airline0, where
here, I've defined inside of a .github directory a directory called workflows,
inside of which I have a ci.yml file. It can be any name .yml. .yml or .yaml are the conventional
file extensions for a YAML file. And here, I'll open up ci.yml. And this now is the configuration for
how this workflow ought to behave. I give the workflow a name. It's called Testing because
what I want the workflow to do is test my airline application. Then I specify an on key to mean
when should this workflow run. And here, I have said
on push, meaning anytime someone pushes their code to GitHub,
we would like to run this workflow. Every workflow consists of some jobs. And so what are the jobs? What tasks should happen anytime that
I try and push to this repository? Well, I've defined a
job called test_project. And this is a name that
I chose for myself. You can choose any name for
a job that you would like. And now I need to specify two
things for what happens on a job. One thing I need to specify is what
sort of machine is it going to run on? That GitHub has its own virtual
machines, otherwise known as VMs, and I would like to run this job
on one of those virtual machines. And there are virtual machines for
various different operating systems. Here I'm just saying, go ahead, and
run on the latest version of Ubuntu, which is a later version of Linux that
I would like for this test to run on. And then for the job,
I specify what steps should happen where I
can now specify what actions should happen when someone
tries test a project when I try and run this job. And here I'm using a
particular GitHub action. And this is a GitHub action written
by GitHub called actions/checkout. And what this is going
to do is it's going to check out my code
in the Git repository and allow me to run programs
that operate on that code. And you can write your own
GitHub actions if you would like. But here, all we really need to do
is go ahead and check out the code, as by looking at what's on the
branch that I just pushed to. And then I'm going to
run Django unit tests. This is just a description
for me to know what's going on in this particular step. And here is what I would like to run. I'm going to first go
ahead and install Django because I'm going to
need to install Django to be able to run all of these tests. But after-- and if there
are other requirements, I might need to install
those requirements as well. But the airline program
is fairly simple. All we really need in order to
run the tests is just Django. So I'll go ahead and install Django. And then I'll run
python3 manage.py test. I would like to test-- run all of the tests. And the way I can do that is just
by providing this manage.py command to say that I would like
to run all of the tests on this particular application. So this configuration
file altogether now is going to specify a particular
workflow, the workflow that says that every time I push
to the GitHub repository, what I would like to happen
is I would like to check out my code inside of the Git repository. So on some Ubuntu VM, GitHub
is going to check out my code, and it's going to run these commands. It's going to install Django. And then it's going to test my code. And it will then give back to me
what the response is after I do that. So let's go ahead and test this. And in particular, let's
run it on a program where the tests are going to fail. So I might say, for example, let's
go into flights and models.py. And let's go to my is_valid_flight
function from before and change it back to that
version that didn't work. That before it was
something and something. I'll change it to
something or something. That as long as the origin is not
the destination or the duration is greater than 0, we'll
count that as valid. But we know that that's wrong. That should not work. So here's what I'll do. I'll go ahead and first say git
status, see, all right, what's changed? And it seems that, all
right, I've changed-- I've modified models.py,
which makes sense. I'll go ahead and git add. I'll add dot. We'll just add all of the files
that I might have modified. I'll commit my changes. Say go ahead and use wrong
valid flight function. That's what I'm going to do. And now I'm going to
push my code to GitHub. I added it. I committed it. I pushed it. That now then pushes my code to GitHub
into a repository called airline that I already have. And now, if I go ahead and go to
GitHub, and I go to my airline repository, what you'll
notice is that we've mostly been dealing with this Code tab. But GitHub gives us
other tabs as well that are quite useful as you
begin to think about working on a project in larger team. So in addition to looking
at the code, we have issues. Issues are ways for people to
just report that something is not quite right, or there
is a feature request that we have for this
particular code base. So the issues might maintain a
list of all of the pending action items for a particular repository,
things that we still need to deal with. And once those issues are dealt
with, the issues can be closed. So I have no issues here as well. Pull requests are people that are
trying to merge some part of the code from one branch into another branch. So you might imagine
on a larger project, you don't want everyone merging things
into master all at the same time. You might have people working
on their own separate branches. And then when they feel confident
and happy with their code, then they can propose a pull request
to merge their code into the master branch. And that allows for
various other features, like the ability for someone
to offer a code review-- to be able to review the code, write
comments, and propose suggestions for what changes should be made
to a particular part of the code before it gets merged
into the master branch. And that's another common
practice with regards to working on a GitHub repository
or any other larger project that you're controlling using source
control is this idea of code reviews, that oftentimes, you don't want
just one person making the changes without anyone's eyes on that code. But you want a second pair of eyes
to be able to look things over, make sure the code is correct,
make sure it's efficient, make sure it's in line
with the practices that the application is using. And so pull requests can
be quite helpful for that. And then this fourth tab over
here represents GitHub Actions. These are the various
different actions or workflows that I might want to run on
this particular repository. And what we'll see here is that if I go
to the Actions tab now, what I'll see is here is my most
recent testing actions. So anytime I push, I get
a new testing action. This one was from 29 seconds ago. I'll go ahead and click on
it and see what's within it. All right. Here was the job that
I ran, test_project. I see that on the left-hand side. You'll notice this big red X in the
left-hand side of this workflow. Means something went wrong. So I'd like to know what
it is that went wrong. I'll go ahead and click on test_project. And here within it, these are all of
the steps, the things that happened when we actually ran this particular job. First the job sets up. Then the checkout action goes
ahead and checks out my code because we need access to my
code to be able to run it. Here was the step I defined-- run Django unit tests, which was going
to install Django and run those tests. It has an X next to it,
indicating something went wrong. And I see down below,
annotations, 1 failure. So all over the place, GitHub's trying
to tell me that something went wrong. It failed two minutes ago here. I'll go ahead and open this up. And what I'll see is the first thing
that happened is we installed Django. And that seems to have worked OK. But down below, what you'll see is the
output of running these unit tests, that we see FAILED (failures-2). And now I can see, here are
the unit tests that failed. We failed the invalid
flight destination test. We failed the invalid
flight duration test. And as before, I can see
in GitHub's user interface what those assertion errors are. I can see a true is not false. True is not false,
those were the problems that happened when I tried to
run this particular test suite. And now others who are also
working on this repository can see as well what the
results of these tests are and can offer
suggestions, can offer ways that I might be able to fix the code
in order to deal with that problem. But now I know that this
particular test failed. And if I go back to the main code
page for this GitHub repository, I'll see that next to this commit,
there is a little x symbol. And that little x symbol
next to the commit just tells me that the most
recent time I tried to commit, something went wrong. They ran the workflow,
and there was an error. And so I'll immediately
see for this commit-- and I can go back and look
at the history of commits and see which ones were OK
and which ones had a tendency to cause some sort of problem. So this one, it appears
caused a problem. And we know why. It caused a problem because of this
condition, something or something else. So I can fix it. I'll change the or to an and. I'll go ahead and git add dot. git commit. Say I will fix valid flight check. If I do git status just to check
out what's going on right now, I'm ahead of the master
branch by 1 commit. That's exactly what I would expect. And now I'll go ahead and push my
code to GitHub by running git push, saying, all right,
let's push this update. And now, hopefully, we're
going to pass the workflow now. Now I go back to the repository. I refresh the page. Here's my latest commit--
fix valid flight check. You notice here, there's an orange
dot instead of the red x as before. This dot just means the
tests are currently pending. The workflow is in progress because
it takes some time for GitHub to be able to start up the VM,
to be able to initialize the job, to check out my code,
to run all those tests. It does take some time. But if I go back to the Actions
tab, I'll see that, all right. This time, for testing,
we get a green check mark. Everything seems to be OK. I go to test_project just to see it. And now I notice the green check
mark next to Run Django unit tests means that the unit tests
have passed as well. If I open those up,
now I see at the bottom the same output that I saw before
when I was running those unit tests on my own machine. We ran 10 tests, and everything was OK. And that tells me that
these tests have passed. So GitHub Actions have the ability
to allow for certain jobs to happen, certain work to happen anytime you push
code, anytime you submit a pull request or on various different actions that
might happen on a GitHub repository. And they're very helpful
for being able to implement this idea of continuous integration
because it means you can make sure that when you're merging code from some
developer's branch into the main branch that everyone's merging
their code into, you can verify that those tests can pass. And you can add rules
to say that you don't want to allow anyone to merge code into
the branch if the tests don't pass, to guarantee that any
code that does get merged is going to pass all
of those tests as well. And so that can definitely
help the development cycle, make it easier to ensure that
changes can be made quickly. But as we make those
changes quickly, we're not going to lose accuracy
and validity within our code, that we can make sure
that our code still passes those tests by automating
the process of running those tests altogether. So other than continuous
integration then, we now talk about this idea
of continuous delivery, these short application
cycles where we would like to very quickly be able to deploy
our application onto some sort of web server. And when we're deploying
applications to a web server, there are things that
we need to think about. We need to think about
getting our program that was running fine on our computer
working on a web server as well. And this can just be
fraught with headaches and all sorts of configuration
problems because you might imagine that the
computer that you are using is not necessarily going to be the
same as the computer that on the cloud, the computer in the server where your
web application is actually running. It might be running a
different operating system. It might have a different
version of Python installed. If you have certain packages
working on your own computer, those same packages might not
be installed on the server. So we run into all sorts of various
different configuration problems where you can be developing,
deploy your code, and realize that it doesn't
work on the server because of some sort of difference between
what's happening on your computer and what's happening on the server. And this becomes even more problematic
if you're working on a larger team, you and multiple other people
working on a software project, but you each have different versions of
various different packages or libraries installed, and those different
versions have different features and might not all work and
cooperate with one another. And so we need some way in order
to be able to deploy applications efficiently and effectively to be
able to standardize on just one version of the environment, one
version of all these packages, to make sure that every
developer is working on the project in the same environment. And once we deploy the
application, it's going to be working in the
same environment as well. And the solution to this comes
in a number of possible options. But one option is to
take advantage of a tool like Docker, which is some sort
of containerization software. And by containerization software,
what we're talking about is the idea that when we're running an
application, instead of just running it on your computer, we're going to run it
inside of a container on your computer. And each container is going to
contain its own configuration. It's going to have certain
packages installed. It's going to have certain versions
of certain pieces of software. It's going to be configured
in exactly the same way. And by leveraging a
tool like Docker, you can make sure that so long as you
provide the right instructions for how to start up and set up
these containers, then if you are working on the application
and someone you're working with, some colleague that's also
working on the same project, so long as you're using the same
instructions for how to set up a Docker container, you're going to be working
in the identical environments, that if a package is installed on
your computer, in your container, it's going to be installed in your
colleague's container as well. And the advantage of this too works
with this idea of continuous delivery. When you want to deliver and deploy
your application to the internet, you can run your application inside
of that exact same container, set up using the exact same
set of instructions, so that you don't have to worry about
the nightmare headaches of trying to make sure that all the right
packages and all the right versions are, in fact, installed on the server. Docker might remind you of the
idea of a virtual machine or a VM if you're familiar with that concept. GitHub uses VMs, for instance,
when running its GitHub Actions. They are, in fact, different. A VM is effectively running
an entire virtual computer with its own virtual operating
system and libraries and application running on top of that all
inside of your own computer. So a virtual machine ends up taking
up a lot of memory, taking up a lot of space. Docker containers, meanwhile,
are a bit lighter-weight. They don't have their
own operating system. They're all running still on top
of the host operating system. But there is this
Docker layer in-between that keeps track of all of these
various different containers and keeps track of for
each container such that every container can have its
own separate set of libraries, separate set of binaries, and an
application running on top of that. So the advantage then
of containerization is that these containers are
lighter-weight than having an entire virtual machine. But they can still keep their
own environment consistent such that you can feel confident that if
the application is working in a Docker container, you can have that Docker
container running on your computer, on someone else's
computer, on the server to guarantee that the application is
going to work the way that you would actually expect it to. And so how exactly do we configure these
various different Docker containers? Well, in order to do so, we're going
to write what's called a Docker file. So to do this, I'll go
ahead and go into airline1. And I'll open up this Docker file. And the Docker file
describes the instructions for creating a Docker image
where the Docker image represents all of the libraries and
other installed items that we might want to have
inside of the container. And based on that image,
we're able to create a whole bunch of
different containers that are all based on that same image,
where each container has its own files and can run the web
application inside of it. So this Docker file,
for example, describes how I might create a container that is
going to run my Django web application. So first, I say FROM python:3. This happens to be another
Docker image on which I'm going to base these
instructions, that this is going to be a Docker image
that already contains instructions for installing Python 3, installing
other related packages that might be helpful. Oftentimes, when you're
writing a Docker file, you'll base it on some existing
Docker file that already exists. So here I'm saying go
ahead and use Python 3. And now what do I want to do in
order to set up this container? Well, I want to copy anything
in dot, in my current directory, into the container. And I have to decide, where in the
container am I going to store it? Well, there-- I could
choose to store it anywhere. But I'll just store it in /usr/src/app,
just some particular path that will take me to a directory where I am
going to store the application. But you could choose
something else entirely. So I copy all of the current
files in my current directory. So that will include things like my
requirements file, my manage.py file, my applications files,
all my settings files. Everything inside of the directory, I
would like to copy into the container. Then I'm saying WORKDIR, meaning change
my working directory, effectively the same thing as something
like CD on your terminal to move into some directory. I would like to set my working
directory equal to that same application directory, the application directory
inside of the container that now contains all of the
files from my application because I copied all of those
files into the container. Now once I'm inside of this directory, I
need to install all of my requirements. So assuming I've put all my requirements
like Django and any other packages that I need inside of a file
called requirements.txt, I can just run the command,
pip install requirements.txt. And then, finally, inside the
Docker file, I specify a command. And this is the command that should
run when I start up the container. Everything else is going to
happen initially when we're just setting up this Docker image. But when I start up the
container and actually want to run my web application, here
is the command that should run. And I provide it-- effectively
it's like a Python list where each word in the command is
separated by a comma, where here I'm saying the command that I would like to
run, when you start up this container is python, manage.py, runserver. And here I'm just specifying
on what address and what port I would like it to run. And I'm running it on
port 8000, for example. But I could choose another port
that I would like to run instead. So what's going to happen then is that
when I start up this Docker container, it's going to, if it needs to,
go through these instructions and make sure that it sets
up the container according to these instructions,
make sure that we've installed all of the
necessary requirements, make sure that we're using Python 3. And anyone using the
same Docker file can generate a container that has
all the same configuration on it. So we don't have to worry about
configuration differences between me and someone else who might not have the
exact same computer setup that I do. And the nice thing about this is that
it can run on Mac and Windows and Linux. So even people running on
different operating systems can still have containers that
all have the same configuration, that all work in the same way
just to speed up that process. Now so far, when we've been
building Django applications, we've been using a SQLite database. SQLite database just being a file that
is stored inside of our application. And this file-based database allows us
to create tables, insert rows into it, delete rows from it. In most production environments,
in most real web applications that are working with
many, many users, SQLite is not actually the
database that is used. It doesn't scale nearly as well
when there are many users all trying to access it concurrently. Oftentimes, in those
sorts of situations, you want your database hosted
elsewhere on some separate server to be able to handle its own
incoming requests and connections. And we talked about a
couple of possible databases we could use instead of SQLite, things
like MySQL, things like Postgres, or various different
SQL-based databases. So imagine now I want to
deploy my application. But instead of using SQLite, I would
like to use Postgres, for example, as the database server
that I would like to run. Well, that would seem to be pretty
complicated for me to test on my own because now in addition to running
my web application in one server, effectively, I also need another server
that's running Postgres, for example, such that I can communicate with
that Postgres database instead. And that's going to be even
harder for other people to be able to work on as well. Potentially, it might be difficult to
get the server to work in that way too. But the nice thing
about Docker is that I can run each of these processes in
a different container effectively. I can have one container that's running
my web application using this Docker file right here. And I can have another container
that's just going to run Postgres. And as long as other
people also have access to that same container
for running Postgres, they can be working in an
identical environment to the one that I am working in as well. And so there's also a feature of
Docker known as Docker Compose. And what Docker Compose
lets us do is allow us to compose multiple
different services together, that I would like to run my web
application in one container, and I would like to run a Postgres
database in another container. But I would like for those containers
to be able to talk to each other, to be able to work together
whenever I start up the application. So if I'd like to do that, in order
to run this application on my computer and have both the web application
and Postgres installed, I can create a Docker Compose
file which looks like this. Here I'm specifying using
version 3 of Docker Compose. Here I specify, again,
using a YAML file. Much as in my GitHub workflows
were formatted in YAML just as a configuration file,
docker-compose.yml is a YAML file that describes all of the various different
services that I want to be part of my application, where each service
is going to be its own container that could be based on a
different Docker image. Here I'm saying that I have two
services, one called db for database, one called web for my web application. The database is going to be based
on the Postgres Docker image, image that Postgres wrote that I
don't have to worry about. Someone else has written
the Docker file for how to start up a Postgres container. Here, though, for the
web application, that's going to be built based on the
Docker file in my current directory, the Docker file that I have written. And then down below, I've just
specified that my current directory should correspond to the app directory. And then I've specified when I'm
running this on my own computer, I would like port 8000 on
the container to correspond to port 8000 on my own
computer just so that I can access port 8000 in my browser and
access port 8000 inside the container. It just lets my computer
actually talk to the container so I can open up the web application
in my web browser, for example, and actually see the
results of all of this. So here, then, I've created two
services, a database and web. So now let's actually try
starting up these containers. I'm going to first go into
my airline1 directory. And I'm going to say
docker-compose up to mean go ahead and start up these services. I'll press Return. And what you'll see is we're going
ahead and starting up two services. I'm starting up the database service. And I'm starting up the web service. And now as a result of all of this,
I've started up the application. And I started it on port 8000. So if I go to 0.0.0.0 slash 8000
or colon 8000 slash flights, that's going to take
me to the Flights page. And now this is running,
not just on my own computer, but inside of a Docker container. Now, of course, right now,
there are no flights inside of this page because I haven't actually
added anything to the database yet. So I could do that if I wanted to. But how do I do that? Well, I needed to go into slash
admin to say, like, let me log in and go ahead and create
some sample flights. But I don't have a log in yet because
I need to create a superuser account. And I can't just like inside
of my airline1 directory say, python manage.py
createsuperuser the way that I used to because this is
running in my terminal on my computer. Whereas, what I really want to do
is go into the Docker container and run this command there,
inside of the container. So how can I do that? Well, there are various different
Docker commands that I can use. docker ps will show me all of the Docker
containers that are currently running. So I'll go ahead and shrink
this down a little bit. I see two rows, one for each container,
one for my Postgres container that's running the database,
one for just my web application that's running as well. Each one has a container ID. So I want to go into my
web application container in order to run some commands
inside of that container. So I'm going to copy its container
ID and say, docker exec-- meaning go ahead and execute a
command on the container-- dash it will make this interactive. Here's the container ID that I
would like to execute a command on. And the command I want to execute
is bash, passing the dash l flag, but bash to say, I want
to run a bash prompt. I want to be able to
interact with a shell so that I can run commands
inside of this container. So I press Return. And now what you'll
notice is that I am inside of my container in the
user source app directory, that directory that contained all of the
information about this web application. I type ls. And what I'll see is here, all the
files inside of this container now. And now I can say something like
python manage.py createsuperuser. And now it's going to let
me create a superuser. So I'll create a user inside of
my web application called Brian. I'll give it my email address. I'll type in a password. And now we've created a superuser. And you can run other commands here. If you wanted to migrate
all of your migrations, I could say python manage.py migrate. And it turns out I've already
done that, so I didn't actually have to do it again. But you can run any commands that
you can run them on your computer. But now you can run them inside
of the Docker container instead. I'm going to press Control D just
to log out, get out of the container and get back to my computer. But now I've created a
superuser, so I could go ahead and sign in to Django's admin. And now I can begin to
manipulate this database, which is a Postgres database running
in a separate container. But the nice thing about
it is that I can start them both up together just by running
something like docker-compose up, for example. So Docker can be quite a powerful tool
for allowing us to very quickly ensure that an application is
running in the environment that we expect it to be running, to make
sure that all of the right libraries are installed, make sure that all the
right packages are installed as well, that the configuration between
my development environment and the environment that's running
on the server are the same as well. So those then were just
some of the best practices for how you can go about
developing a program now that we have the tools to do so. We have a lot of tools for being able
to develop these web applications. But as our programs start
to get more complex, it will be increasingly
important to test them, make sure that each various different
component of our web application behaves the way that it is expected
to behave, and then taking advantage, especially in bigger teams, of CI/CD,
Continuous Integration, Continuous Delivery to make incremental
changes, and make sure each of those incremental changes,
in fact, works on the web application. And then CD, Continuous Delivery, to say
that rather than wait and then deploy everything all at once, let's
deploy things incrementally as well. Let users more quickly get
access to the latest features and more quickly find out
if something went wrong. We can better identify
what it is that went wrong if we've deployed things
incrementally rather than waiting a long time in order to do so as well. So these are some of the best practices
in modern software application development, not only for web
applications but for software more generally. Next time, we'll
consider other challenges that might arise as we go about
trying to make web applications that are used by more and more
users, in particular, taking a look at challenges that
will arise in terms of scalability as the programs get bigger and
also security of what security vulnerabilities open
themselves up as we begin to design our web applications
using Python and JavaScript. So more on that next time. And we'll see you then.
