JEFF DAVIS: Welcome to the
Google Cloud Platform Education Series. My name is Jeff Davis. And in this session we'll
discuss building applications that scale automatically
with Google App Engine. We'll be running a moderated
Q&A during the session, so feel free to ask
questions using the Ask Your Question Here field on
the screen in front of you. Let's get started. So the format of this
session will be-- we'll start with a demo. And this is an in-depth demo
of a relatively sophisticated application that will show
the magic of App Engine in automatically
scaling to meet demand, as well as how easy it is
to manage your application once it's in production. Coming out of the
demonstration, we will have a
high-level discussion about the range of Google
App Engine features and how best to leverage them. We'll take a few minutes
to talk about where you can get more information
online after this session. We'll talk about best
practices for evaluating Google App Engine for specific
use cases in your organization. And again, we'll be doing
a running Q&A. Feel free to ask questions as we go. So let's cut over to a demo. This is a customer success
application running on top of Google App Engine. We work for a company
called betterdev.io. And betterdev makes--
among other things-- a videoconferencing application. So here we have a mock-up of a
video conferencing application. And users can perform calls. So this is a one-way call. You can actually see me
in the studio right now. When they hang up,
it's going to record information about this call
into a backend database. And what we're really
interested in as a vendor is we want to make sure that we know
all the ways that our users interact with our product. And then for each
customer, we want to know where there
are opportunities to improve their outcomes
and optimize our chances at renewals and upsells. So here we have a customer
success application. This is being used by betterdev. They've actually bought
it as a SaaS product from another company
called Success-hq. But you can see this dashboard
gives the customer success managers at betterdev insight
into their various customers. They have a list
of projects here that are underway
that they maybe should be paying attention to,
they have upcoming renewals, and they have trending changes
in usage and engagement with the products. So you can see, for instance,
that the calls per week for Skills Feed-- one of our
customers-- has gone up by 30%. If you click on
any of these, you can actually get
a dashboard that gives you a whole
bunch of detail about how users
within this customer are actually
leveraging the product. So for instance, the
Team Davis account, they've purchased 10
video conferencing devices to go with the software. They have an annual contract
value of roughly $25,000 a year. And we can see the trends
in their engagement with the product. So when I refresh the page,
this is actually loaded live from the backend data that's
been recorded as users interact with the product. So we have purchased
devices, registered devices, registered users, percentage
of devices provisioned, calls per week, stuff like that. Now what a customer
success manager would do with this is they
would look for opportunities to improve the customer's
engagement with the products. So for instance, the first
thing that we notice is only 60% of the purchased devices have
actually been registered. We should get those other
ones plugged in and going. We also notice that
the calls per week is trending down over time. So this shows the last 30 days. And we actually want to boost
this up, not have it go down. On the other hand, we can see
that support ticket volume has gone down and end of call
ratings are slightly going up. So those are positive
things that we could bring up in a
quarterly business review with the customer. Now, this is super helpful
when running a customer success organization, and we
want this application to serve our needs as we grow. As our customer
success organization gets bigger and
bigger, we want to be able to serve more
and more users. And as we have more and more end
users of our videoconferencing product, we want
all of that data to be able to be
ingested into the backend and thus rendered in
widgets on these dashboard without any operations
overhead on our part. So let's take a look. All of these charts
are being pulled out of a backend data repository. We're using Google
BigQuery, which is Google's petabyte-scale,
low-cost data warehousing solution. You can see that
we actually have a table of all of the events. So we have the date,
the type of event-- whether it's a call or a page
load or a comment or a dial-in session-- who did it, what company
they work for, and then some other information. If you look down
here at the bottom, you can see that we have-- 525,000 events
have been recorded. Now, I was just doing
one call at a time, and so that's writing
one event at a time into this backend
data warehouse. But what happens if there
are hundreds or thousands or tens of thousands of users? We want to make sure that our
application doesn't fall over. But I'm not a real expert when
it comes to doing operations, so I would like Google to take
care of all of that for me. And just to show that this is
actually all on the up and up, let me go back-- and
I'm going to bring up the betterdev dashboard. And you can see that this is
showing that we've done 2,600 calls roughly in the last week. If I go back to this UI
and I do a new call-- hi-- and then I hang up,
if I refresh this view, it should actually show 2,633. So you can see that it updated. My use of the product flowed
event data into the backend, and then that backend
is being rendered in a nice visualization here. So let's take a look
at the architecture. What we have here is customers. These are users of the
videoconferencing software. And every time that
they use the product, it's going to write an event
to an event API service. So this is an API,
and it runs separately from the frontend of the
customer success software that shows the visualizations. And all it does is record
data into BigQuery. Now, again, if we have
lots and lots of users, we may need lots
and lots of servers to catch all the
incoming events and write those events into BigQuery. Then the folks who are using
the Success-hq product-- these are the CSMs-- they actually hit
a web application which renders the frontend
as well as all the data visualizations. And this frontend can read data
from a memcache service, which is a stored in-memory data
store that is shared across all of the frontend servers. It reads and writes information
about users and companies into Cloud Datastore,
which is a NoSQL database. And it reads all of the
necessary information to render the dashboard
widgets from BigQuery. This architecture shows
you one thing right away, which is that App Engine is a
great platform for stitching together multiple backend
services from within Google. So you want to use git datastore
and you want to use BigQuery and you want to use memcache? No problem. App Engine is a great place
to stitch those together. Another thing that we'll see is
I've actually broken my product into two different services. These aren't quite
micro services, they're sort of macro services,
but this gives you the idea that you can actually
split your application into discrete
components and then we can manage those
components separately. Let's take a look at what's
required to actually put this all together in production. So behind the scenes,
here we have App Engine. And you can see that in
the App Engine console, I've got my two different
services running. So I've got the API events
service and the default service, which is
my web frontend. And you can see that
I've got instances that are automatically running. So for the frontend,
App Engine is currently running two services. And if you look
at the API events, you can see that it's
currently running one service. What would be great
is if when nobody's using either these
of services, it would spin down
to zero instances, and therefore it doesn't
charge me anything at all. But then I want it
to scale up to dozens of servers or instances
if the load requires that. And in fact, let's try that out. Let's just see how this
autoscaling stuff works. So what I'm going to do is I'm
going to use my SaaS growth generator. If only it was actually just
this simple to generate growth in a SaaS business. So I'm going to start with
generating a low background load of new events. So this is emulating
a bunch of people using the video
conferencing product, and it's writing events
into the backend. What I should see here-- if I look at the
events service-- is I'm going to stay-- you'll notice I've
got 27 requests. There's a little bit of
delay in updating this, but you're going to start seeing
that those request numbers are going up. So it's 40 requests, 45
requests, 50 requests. And we're averaging fractions
of a query per second. This is easily handleable
within the single instance that we have running. So we can look at latency,
we can look at errors, we can look at traffic
or utilization, that sort of thing. So not a ton of load. Let's go back to our
magic load generator and say, hey, now we've
got product market fit, we've got 10,000 simultaneous
users at any given time. I'm going to click
on this button. This is going to emulate
lots more users using the product, thus lots more
events being written in. Now, I haven't had
to do anything, but what we should
see fairly shortly here is additional
instances getting spun up because the
queries per second goes up. So we can see it
continues to go up. It hasn't spun up
another instance yet. So, so far, so good. Well, let's just
drop the hammer. Let's go straight to a
million simultaneous users. This is a lot of
calls on my API-- clearly more than one
instance can handle. Now you'll notice I've
got two instances. And there we go. So now you see we've
got five instances. And this will keep growing
out as is required. So App Engine is using an
algorithm of CP utilization memory utilization, and requests
per second to identify how many frontend instances--
or in this case, how many backend API
service instances-- I need in order to service
all the incoming requests. So we promised you apps
that scale automatically. Here you go--
automatic app scaling. In addition to
making it really easy to scale up or down
depending on load, App Engine also
makes it really easy to manage services
that are in production. So let's say I want
to actually release a new version of the
frontend One of the things that I have found is some
people aren't entirely sure what to do with this dashboard view. And I'd like to give
them the opportunity to ask questions of the support
team while they're using it, so I need to make
a modification. What I'm going to do is
switch to my code editor. And I'm going to integrate
a third-party product. So this is a product
that actually puts a little messenger
box onto a web page and allows the end user
to talk to somebody within our organization. So I'm going to
copy this and I'm going to paste it into
the customer view. What we're going to do is we're
going to put that down here. Now, this is a new version
of the application, so I want to update the
versioning information. I'm going to say
this is version 1-2. Now, this is version 1-2 of
the frontend web app service. Notice there is an API
event service as well. And I'm going to leave
it configured as 1-1. So because I have my app
broken into two services, I can upgrade one
component without touching the other components. So now that I've got
that, what I'm going to do is actually deploy the service. So here's my command line. And I'm going to switch into-- let's see-- Success-hq
and I'm going to deploy. Now, this is all I
have to do to deploy. So it's one simple command line. It's taking all of
my business logic and all of my
application code and it's loading it up on top of the
App Engine infrastructure. It's automatically
creating a new version of the application frontend
keeping the old version. So I have a 1.1
version and you'll see that I have a 1.2
version running now. So what's interesting here is
I go back and I go to versions. And you'll notice that
for my API event service, I only have the one version
because I haven't updated it. But if I go to
default, you'll notice that there are two versions. You've got the big
blue line down here and the big gray line down here. What this means is that
all of the traffic going to my application
is being served from the initial version. Currently, none of
the traffic is being served by the new version. So we've actually
staged this version. People can only get to it if
they use a special URL which prepends the version number. This allows me to test it
in a production environment without actually having it
be live to my end users. I can also choose
to split traffic. So, for instance,
if I click on this and I choose Migrate
Traffic, this will actually move all of the
traffic from the old version to the new version. Or I can choose
to split traffic, and this would
allow me to say, you know what, I want
20% of my end users to be directed to
the new version while 80% continue to
go to the old version. This would be a
great way to test the efficacy of that new
widget that we put in, which I'll show you in just a sec. All right, so let's
try the new version. I'm going to click here. It's going to open
a special URL. And you'll notice I have
a new capability built in. This is asking me
for a free trial because I haven't paid for this. But you'll notice in the
bottom right-hand corner now, I've got this
messaging window which allows the end user
ostensibly to talk to somebody on the backend. If I click through, you'll
see that this actually loads on every single page. I'll dismiss the
free trial thing. And it's having a little
bit of difficulty pulling up the messaging, but you can see
that the widget actually works. If I go back and I try
loading the first version, you'll notice that I don't
get the little plug-in. So I've got two
different versions running at the same time. And right now everybody
will be directed to the default version. If I want some people to try
the newly installed feature, I can send them a
special URL, or, again, what I can do is I can
do traffic splitting. I've been showing you this
running in a canned version that I had already
gotten up and running. What if I wanted to
deploy this from scratch? You're like, Jeff,
OK, it's really easy once it's up and
running, but maybe it was really hard to get up
and going in the first place. And again, not so. Let's take you
through the process of deploying this from scratch. So I'm going to
create a new project, and we're going to
call this success-hq-3. And we're going to
bill it to my account. We're going to create
the new project. Remember, a project in
Google Cloud Platform is a collection of
applications and resources that have a particular
billing account and a particular
security boundary. So I'm going to create a
separate group of resources for this part of the demo. And then what I'm
going to do is I'm going to make a new
App Engine application. So I'm opening up the
Google Cloud Shell, which is a web UI that has
all the command line tools that I need to manage
Google Cloud Platform. So I'm going to create a new
application for this project. And it asks me where I want
to locate the application, and I'm going to put this
in the US central region. So it takes just a moment to
fire up the new application, but it's an empty application. There's no code there. So now I need to deploy my code. So while that's going
in the background, what I'm going to do is I'm
going to go into my code and I'm going to change
this to be success-hq-3 and I'm going to change
this to be success-hq-3. And now I've got
a chunk of code. All I've done is one short
command line in the console to create a new application. Let's make sure that that
has finished running. It has. OK. And now what I'm
going to do is I am going to deploy my
application to my new project. So this is a
first-time deployment of a brand new application. And I built this application
on my local machine. I've got development tools
that Google makes available. It's super easy to do. And once it's up and running,
if I go through to App Engine, I should see the default
service is there. I'm still waiting for the
API events service to run. OK. I refresh here. OK, I can see both
services are running. I can see that by default,
there are no instances because nobody has
tried to access either one of the services yet. And let's try out
the default service. OK, so once the deployment
to our new project is done, we should be able
to click on this. And if all goes well, we'll see
that the frontend is actually up and running. You'll notice that there's
no information for projects and renewals and
trending because this is a brand new instance. There's no data
in the Datastore, so we would have to
populate the data. But we can see that it
was actually as simple as 25 seconds of waiting
with one command line to actually deploy and get the
application up and running. So with that, what
we've shown is that it's super simple
to create a new project and deploy a new application,
that that application can scale servers up and down
automatically based on load, that it's really easy
to upload new versions and actually do partial
releases of those versions. And you can easily
segregate your services within an application so that
you can decouple the release schedule of the API for
events and the frontend for the end users. Isn't that cool? Hopefully you saw
something you liked. Let's switch back
to the presentation and talk at a high level about
the wide range of functionality available to you in App Engine. So as you've sussed
out by now, App Engine is basically a magic
layer of software that allows you to take business
logic and some application code, upload it, and have it
run with little to no operations overhead on top of Google's
massive compute storage and networking infrastructure. You don't have to worry
about defining servers, standing them up,
tearing them down. You don't have to worry
replication, load balancing, any of that stuff. You can do all of this
in Compute Engine, but in App Engine, it's all sort
of done for you with a software layer that Google provides. Now, there are two
different environments available in App Engine. What we showed you in
that demo is what's called standard environment. And what happens in
standard environment is you take your code-- and when you deploy it, Google
creates a containerized version of your application in a quite
locked-down and standardized container environment. They've got four
separate runtimes that you can use with App
Engine standard environment. There's Java,
Python, Go, and PHP. So if you want to
use another language, standard environment isn't
going to work for you. Don't worry, there's
another environment which we'll talk about in a second. But the benefit of having this
really restricted, locked-down container environment
is that Google can then apply all sorts of automation
and management magic to reduce the operations
overhead on you. When you are using standard
environment runtimes, you have access to a
bunch of support services. So for instance,
there's a service that allows you to
send and receive emails from your application. There's a service that allows
you to resize and manipulate images. There's an API-- as
I noted-- that allows you to do a shared
memcache across all of your autoscaled
server instances. These are all available to you. You can do all of this
stuff without using the built-in standard
environment APIs, but you would have to
load client libraries and write a little
bit more code. Now, the upside to these
very tightly constrained standard environment runtimes
is all of the automation magic that happens. The downside is they create
some constraints which are unfamiliar to programmers
coming from other environments. So for instance, you can't
have background processes, you can't write
to local disk, you can't install
third-party runtimes. And all of your requests
to your application have to return and
complete within 60 seconds. So if you're a developer used to
working in other environments, you're going to have to learn
some new patterns to make your apps run in App
Engine standard environment and take care of all of
the automation magic. Now, Google understands
that for some developers, this isn't a viable
solution, and so they've created something called
flexible environment. And what flexible
environment does is it gives you the ability to
use a wider range of runtimes and to customize
your own runtimes. So you have to do a
little bit more work. You have to actually
create a Docker file that describes
the configuration of your container. It does allow you to
do a variety of things you might want to do. So for instance, you can
support additional languages. You can support Ruby
or Node or .NET. You can install
third-party binaries. You can do all the
programming stuff that you're used to, like
accessing the local file system, writing to
the local file system, that sort of thing. You trade off, though,
access to some capabilities in standard environment. So for instance, you have
to do a little bit more overhead defining your
Docker containers, configuring your runtimes. Some of the APIs are
unavailable if you're using nonstandard runtimes. And you cannot host your
flexible environment App Engine applications in Europe. You can host it in North
America or Asia Pacific. You can access it from
anywhere in the world, but you just can't have the app
running in the European data centers. What kind of instances does
your App Engine application have to choose from? So in standard environment,
your App Engine instance classes range from the small end of a
128 megabytes and 600 megahertz to one gigabyte of
memory and 4.8 gigahertz. In flexible environment, you
can use any of the standard GCE machine types, so you
have a lot of flexibility in terms of the
amount of processor and the amount of memory. In standard environment, you've
got three types of scaling. You've got automatic,
basic, and manual. So manual, you would choose
exactly how many instances you want running. You would have to change the
configuration file and redeploy in order to get more instances. Automatic will use that
algorithm that we talked about. And basic is sort of in between. Flexible environment also offers
automatic and basic scaling, but it does scaling the same
way that GCE does autoscaling. So that would be based
on CPU utilization or on custom
Stackdriver metrics. Billing can get a
little complicated. I've got a summary here. I would suggest that you
actually search on App Engine billing calculator. And they've got a fairly
sophisticated calculator where you can type in
all your parameters. Or run test loads
and get an idea of what the costs look like. But the basic thing here is
for standard environment you get billed for instance hours. So you have no
instances running? You pay no instance hour fees. Instance hours range from $0.05
per hour to $0.40 per hour, depending on the
instance class type. And then if you are using
services-- so for instance, if you're making
calls to the Datastore or you're making calls
to the image API, you may pay a fee
based on x number of executions of the API. Flex environment
pricing is comprised of an hourly fee for the
number of CPUs you're using, an hourly fee for the amount
of memory that you're using, and an hourly fee for
whatever persistent disk you have associated with
those instances. And then on top of
the instance charges, you pay for Datastore use and
you pay for network egress. So any data that goes into your
App Engine application is free. Any data that comes
out of the GCP region, you actually have to pay for. And again, that's $0.08 to $0.11
per terabyte in North America and a little bit more expensive,
say, in Australia or in China. We used the Google
Cloud data store-- in the demo application,
we used that to store all the information about
the individual companies and the projects and
the trending information and so forth. Cloud Datastore is a NoSQL
document database that's designed to be very flexible. So it's sort of a
schemaless datastore, and it's also high-performance. So you can scale to millions
and millions of rows and it performs quite well. It supports atomic transaction,
high availability-- so your data is replicated
across multiple physical resources within a region. As a result, what that means
is that if a particular device goes down or even
a zone goes down, your application
is still available and your data is still there. Massive scalability-- so
as I said, really fast across many, many, many rows. Your data is encrypted at rest. And it's a fully managed service
with no planned downtime. So you don't have to worry
about a single point of failure. You don't have to worry about
it being down Saturday mornings from 8:00 to 12:00 AM. Google takes care of
all of that for you and your application runs
without interruption. In terms of architecting
your application, App Engine provides
you a variety of features that allow you to
be more intelligent in the way that you implement, deploy,
and manage your applications. So we talked about services. The idea here is you can
take a single application and break it down into 2, 5,
10, or more micro services. That allows individual teams
to develop each service at a different pace. And you can roll those
out at differing versions and differing times. Once you roll out new
versions, you can roll forward, you can roll back, you can split
traffic between those versions. What we would do at
Highfive, for instance-- which is a videoconferencing
company that I used to work at-- we would actually roll out
new features to, say, 10% of the user base, then
to 30% of the user base. And assuming nothing
caught on fire and there weren't any
screams, we would roll it out to the rest of the user base. If anything went wrong, we
could simply roll it back to the previous version. Task queues allow you to offload
longer running processes. So we said, for instance,
that in standard environment you have a 60-second time limit
to return a response to the end user. What you could do is get
a request from a user, return a response immediately,
but put some long-running task on a task queue and
then that can run for 5, 10 minutes in the background
without a problem. There's also a new feature
that was rolled out just in February of 2017 called
Cloud Endpoint Frameworks. And this is a mechanism by
which you can expose APIs using Google's Open API cloud proxy. And this provides better
security, better scalability, standardized interface,
and automatically generated JavaScript clients that you can
include in your applications to make use of your APIs. So, given all
those capabilities, whether it's
standard environment, flexible environment, whether
you're using a backend data store of Cloud Datastore
or you're using, say, BigQuery, the question that
you want to ask yourself is, what can you do
with Google App Engine? You may have heard of a company
called Snap, previously known as Snapchat. What they were able
to do with App Engine is grow within a few years
from 0 to 700 million plus photos and videos
shared every day with a very small development
team and an even smaller operations team. They were able to focus
exclusively on business logic and application functionality,
and the scale just happened in the backend. You may have seen that they've
made a forward commitment to spend $2 billion on Google
Cloud Platform services over the next five years. So, obviously, it's a
long-term bet for them. Your application probably
isn't going to be this big, but one can hope. So let's say you leave
the session today and as you're working with
App Engine you have questions or you need a little
bit of guidance on how to do specific things. Where do you look? So you start with search-- Google preferably. And you can search for GAE-- short for Google App Engine-- and your topic of interest. So for instance, here
we've got GAE versions. And the first link is
a link to App Engine standard environment. And then you've got the ability
to do deploying versions and so forth. There's also App Engine
and your topic of interest. Sometimes GAE isn't
specific enough and putting App Engine
will work better. The documentation
is a little bit different than what you have
seen with other Google Cloud Platform products. So you still at one point
end up with this nice table of contents, which has got
quick start, how-to guides, API references, concepts,
tutorials, and so forth. But there's a level of
organization above this-- and it would actually be helpful
if I just showed this to you. So if we look at
Google App Engine-- I always start with search. I'm going to go to
App Engine platform. So this is the overview page. If I click on View App Engine
Docs, what you'll notice is it doesn't take me to
the table of contents. Instead it takes me to a
table of contents that asks me what language I'm using. And they've actually got
versions of the documentation for all of the
popular languages. So I use Python primarily. So if we look at
Python, then you have the next level, which
is, choose your preferred environment. Do you want flexible environment
or standard environment? When you click
here, this will then take you to the familiar
table of contents section. But they have a different
one of these pages and different subordinate
sections for each combination of language and environment. So keep that in mind as
you're looking around. Now, as you're considering App
Engine for your environment, a few things to consider. One, follow a process. Figure out what it is
you need to know in order to make a decision
and what success is-- what threshold you
need to cross in order to determine that App Engine is,
in fact, the solution for you. Think about how
you're going to test. Are you're going to lift
an existing application and put it in App Engine? The answer to that is
often no-- certainly not for standard environment
because there are a lot of structural changes
that you might need to make. You could lift an
application from a containerized environment
perhaps and put it on flexible environment. But figure out what you're
going to do-- probably something like a
minimum viable product or a prototype of something new
that you're wanting to build. Figure out who's on the team. You're going to need somebody
to design the test product. You're going to need
software engineers. You're going to need
somebody to project manage the entire evaluation process,
and then decision-makers to whom you report
the results and they help make the decision. Note you need to
be comfortable-- or you need to get
designers or developers that are comfortable-- in order to make this
evaluation worthwhile. If you're going
to get developers who are very entrenched in
another development methodology and they're not
particularly keen to learn how to work with App Engine's
specific constraints, then it's probably
not the right person to have involved
in your evaluation or that's not the right
use case to attempt to move to App Engine. What you want is somebody
who's willing to pick up the specific rules and patterns
for designing and developing for App Engine in order to
get all the benefits out in the operations side. Again, follow a timeline. So typically, you can
make an application that proves out how
well App Engine will work for you within a month. If it's more complicated,
it'll take longer. If you have more people,
it'll take less time. And you want to break it
down into three phases. There's a phase where
you define your criteria and you recruit
your team and you agree on your high-level tests. There's a section of time
where you do some research, you do some point
testing, you make sure you actually understand
how App Engine works and how you're going
to build your thing. You build it. And then ultimately
you perform your tests. And you're testing
probably for performance, you're testing for functionality
and developer productivity, you're testing for pricing. And then you collect
all of those results, you analyze them, and
you make a decision. And hopefully this
gives you a sense of all the things you
can do with App Engine and how you can try App
Engine out for your use cases to make a good decision. And that's it for this session. Thanks so much for joining us. I hope you learned
something and that you found it interesting and useful. We'll be sending you a follow-up
email that has a link to a code lab so you can get some guided
experience with App Engine yourself. We'd love to get
your feedback, so feel free to provide that
feedback on the screen. And stay tuned for
our next session.
