SPEAKER: If you are ingesting
large amount of data for analysis, or
looking to simplify the development of
event-driven microservices, then you need to check
out Cloud Pub/Sub. Pub/Sub Made Easy is
a series dedicated to helping you understand
Cloud Pub/Sub inside out. So let's get started by going
through an overview today. [MUSIC PLAYING] Adam and I have been
collaborating on a project, but we are based in
different parts of the world. I'm here in California. And he is in Australia. So I just do my part
on the Google Doc and assign it to him for review. And he does the same. This allows us to
receive messages and act on them in our
own working hours, essentially asynchronously. We don't have to be
available at the same time and are still able to
get the project done. Cloud Pub/Sub is based
on the same concept but for applications
and services. Pub/Sub helps build
robust, scalable systems of applications by integrating
them asynchronously. Cat-Out is a website to
purchase fun cat posters. When a user orders
a pretty poster, Cat-Out packages the order,
ships it to the user, and notifies them. In a very simplistic service
oriented architecture, they will need at least
four microservices-- a packaging service to
pack the poster, a shipping service to generate labels, and
a notification service to send notification to the user
about the status of their org, all these tied to
Order Manager Service. Granted that there'll be some
payment services, a reporting service, et cetera. But for the purpose
of this conversation, let's assume those all exist. Now, let's trace what happens
when a user orders a piece of art. The order service
receives the order and sends a message to
the packaging service. The packaging service then
does the necessary processing to package the poster and
sends a message to the shipping service to ship it. The shipping service does
the necessary processing to ship the package
and sends a message to the notification service,
which notifies the customer that an order is processed. In the same way, the order
and packaging service also send messages to
the Notification service to send to the
customer the updated information on their order. There are some problems with
this arrangement though. One, it's just complicated. It's a lot of arrows. But as we drew deeper, what
happens if a service, say the packaging service, fails? Well, we need some
kind of retry mechanism that would keep the whole
system from banking. Hm. Can be done, but a lot of work. What happens if a new
service gets introduced like a monitoring service that
logs all of these transactions to a central store? This service needs to interface
with all the other services, which will also require
making a change in all the other services. That makes the developers
work really hard. Think about testing. That would be hard too. We will need mock
services to test each of the individual service. Let's say I need to test
my packaging service. Because it is dependent
on the shipping, order, and notification service,
I need to mock up all of those services just
to test my packaging service, which is a lot of work. What if you want to
scale this system? If there are three instances
of packaging service, there would be a load
balancer involved. You would not know how to
cancel the order, which stage it is in, and can
it still be canceled. This is where Cloud
Pub/Sub comes to rescue. Cloud Pub/Sub is a fully
managed, real time messaging service that allows
you to send and receive messages between
independent applications. Once an event is successfully
published as a message, it becomes the
job of the service to ensure that all
the systems that need to react to this event get it. The idea behind
asynchronous integrations is to react to events
represented as messages. In our example, messages
merely represent events happening in the real
world like order received, package shipped, et cetera. Let's see Cat-Out's flow of
events with Cloud Pub/Sub now. The order service, instead
of sending messages directly to the packaging
service, will just send these messages to Pub/Sub
where they will be stored. The packaging service and
the Notification Service will receive these messages
either by a push or a pull. Packaging service, instead
of directly sending messages to shipping service, would
send them to Pub/Sub. And the event will be
received by the shipping and notification service. The packaging
service did not need to know which services
are receiving this event. This makes it simple. Similarly, the shipping service
sends the messages to Pub/Sub from where the order and
the Notification Service pull or receive those messages. By using Cloud Pub/Sub, we've
solved a lot of problems that we had in the
earlier architecture. Well, one, we've made
each piece simpler since it only needs to emit
an event without worrying about who gets it or receive
an event without much care about how it got there. But more importantly,
this is more scalable. It reduces dependencies. Services can change
and don't need to know about each
other's changes. All that need to be
made sure is that they are subscribing to Pub/Sub and
receiving the right messages. It's also much easier now
to introduce new services. If our new monitoring service
pops up, all we need to do is make sure it
subscribes to the messages from the existing service
via Cloud Pub/Sub. The architecture
with Cloud Pub/Sub has built in resilience. If any service
were to break down, Pub/Sub will store the
messages for seven days or until the service comes back
up, and it's delivered then. And because Pub/Sub is
fully managed service, you don't have to worry about it
being single point of failure. It's scalable, and resilient,
and can handle millions of messages simultaneously. All right. So today we learned
when to use Cloud Pub/Sub and the common
challenges it solves. Join us next time to
learn how it works and some of the most
common use cases. If you like this video and would
like to see more such content, don't forget to like and
subscribe to our channel.
