[MUSIC PLAYING] MARC PAWLIGER: So how did
we get to where we are? So Google's big. It's not really a surprise, but
we have hundreds of buildings. We have tens of
millions of square feet on all of our offices,
and all of our labs. And we have difficulty
managing that. So there's existing
tools and platforms. Have difficulty scaling. Data standardization in
practice is difficult. So what we've done
is we've built tools that allow easy, fast,
and accurate commissioning at scale. Because when you think about
it, building management today is really fine, right? And the answer is,
well, not really. So we built this
platform in order to manage our buildings better. And fundamentally, we want
to run Google on Google. So we want to be a customer of
ourselves so that we can show, this is how we manage our
data and our buildings. So we want to be able
to access our data, we want to be able
to analyze it, and we want to be
able to act on it. So today, we're going to
talk about the platform, how we built it, some of the
data science behind it, and then we're going to
show you a little quick demo of the kind of applications
that you can build on it. So today, when you're thinking
about building management, there's really three personas. The first one is,
if you're either the landlord or the sole tenant. So you're caring
about things at scale. You may not care about
an individual building, but you care about how, for
example, the energy spend and the space utilization
happens at scale, and why this building is working
better than that building. You may also be the
facilities ops manager. That's a little
bit more focused. You care about a
particular building, or a particular campus. And you want to know how you
can actually operate it better. You might be dealing
with trouble tickets. You might be doing truck rolls
in order to do the maintenance and so forth. And then there's the occupant. So all of you here today
probably work in an office. And you care about having
the building be comfortable. You care about the
facilities being able to let you get your work
done efficiently, effectively, and comfortably. So what's the opportunity here? Well, so first is energy. That's the first
one that, any time someone's talking
about smart buildings, they really hone in on. You want to be able
to use less energy, and you want to
spend less on it. And I don't think
we really need to go into too much detail about
the motivation behind there. It's both on cost. It's on efficiency. It's being more green. If you're on the
operational side, I don't think it's uncommon
to say that people really get alarm fatigue. If you have anything to do
with operating a building, you know that there are
literally tens of thousands of alerts that come in,
either weekly or monthly, that are just routinely ignored. You don't have the time to
be able to dig into the ones that you really have to
take care of in order to address issues that you
have either with occupants, or with your facility overall. And they hide efficiencies
that you might otherwise be able to realize if you
were able to dig deeper. And then, on the
productivity side, answering hot and
cold tickets, that means people are
uncomfortable enough not to be able to get
their work done, and they're telling
you about it, if you're managing the facility. And moreover, Google
has tons of buildings. Some of them are
very complex layouts. People have a lot of meetings. And we have found that people
actually spend-- and this is a measured amount--
millions of hours searching on the floor for the meeting
room that they need to get to. Now, there may be
the issue that you're having too many meetings. That's a separate discussion. But fundamentally,
you need to be able to get people to where
they need to go faster. And then there's something
that is very important. I'm not going to be able to go
into too much about it today, but it's cybersecurity. You want to make sure that
the devices that you have-- and it's a growing set of
them-- in your building, whether that's HVAC,
so climate control, whether it's occupancy
sensors, whether it's different kinds of alarms,
water, life safety, that kind of thing, all
of them are safe, and they're communicating
in such a way that they're only communicating
the way that they should, and that somebody
isn't going to be able to compromise your network
through your air conditioner. So there's lots of
stories that you've heard about that
in the news, and we don't want the systems that
we're building over here to be one of those vectors. So our platform is
really to the rescue. And the nut of it
really is, we want to have the ability to stop
having people be your sensors. You want to be
able to use sensors that you have in the
building, and you want to gain the
intelligence from that. So on the energy
efficiency side, you want to be able to gather
and aggregate all your data. You want to be able to use
machine learning, ML, to detect anomalous behavior so
that you can address it. On the efficiency
side, you want to be able to filter all of those
tens of thousands of alarms that you unfortunately
aren't unable to get to, to get to the small
number of them that you actually want
to dispatch a tech or dispatch a truck to
actually take care of. And when you have
people who go on-site to address these
issues, you want them to be better informed. You want them to know
where their devices are, what's actually
going on, and what could be the root cause of
causing them to malfunction. On the occupant
side, you want to be able to not wait for the
trouble ticket to open up. You want to be able to
detect that something is sufficiently out of band to
be able to proactively address it. And you want to be able
to definitively get people to where they want to go. So they shouldn't be lost. They shouldn't waste
time where you're waiting for that last
person to show up for your discussion
or your meeting. And then on the
security side, we want to be continually
monitoring these devices to make sure that they're
only communicating in a way, and only communicating
to the devices that they are supposed to be. And like I said, we can talk
more about that afterwards if you're interested in
that particular aspect of the presentation. So today's building management
systems are really siloed. They've been around
for a long time. But fundamentally,
you're going to have, for each one of the
different silos-- whether it's AV, or energy,
or lighting, or security, they're going to
have their own UI. They're going to have their own
stack, their own storage, which may be on prem. It may be in a
third-party cloud. It may be through an API
to access the devices. They're going to have their
own networking requirements. And really, they're separate. They also tend to
be very rule-based. They don't tend to be learning. So it makes it difficult, if
you get your rules set up just so for one building, to be
able to take them and put them into another building. It's basically a lot
of walled gardens, and that's problematic. And today, it's also
sometimes difficult to be able to take all
of these separate silos and combine them into a more
cohesive orthogonal platform where you're able
to access things, and apply the same kind of
intelligence to one system as to another. So what's our approach? This is the platform
that we want to build. It really orchestrates
all of this together. And it starts with
the fact that we want to be able to take
any kind of device-- like I said, whether
it's a climate control, whether it's a sensor,
whether it's security-- and think about it
in an orthogonal way. So we want to be
able to have an API that your mental model of
how you think about all these is the same. So at the bottom, you have
your devices, and your network, and your connectivity. So fundamentally, that's not
what we're going to go into. We assume that you're going to
have some kind of a gateway. You're going to have some
kind of IP connectivity so that you can talk
to these devices and communicate with them. And they're going to use any
kind of industry standards-- Modbus, BACnet, et cetera-- to communicate with you. And we're assuming that you
have all of that set up. That's kind of a standard
understood way to do things. But then on top of it,
what we're going to do is we're going to take the data
from each one of the devices and put it in a data lake. Now, when I say data lake to
you, that means a lot of things to different people. But fundamentally, to us,
it's a time series database stamped for each one of the
devices, and done in such a way that we understand the ontology,
the relationship of the devices to each other, have
generalized them to different classes
of devices, and we're able to access them
in a structured way. Sometimes, a data lake can be
thought of as just a collection of data. But here, we're putting
order on top of it. And we do that using a
building ontology and metadata about each one of the devices. And Charbel will go into much
more detail about that later. And another thing
that we do which is, I think, a little bit
different than other systems, is we have spatial understanding
of where these devices are, where they are in
relationship to each other. Floor maps for the
building so that not only do we know what
the device is, what it's connected to, its
relationship to other devices, but where it is as well. And of course, none of
this would be useful unless you had APIs in
order to access the data. And we do so using the
same kind of mental model that you do for all of the other
Google Cloud Platform stacks and technologies that we have. This is something that we
did because it makes sense to do it in a standardized way
so that other developers who are developing
apps on top of this are able to access
it in the same way that they do any of the other
cloud platform technologies. And altogether, this
is what we're calling the digital building Platform. So taken altogether, that's the
special black box, if you will, that takes in the
data on one side and provides it to
applications on the top. So this is where we've worked
with some third parties to take their existing systems
that do things like alarming, and analysis, and so forth,
and they use our platform as the source of their data. Or we also have some
groups inside Google that have built
first-party applications to do things like anomaly
detection and alarming as well. So how do we onboard a building? Now, what does onboarding mean? So onboarding-- for anyone who's
set up a building automation or a building
management system-- is taking your devices and
being able to talk to them, and getting them connected
into whatever kind of management software or
platform that you have. And it usually is a
very painful process. Sometimes it can day take days. Sometimes it can take weeks. Well, with the platform that
we built, we started there. And it actually did take us
on the order of days or weeks to onboard a building. But we actually got
it down to the point where we can do it
in a matter of hours for most of the buildings
in Google, which is a pretty big impressive thing. So we start where
you either have greenfield, which
are new devices, or brownfield, which
are existing devices. And those devices
might be individual. So they might be something
like an air handler. They might be something that
you think of in aggregate together where you have
different component devices-- like a VAV, or something
like that-- inside of it, and you think of them as one. And then we create an idealized
representation of them. So you have one where you think
of each one of the classes of devices in a way
that it encompasses all of the special
fields and data points that each one of
the manufacturers' devices might have in it. But we do it in
such a way that you can address them orthogonally. So you don't have to
worry about what brand this is, what type
this is, and so forth. And so if you've ever dealt with
these devices, each one of them has data points. That's a piece of data and
a tag that basically says, what is this, what
does it represent, what units are it
in, and so forth. And in the idealized
representation, we have the same thing. And we do this in
such a way, again, to make it so that you can
treat all of the devices in a particular class the same. And then we need to figure
out, for each one of these data points, what semantic tag, what
tells you what this data is and what it represents, what
unit it has, and so forth. So for example,
this might be a tag that you have in
the actual device. And here are the
different semantic tags that we might
break it down into, because we can do clustering
and matching to basically say, we recognize this kind of
device in this set of points that it has, and this is how
we map it to a set of tags. And then, basically,
we have a mechanism that will go through and
do that matching for all of the other points
and tags in the system. And that way, it saves
a lot of the manual work that you used to have
to do before to do it in a more automatic way. And then, of course,
you have the APIs to access it on the top,
and then the applications. So in whole, this
is basically what the stack looks
like after you've gotten something onboarded
and into the building system. So what do we do with all
this once you're done? It's all great to have the
data, but unless you actually make use of it, it's
not that interesting. So just as an example,
particular problem that you might face is an
outer set point temperature. So if you look at
the box in yellow, that's what the data looks like
raw coming from the device. And it says that
this is a sensor that should be in the range from 62
Fahrenheit to 72 Fahrenheit. It's currently 81 degrees, so
obviously, that's problematic. This has landed
in the data lake. The sensor data captured the
data, and it put it in there. It tagged it with a time stamp. None of this is really
unusual if you've dealt with any typical building
automation system before. But we also have the ontology
and the mapping which tells you where the device is, what
type it is, what kind of class it relates to, and allows us to
get more contextual information about this particular error,
such that when we then dispatch someone to take
care of it, they're able to have more
of this context, and hopefully are able to
address the problem faster than they otherwise would. So the whole is greater
the sum of the parts. And you can use all of
these kind of sensors together to tell
an aggregate story. Now I'm going to
switch over to Charbel, who's going to tell you
some of the data science behind all of this platform. CHARBEL KAED: Thanks. So I want to talk about, how
do we make sense of this data? As Marc mentioned
before, we have all of these silos that
are really in verticals where the data is trapped. So if you take a step back, we
have really different devices and systems that are
provided by various vendors. And they can vary by geographic
region, or by country, because no single vendor
can cover the whole planet. And as we mentioned, the
data is trapped in silos. We saw we have the lighting,
we have the cooling, the badge access reader. All of these data are
really in vertical. And of course, when you're
talking about the building, it's a whole
ecosystem, so we need to do data correlation
for better management. If we know that the
space is not occupied, we don't have to clean it. Or maybe we don't
have to cool it. Or maybe we don't
have to light it. So all of these
simple information can give us a lot of insights
to better managed space. And of course, we want to
be able to correlate data so we can serve it for
these three personas, whether you are a landlord,
an occupant, or a facility manager. So one of the ways of doing
that is by relying on ontology. So before I go through
it, by show of hands, who heard about
this word before? OK. Cool. So an ontology is
a common vocabulary to refer to the
concepts in a building. But we can also use it as a
way to express the relationship between these concepts. Don't frown upon
this definition. I will go through more details. So this is something
we didn't invent. It started with the
Greek philosophers when they started
to map the world. How a mammal can
relate to a human. How a living being is related
to the planet, and all of that. And of course,
computer scientists like to reuse things
from the past. So computer scientists
reuse ontology as a way to represent
information. It can be also in a building. So in a simplified
definition, an ontology is just a way to
represent domain concepts and the relationship
between these concepts. They are usually
represented in triplets, and we distinguish between
classes and instances. So you can see we have a
city and a country, which are classes, connected by
its "located in," which is a relationship. And then we have the
instances, like Paris is located in France. What is nice about
ontologies is we don't need to invent how do
we represent them, because the World Wide
Web Consortium already proposed standardization-- like RDF, OWL, and JSON-LD-- to represent and express
these kind of relationships. So let's take an
example in a building. We have a location,
which is a concept. We can have subcategories, like
a building floor and a meeting rooms that are also
connected together. And we can also instantiate
them with a room, a floor, and a specific building. So this can be seen
as one of the silos. And we also have another
silo, which is the devices. We have a sensor
that is measuring a quantity, a temperature
sensor measuring a temperature, and then we have the instances. So as you can see,
thanks to the ontology, we can link these
two silos together by adding relationship and
connecting them together. So there's a lot of
ontologies out there. We didn't want to reinvent
the wheel at Google, so we looked at
what is out there. I'm going to focus
on the following just on three of them. And as you can see,
with the cloud tags, there's really a lot, like
SAREF, and KNX, M2M ontology. I'm going to focus
only on three of them. So the Haystack Vocabulary
started in 2014, and it was one of the first
efforts that said, OK, I'm going to start proposing a
vocabulary for my building domain. And this is where we started
to see these tags emerging. So it's a tag-based system
similar to the Twitter hashtag where, for a certain point,
like a temperature sensor 1, you can assign one or
many tags together. So it's a tag-based model. It's not an ontology. And back in 2014, they didn't
rely on W3C consortium, but they had their
own Zinc standard. Another interesting progress
was provided by Siemens in 2015, which points
out in the research paper the limitation of Haystack. The lack of expressivity. The ambiguity in assigning tags. You can imagine that you cannot
express relationship with just tags. It becomes really
very hard to manage. And of course, you
need an abstraction when you do queries. The tags are a very flat system. When you are trying
to query, you might want to query by
location, or by an upper class. So you lose that with the
tags because they are flat. So Siemens proposed the
Haystack Tagging Ontology, which proposes to mix
between the Haystack tags, the semantic
sensor network ontology, and the building
models like BIM and IFC. So basically, what
Siemens tried to do is to take these flat tags
and represent it into triples. But as you can see,
it's not really a lot of explicit
relationship just to add has tag in the triplets. We still lack of expressivity. And this is where Bricks
come into the picture. So it's the Brick
schema proposed by IBM and several
academic universities which took into consideration
Haystack, Haystack Tagging Ontology, and
SAREF, and proposed a model that is, I would say,
richer than just a simple tag model. And it's also relied on
a pragmatic approach. They took around
six buildings that are scattered across
the globe, that they support with different vendors. And they try to refine,
what are the concept or the common concepts of these
ontologies and their proposed Brick. What is nice about Brick is
that it's not just out there. It is part now of
the [INAUDIBLE] body which is trying to push
Brick to become a standard. So this is really
an interesting thing you are into buildings
domain, because you can go and influence this
vocabulary and this ontology. And of course, Brick
schema guided our choice for our digital
building platform because of its
richness, and that it builds on prior work
related to the ontology. So now that I talked about how
do we represent information, I would like to talk
about what are the Bricks that we reused from GCP
to build this platform. So we have a cloud IoT
to connect the devices to the cloud. We have Pub/Sub, which is a
publish-subscribe mechanism. We have Dataflow for batch
and streaming processing, and Bigtable and
Spanner for storage. And then I'm going to show a
little bit a simple chatbot that we built in a very
quick manner to show, once you have the ontology
and these APIs in place, how easy it is to
tap into the system and extract whatever information
you would like to have. So first, I'm going to
talk about IoT Core. It's a fully-managed
service for you to connect your devices
and systems to the cloud. And as you can see on
the architecture diagram, you can combine it with
different other components, like Pub/Sub, Dataflow, so
you can collect, process, and analyze your data. So far, in the Bay Area, we
have around 100 buildings that were onboarded. And we have around
20,000 devices that are connected, along
with 120,000 points that are publishing data to the cloud. So once these devices are
connected and are pushing telemetry data to
the cloud, we rely on Pub/Sub, which is an
enterprise-oriented middleware. It's secure, scalable,
and globally distributed, where you have a topic and
multiple subscriptions that can collect data for you. So far, these devices send
approximately 4 million messages per day,
which is kind of a lot, just to give you an
idea about the scale that Pub/Sub can handle. Once this data
comes into Pub/Sub, you can see that you can
have different mechanism to start playing with the
data, do some batch and stream processing on it. And this is where
we use Dataflow, because it gives you a unified
batch and processing service. It relies on Apache Beam, which
is an open source programming model. So you write your logic once. And then you can
pick whatever running environment you would like. So it can be Spark, or it
can be, as well, Dataflow. The cool thing about
Dataflow, it's fully managed. You don't need to scale it. And so it's hassle-free. And of course,
it's very scalable. It supports millions
of queries per second. So Dataflow is grabbing
the data from Pub/Sub, and then doing the
processing on it and putting it in Bigtable,
which is a fully-managed NoSQL database for you that can handle
up to petabytes of data sets. And it's ideal for
Internet of Things data, like the time series. Currently, we store around
500 gigabytes of data coming from our buildings. And as I mentioned, we have only
100 buildings onboarded today. And then my favorite
is Dialogflow. Dialogflow allows you to build
a chatbot or interactive agent that rely on, really,
the knowledge of Google when it comes to the machine
learning expertise related to NLU. It supports multiple channels. You can do chats. You can do voice. Whatever channel you think
about, you can integrate it. And it's one of the
most widely-used tools to build actions on top
of the Google Assistant. So I will go through
the demo a little bit. Oh, I can't point? So as you can see here, we
have a user which can be one of these three personas-- it can
be the landlord, the occupant, or even the facility
maintenance operator-- that will interact with
the chat application that will head the
facility's bot, which will interact with Dialogflow
to interpret whatever the user input was. And then the facility's
bot will actually handle the call with the APIs. So I created a
little video demo. Of course, this is
just a proof of concept to show what is possible. But you can stretch your mind
to support any type of operation based on the data that
we have in our buildings. SPEAKER 1: I'll put it
in full screen for you. CHARBEL KAED: Yeah. I just wanted to
make it high quality. Thanks. So what you will
see here is the user is just typing queries to count
floors on a certain building. So the beauty of that is that
once we put this in place, we can change the building
and tap to any building that Google is operating. So it can be in
Cambridge, in New York. It will not change how
we deal with this query. And this is thanks to
the ontologies, but also the capacity of Dialogflow
to interpret the query. You can search for
meeting rooms information such as, I want to count how
many meeting rooms do I have, or I want to just see them. I want to see the largest
meeting room, because maybe I'm hosting an event,
and I would like to know, where should I book? I can change sites. And if I'm maintenance
operator, I would like to see it
and understand, where are these HVAC components? Because when you are a
maintenance operator, usually, you are
walking into a building, and all of these components
are hidden for you behind the ceiling. So you don't see any pumps. You don't see any fans. You don't see any of
these HVAC components. So you're like, OK. I need to find them. I need to locate them. And this is where as a
maintenance operator, you can use similar capabilities
with a chatbot to understand, where is the device? Where is this equipment
I need to go replace? Where is it located? So here, we're still
targeting the floor level. But you can imagine,
we can show a map and guide the
maintenance operator to go and look up where he
needs to look to prepare the environment for him. So he can locate a fan. Because he needs to change
this type of equipment, he can group them
by floor because he would like to run some
diagnostic operations on them, and benchmark them by floor,
and how they are behaving. And here, we can see more
queries related to the HVAC. So these queries, and the
keywords that you are seeing-- like the fans, the
pumps, the fan coils-- are all taken from Brick schemas
that we talked about earlier. Because Brick allows to lay the
foundation of this vocabulary. So we took these vocabularies
and put them here And of course, for some
who don't want to type, we have also interactive
menus where these queries can be already populated,
and the operator need just to hit a button
or click on one of them. So that's it for the demo. So of course, this was
just a proof of concept to show what is possible on
top of this building platform. I think I'm stuck on-- OK. Thanks. I think it's OK. There's just one slide. Thanks. So another example is also
to extract time series and integrate them
with the chatbot or maybe integrate them
in another third-party application. So with that said, I will
pass the mic for Marc for the conclusion. MARC PAWLIGER: Thanks. Thank you, [INAUDIBLE]. Well, I hope that gives you a
little bit of an idea of what we've been building with this
platform and what it can do. This really is a work
in progress for us for how we manage our
own facilities at Google. But it really is
meant to show you some of the themes that
have been pervasive, I think, throughout the
conference here, particularly-- we're doing this for
ourselves on our own stack. And it's basically
showing that you can build these rich
applications using a lot of the capabilities
that Google already has. So it's modernizing
an infrastructure that frankly hasn't been
changed in many decades. So you're seeing a
lot of new analytics, and applications, and
platforms at the top level, but we're going very deep into
this stack all the way down to the device. You're seeing that we're using
these building blocks in GCP as basically a way to build
an application platform. So you're utilizing and not
having to reinvent the wheel-- as Charbel mentioned-- for a lot of the
capabilities in building out these applications. And we're also going
from analytics to action. So we're using capabilities
like machine learning. We're using a lot of our data
visualization and so forth to be able to take
that raw data-- which is sort of difficult
to maybe discern messaging and action from-- and do the analysis
to be able to come up with a punch list for
techs or operators to be able to go take care of. Whoop. Went the wrong way. Our onboarding
process really is, I think, leaps and
bounds ahead of what it takes to do onboarding
in typical systems today. And this is where a
lot of the intelligence is being developed right now. As we go into new buildings and
discover new types of devices, new kinds of network topologies,
new kinds of gateway boxes and so forth, that's helping
to make this platform richer so that the next time
we encounter it, it's much quicker to
be able to onboard. It's really building out
the fundamental intelligence of the underlying systems so
that it actually applies down the road as we go along. And we're using
structured data here. So we're trying to use the best
of what the industry is working on right now in
terms of ontologies, in terms of terminologies,
so that an operator will be able to take lessons that
they learned in one building and be able to apply them
into an entire campus, or other buildings as well. So what next? Well, that's where
you guys come in. We'd really like to continue
the conversation, particularly if you have device data,
or interesting ways of representing data. So we touched mostly on
things like climate control, HVAC today. But I mentioned that
there's a lot of work that we're also doing
for things like occupancy sensors, and alarms,
and physical security, and so forth. So if those are
the kind of devices that you work with,
we'd like to know, how would that integrate
with this platform? What would you be
able to do if you didn't have to do all
of that plumbing down at the bottom of the stack,
but were able to just apply whatever it is that you do best
at the top for applications? On device security
and maintenance-- I touched on this a little
bit at the beginning, but we have been talking in the
open source community about how to do continuous integrated
testing of your devices to make sure that they're only
communicating with the devices that they need to in the way
that they're prescribed to. And that's something that we'll
be talking a little bit about more publicly. But essentially,
this is something that we want to do
at Google as well to be able to ensure that our
network remains secure as well. And then, again, the third party
apps and other integration. Partner involvement
and so forth. Remaining involved with
the various standards, bodies, like ASHRAE, Brick,
Haystack, and so forth. And if you have
involvement there as well, we'd certainly like
to hear whether or not what you're doing integrates
well with what we're building with this stack, so that
we can make sure that when we encounter your devices
out in the field-- hopefully we will--
that we would be able to integrate
with them as well. [MUSIC PLAYING]
