CRAIG LABENZ: Hello, everyone. Welcome to a very
exciting episode of "Observable Flutter." My name is Craig Labenz. I'm your host, as
always-- so far, at least. Maybe one day someone
will sit in the hot seat and host an episode. But for now, I am very
excited to bring you Erick Zanardo, the TL
of the I/O FLIP project at Very Good Ventures. We've got a lot to
talk about today. It's going to be a
little more interview style than past episodes. But I'm really looking forward
to hearing all the juicy, behind the scenes stories,
anecdotes, all the things. So last note before we
dive into this too much. Remember, everybody, this
is the Flutter community, and on "Observable
Flutter," we're all here to discuss technology,
learn from each other, whatnot, but we're not putting
down other technologies that we don't personally
use, whether that's other frameworks, other
libraries, other state management solutions. OK. I don't think we're going
to have to worry about too much of that today though. Today it's just very fun,
talk about I/O FLIP day. All right. So I'm going to bring
in Erick, and let me make sure I'm on
the right setting here. Here we go. Erick, would you like to
say a few words to you introduce yourself? ERICK ZANARDO: Hello, folks. Yeah. Very, very nice to be here. I'm Erick. I have worked with a great
team of other engineers on the I/O FLIP team,
on the I/O FLIP game. Yeah. Very, very happy to be able
to share some experience here. Thank you, all. CRAIG LABENZ: Nice. So there was-- jeez,
where to even start? I think one thing we
were brainstorming before the episode was that we
might begin by just playing one round and talking about the-- talking about the rules and
whatnot just to make sure-- I doubt anyone has
found their way here without having seen the game
yesterday, but who knows? So, Erick, you're going to
do the honors of walking us through one round of the game. ERICK ZANARDO: Awesome. Yeah. So when you landed at
the page of the game, you first see the leaderboards. You can also check
the rules because we have some elements in the cards
that interact with each other. So like fire beats air and
metal and stuff like that. Air beats other
different elements and so on and so forth. Once you learn the
game, you can just hit Play, accept the
terms, and then you first-- you are greeted by this
beautiful owl, which is the card master of the game. So the lore of the
game is this owl is who creates the card and
all the beautiful images that you see in the game. Then you have to just
select a couple of terms. So first you select the
class of your character. I'm going to pick pirates
because pirates are cool. Dancing pirate seems
like a good choice. I mean-- CRAIG LABENZ: What is this,
"Pirates of Penzance"? ERICK ZANARDO: And then
you'll get a 12 pack of cards to choose from. So you need to-- you
can cycle through them, choose the ones that
you like the most. I'm going to try my chance
here and pick the ones that I find the stronger ones. So we have this Dancy Spark. This Dancy Android
seems good too. 86. 83. Oh, this is better. So I'm going to
exchange that one. CRAIG LABENZ: That's
a big upgrade. ERICK ZANARDO: Oh,
this is even better. CRAIG LABENZ: Oh, a foil. Let's go. ERICK ZANARDO: That was lucky. Yes. Cool. So I'm happy with team. That's a great team. And then I join a match. It will take a couple seconds. Yeah. We are joining the game. So players play at the same
time, so I'm going to choose. I'm going to start
with Dancy Spark. My opening's going
to choose their card. CRAIG LABENZ: Opening with
your weak cards, I like it. ERICK ZANARDO: Yeah. So like I mentioned, elements
interact with each other. So water is weak to metal,
so my card lost 10 points and eventually lost
to the other one. I'm going to go-- CRAIG LABENZ: Pressure's on. ERICK ZANARDO: Yeah,
going to put some pressure and play my stronger one. Yeah. I think it is. So Spark is earth versus fire. I won this hand so it's tied. And my last card. CRAIG LABENZ: Oh, let's go! ERICK ZANARDO: I won! CRAIG LABENZ: Also,
you have water there. Fire had a rough
go in that game. ERICK ZANARDO: Yeah. Yeah. I'm glad because
I didn't have any. And then I can choose
to go to the next match or submit my score. Yeah. That's it. That's a little flip. CRAIG LABENZ: Got a
streak of one going here. So Varie asks-- she said,
I've never seen a holo. I guess it's holo card, right? Holo. ERICK ZANARDO:
Yeah, it's a holo. That's how you call it. CRAIG LABENZ: And
how rare are they? It was originally going
to be 1%, but, Erick, didn't get changed
at the last second? ERICK ZANARDO: Yeah. So every time a
card is generated-- so this happened
12 times per deck-- you have 5% of a
card being a holo. CRAIG LABENZ: So across a whole
deck a pretty good chance, but there is-- it definitely went up a lot. I've seen people get
three holo cards, and then they just play nothing
but hundreds the whole game. ERICK ZANARDO: Yeah. Yeah. I got myself in that
situation, but the opponent got three holos so
that was not that cool. But, yeah, you can
get three holos. CRAIG LABENZ: I'm playing
with my audio here. Oh, no. There. That's in the right setting. Oh, I see. OK. Sorry. Missed a distraction there. OK, so yeah. That's one game. Your streak is up to one. All right. Let's talk a bit about
everything that went into this. You mentioned-- and I heard
a little bit this from Jay as well. And for those who
don't know Jay Chang, he's a marketing wizard,
honestly, on the Flutter team. He was the original brainchild
behind the pinball game that was also built by Very
Good Ventures and a lot of-- just a lot of the cool
stuff that Flutter does. But Jay and folks at
Very Good Ventures, you were all iterating
and trying to figure out, what are we going
to build this year? Take me through some of the
early ideas and different trade offs that you were considering,
and how you ended up actually settling on I/O FLIP. ERICK ZANARDO: Yeah, for sure. So the first couple of
weeks of the project was very cool because we were
just brainstorming ideas, trying to think what could be a
good fit for these years even. Because the only
thing that we knew was that we wanted to use
AI and machine learning, generative things in the game. That was our only requirement
to develop the game. Everything else were open, and
we could try to think on ideas and propose the
game around that. So as you can imagine,
something so broad like that, many
ideas came into play, so we have many sessions of
brainstorming and discussing. We had awesome ideas. One that I really liked was
to build that result of game where you need to find out or
in a crowd and different things. So we thought that
maybe, hey, we could generate these backgrounds
with generative machine learning algorithms. It could be just the
one that we need to find and things like that. We also thought maybe we could
use generative machine learning language models to
create an adventure. So imagine how cool would
be that we could have kind of an RPG game where the
quests are all generated by language models
and the backgrounds, all the places that you travel-- CRAIG LABENZ: The scenes. Yeah. Yeah. ERICK ZANARDO: Yeah. Yeah. Would be generated by
image-generation algorithms and things like that. The card game was also
present since the first ideas from the first iterations
of our brainstorming. Because a card game-- usually, card games
have a picture. They have a theme, and they
also have descriptions, names, and things like that. So we also found it to be a
good fit for this requirement that we had. So we entered throughout,
then we made some prototypes, some POCs. We built a small
result where there's a lot of dashers running
through the screen. It was fun to see that. We started even without
any machine learning, just some procedural
kind of arguments just to validate the idea. So we went back and forth. We have many of the
ideas, and in the end, we went up to choose the
card game because in a way it's a simpler mechanic, right? In an I/O game, usually, we
try to make the game simple so people can quickly get
it, play, repeat it so they-- this was also one
thing that we tried for a lot, which would be
a repeatable experience. And also you can
play one quick match, but that's so cool that
they want to play another, but they never last that long. And that was something
difficult with, for example, the other ideas,
whereas Dash or as other games you can spot what
you are looking for in the first second--
in the first minute or you can spend, like, 10,
20 minutes trying to find it. Or on an adventure like an RPG
game, they are telling stories and don't want to
rush through a story. So those would also
take too much time, and it would not fit that well
into what we were trying for. And so the card game
fit very well on-- we would be able to use machine
learning to generate the cards, to generate the
illustrations of the cards, and it would also
be something that could be designed to be
a quick experience that could be repeatable. So it's kind of key
to show that idea. In the beginning it wasn't
really what we have right now. We run through different
iterations of card games. So the base idea was a game that
I used to play on my childhood a lot that is
called "Supertrump" or "Toptrump" in
English, I guess. Which is basically--
yeah, nobody-- it was fun because I didn't knew
the game in English because I'm not a native English speaker. So I come from
Brazil and the game has a different name
there, so it was hard to explain to the team. And then we discovered
that in the US the game is called "Toptrump." And the game is about
you have a thing-- for example, cars,
then you have a deck of cards that represent cars. Each card has a
set of attributes like horsepower, top
speed, acceleration, wave, and things like that. And each turn a player chooses
one of their attributes. And whoever has the bigger
attribute or the highest attribute wins that round. So that was our first-- the basis for idea,
but in the end we iterate over and over and
over and landed on the mechanic that I/O FLIP is now. CRAIG LABENZ: Nice. Wow, there's a lot there. ERICK ZANARDO: Yeah. CRAIG LABENZ: One
thought, by the way. When your hands
hit the desk it's sending a bit of a thudding
noise through your microphone. ERICK ZANARDO: Oh, OK. CRAIG LABENZ: I'm a hands-- yeah, I'm a hands
talker as well. But OK, so you
settled on this game. First of all, the RPG next
year sounds really fun. I have no idea if that'll be
the right thing or whatnot, but I can imagine if the Flutter
scene situation is ready to go, could do some pretty
fun things there. So you settled on
this card game, and Very Good Ventures
is an either Flutter first or exclusively
Flutter development shop? ERICK ZANARDO: Yeah, so
definitely Flutter first. CRAIG LABENZ: OK. ERICK ZANARDO: I
don't really know if you have any other
projects than Flutter. I don't think we do, but yeah. That is definitely
our focus right now. CRAIG LABENZ: Got it. So Flutter was the
obvious choice, then, for the front end. But tell me about what-- how did you make
all the decisions? What did the rest of the
tech stack look like? And how did you settle
on those things? What constraints did you
anticipate dealing with? Walk me through the
non-Flutter parts as well. ERICK ZANARDO: Yeah, for sure. So, yeah, Flutter was
a given because this was a Flutter showcase. So this was the
choice for the client. We also knew that we
wanted to use Firebase because as always we also
want to showcase Firebase use on those projects. So we also knew that we would
want Firebase for, I mean, whatever we could use from Fire. Firebase has a bunch
of different features. So one that we were for
sure that we would use would be Firestore to hold
the data, to secure this data. All the data would be
managed by Firestore. And we also knew that we would
need a backend for this project because this is a
game where players play against each other. So we need to do validation. So we couldn't leave all the
gaming logic in the client because if we did,
people could try to take a look on
the servers, try to find ways to exploit
the game and take advantage of their opponent. So we knew that this would
need to be a server alternative game, right? So all the logic would be
defined by the backend. So who won-- who wins a
round, who wins the match, it would be the backend
who would know that. So once we understood
that, we start to think, what are we going to
use for the backend? Our first idea was to
use Firebase functions. This was something that we prove
in the past that works great. It's easy to use. But this game, unlike
the other project that we did for
this kind of thing, would have way more need
for it to be a bigger backend than the other. So we wouldn't need a bunch
of Firebase functions. And since at VGV we are very
focused on Flutter, on that, we don't have that much
experience with TypeScript, for example, which is the
language that is usually-- that we see more people use it
on Firebase functions, right? So we were kind of
worried a little bit about going through
building this whole backend with a language that
we are not so used to. So I started considering
to use a Dart backend. On top of this thing
that I mentioned, that team was not that
experienced with TypeScript and using Dart could
give us an edge, we would also be
able to share code between the server
and the client. So when we realized that,
I said, OK, so that is, I think, the right
choice for this project. So let's make the backend
of this project using Dart. And since inside VGV we have
been building Dart Frog, we went with that because we
have more experience with it. We have people that can
support us if we need any help. So, yeah, that's what led us
to go to the Dart backend. CRAIG LABENZ: So you
had a Dart backend, and I think one thing a lot of
Flutter developers, especially those who use
Firebase, long for is, yeah, Dart to be supported in
Firebase functions and whatnot. And you had-- it sounds like
you achieved a similar setup by using Dart in Dart
Frog, but you didn't-- you weren't using
Firebase functions. Of course, it's funny, I
know you deployed Dart Frog to Cloud Run, and
Firebase functions runs on Cloud Run
under the hood. So the difference between what
you did and Firebase functions is not-- there's only a few
differences that remain. But one of those is
the subscriptions. Alert me whenever a
document changes in a given collection or stuff like that. So how did you handle talking
to Firestore from Dart given those limitations. ERICK ZANARDO: Yeah,
that's a great question. So one thing that
we decided to do is that all the sensitive data
operations involving rights and updating and deletions
would be done by the backend, but the client would have full
read access to the entities. So that gives us the
flexibility, for example, to create an endpoint
on our backend that is responsible for when
the player plays a card. So this is a sensitive
operation because that's exactly where players may
try to exploit the game and try to take advantage. So that operation-- so the match
and the card, the match state, cannot be written or
updated from the client. They need to go
through the backend where we have set all the
validations, all the security things regarding the game and
education and all that stuff. But the list should match
stage updates from the client so they don't need
to go to the backend to listen to those chains. So they read access to
those entities in Firestore. They are available
in the client so we can take advantage of the
real time capabilities of the Firestore database. CRAIG LABENZ: Interesting. OK. So what I'm hearing is the
flow would be the client-- when a user plays
a card, the client would send that information
not directly to Firestore but to Dart Frog, and Dart
Frog would verify things like, yes, this card was
actually in your hand, you haven't already play
this card, whatever else. You are who you say
you are, even though it was anonymous authentication. And then that Dart code-- you probably must just
use the raw gRPC stuff to write to Cloud Firestore,
and then the client had a live subscription. So how did you handle-- maybe you just had
two collections, like two documents
for each game? But I'm thinking another
possible way to cheat-- let's say player one
plays their card first, and what you want
to have happen is player two sees that
a card was played, but they don't
know which one yet. Did you actually
send the right-- did you send which card
information to the other client but then simply not render it? Or did you even send more
primitive information like, yes, they've
played their card, but we're not going to
tell you which one yet? ERICK ZANARDO: Yeah, we send
their whole information. But we send them
encrypted and only the ID. So this is interesting
because when we-- that's why we have
two different-- actually, not only two, have
much points in the database. So when the player tries
to get the full state of the match which contains all
the cards of their opponent, all the cards of their own hand,
we send that whole information to the client, but
it's encrypted. And then when a card is
played by their opponent or even by yourself,
that listener is triggered by
Firestore, but you only get the ID of the
card that was played. So that ID is not
encrypted so you can see what ID it is,
but it's just a number-- or actually a
string, hashstring-- CRAIG LABENZ: A
big, long string? ERICK ZANARDO: Yeah. Yeah. So it will not be able to
understand what this is. CRAIG LABENZ: Got it. ERICK ZANARDO: That's how we-- CRAIG LABENZ: So you
use Firestore slightly like a relational database, I'm
guessing, where that was just a pointer to the card, then it
lived in another collection. ERICK ZANARDO: Yeah. In a way, there is a
relation happening there because we send IDs, and yeah. But that's how we manage
to get around those things. CRAIG LABENZ: Yeah,
and that makes sense. The Cloud Firestore,
of course, lends itself to not having fully
normalized data, to heavily denormalizing
data, but occasionally you will have little bits of
normalization on that. OK. So the next thing I was
wondering about your backend is the statefulness of
the 10-second countdown. If you have one thing
that a rest client-- or sorry, a rest server
is not very good at is statefulness because it's
intentionally stateless, and something like a countdown. Once the game starts,
how did you guys-- how did you have the server
enforce the fact that after 10 seconds-- you know, what if
both clients cheated, and they blocked any
communication at 10 seconds? Did the server have
something up its sleeve to force the end of the hand? ERICK ZANARDO: Yeah. So the backend does nothing
towards the counter. This is something that
the client triggers to us. But if they want to
kind of miss their-- trick the backend into
thinking that they did not send a match, the backend, we
will then send in an error, and the match will not go on. So if a player tries to
cheat, their match will be-- it will be left in an
inconsistent state. And so basically they
can break the match, but they will gain
nothing from that. No. So if they try to
do that in a way, there will be no way for
them to gain from that so they will lose their streak. So yeah. CRAIG LABENZ: OK. All right. So cheating would
only hurt yourself. ERICK ZANARDO: Yeah, basically. And we also implement a couple
of measures in the client that, for example, if I
don't get a play, I try to-- I call them back and
say, hey, is there a result that I missed? Maybe our information has been
missed or something like that. So the client does send
a couple of requests to the backend to try to-- to verify if their client is in
an invalid, inconsistent state to try to fix that. CRAIG LABENZ: We just
got one question. I think I know the
answer to this, but I'm going to let
you field it, of course. Did you use Flame? And if not, what were the
factors in that decision? ERICK ZANARDO: Yeah. Yeah. So, yes, we did use Flame, but
it was only for the animations, for some of the animations. So I don't know if
you remember when I showed the game, when a metal
card faced against a ground card? Earth card? I don't remember the
relation of all the elements. There are so many relations. CRAIG LABENZ: I
think it was water. ERICK ZANARDO: Yeah. So the animation
of-- or the water one versus the fire where
the water eats the card, like with a splash
and that sort. The animation is done via
spreadsheet animation, and for that we use Flame. For everything else we
use just plain Flutter, and the decision behind
that was that this is a card game so the game is
kind of a turn based, right? So the turn happens
at the same time, but until a player
choose a card, there is nothing
happening in the screen. So we are just waiting for
the players to play something. So there was not
really a necessity for an active game loop
engine like Flame is, right? So we figured out that there
wasn't much of a reason to use Flame other than this,
for this for the animation ones. Which we could also just
implement a special animation just using Flutter. But since Flame already
have that, so why would we [INAUDIBLE]? So we just use the package. CRAIG LABENZ: Did you turn
off Flame's game loop, or did you let it just
spin and do nothing? ERICK ZANARDO:
Yeah, so in Flame-- Flame is just a widget, right? So when we're building a Flame
game, you have the Flame game, and you have the
Flame gaming widget. The game loop will just
run as long as that game is attached to the gaming widget. So the game loop is running when
animations are in the screen. Once they are exposed,
the games pause together, and Flame, out of the box,
provides a sprite animation widget, which is that sprite
animation that you handle in a Flame game but in
a widget, which is just a convenient helper for you to
use directly in a Flutter app if you want to show especially
animation that is not inside a game but you want
to show in your widget tree. CRAIG LABENZ: Gotcha. We got another question
from a Flame expert. How is the foil effect
applied to the cards? This could be a
leading question. Was it done through
a custom painter? ERICK ZANARDO: Yeah, so I can
show the code for that if you want to, but it's-- CRAIG LABENZ: Sure. Yeah, yeah. ERICK ZANARDO: Yeah. Let me bring my editor here. CRAIG LABENZ: I'm ready for it
when you-- all right, there we are. ERICK ZANARDO: There we are. So before I go into that,
I didn't work very deeply on this, but I can
give a broad idea of how we implemented that. So let me bring the card widget. It is this one, but I want to
show you the file shaded one. So basically this is the
widget that applies the shader to our card widget. We just have some
basic attributes here, the path for the shading itself. And this is done by using
everything that Flutter provides to us out of the box. So we have this
shader builder, which we need to return
a sampler, and then we just used the
animated sampler passing all the
required attributes that the shader requires. And we can see those
attributes if I show you the shader, which I
will not try to explain because that's too complicated. And I don't really
know how shaders really work under the hood. I know how to put
that in a Flutter app. Where is the-- CRAIG LABENZ: Yeah, I was
just actually producing some shader resources earlier
this week that'll come out later this month. They have a steep learning
curve, let me tell you what. ERICK ZANARDO: Yeah. Yeah. They are not that simple. You see, I don't
even have syntax highlighting for shaders. I should have. That's my fault. I mean,
we can try to understand it a little bit. So you see we have this
rainbow effect function method, and this does a bunch of
image processing code. And, yeah, by just
taking this file that's written in shader language
and put it with those-- CRAIG LABENZ:
Wolfenrain says, shaders are easy once you sacrifice
your soul to the shader god. ERICK ZANARDO: Yes, exactly, and
I'm not really ready for that yet. So yeah. That's super simple to put it. Yeah? CRAIG LABENZ: There's
one thing in this file that I do want to just spend
a second talking about. A lot of the heavy
lifting is done by-- there's two widgets here
that, if you haven't seen, are going to be a little
confusing and hard to parse. So the shader builder, that
one's a little easier to maybe imagine. That compiles the
shader and just makes it available to your app. ERICK ZANARDO: Exactly. CRAIG LABENZ: But that animated
sampler is kind of a crazy one. So what the animated
sampler does is first renders something
in your Flutter app. And then it applies a shader to
what that original widget would have rendered. And we can see the animated
sampler takes two parameters. The first one is a function,
and it goes from lines 4 to 15. And so that's-- this could have
been called an animated sampler builder, and that would have
been the builder method. But it's just a
positional argument. You can absolutely think
of it as a builder. And then on line 16, the very
bottom line in the file right now, we can see a
child is passed in. That's the widget that
the animated sampler takes, renders, and then
applies the shader to. So that's how a card
could get passed in here and suddenly have the foil
effect drawn on top of it. And it is honestly all a lot. I happen-- we just released--
the Flutter team released a bunch of talks yesterday
for Google I/O. Mine happened to be about exactly this stuff,
which is the only reason I even know what I'm talking
about right now as I look at this file. I had to learn it to
put together that talk. But, anyway, yeah, if you're
interested in how the foil effect would be done
in shaders in general, you can check that out. It's the next Gen UIs one. Anyway. Yeah, animated sampler-- and
the code inside animated sampler is bananas. Some of the least intelligible-- I mean, it is wild. It is really, really wild stuff. But that's why the
Flutter team made it because the engineer that
wrote that widget was like, he's so deep in the rendering
pipeline and all the things. And he just said
to himself one day, no one will ever figure this
out if we don't offer it, so just stapled together these
kind of primitive helpers. All right, anyway. So shaders. That's how that was all done. OK, let's see here. So we know that you've got
Flutter on the front end, obviously. You've got Dart
Frog on the backend, but Dart Frog's only
half the backend. Firebase is the other half. You're using Cloud Firestore,
and presumably maybe a couple of other
products as well. Probably not any functions. Maybe, though? Yeah, what-- ERICK ZANARDO:
Yeah, we eventually used Firebase functions in the
end to make the scoreboard. So in order to also avoid the
cheating and all that stuff, we made that the scoreboards are
set by a Firebase function that is triggered for on write that
happens on another entity that is also managed by the backend. But we end up using Firebase
functions on this small piece of the whole thing of the game. CRAIG LABENZ: Yeah. I was just going to answer
a question in the chat. Someone said Jonah is a
wiz and I'm going to-- I have to type
here, yes, Jonah is. That's to-- and the
theme is SynthWave '84. But this is not my theme
currently on screen. So what theme are we
looking at right now, Erick? ERICK ZANARDO: That's it. That's exactly what that is. It's SynthWave '84. But this is not VSCode. This is VIM. So-- CRAIG LABENZ: Yeah. What's the name of
this color scheme? ERICK ZANARDO: Yeah,
it's SynthWave '84. It's that. CRAIG LABENZ: Oh, this
is SynthWave as well? ERICK ZANARDO: Yeah. But for VIM. CRAIG LABENZ: Got it. OK. All right. Well, yeah, there's that purple. OK. I feel like I get more
pink in mine, but yeah. I guess it's just a
slightly different take. ERICK ZANARDO: Yeah. CRAIG LABENZ: Cool. SynthWave developers unite. All right. So let's see here. I'm trying to think
about other things. Now, I heard you also used the-- you also started with the-- ERICK ZANARDO: Oh, my god. CRAIG LABENZ: --Casual
Games toolkit. Oh, we got a runaway AirPod. ERICK ZANARDO: Just a minute. My AirPods just fell off my ear. Sorry about that. CRAIG LABENZ: Yeah,
they love to do that. They don't fit in
my ears either. I don't know-- ERICK ZANARDO: Yeah, that's a-- CRAIG LABENZ: I don't have the
blessed ear shape, I guess. The Apple AirPods,
I put them in. They're just like, see ya,
and they jump right out. So I know-- well, I
gather that you all used the Casual Games Toolkit. And, yeah, what was
that experience like? ERICK ZANARDO:
Yeah, it was great. So it's very easy
to get it set up. You just need to
get a team player to get that-- who got
that from the website. Everything is super
well explained. The only unfortunate thing was
that the toolkit's very, very-- the idea of it to help
a lot of the things that mobile
developers will need. So it helps a lot if you want
to have ads and integrate with the Play Store game
services or the one from webs. I don't remember
the name as well. So since this is a web
game, unfortunately, we were not able to take
advantage of those features. But it gave us a
really good edge on the audio setup because
audio setup on web is-- especially on web when
running on mobile phones, it can be a little bit
tricky, a little bit buggy. And the Casual Games
Toolkit already have all of that figured out. We have an audio services class
here that I can even show you since I still have my code on. Here you go. We have this great
audio controller class that handle background music,
handle sound effects ones, to handles-- or it integrates with assets
controller that also came with the Casual Games Toolkit. So all this set
up to make games-- play a music, play
sound effects, and give the ability to the user
to mute and to disable sounds. All of that was figured
out by the toolkit. So that was a great,
great headstart that the toolkit gave, which
is what the toolkit's intended for, right? To not build your game
but to give you a head start and don't need to worry
about those setup things that all games
will need to have. CRAIG LABENZ: Yeah. So, yeah. Nice, by the way, that
you all finally got to take advantage of a headstart
block of code from someone else since you have a very
good start package or template that you offer
for everyone else to use. So let's see. Gosh, there's so much more
that went into this game. So the AI integrations,
one thing-- because a lot of this was done
on very, very, very early AI products that aren't
fully released yet and aren't fully safety tested
yet, an early decision was-- well, first of all,
the earliest vision was quite a bit bolder than
the game ended up being. For those who don't know,
originally the original idea was that users would
kind of type in free form what they wanted
their character to be, so you could really get
imaginative and type different just string answers
for powers, and then images would be
generated in real time. That was the goal. That ended up being
a bit of a challenge, both because the models were
so new and weren't fully safety tested, and so it was moved to
having all of the AI-generated art and character descriptions
that were written by Bard-- well, the PaLM API-- that was all pregenerated. So still done by an
AI, but ahead of time. And then we got the dropdown. So it's like we're
going to pregenerate for these specific things
because you can't pregenerate for the entire universe
of possibilities someone can type in. So what was it like
for the team as you were moving around this-- as the goalposts,
basically, were just being moved on what the
AI story was going to be? Because I know that was
in flux for pretty much up until the final word,
until the final launch. So what was that
like on your end? ERICK ZANARDO: Yeah, so this
is an interesting story for us because at the beginning
it was like you mentioned. So they would be
generated in real time, so we would be given a
service that, given a prompt, we would get an image. So that's quite a simple
integration from an application perspective, right? We just need to call
an API, get an image, probably save that image
somewhere so we could-- we don't need to keep calling
that to get that same image. So that was kind of given to us. On the beginning of the project,
we just mock the service. We create a super
simple endpoint that would return a handle
on the image just to try to simulate what
we'd get in real life, and we move on with the
development of the game. We got very far just by
using those mock surfaces. Then we decided to go through
that generative stuff. And then we had to start to
brainstorm on how to use that. So after a couple of
talks we just realized that, OK, so we're
going to give you some images that will follow
a pattern on their file name. So, basically,
the file names are named character,
underscore, character class, underscore, their location,
if I remember correctly. I think that's the pattern. Then a variation because
you have multiple variations per parameter combination
to give the players the-- look how many images we got from
these pregenerated [INAUDIBLE].. But there's so much
randomness that the idea was that the player would
not even realize that. Once we figured out
that that pattern that we got through
the agreement of how the cards would be made,
it was a simple change because we just need to get
the prompts that the client side would send to the
backend, assemble the URL, and then we would just
fetch on the cloud storage. And we thought,
OK, that's simple. We are going to go. Then we started to
generate the images. And we started to realize that
we had thousands of images. If I remember correctly, we
had 10,000 images or something like that. And then we start-- CRAIG LABENZ: A lot. ERICK ZANARDO: Yeah. A lot. And then we started to get, OK,
we need to put that somewhere-- to host that
somewhere for the app to get or be able
to access that. So this is start
us to think better on another part of the project
that is not even the client side, not even the backend,
but all the things that we did to make the generative stuff,
the pregenerative stuff available to the game. So we made a lot of scripts
to try to upload the images, to upload the
descriptions in Firestore, and we had to start
to find smarter ways because the first script that
we did to upload the images, I realized that would take, I
don't know, a couple of years to upload everything
because there are so much-- so much data to upload. Then we started to
do some research and figure out better
ways to do the upload. We did many scripts that
total normalized the images because sometimes the images
didn't come with the standard that we had agreed
upon with in the start. So we had to make some
script to make sure that the images were correct. Then eventually we
started to see images, cards generated without images. This is not possible because
all the images follow a pattern. They should be-- but
there is network-- there are network errors,
there are human errors that we don't account for. So I actually realized that
we had many missing images, many missing descriptions. CRAIG LABENZ: Descriptions. ERICK ZANARDO: Yeah. So then we went and make more
scripts to validate that. So a script that would run
through our database of prompts and check if there
were images there. And then that was still not
enough because we started to get, hey, we can get-- some images we can
get better variations. So for some prompts
combinations, we have 10, 12, 20 images. But for some others we have
just eight or just four. So that makes our
code a little bit more complex because how can
the code know-- how can our application knows
which combinations has 20 or 30 or just 10? So we figure out a way to
create a kind of lookup tables in Firestore that would
storage how many variations per prompt combinations we had. Yeah. So in the end, the
solution is quite simple. You know? We just have a script that
runs through a folder, creates those lookup tables,
upload them to Firestore. Than the backend simply gets the
prompt, do a carry on Firestore to get those lookup tables,
and return the image. But the process to get
to this complete solution was an interesting one to
go because we were always learning. Because the volume of the
data was incredibly big. I mentioned that we had 10,000
images, which seems a lot, but for description
we had 420,000. So yeah. It took me one hour, then
I have to upload, then hour to production. CRAIG LABENZ: Yeah. I bet. There was a-- I know the folks on Google's
end who were manually looking at all of those
images and confirming that none of them had anything
weird or just inappropriate, those people's brains
turned to mashed potatoes by the end of the process. Yeah. They looked at so many
pictures of Dash and Sparky standing in front of a
Disney castle, basically. It's just like, ah! OK, we got some good
questions in the chat. Randal asks, in
the spirit of this all being an
AI-themed I/O, did you use any code assists
while writing this game? Copilot, Bard, ChatGPT? ERICK ZANARDO: Yeah. I use Copilot
because I have access to it because of the open
source program of Copilot. So yes, my answer would
be, yes, I used Copilot. CRAIG LABENZ: Nice. Yeah, AI to write AI code. Hmm. OK, this is a million
dollar question. Melanie asks, how long does
it take to develop this game? And we could spend
the rest of the time, I think, talking about
the answer to this. I'll rephrase this first. I think this question
could take a lot of forms. First I'll say, how
long did you have? ERICK ZANARDO: Yeah. So the amount of months, we
had three months for everything from ideation, to the first
prototype, to the final thing. But we had many
phases, you know? So we changed the--
like I mentioned, we went through many ideas. We tried many
different approaches. Even when we was sure that
this would be a card game, we still changed the
mechanics a little bit. So I would say that for the
final thing, once we were sure what we would
going to do, I would say one and a half
months, more or less. Maybe a little bit more. Yeah. Something like that. CRAIG LABENZ: Oh, my gosh. Oh. 10 minutes if you fork it. ERICK ZANARDO: Yeah. That's true. CRAIG LABENZ: That's funny. Although that's a
slow fork, got to say. The fork button in GitHub
is faster than that. So, OK, six weeks, I'm hearing,
is how long you actually had. How long would you
have liked to have had? ERICK ZANARDO: That's
not a fair question because now we know everything. Not know everything. That's a bold thing to say. But we learned a lot
during this process, so right now we could make
many things different. We could improve many
things, like I mentioned, all those scripts stuff
that went one week to figure out what
would be the best way. So right now, if I
would do this again, we would probably do
this the right way right from the beginning. So I would say that three
month is a good time frame to build something this size. Of course, everything
that I'm saying is not taking into
account all the work that Google did about the
machine learning stuff, about the models
and the depth area. I would not dare to give a-- CRAIG LABENZ: Yeah,
that's true, because folks on the ML side on
Google's end were working on that pretty much
nonstop, trying to meet you in the middle,
essentially, to figure out how that was going to work. ERICK ZANARDO: Yeah. CRAIG LABENZ: Yeah, that's true. OK, so you had six weeks. And I think one of the
trickiest things at the end was the matchmaking,
and I remember thinking matchmaking
would require, itself, weeks of load
testing and probably would require its own team,
honestly, to completely-- to get matchmaking
totally right. And with so little
ramp-up period, it's like no one knows about it. No one knows about the game. No one knows. No one knows. No one knows. No one knows. Oh, everyone knows and
everyone's playing. And so just figuring
out, did we engineer a system that can go from zero
to light speed in seconds is-- I think it is just
genuinely not possible. So I know matchmaking was one
of the biggest headaches down the stretch, and
I think, honestly, that probably would have needed
its own six weeks, minimum, period to totally get right. But, yeah, that was-- gosh, there was a lot of
complexity in this thing. Just to flip over a
couple of cards and see, is my integer higher
than your integer? Whew. ERICK ZANARDO: Yeah. CRAIG LABENZ: A
lot went into it. ERICK ZANARDO: Yeah. I mean, how much player games-- [COUGHS] sorry. How much player games have
a complex of their own because when you think that-- when I analyze things
the way you mention, it's just two
integers to compare. Yeah, but those integers
will come in hand on times of periods of time. So the timeline of the
things are the tricky part because we need to take
account about inconsistencies. One interesting
bug that we had was that it seemed that
your opponent played the card before you, but
then it showed after, and then the game state
was all messed up. That was happening because
something that we didn't even know could happen. Like, a state that was
triggered by Firestore first arrived after
in the client. So the client would be
receiving two triggers but in the wrong order. So that completely messed
up with the game logic, and we had-- CRAIG LABENZ: How
did you fix that? How do you account for that? ERICK ZANARDO: It was actually
a quick, simple solution. The tricky part was to
realize that was the bug. So once we realized
that was the issue, it was simply a
matter of, OK, let me check if this
state is outdated. If it is, I just discard it. Don't even consider
for the game, because the states are
always triggered correct. So I wrote this state first,
then I wrote the second one, so this second one is correct. It contains all the data
from the previous one. CRAIG LABENZ: I see, ERICK ZANARDO: But they
arrived in a different order. So what was happening was
that the client was being-- the correct state would be
overridden by the wrong one. CRAIG LABENZ: Yeah, yeah, yeah. Yeah. ERICK ZANARDO: So it just had
to discard the incorrect one. But the trick part was to
realize that was the issue. CRAIG LABENZ: Right. Yeah. It's a-- oh, man. If your game state
is terse enough that you're able
to send the whole-- the whole universe goes out
with every communication, then, yeah, that certainly
makes that a lot easier. But the trick with sending
the whole universe out with every communication,
it sounds like, OK, well, that's easy. But I imagine you may
have also had things where if the server didn't get-- if the server screwed
up in any way, like you said, just like a write
fails or something goes wrong, there's no way to reconstruct
what should have happened. You're just lost at that
point, I'm guessing. ERICK ZANARDO: Yeah, and
that actually happened. So there was one edge case
where the game would be over, but that would be no
result on the document that the client received it. So we did some things to
resolve that which was basically the client-- when the client gets the
game state, it checks. Because when-- remember
that I mentioned that one factor to
use a Dart backend would be to share code between
the client and the server? So that helped us-- that gave
us the ability on the client to do some verifications. So the client knows
when a game is over, but it cannot calculate
the result of a match. So when the client realized,
OK, the match is over but I don't have a result,
I just call backend. I say, hey, can you
give me a result? And then the state
would be rebuilt and everything would work. CRAIG LABENZ: OK. OK. Yeah, it's funny. That's like the allure of
nonserver authoritative logic, right? You know? It'd be so tempting to be
like, well, the client can't figure out a result. Really? It can't decide if it has more
points than the other player? It's like, yeah,
well, if it decides then someone's going to cheat. ERICK ZANARDO: Yeah. And super easy. You know? It's not that
complicated to cheat. CRAIG LABENZ: Right. Right. OK, let's see. I want to look at some more
questions from the chat. Someone a lot earlier
and then again just now-- oh, here's the
one from earlier-- asked, yeah, where is
this code, by the way? What's the repository? ERICK ZANARDO: It's Flutter
slash I/O underscore FLIP on the Flutter organization. CRAIG LABENZ: Oh,
it's a Flutter repo. ERICK ZANARDO: Yeah. CRAIG LABENZ: Flutter
I/O underscore FLIP? ERICK ZANARDO: Yeah. CRAIG LABENZ: Oh, there it is. Huh. I'd assumed it was a VGV repo. ERICK ZANARDO:
Yeah, it was, but we transferred before the event. CRAIG LABENZ: Oh, OK. Got it, got it, got it. OK. Typing. All right. Well, that's all done. This question is
I guess maybe more for Google's end of things. How much time do
you estimate was saved using artificial
intelligence to generate the images and descriptions? Well, infinite. ERICK ZANARDO: Yeah. CRAIG LABENZ: You said it
was 10,000 images and 420,000 descriptions? ERICK ZANARDO: Yeah, that's the
final number of the collection of the game right now. CRAIG LABENZ: Yeah. So writing 420,000
descriptions is, I think, basically impossible. Any human beings
that you assigned that task to would
probably go mad and quit before they completed the
job, so you'd just never-- ERICK ZANARDO: Maybe
a few hundred people may be able to, but-- CRAIG LABENZ: Yeah. Maybe. ERICK ZANARDO: --too much. CRAIG LABENZ: And
for the images, the interesting thing was
the team on Google's end that was making
those images really wanted to make sure
it wasn't going to generate crazy nonsense. We didn't want very, very
loaded, very inappropriate images ever appearing. So it was a model-- so Google has this model
that's trained only on responsible, safe,
royalty-free, essentially open source images. And so it, in theory, should
be incapable of rendering some of images of the worst
things from human history and whatnot. You can't ask it
to be Dash holding insignias from terrible
governments of the past and whatnot because it just
doesn't know what those are. But it doesn't know
what Dash is either. They trained this
image on artwork that was commissioned
by this design agency that Google and Flutter
work with all the time. And they produced these
really just stunning images of all these characters. They had quite a few of them. And so then this pretty safe,
responsible image-generating model-- which I believe is called Muse-- was tailored or was
fine tuned, essentially, to learn about Dash and Sparky
and the other characters, and then it started
spitting all those out. So, anyway, all of this is to
say the answer to this question is, however long-- 10,000 images. So how long would it
take for that design firm to generate 10,000 images
minus how long it took them to generate a hundred or however
many they made for the training data, and then the
time spent training? I think, again, just a
crazy amount of time. But they got-- it's a pretty
cool thing to teach this model-- basically, all it
knows how to do is plop these four characters in
fantastic fairy tale locations and it's pretty cool. Yeah, pretty neat. All right, let's see here. So we've talked about
some of the bugs. I'm guessing there's a
lot more fun bugs that are fun in hindsight
or that were driving you a little mad in the moment. What are some of your other
kind of just best battle stories from developing this? What bug had you most
confused, actually? I'll start with that one. What one made you go,
how is this possible? ERICK ZANARDO: Yeah, so I
have one that I think is-- it is not confusing in the way
that it was difficult to find or it was a tricky bug to find. But we got to one bug
report that the game was not working on Pixel 7. We were in a rush, we
were trying to think, so we had so many things in
our heads to figure it out that we didn't even realize that
Pixel 7 was not even out yet. So that was the most funny
story, I think, for-- we tried to debug this bug. We didn't have a Pixel 6. So we tried to debug
it on that one, but we weren't able to figure
out what was not working. And then we realized that
the Pixel 7 was not out yet. And then, yeah, that
was the most funny. CRAIG LABENZ: Cannot
reproduce, send Pixel 7-- ERICK ZANARDO: Yeah, exactly. CRAIG LABENZ: Yeah, that-- and that was like the owl
wasn't showing up, right? Is that the-- ERICK ZANARDO: Yeah. The owl and the
card pack animation. So none of those were
being show on Pixels. CRAIG LABENZ: Well,
that is weird. ERICK ZANARDO: Yeah. CRAIG LABENZ: I don't-- it's
like the intersection of two very specific things. Something is going wrong. ERICK ZANARDO: We tried
to figure out the-- CRAIG LABENZ: Yeah. Chloe and I, who had that
little game, the promo video, we were playing on Pixel 7A's
brand new phones, which is-- I think the bug report
you got was from us. And I hadn't played
it on other phones. So I didn't know that it
worked on other phones. So I thought it was
a mobile issue, and-- yeah, but we were
playing on that. And then for the
video, the owl was just added in after the fact. It was edited in post because
I don't think you ever figured out what was going on
with the Pixel 7's. ERICK ZANARDO: It should
be showing now, by the way. We've made a simpler
version, and it should be showing for all mobiles now. CRAIG LABENZ: OK. ERICK ZANARDO: Yeah. CRAIG LABENZ: Nice. Nice. All right. I'm just looking over my
notes to see if there are any other thoughts that I had. I think we've pretty much
gotten to everything. We talked about how the
game was settled upon, the nature of building it in
Flutter, having a Dart backend, integrating that with
Firebase, server side authoritative aspects. I'm just running through it
all out loud here and make sure we didn't miss anything. Talking to Firestore from Dart. That's always a trick. ERICK ZANARDO: Yeah. CRAIG LABENZ: But a
very interesting detail there about how your
server doesn't need to have an open connection. One of the coolest
things about Firestore is you can watch those games or
you can watch documents, sorry, but the server doesn't
need to do that. Certainly, unless
it's a-- if it's a RESTful server, if you
have something long lived, then it could. Oh, scaling things. Scaling thing. So you got to work-- maybe this can be the last
thing unless we get some more questions from the chat. You got to work side by
side with Firebase SREs. And I imagine how much you know
about the internals of Firebase and how to profile
a Firestore database has gone up an order
of magnitude or two. So what's some kind
of fun stuff that you learned in that regard? ERICK ZANARDO: Mm-hmm. Yeah, so I still
have a lot to learn about in terms of
Firestore and Firebase, but I did learn a good deal. I talk a lot with Berf. He helped me a lot trying
to understand how to make the Firestore scale better. I think the thing that
I really liked to learn was how Firestore collections,
they scale Firestore individually. So if you-- so Firestore
scale everything for you automatically, right? When it hit a certain load, it
starts to scale to a new tier. So when there is a surge
of excess interconnection, some users may face a
little bit more of latency because Firestore is scaling
the collection for you. So one thing that we can
do to avoid that-- and we tried and we did
that for the launch-- was to warm up our database. So even though it would not make
the game unplayable for anyone, it would make
everyone's experiences better if you could
warm up and avoid those latencies while
the Firestore is scaling your collections. So he showed me this
interesting tool that is called Key Visualizer
that you can access it from the Firebase Console. We will see more
information in Google Cloud. It will take you to a section
of Firestore in Google Cloud. And you can see this
beautiful heatmap about access of
your collections. And it shows how much
requests per second you have on each collections. As your database is getting
more and more and more requests, it starts scaling
collections differently. So it starts to see what is the
attire of the card descriptions collection, what is the attire
of the card description. And with that, you can
decide what collection to better scale. So you can identify,
OK, so we know that card collection is one
that will be accessed a lot. We'll have a lot of
rights on that collection because people will
generate cards, and that's the core
thing of the game. So you can go there and
decide which collections you can use to scale. CRAIG LABENZ: And when
you say warm it up, I imagine you mean just spam
records into the collection. ERICK ZANARDO: Yeah. Yeah. So there is a-- CRAIG LABENZ: Just
fill it up, whatever. ERICK ZANARDO: Yeah,
there's an interesting rule. So it's 55,000, and then you
can use that operation for more. So you keep doing 50
operations for five minutes, then you go up to 500
per minute and you keep going if you need to. Then that is usually
what Firestore needs to know that they need
to scale up your collection to the next tier. And the cool thing is that
once that collection is scaled to our upper
tier, they never go back. So just need to do that once. So if you are really sure
that a collection will have a lot of throughput, a
lot of operations per second, you can do that beforehand. CRAIG LABENZ: OK. Interesting. Interesting. ERICK ZANARDO: Yeah. I loved to learn that. CRAIG LABENZ: Yeah. One thing I heard from
one of the Firebase SREs was that you can have-- Firebase does great at
logarithmic scaling, even exponential scaling,
but what it will still struggle with is a cliff. So if you ramp up,
Firebase can ride with you. But if you go from
very little to-- [IMITATES ACCELERATION]
because the keynote just dropped your link, then, like
anything, it can struggle. So yeah, the warming up
sounds pretty important. ERICK ZANARDO: Yeah. CRAIG LABENZ: All right. Erick, you have been
thinking about nothing other than I/O FLIP, I expect,
for the last three months. And now you've agreed to
think about it for one more morning by joining me here
to chat about it, so I-- gosh, thank you. Thanks to the whole
Very Good Ventures team for pulling this off. Gosh. Everyone was working around the
clock long nights, weekends. I know your whole team was. We were on the images. Just like on my end,
it was the video. Everyone was just kind
of hitting it nonstop. Yeah. PTO, Erick, that's the-- ERICK ZANARDO: I will. I will. CRAIG LABENZ: That's
what the people want is for you to get a break. Anyway, gosh, I just
had the best time working on this whole
thing, and I don't know. Any closing thoughts
from you, Erick? ERICK ZANARDO: Yeah, just
want to say thank you for the opportunity to be here. It's super great for me to-- having the opportunity to work
on this project was already super great and having
the opportunity to share our experience of that
project for everybody to tell our learnings, our
thoughts, things that we-- everything. It's a great honor to
me, so thank you so much. And if anybody has any
additional questions that I didn't cover--
because there's so many things to cover. I'm pretty sure I
didn't went through all. Just hit me on Discord, on
Twitter, on whatever network that you may find me. I'm happy to talk more about. CRAIG LABENZ: Nice. And your handle is
CPTPixel, right? C-P-T. ERICK ZANARDO: Yeah. CRAIG LABENZ: Pixel, P-I-X-E-L. ERICK ZANARDO: Exactly. CRAIG LABENZ: Yeah. Well, everybody needs
to follow CPTPixel for the hottest Flutter
engineering competency you'll find anywhere on Twitter. OK. Erick, thank you so
much for joining. Everybody who tuned in, a lot of
great questions from the chat. I really appreciate that. Exhausted all the
questions that we had lined up before I thought
we would, so I appreciate that. And, folks, next week
I hope to be here, but then "Observable
Flutter" is going to go on holiday
for a little bit because I'm going to be
in Europe for a long time. I'll be in Paris for
the Flutter Connection, then I'll be in Amsterdam
for I/O Connect, and then be in Berlin
for FlutterCon. I'm just staying in
Europe the whole time. So next week I think might be
the final "Observable Flutter" until later in the summer. So the show won't be
off the air or anything. We're not pulling the
plug, even though it's going to be going to be
quiet for a little bit. But I hope to see
you all next week. Erick, thank you so much again. Please go sleep. ERICK ZANARDO: I will. CRAIG LABENZ: Go take a nap. And, everybody, I'll
see you next week. ERICK ZANARDO: Bye-bye.
