[MUSIC PLAYING] CAMILLE SIMON: Hi. I'm Camille, a software
engineer on the Flutter team. KHANH NGUYEN: And I'm
Khanh, developer relations engineer also on
the Flutter team. And today we're talking
to you about a fun project that we've been working
on with Flutter, Dart, and Raspberry Pi. CAMILLE SIMON: If you're
not already familiar, this is a Raspberry Pi. I'm sorry if you were
expecting a more delicious kind of Raspberry Pi. It's a single-board
computer that was released in 2012 to
teach kids computer science. It's grown in
popularity since then. And it's now used in a variety
of fields for various use cases because it's just
so cost effective. A Pi can range from $5 to $75. And you can run all
kinds of things on it because, well, it's a computer. KHANH NGUYEN: I caught the Pi
bug myself a few years ago. Inspired by the hashtag
#NowPlaying trend back in the day, I built a
dashboard to display the song that I was listening to
on my music streaming service and a dashboard to show
the upcoming commuter train status. And since the Pi
can be hooked up to all kinds of
gadgets and gizmos, like sensors and buttons, I
also built a disk drop game with a digital UI. You drop a little puck. It'd fall down, hit a bunch
of pegs along the way, and trigger one of
the sensors that was installed in a slot down below. The UI would update
to show whatever prize the player had won. You could say I've been
really into building my side projects on Raspberry
Pis for a while now. Every time I build
a new project, a little voice in my head likes
to ask, can I run this on a Pi? Fast forward to last summer
when a teammate and I built a game in Flutter. Doodle Dash, as we
like to call it, is a platformer
game, where players try to jump as high as
possible and rack up points. There's powerups, enemies,
the whole nine yards. The game is built with Flame, an
open source game engine that's been built on top of Flutter. It was a lot of fun. And that voice in the back
of my head popped up again. We started wondering, can we
build a game console experience with Dart, Flutter,
and Raspberry Pi? With some code, a Raspberry
Pi, a screwdriver, and a little Pi accessory called
a game hat, turns out we can. Before we jump into how
we brought this Dream project to life, here were our
three big project requirements. First, we wanted a gamepad. This was a requirement
because, at least to me, a game console experience
requires a gamepad. Second, there are gamepads
that plug in via USB. But since we have access
to the Pi's GPIO pins, we wanted to use them. Personally, my
favorite Pi projects have always included working
with GPIO pins for input or output, sometimes both. As much as I love writing code,
having the hands-on aspect of putting the pieces together
is always a ton of fun. If you're not familiar with
GPIO pins, don't worry. We'll cover this topic in more
detail in just a few moments. And finally, we wanted as much
of our code to be as reusable and extensible as possible. Given how community-driven
both the Flutter and Pi communities are, we want to
share our work with everyone. This means the
code that we write shouldn't just enable Doodle
Dash to work with the game hat. It should enable other
Flutter developers to integrate their games
with their game hat as well. CAMILLE SIMON: Awesome. Now that we've covered our
requirements, let's talk setup. We installed a Debian-based
operating system for the Pi, Raspberry Pi OS 64-bit
on a micro SD card using the Raspberry
Pi imager that's available on the
Raspberry Pi website. Once that was loaded up, we
popped the card into our Pi and booted it up. Then we used Snap, Canonical's
app store for Linux, to install Flutter. Now for the fun part,
assembling the game hat. The first step was inserting
the micro SD card into the Pi and attaching the game hat
module to the Pi's GPIO pins. Next was connecting the game
hat's pre-installed screen to the Pi's micro HDMI out
using the included adapter. Finally, positioning
the phase plates, putting all the screws and
support pieces in place, and attaching the
joystick to the game hat. Our development cycle involved
writing code on our laptops and building,
executing, and testing the game on the Pi itself. For portability
during development, we used a mini keyboard to
run commands on the terminal. And the built-in
trackpad let us interact with the Pi's desktop UI. KHANH NGUYEN: Now that
we've got everything set up, let's get building. First, reading the
GPIO signals with Dart. As you'll recall from
a few moments ago, we attached the game hat
to the Pi's GPIO pins. GPIO stands for general
purpose input output. They're signal
pins on the Pi that can be controlled with code. As their name
indicates, they can be used for input or output. Like I said, you can connect
it to all kinds of things, from buttons or sensors to,
well, in this case, a game hat. The game hat itself is really
a pre-configured set of buttons that we want input from. Taking a look at the
manual for the game hat, it mapped out exactly what GPIO
pin is used for which button. We made this more
manageable in code by creating an enum to represent
the game hash GPIO input. Each enum has a pin with
a game control event. We then used the rpi
GPIO package from pub.dev to get the input
values from the pins. We created a new class called
gamepad to represent, well, a gamepad. On initialization,
the code loops through the list of
game hat GPIO enums, initializes each pin, and
starts listening for input. The code listens for
a change in pin state from true to false
or false to true. A value of false means
that it's been pressed. And a value of true means that
the button has been released. The game will
listen to the event stream for new gamepad input. So when there's a
change in a pin state, a new game input gets added to
the event's stream controller. CAMILLE SIMON: Now, let's
talk about a small detail that we glossed over
before, turning the signals into game input. The question we need to
answer is, how should our code translate raw button press
events from the hardware into Doodle Dash commands
my game can understand. To translate between the
two, we created a few classes for the Doodle Dash game to use. There's game
control event, which represents what game control
was activated, left, right, A, B, et cetera,
and event state, which represents whether the
button was pressed or released. Together they make up
a game input event. Like Khanh discussed
before, the gamepad will listen for events from
the physical gamepad buttons and add game input events
to its stream controller. To test this out, we
can use the gamepad. To use the gamepad we'll need
to initialize it in the app. And now we can run the app while
logging the joystick movement, button presses, and releases. Since we're able to confirm that
the button presses are coming through, the remaining
work left for the project is having Doodle Dash react
to the gamepad event stream. KHANH NGUYEN: OK, so the button
presses are being registered. How do we ensure that
Flame recognizes game input and allows players to
actually control the game? Let's talk about
Flame components. Flame components are all
the individual pieces that make up a Flame game. I like to think that components
are to Flame as widgets are to Flutter,
like these assets you see in the game, and the
background, and even the things that you don't
see on the screen, like the game manager, which
is responsible for spawning on screen objects, similar
to state management widgets. Flame components
start off pretty bare. But you can enhance
their functionality by using various mix-ins
offered by the Flame package. For example, a popular mix-in
is the keyboard handler. This mix-in gives
Flame components access to key press events. You can add the keyboard
handler mix-in to a component and override the
onKeyEvent method to do whatever
you'd like whenever there's a new key press. With Doodle Dash, if the left
or right key has been pressed, onKeyEvent will make
Dash move left or right based on the arrow key
that's been pressed. We followed a
similar architecture for extending Flame to
work with the game hat by creating a mix-in
called GamepadInput. This mix-in has three parts. onMount initializes
a stream subscription that listens to the gamepad
broadcast stream for new game input events. It calls a method named
onGamepadEvent whenever a new event comes through. onRemove ensures that the
subscription is cleaned up when the component is removed. And finally, onGamepadEvent
which is called every time there's a game input
event from the gamepad. Then in the Player class we
add the gamepad input mix-in and override onGamepadEvent. When a new game input
comes in and the player has pressed the joystick
either left or right, then the character will
move left or right, just the key presses. The only difference
here is that we listen to all game input
events, not just button presses. So we had to introduce
a conditional for EventState.PRESSED
so that it filters out EventState.RELEASED. With all that coding
done, we can now move Dash left and right using
the joystick on the game hat. CAMILLE SIMON: Finally, the
missing piece in our game console is menu navigation. Flame uses regular old Flutter
widgets for overlays and menus. That's great. But we don't have
a keyboard, so that means we don't have
access to Flutter's built-in keyboard-based
focus system. We came up with our own solution
backed by a class called GamepadHandler. A GamepadHandler
has five callbacks that can be called, on up,
on down, on left, on right, and on confirm. These callbacks can be used
to navigate the menu screen. For example, this
nextCharacter method that changes the selected character. You'll see it being passed
to the GamepadHandler's on left and on right
callbacks in the next slide. Within our menu widget,
a new GamepadHandler is initialized for
each section of the UI that requires focus
and directional control from the gamepad. Upon game initialization,
the characterSelector GamepadHandler is active. On left and on right calls
the nextCharacter method to change the
selected character. While on down changes
the activeSelector to the next section
that wants focus, the difficultySelector
GamepadHandler below it. The neat part is
the activeSelector can also be used to
conditionally style the section of the
UI that is in focus. We can check for which
selector is currently active to put a border on
that section of the UI. Finally, the widget
initializes a subscription to the gamepad's
broadcast stream and listens for
game input events. When a game input
value comes through, it activates the selected
gamepad handler's callback. And there you have it, a fully
functional menu navigation system. You can use the gamepad
controls to select a character, change the difficulty
level, and start a new game. KHANH NGUYEN: Once we got the
core functionality working, we had a few things to work
out, like making sure to adapt the UI to the smaller screen. And this is a game
console after all. So we wouldn't want to carry a
keyboard with us everywhere we go. So we also made sure
that the Doodle Dash app launches on boot up. And finally, for
the finishing touch, we also added a 3D-printed case. How cool is that? CAMILLE SIMON: So cool. And there you have
it, a Flutter, Dart, and Raspberry Pi game console. This project might
be a prototype. But it sure does show so much
potential for Flutter and Dart on Raspberry Pi. KHANH NGUYEN: For sure. Camille, this was such a
fun project to work on. I mean, I truly felt like
I was five years old again, getting my very
first game console. It also makes me very
excited looking forward to what's coming for Flutter
and Linux in the future. In fact, the new Ubuntu
23.04 desktop installer is built with Flutter. So that's pretty cool. I also want to say get it
to work with the open source community throughout
this project has been such a
great experience. Plus, they're always doing
awesome things like this. I mean, I've seen a Flame
game running on a smartwatch. So that's all to say I'm so
excited to see what the Flutter community continues to build. CAMILLE SIMON:
Completely agreed. But wait, Khanh, do we
have the source code to share with everyone? KHANH NGUYEN: Good question. You'll notice that this code
was made to be very modular, given our goal to make it
reusable and extensible. We'd love to see you use it
with your own Flutter and Flame games. We're polishing the code
for this project right now and plan to share it
on the Flutter Samples repository in GitHub very soon. Be sure to follow
Flutter Dev for updates. CAMILLE SIMON:
That sounds great. All that said, I think
that's about it for us. Thanks for joining us on this
Flutter, Dart, and Raspberry Pi journey. I'm Camille. KHANH NGUYEN: And I'm Khanh. CAMILLE SIMON: See you later. KHANH NGUYEN: Bye. [MUSIC PLAYING]
