JASON MAYES: So
we're heading over to meet Paul, who's
going to tell us more about a very fascinating
open source project called Node-RED. Now first off, Paul, tell us
more about your background and who you are. PAUL VAN ECK: So hey,
Jason, thanks for having me. So hey, everyone, My
name is Paul Van Eck, and I'm a software developer
with IBM primarily working with open source
AI technologies. So currently I am working
on the Kubeflow project on the ML operations side
of things, but in the past I did spend quite a lot of time
in the TensorFlow community, working on a variety of projects
like TensorFlow.js, which really garnered my
attention, because I'm a bit of a web.dev guy, so I
kind of like the intersection of ML and web development. JASON MAYES:
Excellent, you've been working on this project
called Node-RED. What exactly is that? PAUL VAN ECK: Yeah, so
Node-RED is an open source visual programming tool
that offers a browser based flow editor for wiring together
devices, APIs, and services. So you can say it kind of
provides a low code approach for event driven programming. I think it really lowers
the barrier of entry for making a variety
of these types of apps. And since it runs
on top of Node.js, Node-RED can run on
a variety of devices, like your laptop,
the Cloud, and even on low cost hardware
like the Raspberry Pi. So if you're working
in the IoT space, this is where Node-RED
really shines. So to showcase this, here I have
Node-RED running on my laptop and the visual editor
opened in the browser. So on the left side,
you can see the pallet which contains all
of the nodes that are installed and available to use. Each Node-RED node has a
well defined purpose and acts as a building block
for constructing flows. You simply drag over
nodes to the workspace, like I'm doing here, and
string together enough of them. And you can make some
pretty powerful stuff. So here I'm making
a simple flow that sends an input to an API
for interesting facts about numbers. And after you were done
constructing the flow, I deploy it, then hey,
here's some interesting facts about the number 42. JASON MAYES: That's super cool. I like that. And how does TensorFlow.js
fit into all of this, then? PAUL VAN ECK: Yeah,
so good question. So TensorFlow.js and Node-RED
fit together almost seamlessly as a kind of both leverage
the JavaScript and Node.js eco-systems. So users can take a
TFGS model, package them into a Node-RED node, and
use them in their flows to build a AI enabled IoT apps. So what my colleagues
and I have done here are built some TensorFlow.js
based Node-RED nodes that can be installed and used
by users looking to bring ML capabilities to their apps. JASON MAYES:
Awesome, sounds good. Let's see that in action. PAUL VAN ECK: Yeah, sure. To showcase some of
these nodes, here I have a flow that takes
an image as input, and performs some object
detection on it, displaying an image with the
detected objects outlined by bounding boxes. Going over the flow,
the image input is passed to a tf.function
node here, which allows you to use
a TFGS node, API, and JavaScript for arbitrary
scripting like decoding an image into a Tensor. This is then passed
to the tf model node, where we loaded a TFGS
COCO-SSD object detection model. This could be any
TensorFlow model. But from the model, the
output from this model will be then sent through a
post-processing node, where you specify a link to
your list of classes. And this will turn the model
output into a nicer format, like we see on the right. Can use various forms of input. I was wearing a panda
mask for that so. JASON MAYES: Good stuff. Cool, so essentially
you've created a really powerful visual
editor that's not only able to program these
really amazing things, but you can also
experiment with a number of different TensorFlow.js
models, and even better, you can deploy to devices
like the Raspberry Pi. So maybe you can tell
us more about some of the projects you've
created using this system. PAUL VAN ECK: Oh yes, sure. So there are several
cool things you can do with Node-RED
and Tensorflow.js, especially when
hardware is involved. And so I'm excited to
share some of the things I was able to do with the
Raspberry Pi and peripherals like, for instance, this
motion sensor and speaker, USB speaker. So here's the first one. So my cat likes
to hop on surfaces he isn't allowed to be on,
so I wanted to deter him. [DOORBELL RING] So here the flow is triggered
by a small motion sensor as I showed before, and then
using the pre-trained COCO-SSD model I used before, I
check if a cat is detected. And so if so, I
play a sound the cat doesn't like through
an attached speaker. In this case, my
cat is terrified of the doorbell sound, so [CAT MEOWING] That's one way for
me to deter him. JASON MAYES: Amazing, very cool. I like that one. What else have you got? PAUL VAN ECK: So
now this next one is a fun one, where I use
the same hardware as before. But this time, I
add a spray bottle with a Servo Motor attached. The Servo Motor will be
able to pull the trigger to actually spray
some disinfectant, or in this case, water,
on an unsuspecting person. So the idea here, for this flow,
is that the object detection model will check if a
person is wearing a mask. And if not, well, let's
just see what happens. [DOOR CREAKING] [COUGHING] [ALARM] Got my mask! JASON MAYES: Brilliant. That'll definitely
get people more motivated to put their masks
on before entering anywhere. PAUL VAN ECK: Yeah, so
that was a fun one for me. So definitely wear
your mask, people. So for the next
flow, a colleague of mine at IBM, Yihong, decided
he wanted to use a smart garage opener with a TensorFlow
model for license plate number recognition. So using Node-RED,
on his Jetson Nano, he was able to create this flow. So a car pulls up
into the garage and a camera will take
a photo of the car. Here, a GoPro is mounted
above his garage. The license plate and
any image in the image is isolated and then
processed for characters. If the license plate characters
match a specific string, then the signal to open
the garage is sent. And voila. JASON MAYES: That's
super cool, I love that. PAUL VAN ECK: Yeah, so I believe
he's using a myQ Smart Garage Opener project to facilitate
this, but there's-- well, in the world of
IoT, a lot smart things you can connect to
a variety of things. JASON MAYES: As long as you
can communicate to it somehow, then you're
basically good to go. So that makes sense,
that's awesome. Really cool. Amazing projects there. And I think as
JavaScript developers, we can sometimes forget all
the places we can actually execute JavaScript. It's great to see
some of the potential here for controlling real
world physical objects beyond even the web browser. Do you have any other ideas
you'd like to create in mind, or maybe worth exploring for
our viewers, even, to try out? PAUL VAN ECK: Well,
yeah, so before that, I think it's
important for viewers to note that with
our Node-RED node, you can pretty much use
any TensorFlow model that can be converted to
a TFGS model format. JASON MAYES: Nice. PAUL VAN ECK: So
this kind of opens up a wide range of possibilities. But I do think people
should definitely branch out from just
image based models like what I've shown here. Perhaps consider ones for
things like speech recognition or natural language processing. For example, here's a flow I
made using a natural language processing model. So here a Twitter hashtag,
in this case #coronavirus, is live-monitored using a
Twitter node for tweets. These tweets are run through a
BERT based model for sentiment analysis, and then using
Node-RED dashboard nodes I can get a simple visual
picture of the sentiment in the live updating graph. JASON MAYES: Awesome,
very good stuff. And I guess if people who
are watching right now want to go and try this
out for themselves, what links or resources
would you recommend? PAUL VAN ECK: So
there are a number of links I'd like to share. So first, if anyone wants--
is interested in learning more about this, how we can use
Node-RED and TensorFlow.js together, definitely
check out the tutorial that my colleagues and I
made at developer.ibm.com, and its corresponding
video on YouTube. This should definitely get
you started using Node-RED and TensorFlow.js together. So I believe the links will
be in the description, so definitely check those out. JASON MAYES: I'll put
those in the description after the show for sure. And yeah, everyone go check
those out and, of course, let Paul know your feedback. So yeah, once again,
thank you so much, Paul, for being
on the show today. It's great to have you with us. And some really, really
innovative demos there, and it's great to see how we can
apply this stuff to hardware as well. So thank you very
much and see you soon. PAUL VAN ECK: Thanks, Jason. [MUSIC PLAYING]
