- Suppose that you manage a
factory that makes widgets, you've got a spinning thing over there, you've got an up and
downy thing over here, it's a complex setup that
produces a lot of vibrations, so you're quite reasonably
concerned that one day your factory could shake itself to bits like a small vibration here
may cause a bolt to come loose, which causes even more vibrations that leads to metal fatigue over here, which leads to even larger vibrations, it's a snowball effect. So you wanna catch the vibrations early before they become a problem, before they're even
visible to the human eye, so what do you do? Well, you could put some sensors on things that you're worried about, so maybe you put an accelerometer
on this pipe over here that you're worried might be
moving up and down too much, and that sensor supplies you with data and you plot a graph and you
can see it moving up and down and it's within limits
and you can monitor it, and it's fine. But what if there's a part
of the factory you forget to put a sensor on because you
don't think it's a concern? What if you could take millions of sensors and just throw them across your factory and capture everything? It turns out you can, and in fact, you carry a set of these
sensors with you all the time, I'm talking about your phone, specifically the camera in your phone, that chip that gathers
light for the picture, the CMOs chip is made of
millions of individual sensors, one for each pixel, well I suppose three
for each pixel actually. In reality, you're probably
gonna wanna use something better than your camera phone selling industrial with a global shutter. So you get a camera and
you take some videos of your factory, some really
wide shots to begin with, make sure you capture
absolutely everything, but if what you're
looking for is vibrations that are so small, they're not perceptible with human vision, then how can a camera
see these vibrations? That's where motion
amplification comes in, in fact, all the footage
I've shown you so far is motion amplified, for example here, you can see side-by-side
the original footage and the motion amplified footage. The original footage doesn't
look like it's moving at all, but there is a subtle
movement information in there that's been amplified on the other side. In fact, an original video can
contain sub pixel movements and still have that information amplified, so you don't even have
a hard edge sweeping across several pixels, instead, you've got a line of
pixels that are getting darker while the neighboring
pixels are getting lighter and you can interpret that
information as movement and amplify it, it's incredible really, especially when you consider if you're extracting this information against the background of noise, all digital images and videos
contain a bit of noise, which is a fancy way of
saying random fluctuations in brightness for individual pixels. I spoke to Jeff Hay at RDI Technologies who created emotion amplification system that's used in industrial sector. - I started in astronomy, thought I was gonna be an astronomer, but I think I had a
conversation with my advisor that was about being gainfully employed, you know remarkably the
problem sets kind of the same, you know what I mean? You're using the camera
to figure out something about some object that's, you
know, kind of at a distance, the other hobby that I had, I was the photo editor for
my university newspaper, and so I've got a photography
background as well, so all these things sort of
just started to come together, you know it's no secret where I ended up. Even a megapixel camera,
which is by today's standards, kind of small and in resolution, that has a million sensors in it, you can blanket an object with sensors and measure everywhere, but when you have sort of
like a motor and a pump, you know, they're connected,
the shafts are connected at something that they call a coupling, which is just essentially connecting those two pieces of equipment, and if one side, if for example, the pump is going up while
the motor is going down, that's misalignment and that's bad, you know, so lots of times that
things are moving together, that's some indication that that's good, that's all right, and sometimes when they're opposing each other, that's where the problems come in. - If you've got a 50
frames per second camera and you've got something that's
vibrating 15 times a second, you may not pick it up, do you have to worry
about that sort of thing? - Yeah, so usually the rule of
thumb is you have to sample, you have to take data at a rate twice the frequency you're interested in. - With this system you can
look at the raw data as well, like you can say, look,
tell me what's going on with these pixels here, I'm interested in the
vibrations at that point, and you can see a nice graph
and that's really important, but I'd also say that this
summarized visual representation, that's so easily digested
by the human visual cortex is also really important
because human brains are better than computers at certain
tasks, maybe not forever, but it's true for now, which
is why we use human brains to train artificial intelligence in things like computer vision. This is a bit of a diversion, but when you log onto a website and you have to prove that
you're human and not a robot, you fill in a CAPTCHA, and in the old days that
used to be two words that were garbled in some way, and you had to type out
what the words were, an easy task for human vision,
a hard task for computers, but actually only one of those words was to test whether you are a human, the other word was something
that Google was struggling with as part of their book
digitization project. So here's a word we don't
recognize in a book, we're gonna outsource it to a human through the CAPTCHA process, and it doesn't just
help with that one word in that one book, it also trains the AI to be able to recognize
that kind of word in future. By outsourcing word recognition
to humans in this way, Google were able to blaze through their book digitization projects, which is why you don't see
these word CAPTCHA anymore, so what do you see now? Well, for a while it was
numbers if you remember, and it would be a picture of a number that look suspiciously
like a blurry photograph of a front door, and guess what? That was helping Google
maps to identify addresses through pictures, taken
from Google street view, these days the CAPTCHA task, it tends to involve something
like click on all the images that contain a road sign or click on all the
images that contain a bus, and guess what, that's because
Google's parent company has an autonomous vehicle project and they need human brains to help them train their
artificial intelligence, to recognize things like
road signs and buses. Interestingly, those
CAPTCHAS are real time, so if you're asked to identify
a road sign for example, that's because there's a
car out there on the street at that moment that needs to
know that that's a road sign, so if you see that you
need to act quickly, otherwise someone could get
hurt, that last bit isn't true, or is it? It isn't. But anyway, the point is, if you can take your data and
turn it into something visual, you might be able to outsource some of the processing of
that data to a human brain that can produce remarkable insights without even breaking a sweat. And by the way, having
this visual representation really helps when you're
trying to persuade superiors to spend some money. - So if you go to somebody
and you show them a plot on a graph and you say, man that machine is gonna
come apart, you know, that's a hard, you know, but if you can show them that
it's lifting up off the ground or it's cracked, or you know, that's kind of an element to it as well. - You talked about having, you got some footage of
engines and things like that. - We've got car engines, we've got some more common, you know, sort of everyday stuff that we could do, I think it would be a little
bit more universal to people. - Jeff is understandably reluctant to talk about the finer technical details of how the technology works, but I really wanted to see
if I could create emotion, amplified video of my own, I did find some research
from scientists at MIT, they're calling it video magnification, it's the same kind of idea, you can amplify motion in a video, you can also amplify
color change in a video, let me show you what I mean. This is my wrist, you can
just about see a pulse there, but look here it is with
motion amplification applied, I'll link some links to their
research and description, they've got software you can download, they've even got an online
system where you can upload video and play with algorithm, it's really cool, you can even detect pulse
through color amplification. The color of my face is
changing very, very slightly with each beat of my heart, you can't see it with the naked eye, but here it is amplified, and then I blink. Actually you can detect pulse
just from me standing still here I'm trying to be
as still as possible, but every time blood is
pumped through my neck, my head wobbles slightly,
here it is amplified, there was a lot more motion in my body than just the wobbling of
my head due to my pulse, but the algorithm that
generates the resulting video is really clever, and you
can filter by frequency. So I can say I'm only interested in motion that has a frequency between one Hertz and one and a half Hertz, because I know my pulse was around that and it will amplify just those motions, and the same goes for the footage of the factories by the way, you can filter by frequency
to pick out different things that you're interested in, this method works even
if I'm wearing a mask. I'd love to talk in detail
about how the algorithms work that generate the videos, but honestly, I get the feeling that you'd
need to study this stuff for a very long time before you
could even start to describe how it works. Here's a choice phrase from the research, it's to do with phase variation of a
complex steerable pyramid, what does that even mean? So I'm gonna declare that
beyond the scope of this video, you may already be imagining
medical applications for this, like taking someone's pulse noninvasively or without having any
physical contact with them. If you've ever looked after
very, very small children, you probably watched them sleep and you've probably started
to worry at some point, have they stopped breathing? So you poke them and
they move and you think, oh okay, phew, they were breathing, I mean, now they're awake and crying, but at least I know they're breathing. Imagine if instead you
could train a camera on them and amplify the up and
down motion of their chest and know for sure that
they were breathing, I'm gonna put a link in the
card and on the end screen to a very Tassian video
that's somewhat related that shows how you can
extract audio information from a video by looking at
the subtle movement of objects in the video that have sound
energy impinged upon them, it's really good. Thanks again to Jeff Hay
from RDI Technologies, for all the motion amplification footage, details about his company
are in the description. Segue, I can snowboard, maybe
it's not that interesting, but what is interesting
is the way that I learn, I basically taught myself
with a little bit of help from a couple of people who were with me, who could already snowboard, the issue with that approach, which I've learned over the years is that I have a few habits
that are now really ingrained, and if I want to get any
better at this point, I really need to unlearn those bad habits, which is a total pain in the ass, and that was my approach
to life back then generally like if there's something
you need to learn, just muddle through, figure it out, but more and more I'm of the opinion that that's a false economy
in a lot of situations, like often it's better to invest upfront, spend some time on a proper course or, you know, take a class, and that way, all the time
that you invest subsequently is really quality, like
you're building on a framework of good practice, it's really
gonna accelerate your learning and your improvement. So these days I'm a real advocate
for online video learning like the sponsor of
this video, Skillshare. Skillshare have courses
in things like web design, photography, interior design,
animation, illustration, I took the styling your space interior design course recently actually, because we're remodeling our kitchen, broadly the categories
are like creative stuff, artistic things, but also
like business-related things like leadership or productivity, really there's a huge
spectrum of stuff on there, and you know, maybe you find yourself with a bit more free time
in this current situation, maybe it's time for a bit of
well-directed self-improvement or maybe you've been
mucking about with an idea, and it's time to kickstart the idea with a proper chunk of formal learning, but whatever your story is, I recommend you have a
browse through Skillshare because you might get inspired
to start something new. The whole thing is just $10 a month, you get access to absolutely
everything for that, and because they're sponsoring this video, the first 500 people to
go to skl.sh/stevemould8 will get two months absolutely
free, no strings attached, so check out Skillshare today. I hope you enjoyed this video, if you did, don't forget to hit subscribe and I'll see you next time. (upbeat music)
Used RDI at work, did what it said on the box.