This is NOT a Graphics Card

[Music] this might look like a gpu smell like a gpu stick in a pci express 16x slot like a gpu and even talk like a gpu but i assure you its intended purpose is very different and to some potentially terrifying it's dubbed the asus iot ai accelerator pcie card or crl-g160new-p3df for short and it's designed for artificial intelligence computation speaking of intelligence you'd be intelligent to know i'm going to tell you about our sponsor honey honey is the free to use shopping tool that helps search for the best promo codes on tons of your favorite sites get it today at joinhoney.com ltt [Music] historically we've mostly shied away from covering ai stuff and there's a couple of good reasons for it one is we're not ai or machine learning developers so finding practical use cases that we are able to set up and demo for you guys can be a little tricky and second a lot of those use cases are extremely technical behind the scenes kinds of things at least for now so hardware like this isn't too applicable to end users like me and you that is until today while doing research for and testing the home automation setup at my new house get subscribed by the way you don't want to miss any future content around revamping that place we stumbled into a bit of a roadblock regarding presence detection or the ability for the systems in the house to be aware of whether or not anyone is actually home presence detection provides us with a number of benefits for example we can turn off lights and set the hvac to be more energy efficient when there's nobody at home and this is relatively simple to implement on a whole house level but therein lies the challenge we put a lot of effort into making my house a lot more granularly controllable practically every room has its own independent hvac and lighting which is awesome for personal preferences like the kids might want their rooms cooler than my wife and i do for example or vice versa but it can also be used to improve efficiency and save cost if no one's in the giant rec room downstairs what's the point of heating and cooling it i mean surely that can't be that hard to automate well as we found out it can be really freaking hard one idea we had was to install bluetooth low energy beacons in every room and read people's presence through their phones the only problem with that is that people have this tendency when they're at home to put their phone on a charger and oh i don't know walk around their house so that won't work another idea was to use motion sensors but those kind of crap the bed as soon as you decide to take a nap on the couch and then you wake up in a sauna of a room in the summer which sounds like a pretty bad time to me so this is where our ai card comes into play this is an asus designed product but all of the important ai bits are actually made by google of all people under the brand coral we've got both of them linked down in the description below let's take this thing apart this specific card has 16 onboard edge tpus as google calls them and the tpu stands for tensor processing unit in the most simple terms these tpus are hardware processors that are physically designed and optimized to run a specific application you may have heard of something called an asic as something that's really good at mining a specific cryptocurrency well these are the same idea except that instead of mining bitcoin they're for running machine learning inference calculations so what we're looking at here is essentially just a pci express 16x interface here going you can actually see the traces going right into this pci express switch which appears to be just splitting out these lanes into an m.2 2x interface for each one of these little dual tpus here i was expecting something less kind of modular and science fair project now each of the edge tpus of which there are two on these m.2 cards can do around 4 trillion operations per second with a power draw of only 2 watts so 2 watts times 16 tpus brings us to 32 watts for the card you add in your pcie switch and fans and that brings you to 52 watts which is well within the 75 watts that this pci express slot could power but for some reason it still requires a six pin power connector and it's got this gigantic copper fin cooler on it now you're probably thinking wow that's really cool linus but most of this sounds like the boring technical stuff that you were talking about earlier but if you were following closely while we were researching better ways to detect presence in my new house we stumbled upon some software called frigate that uses cameras plus this hardware to really efficiently detect presence it's really freaking cool let's take it for a test drive i gotta put this back together and we're up this test bench is running the latest version of unraid and it's gonna stand in for my home nas now you might notice there is also a gpu installed in it so we need that as a video output because this is a non-apu ryzen processor but it's also useful for offloading the camera video decode process from the cpu now in my home deployment i'll probably just use the cpu because i'm going to have 24 epic cores that are otherwise going to go unused but for now this is the better solution now once on raids booted up you can see that we have just a single array disk with no redundancy that's not what i'd recommend as a production deployment but it's good enough for this video now we just pop over to the community apps tab and installed apps so we've got our coral accelerator module drivers we don't need this if we're using a usb version of google's ai processor we only need it for the pci express version and then we've also got our nvidia drivers which are for our gpu now it should be noted that you can use your cpu for frigate but performance is not good next we install frigate now we can't just launch it we've got to actually configure frigate otherwise it's going to have no idea what devices it's supposed to be using so step one we're going to copy the gpu uuid from our nvidia driver and paste that there then for our tpu mapping we've mapped it to our apex underscore zero device and that is only one of the 16 tpus that are on that card because we're only going to have a couple of cameras we're not going to bother mapping all of them but we did map five of them so that that should be enough for our demo for now wow this is a great password jake i like it don't get overwhelmed most of this is included in the template but there are a couple things that we had to tweak so one is we had to set up our camera so you can see we've pulled our camera from the writer's den and we've got our network path for it right here we're just using the one camera for now we actually didn't take all five of our npus we've just got four of them configured here again total overkill for what we're doing and then we've also got this right here so we are limiting our frames per second for the camera feed that we're using to 5 fps that is all we're going to need in 99 of situations to have functional detection and adding more processing to this would serve no real purpose one other critical thing back up at the top here is that you can plug this into an mqtt server which is basically its way of communicating back to your other home automation devices so this mqtt server 10.20.0.71 is running on our office home assistant instance hey 5fps of green i can see why you said we didn't need more fps there's not a whole lot of movement there it's peanut butter demo time peanut butter demo time peanut butter demo time wait yeah well yes okay that one camera is set up we have another one too but let's just okay hi there oh wait hold on i screwed up oh all right do we not have the detection on uh we do but i i didn't turn on like the the bounding boxes bounding boxes are just little boxes that show us what the software is detecting and the degree of certainty that it has for what that object is humans aren't the only thing that it can detect but they're kind of the only thing that we care about for home automation presence detection you're telling me we don't have to detect how many apples you have in your fruit bowl oh yeah oh yeah oh yeah you are there buddy oh it's kind of detecting your feet does it know who i am that part we have not done yet what if it only sees like this much of me yeah it still thinks you're there what if my legs show oh it lost you you lost me what about a hand oh yeah it sees you again no really okay what if it just saw an elbow oh yeah it's detecting what really yeah okay can it can it see me it's confused right now it's a little confused what about if there's just like a casual like hand there it's kind of freaking out so sometimes it actually sees two people um when he's kind of obscured but that's totally fine oh like it thinks i might be like two people having a sex orgy behind the curtain uh yeah sure but it's fine because all we need to know is if there's anyone right right yeah and sex or gsi you don't want the heat to cut out during your sex or do you have it all right it's cold man you can do so much cool stuff with this but wait there's more what if you have a common area like a kitchen or a rec room and you want to be able to apply personalized preferences when you're present like say have your spotify music with your specific playlist play when you're in the room and not when yvonne's in the room right oh i never even thought of that you could walk around the house and have spotify playing here oh i'm over here now and now there's my spotify over here oh my god that would be so cool turns out someone had that exact thought and created double take which is a piece of software that allows you to determine the face this is more creepy yeah but it's all local i mean we could just configure this to not log any of this right yeah did a linus appear oh wow yeah nice now double take isn't available on the unread community app store at least not yet so we need to manually set up the docker container ourselves then once it's set up we need to configure it to listen to frigate over mqtt it should be noted that doubletake itself doesn't do facial recognition it's just an api and interface to make hooking into and training facial recognition software easier so the first thing we need to do is grab some pictures of me which are going to be training data for our facial recognition so i guess i need to go over there right we could go over there or we can just upload some photos we already have double take currently supports deep stack compreface and facebox they're all machine learning based facial recognition models deep stack is on the community app store so that's the easiest to set up but i tried it out and it wasn't really that great with the limited training data i had compre face on the other hand it requires docker compose to have to run into vm but it's scary good i put a couple photos of linus and secretly kind of had him walk by without telling him and it was like linus and then i walked by it was like jake with like 10 photos of each of us all right let's try it why do you have so many pictures of me on your hard drive well you make my my drive hard why cpu when i can see you pee why talk to you when i can leave can i zoom how do i use a mac do you know no these stills are horrible it's scary good luck see you without a beard oh wow yeah yeah oh my god 95 percent how can it be that sure on unbearded linus training there was only one that it is actually confident about i mean 94 seems pretty damn confident is take all of these yeah and say train off of those data now you want to walk over there again yeah for sure okay that's plenty there's so many linuses now linus linus line it's linus there's a couple times it thinks it's james i think like once you trained a good model with compreface i'd probably say that like it would only accept it as as that person unless the confidence was like 99 some of these other models medicine i also made with people's photos with them wearing a mask so it literally has the eyes okay go back run over there all right i'm going i'm going to see if it makes more mistakes i'm going to pull up a chair line of style it's very confident about linus now what who else would behave like this i don't i don't think that's what it's looking for but man so you could just kind of wander around your house for 10 minutes find anything that's wrong boom train it part of it is also gonna be specific cameras because you know each camera looks a little different one thing i asked jake about and he said it doesn't have this functionality for now is can it morph over time and can it continue to self-train the thing with facial recognition is like the more data you feed it the more uncertain it becomes all of this leaves us with some questions do you even need this thing could a bog standard cpu do the same calculations the answer is yes it could but way slower to put it in perspective your average quad-core cpu would probably only be able to handle a few frames per second on a quad core but because these edge tpus are so optimized for running these specific calculations they are stupid fast at them a single coral edge tpu can handle around 100 frames per second of people detection in frigate and we've got 16 of them so if my rough math is correct that's around 1600 frames per second we might actually be pushing that once we've got all the cameras place it's taking five fps for me still it's a lot of cameras i don't know five fps yeah five okay yeah man that's crazy so a single one of these tpus could probably handle 15 to 20 cameras at 5 fps another question this raises is are we worried about the security implications fortunately since we have all of our own hardware and all of this is open source and locally hosted none of the data goes outside of my house so i'm actually pretty happy with the level of security and safety as for being worried about ai taking over no one's asking that question so it's in here anyway the answer is no leaving only one question then who's our sponsor freshbooks freshbooks is the simple accounting solution that is designed specifically with you in mind the small business owner it features built-in automation that allows you to spend less time tracking projects and more time growing your business so whether you're a tradesperson creative agency or a youtuber you can choose the plan that's right for you with freshbooks they have an award-winning toronto-based support team who are always happy to help you if you need it and you don't have to take my word for it you can try freshbooks for free for 30 days today with no credit card required at freshbooks.com linus if you enjoyed this video maybe go check out the video where we built the server that is going to be running all this stuff we're going to have to get a little bit creative with pci express expansion on it um but that's the machine that's going to somehow run this full height card yeah yeah so good luck everybody are we going to do an external pcie box we should totally do it we should totally do it yeah that will be a video get subscribed yeah we get water cool and everything

Channel: Linus Tech Tips

Views: 1,239,983

Rating: 4.9253783 out of 5

Keywords: google coral, edge tpu, facial recognition, ai accelerator card, asus ai accelerator, graphics card ai, frigate nvr, double take, unifi cameras

Id: B635wcdr6-w

Length: 15min 24sec (924 seconds)

Published: Tue Oct 05 2021

Reddit Comments

[u/jptuomi]

Getting attention from LTT is big, we already knew it was big - anyhow congrats /u/blackbear85!!

BTW Linus just always has to go big doesn't he? 16 friggin TPUs, NOICE!

[u/idontknowwhattouse33]

Who bets we see this monstrosity + a Pi CM4 in 9-12 months time.. /s

[u/Vertigo722]

He is going to be disappointed when frigate stops detecting him when he falls asleep on the couch.

For non frigate users: as of now frigate will only detect objects/people when they move. So its really not a whole lot better than PIR motion sensors for this stuff, it wont be able to tell you if a room is empty or if someone is sitting still.

[u/SuperLuks_]

god damnit as if these weren't scarce enough already.

solid video though.

[u/WhozURMommy]

I have Frigate and LOVE IT. I tried Blue Iris and it just felt bloated and slow to me. Frigate on the other hand is crazy performant. I'm still a relative HomeAssistant and Docker newbie, but I found it to pretty easy to setup. And it's by far my favorite HA automation. Here's what I have setup:

I have two Amcrest 4K cameras setup on my driveway and front porch, feeding a old PC I bought off Ebay running Frigate within a Docker container. I bought a Coral AI USB dongle to add to the mix. You don't need any of this crazy stuff Linus was showing off. It cost ~$60.00

When someone enters my front yard or driveway Frigate immediately detects it. Using NodeRed I kick off an automation that lights up my cool steampunk light as Alexa tell me "A person was detected in the driveway"

[u/EuphoricAbigail]

I didn't know about these facial recognition apps. Definitely very cool, am more inclined to try the USB HW acceleration device rather than the card due to the price though. Need to do some research to see how much that is actually capable of.

[u/stingbot]

Remind me in 2023 when it looks like we might get these in Australia :( price might have come down from 4 figures by then.

https://www.hiome.com

Seems like that is a pretty common sense way of determining if a room is occupied.

[u/Saars]

I have thought about doing this, but putting cameras in kids room, especially girls, is a massive moral query... even if you assure everyone that you don't have access to view them

[u/darkamikaze]

Wow, I didn't know how much more EdgeTPUs the PCI variety offers compared to a USB. Pretty cool Linus is doing vids about these though.