AI in less than 100 mW! Power efficient tech at CES 2024

I'm at CES 2024 and I'm on the hunt for products using arm well it's not that hard actually so much runs on arm these days from Tiny iot devices up to massive Cloud servers running AI models I interviewed experts from the automotive industry to an edge AI chip manufacture all the way to a company that makes flexible solar cells for headphones that you don't need to plug in to charge I'm going through all that and more in this video and thanks to arm for sponsoring this video this is the first time I've ever done a full sponsored video I really want to make that clear since you deserve 100% transparency sponsored or not CES has a ton of Great Tech and I got to dive into the things behind the scenes for all the fancy TVs self-driving cars and Even pinball machines if you look under the cover chances are you'll find arm so let's Jump Right In and talk Automotive cars were everywhere on the show floors from this MC Mery Hill Climb car to a fully autonomous race car to this flying car and even a tank every vehicle on the road today runs on software and immed immediately when I was passing by the automotive grade Linux Booth my Raspberry Pi detector went off these demos aren't as Cutting Edge as the flexible solar panels that keep headphones charged forever but it's cool to see how the same code on their Pi demo units runs on real Vehicles arm was everywhere I even found this Pi 400 tucked behind a sign and I always love seeing companies like aalia supporting open source projects like they do with chromium and Vulcan that's one end of the spectrum on the other end Greenhill software has been building Automotive software for over 40 years longer than I've been alive they've been involved in the evolution of the softwar defined vehicle but what does that term even mean so I'm I'm Joe Faber I'm the global technology director for Greenhill software one of the things that I think is interesting is like you have a different perspective on software defin Vehicles how how has that evolved out of you know whatever it was before which was a lot more hardware and now it's Consolidated and there's a lot more software well I think um software defined vehicles at least what it means to me is the ability to add features after the vehicle's been you know comes off the production line so your vehicle can grow and you can have new exciting features for the OM I think it's important because they often are trying to monetize those features but it's also bringing a lot of convenience for the consumers a lot of people love the fact that their car can get an update and have some new feature on it so I know for me and a lot of people who watch my channel we're we're all about the security side too and we're a little nervous when your when your car can get an upgrade over the air so how do you guys make sure that the the updates are rock solid and reliable and all that that that's a pretty deep question actually there's a there's a lot of moving parts to that question um but Green Hills has been focused on Safety and Security since we really introduced our integrity realtime operating system in the mid90s it was sort of the the driving goal of integrity was to address those challenges with embedded software and so what it comes down to is having an operating system that can safely separate critical functionality from non-critical functionality we have integrity multiv visor it's in lots of of oems today deployed on the road hosting Android virtual machines like you see in the the demo behind you um and that Android you know can be connected to an app Store can take updates and all that stuff is isolated and protected by Integrity underneath um keeping access to the can bus the vehicle networks safe and secure what does arm give you in terms of like why why do so many people seem to use Arm processing there's I think it's largely the um performance per watt that that arm is able to deliver um and the scalability so you know arm offers from cortex m0 cores all the way up to Cortex you know a76 AE and I'm sure Beyond maybe we can't talk about those yet even the server grade course do neoverse and all that yes exactly and that's an area where I think arm and Green Hills complement each other well because Green Hills offers compilers debuggers hypervisors including for things things like the the cortex r52 you know that's an Rass processor that added virtualization extensions we're seeing that in a lot of Zone controllers like this s32 Z from nxp and there's an sr6 over there from St um and we've built a hypervisor so it's it's that scalability and you know again Green Hills an arm all the way up the product line we have complimentary Technologies to address functional Safety and Security things have changed and evolved quite a bit in the automotive space one of the changes is that traditionally the ecus were the hardware and the software was fully developed by a tier one supplier and then hand it to an oem and they put all the pieces together and built the car a lot of the oems want to own more of that software experience and software to finded vehicle and be able to update it over time and so there's been a shift in where is that software happening um that's been an interesting thing that that we've been dealing with and as a result we have a lot more direct relationships with oems these days than we used to another company that's had a deep relationship with car manufacturers is Amazon I dug deeper into the idea of softwar defined vehicles with Amazon's head of Automotive Stefano marzani traditionally Automotive worked uh for many many years like uh in electric electronic architecture of the vehicles themselves they have a lot of ecus electronic control unit with a lot of different processors different operating system and uh the development was really starting with having one of these boxes to develop four right so and uh but that uh do you think that enables the community of developers uh big one to develop software no it doesn't so consolidating uh that is happening right now in the sector luckily and we are very happy about it I keep hearing this word so ay or so Sophie or how do you how do you say that and what does that mean Sophie Sophie yeah it's a Sophie service oriented architecture for embedded Edge and it's an initiative have U arm and other including AWS launched uh couple of years ago because uh really how do you achieve software defined Vehicles yeah you need to Define some really basic elements right and the main objective for shopi was to challenge a problem that we again uh we call environmental parity what does that mean it means uh you develop software wherever it will just work wherever so just respect the instruction set architecture and that's why we collaborated with arm a lot in the past couple of years Stefano took me on a tour of a modern developer workflow showing me how devs can program virtual dashboards on their workstations and in the cloud and then deploy that code to real Hardware I even spotted one of those adlink developer workstations like the 128 core Beast I'm testing here but the key is being able to abstract the hardware in a modern vehicle so before you need to have the hardware and how many Hardware you you can have in Hardware in the loop 100 500 but you can't have millions we can absolutely have tens of thousands of virtual ecus running in cloud and perform all kind of weird stuff in there fault injections uh mutation permutation of conditions uh attacks right uh because uh it's just a virtual ECU but it's a realistic one in fact if you look at the demo each of these virtual ECU is assigned a VIN like a real card so it's there's an extreme level of fidelity that you can leverage to boost security studies so that's that's pretty exciting front so if you your target is arm on the car you need to use an arm processor in the cloud because it's the same instruction set I dug into Amazon's graviton processors a little deeper I really wanted to know why did Amazon build their own silicon for their data centers our data centers are big so and they consume a lot of energy it was very important since we have several objective on sustainability to reduce the energy amount and the arm architecture is fantastic for that so it really helped us to reduce significantly the amount of energy that is used in the data centers and it's an efficient instruction set so arm silicon can perform and it's efficient but you really need a solid OS for your car too one great option is Ubuntu core and I asked JC from canonical about it so I'm uh JC for I'm from France and I work with canol as the head of the field engineering team for EMA yugun is a product that has been designed and engineered by canol 8 years ago back in 2016 and it was uh out of the fun project that has been uh abandoned and we took back from uh from this a few technologies that we sold made sense in the world of NBD system one of them was making an immutable system so it's a ronly system fully secure that you can only update through OTA with a sign system and packages so we use the snap package that have been designed for the phone and that has been now deployed in the desktop and every kind of environment but we went further and uh the Snaps are strictly confined in ubun which means that uh unlike on the desktop where you can do everything except what you prevent the system to do on you to car you cannot do anything you have to open up connection you have to allow the snap to access the network to access the screen uh you have to allow snaps to communicate each other which make it even more secure I spotted another m devkit board stashed in their Booth but I wanted to ask how does canonical certify their software on so many different arm boards even including the Raspberry Pi I saw on the website that there are certified systems like it will work with this how does that work do you have like a Testing Lab do you work with the vendors directly how how do you get those things certified so the approach of of canonic is to uh to provide a solution to the entire food chain if I may say so we have customers coming to us with uh their their their Hardware already designed their solution already packag and then we will help them to have to get the best out of Ubuntu so we will optimize the kernel for our platform we will integrate whatever drivers they need we this is what we call enablement and we also discuss with odm and odm as you know is a designer platform based on the CPU and we work with them to have a pre-certified Ubuntu available Ubuntu and Ubuntu core yeah of them available on the platform so that people who are not yet decided on using a platform they can just pick up a random odm and get something out for them I also spotted this pie running one of the demos and asked how canonical sees pie in their iot efforts the beauty of Raspberry Pi is that well it changed the way people are are thinking about embedded systems are thinking about home appliances and a lot of things there there are there have been so many very surprising things done because of it's a great platform to hack on and to bring you up into the EOS system but arm wasn't just in cars all over the floor I spotted arm powered devices it makes sense I mean 70% of the world's population uses products powered by arm and at CES I spotted devices like the one court that uses a Raspberry Pi and 5G to deliver realtime Braille updates to make it so people who can't see can watch a football game I also spotted an arm processor on this smart radar drone that lets it process radar data in real time and everywhere you looked people were slapping AI in front of the word to make it sound impressive but one company that was doing real AI work was himx and they partnered up with Seed Studio to build a powerful second gen Grove Vision module powered by their highx Wii 2 AI processor we actually have two uh different generation the second gen we use Arm so we have three cores so one that's uh M50 y one on the 150 MHz one is 400 MERS the other one we have dedicate mpu so that's the called u55 so that has a beauty beauty of our architecture so we can really uh run in very uh low computing power with a 150 mahz one and we uh use the AI whenever we need okay so you can save power and turn off one of the C need all that another feature that was interesting I saw the physical unclonable security functionality how how does that work cuz it is important if we're using these for security purposes and things that you don't want people tampering with the chips yeah how does that work what's what's the import there exactly so especially for AI the iot uh the security will be primary uh consideration for the chip so we have like a a puff like a IP security IP that's typically just like a security engine bu into our chip so you can consider as a fingerprint in the chip and what kind of battery life could you expect could you run it off of solar power those kind of things well yeah that's because highx wize solution we really focus on AI very power powerful AI running on the very ultra low power scenario so most of our consideration will be Ai and low power so like is the thing right here the skeleton this is running like a four time AI only like a double digit MW so less than 100 MW so normally our aage time like this we have this guy right here a uh p security camera this have a three years life spend by using our uh sensor and our Air processor so typically like around like 2.6 M uh power consumption in average that would be the goal we we looking for like we kind of greedy we want to try to uh reduce uh down to like 2 m so we working on that so you could have this up put it up for 3 years you don't have to go out there on a ladder every day batter you n soda hax's aitech can run for years on just a battery but what if you could run it practically Forever Without ever changing the battery that may sound like magic but that's exactly what exiter is doing with power foil putting this Tech in wireless headphones and headsets remote controls and even iot buttons that always stay topped up from light energy but how well arms efficiency is one of the big reasons this is possible today how does that work with solar like you still have to plug it in right no I would say for most users you don't actually have to plug it in there were going to be some extreme edge cases where maybe yes but uh it really becomes a duty cycle issue if you use them outside for instance you gain more power than you consume but even if you use them in low light conditions if you store them in the window or you have them around your neck when you're commuting to work there's enough light to to power them so I would say most users no they don't have to charge them and another problem that I think of is like solar panels on roofs they're kind of fragile glass and all that like how did you make it so that you can have a solar cell in a headphone or in some consumer product where people are just throwing them around put them in their bags maybe stepping on them no but you're right I mean solar cells as as a te is been around for a long time what we did and that has been to develop a a flexible solar cell that's also very durable so it can take you know a fair amount of mechanical beding and temperature variations UV exposure so you really make a a a durable product and we're not trying to compete with you know silicon solar cells for power generation this is really for product integration to replace batteries and charging on on products the other part that I think of like solar is great and you can get some energy and the solar efficiency is getting better and better and you guys seem to have done a good job but there's the whole back end of how do you use that power effectively and that's where I think arm comes in yes can you explain why why do a lot of the devices that do this use Arm yeah know I think you're entirely right I mean so we have a fantastic solar cell but there are limitations right I mean we can only generate so much energy and to make a good product that energy has to be used efficiently and that start starts with processing and communication so we only make the solar cell but we try to partner with companies and build referen in signs and solutions to make energy efficient products and then it becomes quite natural to work with arm because arm has maybe or probably the best power efficiency in semiconductors I know as a programmer I always I'm I'm jealous I'm always guilty of putting in things where it's like I can do that and you end up like you have a loop that's just consuming energy yes yes and yeah the hardware is one thing but you're absolutely right I mean energy efficient coding to make sure that you go down into dip slip when you soon as you can and you know for a tracking device for instance how often do you wake up to send data do you wake up on a time interval or do you work up on a low power you know accelerometer or something right so yes you should think of power efficiency on a systems level uh to build great products and I think that's one of the great things with these Dev kits is it's a low threshold to start working on that it's awesome to see how many new use cases are uncovered just by cramming more performance into smaller and smaller power envelopes a pinball machine plugged into the wall running on armcor doesn't really need that same efficiency but a wireless AI security camera that only needs a battery change every few years arms all over that and I wanted to find out more about where arms headed so I got to speak to Chris Bergie the SVP of arms client line of business when I started out working in in you know small electronics and things it seemed like arm was small you know low-end small devices TV boxes and phones and things but nowadays I see it in servers Automotive everywhere every kind of device where is arm the best well so you know our a lot of our Legacy it does come from Energy Efficiency right and you know some of our earliest um company defining designs were earlier early Nokia phones or nintend Nintendo game boys and and these kind of mobile devices so really efficient risk design is fundamental to the company um however we've seen those types of products and what kind of a smartphone can do today and the types of performance that is required power is becoming a challenge in all kinds of applications in fact even you take a data center right which you think about a data center why do they care about power well actually one of the biggest challenges especially as we get to Cloud scale data centers is how much power can they deliver and get to that data center that huge Warehouse side data center you know 500 megawatt type Data Centers of of course power is super expensive and we all want to be you know good consumers of power and so once you have that 500 megawatt data center how much compute can you get in there so it ends up being a well we want really efficient Computing and you can see that with many of our neoverse customers that have taken our arm processors to data centers great example and I know you do a lot with uh ampere being able to scale that out to 128 256 cores that's more jobs that can be servicing you know providing services to people providing Computing Services so we see it in in data center in automotive right we have a night nice picture of a car here I'm not sure if it's in frame but uh in automotive electric vehicles and the amount of computing and especially as we talk about all the AI and all the autonomy so really efficient compute is just a fundamental requirement in today's devices and then you know we can go from those big Computing devices we talked about to super small products now obviously arm is bragging a bit but it's not a lie having an architecture scale from the tiniest devices to massive servers is no small feet and that makes the arm ecosystem easy to tap into we have 15 million developers at this point in time developing on arm um relative to Applications I think so many of them are your users you know your listeners whether they're using Raspberry Pi or other types of products so it so it's both the technology scale that we're able to provide with these volume high volume markets and be a you know take that architecture and scale it up and down and then on the software side the huge scalability you get um from osses to Linux to you know all kinds of application development like what does arm give to continue advancing the industry in terms of performance and efficiency and everything so it's a great question Jee um so yes uh you know lots of discussion as Motors law dead or whatever we are absolutely continuing to push the envelope on technology but we're absolutely having to be a little bit more um thoughtful not just counting on lithography or the Fab to just make things better for us so there's a lot of work that's being done now around looking at those Next Generation Technologies and figuring out okay what types of decisions you might want to make what type of caching decisions you might want to make what kind of scale decisions you might want to make and how do you want to deal with that we also discussed chiplets and mixing multiple types of cores and Chris highlighted how arm pioneered the big little architecture behind some of the most efficient and flexible processors today arm was running behind the scenes all over the place at CES and I'd like to thank them for sponsoring this video and for sending me to Vegas for my first CES experience until next time I'm Jeff Garling

Channel: Jeff Geerling

Views: 116,806

Keywords: ces, arm, ces2024, advanced, risc, machine, acorn, history, efficiency, power, compute, computing, datacenter, embedded, powerfoyle, solar, efficient, raspberry pi, cpu, gpu, npu, neural, ai, battery, exeger, green hills, software, hardware, design, canonical, ubuntu, core, odm, oem, automotive, car, himax, seeed, studio, grove, vision, module, v2, microchip, development, aws, graviton, cloud, security, connected, interview, chris bergey, vegas

Id: IIk8uyBj76Q

Length: 21min 22sec (1282 seconds)

Published: Fri Jan 26 2024