Apple’s M1 chip with the neural engine — what is it, and why is it so disruptive?

on november 10 2020 apple rocked the computing world when they revealed that their engineers had designed an entirely new cpu chipset based on the risk arm architecture not only is the cpu itself incredibly powerful and energy efficient it has an entire neural engine built in that could revolutionize how ai and machine learning work let's find out more [Music] hey y'all it's doctor know-it-all i have wanted to do this episode for quite a while now but i wanted to really take the time to understand apple's new cpu architecture before doing an episode as i said in the intro the m1 is based on the arm risk or reduced instruction set computing architecture that has had massive success in everything from embedded car processors to most of the smartphones that people have today but arm-based silicone has never powered a desktop or laptop class machine before and no personal computer architecture has ever had an npu or a neural processing unit built in and you can see my episode on npus versus gpus up there if you want to three things are going on simultaneously with the apple m1 that is causing such a stir number one apple is attempting to break from intel and the x86 architecture hegemony which is highly disruptive and in my opinion a good thing 2. apple is disrupting our expectations of laptop battery life arm-based chips sip power compared to sisk intel chips which are complex instructions at computing chips and number three apple is disrupting the likes of nvidia and even google by bringing a neural processing engine on board in a consumer class computing device will this be powerful enough and have enough resources to do anything useful i'm really curious to find out let's talk about these three areas of disruption especially the neural engine in just a moment but first if you enjoy this episode definitely like it so other people can find it because that's how youtube works and of course subscribe for more of these things i will be doing another episode comparing the tesla inference chip to the apple m1 chip so i think you'll find that really interesting so definitely subscribe if you want to see that also a big shout out to my patrons on patreon you guys are awesome it's been really fun having discord conversations and you know just i'm looking at doing some t-shirt designs and so forth and i've been getting feedback so it's been really cool so thank you i especially want to give a shout out to my new patreon patrons armand vervek i hope i said that right jeffrey l carver bear trump ander and will depew thank you all so much for your help also a quick thank you to zenli music for doing the intro and conclusion music if you want to check him out definitely look up his link in the description and of course if you're in the market for a new tesla you can use our referral link below if you purchase a car through that link you get 1 000 free supercharger miles and so do we let's look at apple's attempt to break intel and the sisk based x86 hegemony first that is a lot of acronyms this is not the first time in fact apple used motorola chips they were the 68 000 series chips starting with their lisa and then their macintosh computers in 1983 and 1984 respectively ibm of course went with intel and their 8086 architecture and thus a long-standing rivalry between apple and ibm intel was born and actually this isn't the first time that apple has tried to go with risk over sisk processors in the 1990s apple and ibm teamed up to create the risk architecture and apple based their computers on the powerpc cpu chipset at the time for a while this chipset and i remember this it actually outperformed intel's x86 computers but intel caught up and surpassed apple and ibm mostly simply due to the scale and manufacturing ability and and really focus that they had in the 90s thus in 2001 steve jobs basically had to eat his hat and announce that mac was going all in on intel and since then macs have been based on intel cisc processors three side notes here number one jobs asked paul adelini the ceo of intel at the time to make chips for their iphones which were released in 2007. adelini rejected the idea thinking they could never sell enough iphones to justify the development expense it's like oh what a mistake thus apple went with off-the-shelf arm chips from arm holdings uk and eventually licensed the architecture so they could build their a123 etc chips that still power iphones this one's got an a14 inside it this was a great move as arm is so crazily power efficient compared to x86 chips second side note apple with tsmc has gotten five nanometer production working with their m1 chips which is insanely small intel is having real problems hitting this die size and samsung actually might have to step in and manufacture some of these m1s in the future as well because apple is requiring so many of these things to be built and note number three what is sisk versus risk just a super quick intro here uh you can check some linked articles below for much more it's all in the name risk is reduced or simple instructions while sisk is complex instructions this basically means how many commands are in one high level instruction so risk only has a handful of instructions while cisc has many times that so sisk is great if you have to do something really funky it has you know basically like an iphone or an android it has an app for that and it can do those things pretty efficiently but those complex instructions are hardly ever used and it makes the chips nastily complex and really big and power hungry and hot so risk on the other hand only has a handful of instructions which is detrimental if you have to do a particular complex wonky instruction it can take many more steps but most instructions that happen for cpus are simple things so most of the time it's more efficient than cis and the architecture is much cleaner and simpler in other words smaller and it doesn't eat power or get hot like a sisk chip does for laptops or the mac mini or a phone or an ipad or something like that especially one can see that apple going with a full bore risk chip has major advantages which is exactly what we're seeing in all the tests that have come out recently it's got great performance and very low power usage compared to intel and while this is first generation it's based very closely on the iphone bionic chips of which they have had numerous generations so this architecture is actually well understood and designed by now so point number one that apple wants to disrupt the intel x86 hegemony is complemented by point number two that apple wants to improve their laptop's battery life drastically similar to how tesla has built their own chips for self-driving making the m1 is crazy expensive but you get massive market advantage if it does work after all having all of the silicone in one place sharing memory or an soc or silicon on chip is hugely efficient both for processing speed and for power efficiency integrating all of the traditionally separate parts of a motherboard into one chip is a huge advantage you can see the mac rumors link down below if you want to read more about that so what you get an effect is battery life that no pc or intel based laptop can match you also of course get extremely good risk based computational performance under normal circumstances and you get a whole bunch of marketing buzz which is an added bonus of course there's a lot of difficulty doing this not the least of which is software they have to build a whole new version of xcode and actually big sur their new operating system is designed to work with both intel and the m1 architecture so basically you have to build a development environment that can compile effectively to both x86 and m1 at the same time and their laptops and desktops are now split between architectures which is really inefficient for a period of time but i expect apple to move their imax to a new m2 chip you know m2 something like that m3 who knows chip in a year or so at which point i will likely upgrade i'm not so sure about their pro desktop lineup now but it could eventually go that way too in fact actually the more i read about it the more i think it's actually likely it will as a weird extra bonus m1 based laptops can now natively run iphone and ipad apps which is actually kind of cool so you can download things from the app store for an iphone and run it on your computer and of course as more software gets developed for the m1 architecture specifically we will start to see its real potential like you know become more apparent so apple is disrupting intel and apple is disrupting battery life expectations for laptops and as a bonus apple is disrupting integrated graphics the m1 destroys intel chipsets with integrated graphics in them the only real way to beat an m1 chip graphically is to have a dedicated gpu which is both expensive and also extremely power hungry which you know of course really matters if you're on a laptop and you can see the mac world link below if you want to read more about that all right but what about point number three what on earth is a neural engine doing in a consumer-oriented product i had to ask this one myself when i heard about it in fact i honestly didn't even believe it when i was told about it i had to go look it up for myself so here's what apple claims 16 cores 11 teraflops of performance machine learning accelerators good for video analysis voice recognition similar to the a14 bionic chips that has voice recognition for acceleration for siri for example also image processing and they say it's great for machine learning but this is really kind of muddying the waters it doesn't appear to be the best suited for training models though it might be but rather it's good at running models that have been trained previously this is still super important but the wording makes it sound like these chips are ideal for training models which again they might be but i'm going to get into reasons why i don't think that's ideal yet but i believe the primary focus of these is on running models efficiently this is what medium link below terms quote edge computing or running models on low power processors in real time as opposed to feeding data to big centralized computers to get a result as is traditional quoting from medium quote a lot of m1's efficiency in ai computing owes to the neural engine a type of npu or neural processing unit unlike a cpu or gpu this unit is focused on accelerating neural network operations like matrix math you've probably heard of another famous npu out there google's tpu or tensor processing unit end quote so again you can check out my video on cpu gpu tpu and npu in the link if you want to by the way does this sound familiar to anybody i am definitely going to do another video on how the m1 and tesla's inference chips stack up against each other of note the iphone 12 has the a14 bionic chip which also has a neural engine and upon which the m1 is actually built again quoting medium quote instead the iphone 12 has what's called the a14 bionic chip an 11.8 billion transistor powerhouse that has a fast neural engine a new image signal processor and 70 faster machine learning accelerators end quote so let's think about a traditional use case of a machine learning model responding to a voice activated question something like hey siri what's the oldest known human artifact when someone asks this the voice data would be pre-processed and then uploaded to the cloud a massive computing cluster with a really big speech analysis model would then decode that and possibly even do the legwork of looking up the results online then either the interpreted command or the web results would be sent back to your phone this is of course time consuming and it breaks if the phone is not connected to the internet for voice recognition and i know you can't get web results but you could get the answer to something like hey siri what is the square root of 12 times 2 to the x when x is 27.5 and by the way the answer is about 47 726 it answered for me so what happens with these new chips the a14 chip inside an iphone 12 can process a natural language model and do the analysis itself and thus the phone itself can determine what a user is asking and act upon it this is edge computing as we talked about when a model can be processed at the edge of the network in other words on your phone or laptop rather than at a central compute cluster combining the neural engine with an 8 core built-in gpu with 25 000 simultaneous threads the m1 can really crank through large and complex machine learning models and and there's some evidence that it can do training too for example here's a quote from volico quote results show that an m1 powered macbook pro with 8 core gpu 8 core cpu and a 16 core neural engine had better performance as far as ml capabilities go on tensorflow than a 1.7 i assume gigahertz quad-core intel core i7 based macbook pro proving that its new version is better than the previous one is to be expected but the real eye opener came when the m1 powered macbook managed to outperform even the best versions of mac pro systems powered by intel end quote this is all great but much of machine learning depends on massive memory and massive memory throughput you know sure you can train small projects on a laptop with 16 gigabytes of ram but that's not really adequate for anything big or cutting edge right now and laptops and mac minis are severely memory constrained so sure they can do some actual machine learning training but they are going to be best suited to running complex models rapidly and using minimal power while doing so and that's fine being able to efficiently process the ever more complex machine learning models that are being put out into the world is a laudable goal and something to get really excited about i just wish they didn't make it sound like you could train gpt 3 on a mac mini or something that's just not going to happen without some crazy add-ons now when we get a true desktop class m2 chip then let's revisit how useful an apple mac pro m2 will be i'll probably be the first person in line to buy one okay i hope you found this episode interesting and informative if you did definitely make sure you like and subscribe and by all means ask me questions in the comments or at my email address which is doctor know it all knows gmail.com until next time bye [Music] you

Channel: Dr. Know-it-all Knows it all

Views: 62,312

Keywords: dr know it all, dr know-it-all, Telsa, elon musk, autonomous driving, dojo, supercomputer, npu, gpu, tpu, artificial intelligence, trajectory prediction, cpu, difference between computer chips, apple, apple computer, m1, apple m1, apple m1 chip, neural engine, apple neural engine, what is the neural engine, why does apple m1 have neural engine, machine learning, ai, gpt-3, model, models, neural net model, neural net, neural network, why is m1 better, inference engine, inference chip

Id: 5dAuxkKix4w

Length: 14min 33sec (873 seconds)

Published: Fri Dec 11 2020