The World’s First WoW Processor explained

hi friends my name is anastasia and i work in chip design on this channel i cover the most exciting processors out there and today i have something really special for you some really exciting processor came out just yesterday graphcore just announced their new ai accelerator so-called bow processor the thing is this processor fabricated in seven nanometer technology node by tsmc basically the same node as the previous generation of the ipu however it contains some interesting and special tweaks that enable 40 performance improvement at 16 percent lower power consumption let's find out where the magic comes from graphcore has previously released two generations of ipu's intelligent processing unit the last one being in late 2020 and now they've just announced their new bow processor and this chip is the first in the world to use wafer and wafer technology basically is taking one silicon wafer on top of another you know i talked a lot about chiplets and our 3g future of processors but this one is actually the next level so basically it's taking not chips but the whole wafers this technology is actually wow technology is called waveform wave technology it is a special packaging developed in the cooperation with tsmc you know it is very famous taiwanese semiconductor fab which is i would say a couple of steps ahead of the rest of the world in the advanced vertical packaging so this chip we call it the bow ipu it has a logic wafer seven nanometers and we bond to it a second wafer um which actually made with a um a process that's good for dram cells so it's a completely different process technology we take two wafers of the same size bomb them together and the type of bonding is called hybrid bonding and bonding two wafers like this allows for a very very fine pitch for those bonds so initially in the bow ipu we use that fine pitch to achieve a low power supply impedance between the capacitor wafer and the logic wafer have a look here the power comes through chessv so through silicon vice the top wafer also integrates high density deep range capacitors which helps to speed up the clock and why is that that's because in this case the dynamic charge doesn't have to travel all the way from the actual power supply but it comes from high density capacitors which sits on the top of ai cores so right on the top of the circuit meaning it's just right where it's needed and this allowed to push the clock speed from 1.3 something to 1.8 gigs and that's cool this is much more shorter and more efficient way to deliver power right and power is a big deal for ai accelerators actually just by this new wafer stacking graphcore achieved 40 better performance at lower power consumption and this is for the same architecture and the same seven nanometer process note and this is truly amazing this means every processor now delivers 350 teraflops of compute anyway this way first taking technology has many more advantages than just with respect to power delivery but in future um the technology will certainly be used to achieve a very high bandwidth of signal interconnect between wafers and by high bandwidth so there are two advantages one is obviously you you now have an area over which you can send signals instead of just a beachfront around the edge and the second is that the fineness of pitch of the signals you can send is much finer than it is breaking out at the edge so for example much finer than the traces of a high bandwidth memory interface the technology if used for signals is certainly capable of passing tens or even hundreds of terabytes per second of information between wafers vertically and there's no technology that would allow you to do that between ships horizontally new boat ships are currently shipping to the customers including us department of energy where it will be used for applications like cyber security and computational chemistry graph core processors will help researchers to reduce time to train ai models from days of training to hours of training similar to the previous generation graphcore's bow ipos will be available in a four ipu configurations which is capable of 1.4 beta flops both parts can scale from 16 to 1024g chips for instance superscale bow pod 1034 packs 350 beta flops of ai compute which will allow to train huge ai models and make new breakthroughs in ei so what comes next graphcore is already working on the next generation of ipu they want to build a brain scale good computer and that's cool it is called after a computer science pioneer jack good who proposes an ultra-intelligent machine basically a computer which has more intelligence than the human's brain but we have teased this good computer as we call it named after a very powerful ai machine that most of our engineers are now working on that will deliver in two years time that does include a new generation of ipu's that new generation does include architectural update and we've also said that it includes taking the 3d wafer and wafer technology further by by stacking logic wafers on top of each other so so those are the things we're public about graphcore plans to build a 10 exa flops system with 8192 graph core chips this one will support ai models with up to 500 trillion parameters which actually exceeds the parametric capacity of the brain the expected cost of this computer is 120 million dollars so i'm really looking forward to see it so how do you benchmark your new ipu uh to the cerebros vapor scale engine too well well we don't um so we don't spend any time at all really benchmarking ourselves against other startups um i have great respect for some of the engineering that sarah brass have managed to pull off and also some of the other players like san benova and they will all have different strengths and weaknesses and i think ai is going to be broad enough as a well it's going to affect all computing so it'll certainly be broad enough to support multiple architectures for quite some time when we're trying to judge how good our technology is uh i mean really we judge it against the default incumbent which is nvidia and and also against other big players who participate for example in ml perf so intel are there with the havana technology and google are there with their tpus and those results are public and we do pretty well uh in our latest results we've we've managed to outperform in video with our chips which is not bad um and i think one of the things that's very interesting about these architectures is the question of how they're going to scale because the the shift to ever bigger models in ais is continuing unabated and i think you can sort of you can map out that there have sort of been several steps of growth of models the first was the breakout of deep learning suddenly models of maybe 100 million parameters were useful and and learnable but they were limited by how much supervised data you had so then there's the emergence of things like burton gpt that could train without supervised data and suddenly you can make your models bigger because all the data's cheap read the internet or something like that and so you saw model sizes jump from hundreds of millions to hundreds of billions now what stops them from going to hundreds of trillions which is brain size the answer so far has been the amount of compute necessary because as the models scale the amount of computers scaled but actually that's not the way your brain works when you see something not all of your neurons fire your brain has the ability to steer the information to the relevant parts of your brain to affect the relevant synapses now this property is now being harnessed in artificial neural networks with uh structures like sparsely gated mixture experts and g shard and things like that and switch transformers and what this means is that we can now increase the model size further so we can build more intelligent ai if you like without having to increase the compute because the computer's already very expensive from a machine point of view this means that suddenly you need a really really big memory system that you can access quite quickly so so by really big i mean if you want something that holds a similar number of parameters to your brain like 100 trillion you're going to need something with maybe a petabyte of memory and many of the companies have recognized that they're sort of thinking okay how do we build a machine with a petabyte of memory but not all of them i think have got a credible architectural approach to it there is no question that the future is going to be 3d and performance burst which graphcore was able to achieve just by switching to this new tsmc sosc wow waveform wave first tekken technology is truly impressive i'm really excited to see more updates on graphcore processors we should definitely keep an eye on them now this wavefront wafers taken technology proven to be a good working solution so i think more and more companies are going to consider using this technology i think maybe cerebrus should give the try to stack a wafer on top of a wafer and then just cut off the sides i hope you guys enjoyed this video now you may like to watch another video on my channel where i talk about one of the main graph core competitors cerebros i will link it here thank you for watching and see you in my next episode ciao [Music] uh [Music]

Channel: Anastasi In Tech

Views: 135,799

Keywords: graphcore, bow processor, wow stacking, wow, wafer on wafer, tsmc wafer on wafer, graphcore IPU, AI processor, AI accelerator, anastasi in tehc, Simon Knowles, Graphcore processor, bow, good computer, new ai, ai, world of warcraft

Id: -NeRIrRSFs4

Length: 11min 54sec (714 seconds)

Published: Fri Mar 04 2022