Brain-Like (Neuromorphic) Computing - Computerphile

we're going to talk about actually this neuromorphic nanoelectronic materials which is a hell of a mouthful so you got in touch on i don't know about a month ago or so i've lost all track of time under these circumstances you said it's a while since we did something in the links between nano and computing i said it has it is indeed and then actually just before that i've been reading this month so at the time that month's issue of nature nanotechnology which was one of the key nanotechnology journals and this is a review paper in that there's a massive push to integrate nano electronics with computing of course to push down the limited computing but to actually integrate different ways of processing information that go far beyond what's known as the von neumann architecture i think you've done the von neumann architecture before on computer file i think dave did a video on it so i'm sure sean will put the link in somewhere the key bottleneck when it comes to von neumann is that we've got a central processing unit and we've got memory the bottom neck is that we've got a transfer between those two units that's not what our brain does our brain doesn't have a cpu over here and memory over here and transfer between the two and that transfer is not only limits bandwidth it also limits the ability to train these systems and although there's a vast amount of work in machine learning countless computer video computer file videos on machine learning i think we've even done one in the past shown um on on machine learning but it seems a little bit strange in that what we're doing is we're taking the machine learning idea artificial neural nets and we're imposing it on the von neumann architecture which is very different from how our brains or work how our brains work so then the question is well can we change the substrate can we change the architecture so we get something which is a bit more brain-like the other key issue with von neumann is it's incredibly energy hungry in this paper there's a study that's quoted that says by 2040 we're going to need 10 to the 27 10 to the power 27 jewels to continue on doing cmos logic as we currently do that's a real problem because that's more than the energy budget of the entire world at the moment but um surely the energy's efficiency is driving down as well in cmos right the energy efficiency is getting you know sorry i meant to say yes yeah it's getting better yeah so the energy it's it yes so we're making improvements but we've still got this key bottleneck and key latency that's the other thing there will always be an inherent latency with the von neumann architecture because you've got to transfer information between these two so the key thing is why can't we co-locate why can't we have the memory and the processing in the same place because that's what our brain does and moreover this is incredibly energy efficient on no more than i don't know a banana and a cup of coffee and sean got me a cup of coffee this morning so i'm fairly fired up at the moment and no more than that i can solve problems in terms of face recognition and in terms of identification of different people or different objects that it takes countless cpu hours to train a machine learning algorithm to do so in many ways we can beat a computer and the key issue the key reason for that is because we've got this different architecture in our heads one comment that always comes up is why didn't he discuss this why didn't he discuss this why didn't he discuss this it's a five or ten minute video we can't discuss everything and i just want to try and get how large this field is a survey of neuromorphic computing and neural networks and hardware 2 682 references the reference list is much much longer than the paper itself so this is a huge field i can only scratch the surface hopefully if computer via viewers are interested in this we can do maybe more videos on this because it is it really is a fascinating area anyway where will we start so one thing that comes up time and time again is this word memristor memorists are synapses for neuromorphic computing nanoscale memorists are devices synapse and neuromorphic systems review of memories to devices in neuromorphic computing memorists with diffusive dynamics of synaptic emulators from neuromorphic computing etc etc it comes up time and time again so we've got two words there synapse and memristor how does the sort of processing unit in here work how does the brain work how does any brain work how does any biological brain work well you have neurons and you have synapses and synapses are the gaps between neurons and those neurons communicate with each other via well how do they do that they do that via what are called neurotransmitters and you you program these you set the state is probably the best way if we're going to talk about computing language you set the state of those neurons by controlling the ion flow so in a cell you've got ions inside you've got ions outside what's an ion an ion is a charged atom so you've got potassium you've got chloride you've got calcium got a range of different ions and a neuron is really and certainly a synapse is really an electrical device so what it's doing it's it's we're controlling electrical current due to those ions the charged particles we're controlling how they flow therefore we have an electrical device that's how a computer works what do we do in a computer we control where the electrons go we control where the charge goes so the question is why can't we take this architecture and put it into the solid state and that's exactly what neuromorphic computing is all about the question is now can we develop an artificial synapse and to develop an artificial synapse we need an electrical device that's got a memory effectively that instead of just we pass a current we put a voltage across and we pass a current through it in a standard resistor there's no memory of what's happened there's no memory of that current to really get what the brain is doing and certainly to get learning and to get memory we need to have a memory we need to have the the device the synapse the artificial synapse remember what happened to it in the past we need a component with it with us a memory that's a simple component that we can synthesize that we can fabricate moreover not just that we can fabricate but that we can fabricate easily and also scale so we can get lots and lots and lots of these on a chip we also don't just want to mimic what's happening with cmos we don't have the same we face the same problem again in terms of energy dissipation so we want something that doesn't have the same level of energy dissipation and causes the same amount of heat because this we can do a lot as i said on 20 watts gpus are taking hundreds of watts and you've you know it's an awful lot of heat energy being generated it's an awful lot of energy being wasted so we've got to try and push that efficiency up back in 2008 and i've scribbled all over this i apologize there was this paper which has caused a great deal of controversy as you can see the missing memristor found so i'll write a blog post on this so to fill in the details some of you i know are really interested in maths some of you are not interested in maths i'll write a blog post that puts the mathematical detail in but basically up until this point the argument is we had resistor capacitor and inductor we've got four key variables here voltage current charge and the magnetic flux because once we've got moving charges once we've got current we've got a magnetic field this group claim as i say it's controversial claim that they found the memristor they found this component we were looking for for years and it turns out that that's exactly what we need for an artificial synapse and it's why all those papers have neuromorphic computing with memoristors because the memristor is a really simple way and it's a really simple to fabricate device that allows us to make an artificial synapse 12 years of research since that paper though i mean surely somebody's come up with something better or something that people isn't controversy you said yeah it's controversial in terms of what is that really a fundamental component but what absolutely isn't controversial and even those who will argue against the fact that it is a fundamental component or not will say this in terms of neuromorphic computing in terms of mimicking here it is a massive step forward how does a memristor work i would say an extreme example of a memristor is a fuse you pass a current through it and it blows it's got a pretty extreme memory of the last state before it blew that's not particularly useful because it's dead obviously but you can have a much less extreme version of that whereby you do something to the material as you pass a current through it so when you stop you know you don't apply the voltage anymore the current isn't flowing you halt it at that particular point so it's got a memory and what do you do in terms of controlling that resistance you do in many ways exactly what the brain does or what the neurons and the the biological architecture of the brain does is you control iron flow and so in that particular hewlett packard paper they had titanium dioxide which is an insulator usually so very well used material in particular when it comes to solar cell technology it's very important and they had regions of that sample which were doped which means they had impurities which changed the conductivity and regions which were undoped and the important thing is what happens is when they put the voltage on they cause those impurities to diffuse right and so that means they change the conductivity those impurities move under the influence of the electric field so we've got a voltage across this sample and that creates an electric field this is a tiny variable resistor that's exactly it that is a very good way of thinking about it showing wonderful it's a tiny variable resistor potentiometer like that obviously if we have a volume you know you just turn it to a certain volume and that's it's that that's what we're doing in in essence we're changing but we're doing it at the level of changing the material properties this is a really simple very unsophisticated demo but what we have is the box represents our sample and these yellow balls represent the impurity atoms in that sample what happens in the real sample is you have a voltage and that generates an electric field across the device and that causes these impurities to move we're not going to put an electrostatic potential on this we're going to put a gravitational potential on this but it's the same general idea you have a gradient in the field and that causes the particles to move and by causing the particles to move you change the resistance of the device so with a normal resistor we're just going to do ohm's law we're just going to do v is equal to ir you'll allow me just one equation shown linear so we've got current versus voltage and you do that current depends linearly on voltage but more importantly you increase the voltage you decrease the voltage you increase the voltage you decrease the voltage you follow this line back and forth so that's for a traditional resistor normal component bug standard resistor that we find in practically every circuit for a memristor the most important difference is that instead of it being exactly the same curve every time because you are changing the device when you apply a voltage to it when you change those impurities and move them around it means you have hysteresis it means you have a memory so the current voltage curve you get on the way up is not the same as the current voltage curve you get on the way down and we have what's called a hysteresis loop and that gives us memory and it's really really elegant because it's controlling it's exactly analogous in many ways to the biological system because the reason we're getting this hysteresis the reason we're getting this memory is because we're controlling iron flow but not in a liquid you can do it in a liquid and there are examples of neuromorphic computing in a liquid but with a laptop you know you don't have a lot of liquid in a laptop it's a solid-state device and what we're doing is we're translating that biological computing to a solid-state platform and we've got memory from a practical point of view how would that hysteresis be used just like neurons spike so neurons spike they send a potential spike um and that controls the that's how you learn you have what's called short-term plasticity and long-term potentiation so what will happen is you will have connections that last a short amount of time with this what you do is you spike you create a snapshot of where the iron concentration is and you remember that and then if you keep going what you can even do is to mimic long-term learning in the brain is what will happen is that those ions will connect and you'll have a connection um throughout the device which forms a long-term connection just like you form neural pathways you form pathways which are due to those atoms interacting and actually bonding all the way across the across the gap it's like what i find amazing is when you're learning to drum even simple things like just putting those bun john bonham style triplets where you put one drum after the other da da da da da da da da da da da da da you start off with that and you do it at 80 beats a minute you do it at a particular slow rate and then you build it up gradually up until the point where i could never have done that two weeks ago and that's because you're burning those pathways into your brain you're setting those you're solidifying those neural pathways that's what's happening you can also do that in these artificial synapses which is amazing to an ip and then i can actually communicate with that server to do whatever it was i wanted to do so just to clarify if i put in computer file.website or something like that something somewhere needs to know where to find that there will be usually one or two i'd be removed 25 and 35 then i get seven and so on so we repeat

Channel: Computerphile

Views: 173,404

Keywords: computers, computerphile, computer, science, Computer Science, University of Nottingham, Dr Phil Moriarty, Sixty Symbols, Von Neumann, Architecture, Memristors, neomorphic, neuromorphic computing, brain, neuron, synapse, spike

Id: Qow8pIvExH4

Length: 13min 58sec (838 seconds)

Published: Thu Sep 10 2020