Jeff Moehlis - Learning How to Control Populations of Neurons

is currently the vice chair he was also recently the chair in the program in dynamical neuroscience at uc santa barbara he has received a sloan research fellowship in math and a national science foundation career award and was program director of the cyan activity group in dynamical systems from 2008-2009 he has supervised 10 students to the completion of a phd four students received a terminal ms degree and three students who received a bsms degree jeff's current research includes applications of dynamical systems and control techniques to neuroscience cardiac dynamics and collective behavior he's published over 100 journal and conference proceedings articles on these and other topics including sheer flow turbulence microelectromechanical systems energy harvesting and dynamical systems of symmetry so uh we're really excited to have jeff uh speak today and with that i'll hand it over to you all right uh thank you very much to the organizers for inviting me i'm going to share my screen and hopefully this all works okay all right so let's see here we go okay you can see my screen okay i hope oops okay so uh today i entitled my presentation learning how to control populations of neurons and i'm kind of using learning in two senses here one is you know we are learning how to do this and as you'll see i'll be presenting uh some of uh the older work that got us to where we are today and also in the spirit of uh the theme of the seminar series um as you'll see you will do some machine learning for how to control populations of neurons as well okay so i want to start with the motivation which is parkinson's disease so parkinson's disease affects about a million people in the u.s including some well-known people the estimated cost of parkinson's disease is about 25 billion dollars a year in the u.s and this includes treatment costs it also includes things like lost productivity the symptoms of parkinson's probably the most familiar one is tremors the shaking of the limbs but there's other symptoms as well that can be equally or even more debilitating including a postural instability slowness of movement rigidity things like that so parkinson's there's at least a hypothesis that it's associated with a pathological synchronization of neural activity in the motor control region of the brain and it's also associated with elevated beta-band power and as you'll see the control techniques that i'll be considering are to try to combat one or both of these either the pathological synchronization or the elevated beta band power now the treatment techniques for parkinson's are either drugs or deep brain stimulation i'll be focused on deep brain stimulation and here a neurosurgeon implants an electrode into the brain of the patient and there's a wire connecting the electrode to a pacemaker that is implanted in the patient's chest and the pacemaker generates an electrical signal that's injected into the brain tissue and if you choose the correct signal then it can make some of these symptoms go away and i'm going to show you a video of a patient that had deep brain stimulation implanted and you'll see kind of you know how this works so this video is an individual named andrew johnson i'll tell you i was not involved at all with andrew's treatment uh he's an individual that had early onset parkinson's disease and uh he has a little bit of a sense of humor so he has a blog called young and shaky.com and the sound on the video may not come through very well so i'll kind of narrate it as it happens but i want to you know just orient you a little bit about what's going on so first of all he's excuse me he's going to um hold up a little device a yellow device that he can use to communicate with the pacemaker which is implanted in his chest and the at first he's going to hold it up to his pacemaker to show that the stimulation is on and then he's going to use the device to turn the stimulation off and you'll see that he starts to develop some symptoms of parkinson's and um you know i'll warn you it gets a little uncomfortable to watch because it's clearly not a kind of pleasant situation for him i do assure you that he will turn the stimulation back on so bear with that okay so as i say i will kind of narrate the video here so you can kind of hear what's going on okay so this is the remote control this checks that i'm on okay so the device is saying that the stimulation is currently on now he's going to turn it off so he says i can't control this he's deliberately holding his arms out but the shaking he can't control the right way so he says i can shake some quality cocktails with my right hand the left is the royal wave his voice is gone harder for him to get words out and speak he's completely rigid he tries to stand up and fall if anyone thinks i'm faking do you think he's faking it he's not controlled but tremors in dysphonia the twisting underneath is called dystonia and it is really comfortable so you saw his neck twist that's called dystonia so as you can see the trimmers are getting much worse the tremors are getting much worse so he says he's gonna he's afraid he's gonna drop the remote control so he's gonna turn the stimulation back on [Music] [Music] so with the power back on he can you know move normally so in case you couldn't hear that he says you know if anyone is considering deep brain stimulation he thoroughly recommends it it's life-changing so you know i've watched this video many times it still is amazing to me how you know there's this procedure seems to be so effective and almost magic you know as soon as he turns the stimulation off the tremors start to kick in and they get worse and worse and then when he turns the stimulation back on suddenly he's kind of able to move normally okay so the uh stimulation that was being used in you know in his treatment is something like this i don't know you know the exact specific details but this is kind of the standard deep brain stimulation treatment uh there's electrical signal at a constant frequency about 120 hertz and it's a pulsatile stimulus so it's kind of you know a short big input here i drew it as a positive input and then there's something they call charge balance so the amount of current injected here and injected in this negative part balance each other and so this is always on as long as you know the the pacemaker is on he can turn it on and off manually if he wants to now what frequency what stimulus would one use well this has to be tuned by a neurologist and it typically requires multiple sessions and it may be necessary to re-tune as the disease progresses but this is kind of the standard and you know i can say that you know i've asked some of the people that do these um you know the uh implants the electrodes and so on you know it asks them why 120 hertz and it's kind of like well you know the first people who tried this that's what they tried and it seemed to work you know so there's been you know definitely a lot of thought about how it works but it turns out it's not really well understood how deep brain stimulation actually works okay now what i've shown you is a video that you know pretty much demonstrates that deep brain stimulation does work and so you might say you know why are we still talking about this um so a big question is is there a better way to do this and what does this mean well it can mean a couple different things one is can we do this using less power so the pacemaker implanted in his chest has a battery and the battery has to be replaced every two years or so and to replace it you have to have surgery and of course surgery always has risks associated with it and so if you could use less power the battery could last longer and you know you may not need to replace it as often also if you think about what's going on here there's an electrical signal being injected into the brain tissue it seems probably it would be prudent to use as small of a signal as possible of course the signal is small enough that it's not somehow frying the brain tissue but you might imagine that the signal kind of you know spills out a little bit into nearby brain regions and if you use a smaller signal there's less of that happening and hopefully that means there's less side effect and also you know this in the control world we would call this open loop control and we know that there's a lot of advantages to something more you know closed loop kind of feedback control and so you know maybe by thinking in those terms one can come up with a better way to do deep brain stimulation okay so with that as kind of the motivation here's just a quick outline to give you an idea of what's coming the first half of the talk will be on trying to control a population of neural oscillators where we're trying to desynchronize the neural activity the control methods here i'm going to just focus on a few of these we've worked on chaotic desynchronization optimal phase resetting and phase density control and then the second half of the talk will be a kind of more uh detailed model for the basal ganglia neural circuitry so the basal ganglia is the brain region that's associated with parkinson's and motor control and things like that this gets more into the data-driven aspects and here the goal is going to be to reduce the so-called beta-band activity and the control method will be a data-driven machine learning approach okay so the challenge for both of these halves of the talk will be that you really only have one input but you're trying to control many neurons so there is a single electrode implanted into the patient's brain well technically usually on each hemisphere there's one electrode but the point is you have one electrode trying to control probably at least 100 000 neurons maybe even a million neurons and that presents of course a huge challenge it's it's severely under-actuated it's not possible to control each neuron individually so i think what this means is one has to be a little bit clever in coming up with a way of controlling these okay so i'm guessing that not all of you have done kind of mathematical neuroscience before so i want to just give you a quick overview of how how mathematical neuroscientists view neurons so the biophysics is kind of captured in this picture here the membrane of the neuron is a lipid bilayer that electrically acts as a capacitor then you have protein molecules that kind of go across the membrane and they act as gates for ions to flow either into out or out to in and electrically these act as nonlinear resistors so hodgkin and huxley back in 1952 published a very very influential paper on the neural circuitry associated with this membrane here so this is you know just um i mean you could view it as a single neuron or even less than that it's just kind of a patch on the axon of a neuron and the way i like to think about this is suppose you put an electrode inside the neuron injected some current i and you wanted to know how could that current move from the inside to the outside well you could have a capacitive current associated with changes in the voltage so it turns out capacitive current is c the capacitance times dv dt then you can have different ions flowing across you could have sodium ions potassium ions and here they call this the beak currents it's basically all the other ions that that could be crossing the membrane there are these nonlinear resistors associated with the ionic flow and then these batteries you can think of as just ionic pumps that maintain a concentration difference between the inside and the outside okay so we have an electrical circuit and you can then write down some different equations that describe the dynamics associated with this now this first equation is essentially just kirchhoff's law that says that the current coming into this node is equal to the sum of the currents leaving the node just rearranged a little bit the c dv dt is the capacitive current i mentioned so you have the injected current you have a sodium current potassium leak current and you know these are called conductances so they're like one over the resistance but you'll notice there's these extra variables n m and h these are called gating variables and they obey their own differential equations as well with some functions here and these functions have you know somewhat complicated forms and you have a lot of parameters so this you know you can do this as the equations for one neuron and you see that you know you have four different equations coupled non-linear with a huge number of parameters it's a pretty complicated system and this is just one um i should mention that this the neurons that they were modeling were from a squid so they called this the squid giant axon and at the time the this was used because it turns out the neuron is big by neuronal standards and so it's easier for them to do the experiments that were needed to come up with these equations and i want to mention these were not derived from first principles although there's a lot of insight based on kind of you know how a circuitry uh how an electrical circuit would work and one last comment they won the nobel prize for this work which uh i think was probably well deserved because in some ways this is like the hydrogen atom of neuroscience most modeling of neurons is based in some form on these equations okay now if you were to simulate these equations this is the type of voltage trace that you would find so the voltage builds up slowly and then very rapidly spikes up and then drops down and builds up slowly and so on and here i've chosen a parameter this would be the injected current coming say from an electrode inside the neuron and we get periodic spikes or they're often called action potentials okay so this is just to give you an idea of you know the sort of model that i'll be talking about here in the first half of the talk now the equations are you know as i mentioned they're somewhat complicated so it's a system of four coupled nonlinear equations i'm going to go through a reduction procedure that is quite useful for understanding the dynamics and also coming up with control algorithms so imagine this equation dx dt equals f of x is my hodgkin-huxley equations so i've just kind of abstracted it or it could be if you're looking at neurons in another brain region or for humans you would have a different set of equations but i'm going to assume that these equations have a stable periodic orbit so this would correspond to the periodic spiking that i just showed now we can define something called an isochron which is a set of initial conditions that share the same so-called asymptotic convergence to a periodic orbit and associated with the isochrones you can well you could view them as a level set of a phase variable theta which is a function of the state of the system and we parameterize theta such that on and off the periodic orbit we have simple dynamics d theta dt is just equal to two pi over the period which we defined to be omega and here i have a little cartoon to illustrate what's going on with isochrons so i'm plotting the voltage here and a gating variable on this axis the black curve is the stable periodic orbit associated with spiking so when v becomes large that's when the neuron is spiking and the blue curves are the so-called isochrones and what i'm doing is i'm starting with three initial conditions on the same isochron and it's a stable periodic orbit so the trajectories all approach that stable periodic orbit and you can see they approach in phase with each other and so when these different neurons spike they're all spiking at exactly the same time and this is just we call the phase circle so you know theta would be the you know the phase variable that i'm talking about and remember d theta dt is just equal to a constant so here it's uh you know it's just moving around at constant speed okay and oops if we start with initial conditions corresponding to different isochrones again the trajectories all approach the periodic orbit but they approach out of phase with each other because they live on different isochrones so you can see as far as the spike time is concerned these three neurons are spiking at different times okay so isochrones are related to something called phase response curves and a phase response curve is defined i like to look at this definition here you can imagine suppose you did a voltage perturbation to your neurons so you let v go to v plus delta v associated with that perturbation can be a phase change a delta theta so kind of going back to this here you know if i let v go to v plus delta v i've kicked myself to a different isochron that has a different theta value remember isochrons are level sets of this theta variable and so i have a delta theta associated with that and then i take the ratio of delta theta divided by delta v take this limit as delta v goes to zero and that gives me this so-called phase response curve and the phase response curve captures the effect of impulsive perturbations in the voltage this is the phase response curve associated with the hodgkin-huxley equations and what i think is really cool about this is if i do the same exact perturbation at different phases of the periodic orbit i get a different result so if i do a positive delta v perturbation at this phase of the periodic orbit it actually kicks myself to a lower phase it makes it take longer for the neuron to fire whereas if i do the same exact perturbation here it kicks it to a higher phase and it makes it fire more quickly so a lot of our control algorithms based on this phase response curve are kind of taking advantage of that that the timing of the stimulus matters now i want to emphasize that z this phase response curve can be measured experimentally and so in some sense and this is maybe not what you imagined when you heard this was going to be a data-driven approach but this is a lot of the reason why we developed control algorithms based on the phase response curve because it can be measured experimentally not saying that it's easy so you have to get a neuron you know a living neuron in a petri dish and do something called patch clamping and uh you know make measurements before the neuron you know turns terminates uh passes away but in principle it can be done and so as i say this is a form of you know let's use data uh in a smart way in order to um you know generate something that can be used to do control okay so i'm going to go through a couple control algorithms based on this phase response curve and then as i say i'll talk a little bit about this kind of more machine learning approach okay so this first one i'm going to talk about is called optimal chaotic desynchronization so i told you about this phase reduction d theta dt equals omega if you have an input something like a deep brain stimulation input the way it comes in is like this so theta i would be the phase of the ith neuron and i'm imagining i just have two neurons here to derive the idea both of them have the same frequency and the same phase response curve and they're both feeling the same input okay so this is how when you do the phase reduction you account for the inputs phase response curve if we let phi be the phase difference between these two neurons and do a taylor expansion we get d phi dt is given by this expression and if i get rid of this nonlinear term then i can say that phi goes as e to the capital lambda times t where capital lambda is a finite time lyapunov exponent that depends on the derivative of the phase response curve and the input now for those of you who have studied chaos you know that a positively up and off exponent is associated with chaos and what it means is that you have this exponential divergence of nearby trajectories so here what we're going to do is an optimal control where we're trying to make this lyapunov exponent as large as possible okay positive and as large as possible so that the phases of the neurons exponentially move away from each other and remember there's this hypothesis that some of the symptoms of parkinson's are associated with pathological synchronization that means all the neurons have phases close to each other here with this control it's going to move those phases apart so it corresponds to desynchronization okay so i'm going to skip the details of the optimal control i'll just show the results now here i'm showing some simulations with no control just to kind of give you an idea of you know what the behavior would be so here i'm simulating 100 neurons they're coupled the coupling is trying to bring them all in phase with each other they're also noisy which is trying to spread those phases apart there's no control here and the neurons are spiking when they cross over here okay and i should mention i i don't think i mentioned you know the way we parameterize the phase variable theta equals zero would correspond to this maximum voltage okay so that's in the absence of control here i'm starting with the neurons fully synchronized with each other but using this optimal chaotic desynchronization when the dots are red that means that the control is on when the dots are blue we've decided that the system is desynchronized enough that we don't need any stimulation and so you can sort of see that you know the dots are crossing through this this maximum voltage you know separated from each other so they are in fact desynchronized so here you know as i say we judge that they're desynchronized enough so we turn off the stimulation but then when they become too re-synchronized due to the coupling then we turn it back on okay so and it turns out that uh you know what we're optimizing is the leopardoff exponent but we're also trying to minimize the amount of power that's needed to do this and this approach uses at least a factor of 100 and possibly as much as a factor of a thousand less power than kind of the constant frequency pulsatile deep brain stimulation that's normally used okay so another method that we've developed um is again inspired by these isochrons so remember i showed you a plot of the isochrones it turns out that if you follow them they spiral into an unstable fixed point for the system and this we call a phaseless set because you cannot define the phase at that point the control method we used here was we call optimal phase resetting this was inspired by art winfrey but he did it not so much as an optimal control problem the idea is suppose your neurons are all kind of in phase with each other let's design a control that brings us into this neighborhood of the faceless set then the system is very sensitive to noise at that point because you can imagine you know if every neuron is feeling a different realization of noise it gets kicked to a different isochron and then when the stimulation gets turned off and you return to the periodic orbit they return out of phase with each other and therefore the system is desynchronized so again you can formulate this as an optimal control problem here i'll just show a movie so when the dots are red that's when the control is on remember you know every neuron is feeling the same control there's only one electrode that that you have here and so this control method is basically trying to steer all of them into this region of phase space where you're very sensitive to noise and as you can see the dots nicely separate from each other and that's you know desynchronized okay the last method i want to show you based on the phase response curve is what i'll call phase density control so you might imagine if you take the number of neurons to infinity you can come up with a pde which describes the density of the phases for your system and then what we did was we formulated a control acting kind of directly on that phase density here just to orient you the blue curve i'm imagining is the current phase density for the neurons so you know theta equals zero corresponds to the neurons firing and then the black curve is our target distribution and we're going to design a control that'll be shown here and then this is the l2 difference between where we are the blue curve where we want to be the black curve okay so here you know given this control we're able to drive this blue curve towards the black curve and you know the longer you wait the closer you get okay so here we started with a synchronized system where there's kind of this hump which means the neurons are all kind of in phase with each other and we're targeting this final distribution where it's flat and this would correspond to desynchronization and one nice thing about this method is you can use other target final distribution so here i'm kind of doing the reverse i start with a completely desynchronized distribution and i'm forming that we call this clustered state where half the neurons are in one cluster half the neurons are in another cluster so there may be some advantages to steering your system towards that sort of state rather than a fully desynchronized state and here i'm just showing if i have a population of oscillators feeling this signal you can see that indeed they do separate into these two clusters okay so all of these methods that i've shown you are based on the phase response curve which i said is experimentally measurable so you might say hey the problem's solved we can stop the talk early we're only halfway through and i'll tell you no sorry the problem is not solved so we were excited by these results and you know we've published papers on these and my students have gotten uh you know publications and a few of them have won best dissertation awards and you know it's all very exciting right so we were thinking we want to apply this to a real system and so i got in touch with philip starr who's up at uc san francisco he actually does these surgeries and he's also i think his phd is in chemistry so he's also you know very well versed in modeling and things like that and so i said hey can i you know tell you about the work i've been doing then it'd be cool if some of this could be translated to real patients and so i kind of gave him a talk like what i just gave you so far and he you know politely kind of smiled and said yeah you know that's very interesting work and they said but there's no way that any of this would work for a real patient because it it just assumes too much information and it's too idealized so you know we're considering identical neurons so no heterogeneity sometimes we're considering systems without noise and of course the brain is a very noisy place so it and it's just much more complicated you know there's neurons interacting with other neurons different brain regions and so on so what can you actually measure in a human patient you can't measure phase distributions unfortunately although we have some ideas for how you might be able to extract out the phase response curve from some kind of population level measures but this is more honestly what you might be able to measure and even this is not trivial for a human patient so this is something called the local field potential and it is essentially you stick an electrode into the brain region of interest and you just measure whatever electrical activity can be detected so it's some averaged filtered activity of whatever neurons happen to be close to the electrode and here i'm showing two different traces of this lfp one for a healthy patient one for a patient with parkinson's you know you might be able to say well it looks like some oscillations are bigger but i can assure you we could find a different you know section of this time trace where things would look almost flipped from each other and so it is in fact very difficult by eye to tell the difference between a healthy patient and a patient with parkinson's disease okay now these traces it turns out these are not from real patients although they do capture the dynamics well for real patients these are coming from a more realistic model than the models that i've shown you up to this point so up to this point the models have kind of been a single population of neurons that are all identical and so on here this is more realistically what's going on so you have the different brain regions so this box is the basal ganglia this box is the thalamus then you have the cortex and you have projections from say the cortex to the subliminal nucleus then the subthalamic nucleus projects onto the globus pallidus and turnus and and so on and you see it's it's quite complicated right so this model here and we're using the implementation from from this paper here much more accurately is capturing the dynamics associated with parkinson's disease and to orient you the stimulation goes into this subthalamic nucleus so this is the deep brain stimulation would go here and the measurements of the local field potential are in this gpi region okay so what we were interested in doing is you know taking this model which is much more realistic and it outputs something that is like what you could actually measure for a human patient can we develop a control algorithm which now does something useful and here we're going to be looking at the power spectrum of the lfp and in particular the beta band activity and i'll say a little more about this and you may recall that there's some evidence and a hypothesis that parkinsonian symptoms are associated with elevated beta-band activity so what we want to do is design a control that brings that beta band activity down okay so here's our you know kind of big picture our control strategy we call this adaptive deep brain stimulation so there are bursts of beta band activity the beta band is you you may be heard of alpha waves and beta and gamma and theta and so on so the beta band is roughly 15 to 35 hertz the bursts of this activity seem to be associated with the symptoms what we want to do is a short term prediction of whether or not one of these beta band bursts will occur in the next time window and then based on that information adjust the frequency of the stimulation okay so we're going to be using the same type of pulsatile stimulation as what i showed you right after the video of the guy from youtube but we're instead of using a constant frequency we're going to adjust it as needed according to this prediction and what are the advantages over constant frequency dbs well it can save power so you're only using a you know the level of stimulation that's necessary we believe that it could reduce side effects because again you're only using the stimulation that's necessary so one might expect that lower frequency stimulation is less likely to cause side effects and higher frequency and also well we want to adapt to the status of medication so a typical patient will be on medication and also some of them will be using d-print stimulation as well and of course the medication can kick in and wear off and that will affect the dynamics of the neural circuitry and therefore maybe you want to adjust the frequency of the dbs in order to do this so we scratched our head a little bit you know looking back at this model and saying well this is pretty complicated i don't know if we can do a you know a phase reduction of this to use the techniques we've done so far so that brought us to machine learning and you know this is my you know somewhat humorous somewhat cynical view of machine learning where you have an input that goes into some black box here and then suddenly you get this wonderful output which somehow takes you know a signal where you can't say much about it and splits it into something which maybe you can say something useful about now i should say you know i i of course appreciate the power of machine learning and i'll be honest probably you know many of you listening to me right now are more experts in machine learning than i am so i'll try to do my best to convey the approach that we've been taking so we're trying to as i say you know do this realistically where what you could measure is this local field potential so we're taking as input this lfp and more precisely we're taking the power spectrum of the lfp feeding it into an encoder so this is our auto encoder feed it into encoder to develop some encoded variables and then we to determine these encoded variables we then have a decoder and reconstruct it and train it and then once we've trained it well we cut things off and then just look at these encoded variables and the hope is this gives us a lower dimensional representation of the lfp that we can then use to make predictions okay so just to give you an idea so these encoded variables are again you know the power spectral density over some frequency range here you know i'm just trying to show you you know for some different inputs of the power spectral density the red corresponds to the lower values of the encoded variables so encoded variables 0 1 and 2 we found three encoded variables work pretty well so you know what exactly are they capturing well you can see here they're capturing some elements of the shape of this power spectral density okay so now we're going to do supervised learning of this adaptive deep brain simulation so we're going to train a deep neural network with inputs being the encoded variables for the power spectral density over a two second interval and then we're also inputting some candidate control input frequencies and the output from this neural network are going to be the best stimulation frequency to keep that power spectral density below threshold for a desired fraction q of the time for the next two second interval so remember this beta band activity if it's elevated it's associated with parkinson's what we're trying to do is get that power spectral density below some threshold at least you know maybe half the time for the next two second interval or this q will be a parameter that we can change and uh train our network in order to do so to give you an idea that this looks promising so here and i should say we start our neural network with you know just a random initialization so it's learning you know on the fly and here we're targeting that we're below threshold point five of that uh upcoming window and we're just doing two different trials here and you can see that the frequencies do you know change quite a bit and they change differently for the different trials but we're able to hold things pretty well around this 0.5 level okay and as i said q is something that uh you know we can tune for our network so these are all again cases where we're starting with just a random initialization for our our network and you know for different values of q so here this would be the 0.25 and i should say the thin lines are individual realizations and the thick line is kind of an average over those realizations so when we're targeting 0.25 we can do well or target 0.4 0.5 0.6 0.75 and then these are the frequencies that are necessary in order to bring you there and you see that okay you you might have to adjust the frequency quite a bit in order to hold yourself at that level but what we're imagining is that one could you know use this q as an extra parameter to kind of say you know how carefully do we want to control for this patient the level of beta band activity above threshold and so using this algorithm you could then you know given that cue adjust the frequency accordingly and here's one thing that we think is very cool so when a patient is on medication you would expect the parameters associated with their neural circuitry would vary and this is kind of a busy plot but we're just choosing different frequencies of variation of the parameters and so basically in the model here we're varying the patient from being healthy to being having parkinson's disease and a sinusoidal fashion over some time intervals so this is time in seconds and then this uh this control algorithm that we've developed will adapt because it's always taking in the new information and adjusting the weights in order to make that happen so here we're trying to do this moving probability of the q value at 0.6 it's just showing for all these different periods of variation it's possible to choose the frequencies you know for the next time interval in order to keep yourself very close to that and this is something when we talked to phil starr about what they felt you know would be a nice uh target for this adaptive deep brain stimulation they felt it would be you know very good to have something that could adjust to this medication level of course we're not making the meds of the medication it's just based on uh kind of the dynamics of the model and the uh the deep neural network learning what level of stimulation or what frequency of stimulation to use to make this happen okay so we feel good about how this is happening with the model and so the next step is to apply this to human data and so phil starr's group was very generous in sharing some data with us and you know we we don't have this done yet but i i'm you know i'm excited to show you some kind of new results that we have with the human data that we think is really pointing in a good direction so the data here is lfp recordings from humans the recordings tend to be about one to two minutes long they're all recordings for a single patient but recordings over different trials over several years and there's data for the dbs stimulation on or off and medication on or off and so there's four different combinations you could have dbs on medication on dbs on medication off and the other two combinations so it's not a lot of data in all honesty so you know 22 one minute trials for all for one patient so we wanted to see you know what can we extract from this so we've tried a couple different methods the thing that seems to be working good working well now is regression so this is kind of a you know neural network style regression so the inputs are our encoded variables so i showed you these encoded variables before and actually i i didn't mention this deliberately but these encoded variables actually are from the human patient data and we use the same encoded variables for the model uh so you know we feel pretty good about you know the using you know real data and coded variables and the model still works well but so here for our regression problem on the real data we input the encoded variables then this is similar to what we did for the control but a little a little different so we're essentially looking at a sliding window inside the current interval so let's say the current interval is one second we look at a sliding window of 100 milliseconds and we slide through that whole interval and we look for the maximum proportion of time that the peak of beta activity is above the threshold inside that current interval okay so this is we believe a nice way of you know characterizing if there's a burst of beta-band activity and then for a regression we also input is the dbs stimulation on or off or is the end is the medication on or off and the output is going to be the maximum proportion of time that the peak of the beta activity is above threshold in that sliding window so basically we're taking what's happening in the current interval and predicting what's happening in the next interval and then the idea will be we'll adjust the frequency of stimulation based on that okay so does this work so here we're doing a comparison between a no change hypothesis this would basically be just assuming whatever happened in the current interval is going to happen in the next interval and it turns out when you look at the data that's a very good prediction it turns out things don't change a whole lot every once in a while you get this burst of beta band activity these um the oh did i mislabel these the dashed i mislabeled these sorry the solid is oh no this is correct i apologize this is hot off the presses here the solid is from the regression the dashed is from the no change hypothesis we're plotting the error here and you see that using the regression is better and the way to interpret these lines so for example this curve here corresponding to the third quartile absolute error 75 percent of the data has error below this line so you can see that the regression is giving a you know pretty significant smaller amount of error than the no change hypothesis and um we're also considering you know other things kind of based on pca and so on i don't have the results to show you just yet but you know we believe that this uh this regression algorithm is giving nice results and if we look at you know how is it oops how is it improving things so here i'm plotting in the current window the max average bursting activity so if this was zero that means in the current window there's no bursting activity if it's one in the current window it's all above threshold so there's kind of a beta band burst and then the mean absolute error in in our prediction well and over here this would be for the no change hypothesis maybe it's easier to start here so if there was no beta band activity in the current window and in the next window there was again no beta band activity then there's no error associated with the no change hypothesis but if it turns out that there was you know burst inactivity over that entire interval then you would have an error of one and then you have all kinds of errors in between you can see you know sometimes in the current window there's you know maybe point four percent uh above threshold and then the mean absolute error well if in the no change hypothesis you say it'll be 0.4 again but in fact there's zero then you would have this .4 absolute error so if you look our neural network regression approach gets rid of kind of those x's so it makes much better predictions uh than the no change hypothesis if there's you know a level of activity above threshold between zero and one and it also does better up here you know so it's driving uh you know these if you look at the color scale you know the no change hypothesis you have uh you know more realizations up here here when it's green basically you're pushing the error down here okay so at any rate we feel pretty good that this regression is giving us less error this as i say is very hot off the presses the graduate student tim who is doing this sent me these figures i think certainly within the past week and probably within the last few days so what are our next steps here well um we already have some data for more patients we're interested in applying these ideas to those to see if we can get comparable level of improvement in making predictions we would also love to have and this is very difficult we would love to have patient data for different stimulation frequencies so i told you that some of the data is for stimulation on this would be for constant frequency dbs for training our algorithms it would be great to have patient data for different stimulation frequencies if that's not available then you know we would do the training using the models such as i showed you and of course one of the big questions is how to implement this for real patients so we're in touch with our collaborators at ucsf and we're very excited to you know see where we will be able to take these ideas okay so um just to wrap up you know to summarize what i've told you about so you know this is all motivated by parkinson's disease uh deep brain stimulation our goal well in the first half it was to desynchronize the populations of neural oscillators the second half it was to reduce beta band activity these are you know related to each other certainly uh i talked about the control based on phase models which i will emphasize again in some sense this is data driven because it's possible to measure these phase response curves so i told you about chaotic desynchronization optimal phase resetting phase density control and then the newer work the data-driven machine learning approach to adaptive dbs using our autoencoder deep neural network regression also when we're doing the real patient data one thing we're very excited about is this algorithm adapts to slowly changing parameters so we feel that this could be useful for situations where a patient is on medication and dbs how can we you know adapt to what they need based on whatever their current state is okay uh i'll mention you just very quickly we've been thinking about this problem for a while we have other ideas for how you can control this is more based on the phase sorts of models but we also did some supervised learning for binary control some reinforcement learning and then other types of related research and i'll just mention you know we also have other experimental collaborators uh tay netoff's group and ken showalters group where we've applied some of these ideas to chemical oscillators also controlling circadian rhythms if you like this phase sort of stuff here's a tutorial we wrote recently that kind of talks about how the phase reduction works and how you can develop control algorithms based on that perhaps the more important slide is this one which is the collaborators so of course i didn't do all this by myself uh the work on kind of the machine learning uh adaptive dbs was done by tim matchin some of the other work on the phase control was done by dan bratt and ali over the years uh para was another student who kind of started with this he was kind of the brave first mechanical engineering student to work on neuroscience applications in my group here are the collaborators from uc san francisco we've had high school students working on this my experimental collaborators at minnesota and west virginia and some other uh people who have been you know useful for different aspects of this of course thanks to the nsf for funding this work and i'll end that here with uh you know the mad scientist slide here uh is my attempt at brain control uh so with that i thank you for your attention and i'm happy to answer any questions that people may have so i will stop sharing so maybe you can see me and i'm not sure how you do questions if you want to uh let people ask me or whatever will they open what we've been doing is have people type them in the chat and then uh i would convey them to you okay so um yeah feel free to type your questions in the chat um because while we're waiting for people to type out their questions i have one um so this is more on the mathematical side just based on my interest and about the phase density control to what extent have you thought about coupling including coupling in that problem so uh we have thought about that and actually there's uh a group in iran who got in touch with me and wanted to study that problem so we've been thinking about it a little bit i've i've been too busy to kind of you know be giving them the feedback they deserve but that's something that's kind of next on my radar um wanting to work on that and so uh just to give kind of an idea of how we want to do that so you know we want to you know still do phase models but incorporate then you may know so for phase models you can get phase difference coupling when you kind of average uh the system and so we're kind of interested in looking at systems that have that coupling and also have you know these external inputs and you know we believe that we end noise you know so why not throw noise in as well so we believe we have some ideas for how that can be done but you know that certainly i think is worth exploring further because the real systems will have coupling and you know often the way we've derived our control algorithms is we assume there's no coupling and then we develop the algorithm and then we try it numerically on the system with coupling and verify that it works but you know we've only done a few cases where the coupling was actually included in the development of the control algorithm and i i think that's a very ripe direction but it's also very challenging direction gotcha thanks uh okay so we've got several questions in the chat um i imagine there would be some power costs to actively monitor brain signals and adapt the applied signal power in response to that uh given that how much power can you expect to save from adaptive brain simulation yeah that's a great question and in fact our collaborators at ucsf have asked us that same exact question and we don't quite know the answer to that yet um but i think that um you know certainly there would be a power cost associated with that i think it's even possible that it would use more power than uh you know just running the constant frequency stimulation but though that's not the only reason why we think adaptive dbs might be useful we think that you know adapting to the medication state could be a very beneficial thing and also just you know there there may be a and a lot of this needs to be explored but there's a sense maybe that higher frequency stimulation is more likely to lead to some side effects and so maybe the higher frequency stimulation you're willing to put up with side effects because it's needed because you're trying to suppress this beta band burst but then when it's not needed you're doing lower frequency stimulation where there's less of a chance of the side effects being important and so you might be willing to use more power even though you know it's like on the average you're using more power but you're only using it at the times when you really need it and i should mention that you know phil stars group part of the reason why i got in touch with them they have another form of adaptive dbs that is currently you know being used in some human patients i don't think it's being used kind of out in the field so to speak but um where it's more of an on or off so sometimes they turn the dbs completely off when it seems the the patients don't need it and um you know again there could be similar advantages to what i just sketched out for that but that's that's a great question cool thanks uh next question are there approaches to control of unstable periodic orbits in neural models do the unstable periodic orbits appear in neurodegenerative diseases yeah that's another very good question and the answer is yes i've seen some you know kind of chaos control uh sorts of algorithms applied to neural models i'm not sure if they're important for neurodegenerative neurodegenerative diseases or not they may be um but so there could be for example a separatrix of sorts between different states and the the unstable periodic orbit is separating those uh some of the work we've done has been kind of controlled based on those sorts of ideas i should also mention you know we've worked on cardiac systems and some of our cardiac control is trying to control the system towards an unstable periodic orbit so there's a phenomenon called alternans so you also have action potentials for heart cells and for a healthy patient you would have all of these action potential durations are called being roughly the same for subsequent heartbeats for alternans it turns out these durations alternate between short and long and this can lead to bigger problems bigger arrhythmias like fibrillation and so what we've done it turns out there's for the dynamical systems people there's a period doubling bifurcation where the periodic the period one solution becomes unstable and then the period two solution becomes the stable alternate solution and so there what we're trying to do is control to the unstable periodic orbit which corresponds to the the healthy state and so we have developed a an algorithm for that it's maybe a little different from what you might expect because here the periodicity only comes from pacemaker cells not a implanted pacemaker but there are cells that are firing periodically and they then kick the excitable cells and make them fire periodically and then that's where you get the period doubling bifurcation um with that but yeah i i think you know i i'm very excited by these approaches that you think about kind of from a dynamical systems perspective what's going on like are there unstable periodic orbits are there uh you know are you trying to change the frequency of uh the periodic whatever you know things like this and uh can you then develop control algorithms that kind of exploit that insight from dynamical systems so yeah that that's that's a great question and and i i think i saw my my friend predragstanovich pop up i don't know if he's there or not but he's worked a lot on unstable periodic orbits and uh you know i i think the interplay between unstable periodic orbits control that's something that's very interesting and probably worth investigating more cool thanks so there's two more questions in the chat right now um did you try for the data driven and the prediction any type of common filtering such as the ukf yeah so ukf i assume unscented coleman filter we have not tried that but there is an another great question um i i particularly like there's this work by uh steve schiff and tim sauer i forget who else was involved but on you know applying an unscented coleman filter to um kind of fit a non-linear model to uh to data i think they were using numerically generated data not um patient data but i i was very impressed by that work and i think what that's doing is you know it's it's kind of a system id approach i'm not sure maybe this is what what the person asking the question is or maybe not but my interpretation it's kind of a nice system identification approach where you're assuming a certain form of the equations kind of a tattoo nagumo style model and then the unscented kalman filter is just looking at the voltage trace and figuring out what the correct parameters would be in order to you know get a good model which then could be used for prediction uh we have not tried that i think that's a great idea and uh you know maybe one of my next students that would be a good good project for them to think about and and i should say you know there's a huge number of uh approaches out there i'm currently reading the nice book by by steve and nathan on uh data-driven dynamical systems and i'm learning a lot from that uh you know some of these things i've looked at before and some of them i haven't and you know i think that you know it's an exciting time to be able to you know think about all these different approaches and what would be the best one for the particular problem at half great yes the final question we have in the chat here is are you envisioning closed loop control strategies that incorporate sentencing of parkinson's disease symptoms like hand shakiness and are there available data on that yes um we are not actively doing that but that's another great idea there's a group at oxford that i think has done that a little bit and you know so you could mount an accelerometer on a patient's finger for example and then you have uh you know a measurement of how how the tremor is and then based on that you could you know presumably adjust the frequency or turn the stimulation on or off i think that um you know what they were doing they i believe they were actually measuring phase response curves based on accelerate accelerometer data which is very cool um i don't think they were kind of closing the loop and doing it for feedback control but i i can't say for sure i i think that's a great idea uh one thing i would point out though is that tremor is only one symptom and often patients i'm told find the rigidity to be the most frustrating symptom the fact that you know they're not able to initiate movement in the first place and i'm not sure i mean there might be other ways of detecting rigidity using you know accelerometers or you know other who knows maybe uh eeg signals or something like that that could be used i i think this is a wide open field though i mean we're just scratching the surface at ideas of what could be done i think especially now that these machine learning techniques are coming on board you know you could perhaps i mean i just saw it you know there was this was on facebook so but i think it's probably true a group at mit was looking at people's coughing and you know being able to detect is that cough associated with asymptomatic coved or just a normal cough you know using a machine learning algorithm and you know i've heard of you know analyzing people's speech you know and being able to detect if if there's maybe some disorder for that person and some of this you know you always have to take with a grain of salt and is it being over hyped are you just fitting to you know the training data and then you try it on new data and it's totally different you know so one has to of course be very careful about these sorts of things but i think you know we're in that data era now where there's a huge amount of data available can we sense something useful from that data you know what we're doing in what i presented today is you know from the lfp data can you you know at least extract out some information that allows you to make a short-term prediction about what's going to happen and then do your control um accordingly you know maybe yeah maybe somebody can think of other type of data that would give you something you know either you know comparable or even better maybe that comes from an accelerometer amount of someone's finger maybe that's an analysis of their breathing pattern or an analysis of their speech pattern or something like that and then design control based on that so this is all wide open and hopefully you know we can as a community work together to you know bring a better quality of life to people that have these disorders so yeah i'd love to hear if you have ideas of how to do these things i'd love to hear about them great okay well uh i think that that's uh all the questions we have thank you so much for uh being with us and sharing your research it was great talk uh thank you for inviting me uh very nice for i i don't consider myself to be a hardcore machine learning person so so i appreciate that you gave me the chance to talk to this this audience great okay and we'll see everybody in two weeks okay okay bye all right thank you

Channel: Physics Informed Machine Learning

Views: 348

Keywords:

Id: i9xMTZRFDSE

Length: 71min 58sec (4318 seconds)

Published: Fri Oct 30 2020