Brain Computer Interface w/ Python and OpenBCI for EEG data

what's going on everybody and welcome to a video series that might have me more excited than any other series before most of because I don't know what I'm doing here I know some of the things that are possible but I don't know everything and it's gonna be a lot of stuff that I have to learn and hopefully we can learn together so what I've got here is the open BC I think what I've got is the starter kit I could be mistaken we'll break it down in a second but basically the idea is so you can read EEG data from your head I've think neural link just without needing this machine to embed electrodes into your brain it just sits right on top non-invasive and you can read brainwaves I believe also in here we've got things for ECG and EMG like reading muscular movements and stuff like that mostly I'm interested in the EEG stuff but honestly I'm just super psyched about checking out all of this technology so let's dive in so inside the box we've got two for some electrodes I think these are just for use with conductive paste though then we get the actual conductive paste some electrode wiring they also sent me their headband kit if you're looking for a cheaper entry into EEG data this is probably your best bet sticker next we get the battery pack the sight on board and the Daisy board the battery is for headset power though we're gonna wind up using something else the sight on board is eight channels and then the Daisy board gives us the other eight channels for sixteen total so now for the main act of the show we have the ultra cortex mark 4 headset on assemble but 3d printed - pretty good perfection here each octagon is a spot for an electrode and the unit basically just rests on your head headset electrodes jumper wires for them inserts for the headset electrodes and then the last of my electrodes oh and also some ear clips and that's it now we're ready to actually assemble the headset so the official guides simple enough they've got a video and a text write up lots of photos it's pretty useful the kid overall is pretty expensive you can go down from 16 channels to eight to save some money or just go with the headband kit that I referenced before for 200 there's also like toys on the market with EEG like you know to control cars you can convert those maybe for 50 bucks you can even try to source parts on your own and build everything from scratch I estimated I could do it somewhere between 10 and 20 dollars per electrode circuit and maybe even go lower but it's just the quality was gonna be an issue plus the whole I really don't know what I'm too weakened part so the only issue is with this battery as you can see here this is if you have a standalone battery it can kind of go in real nice and clean there but the battery doesn't come with the headset and I'm not really sure why made for shipping reasons so they give you instead of battery pack and this is it it's just taped to the top that's not acceptable so so open bc i does link to a battery from Adafruit but it's sold out so luckily you can buy these little lipo's like all over the place I even have a few lying around for small drones so which brings me to my next problem and here you see two identical plugs one is the battery pack that they sent me and one is my lipo battery notice anything catastrophic how about now I've no electronics expert but I think rule number one is don't reverse polarity so I don't know if there's some stand where standard somewhere for these plugs but if there is someone messed up we need to flip the lipo plug like the lipo battery plug because these plugs can only go in one way you know so people don't reverse polarity to do this you just have to lift up on some small tabs usually people use like a screwdriver but I actually just stuff a small thing in like an Allen wrench where the wire goes and I'll do the trick to pull it out long story short always double-check the polarities before you connect things this could have ended in tears even though the adafruit battery whistled I did look at the pictures and it looks like the plug was indeed opposite from mine really weird I've got multiple brands of these batteries and another brand charger all of which are the opposite of this headsets board and the Adafruit battery but anyway once you got everything together you can use this little plug here and it's just a bluetooth dongle they've got Mac Linux and Windows support for their GUI that you can just download once you've got that go ahead and open the GUI you pick various settings like which board you're going to use how many channels and then make sure that you can actually make a connection to the headset so obviously make sure the headset is you know turned on once I sign go ahead and put on the headset and clip on the ear clips you might find the headset to be slightly uncomfortable you just need to it like adjust the electrodes in and out so if you go too far in it might hurt or lift up other electrodes and then too far out you won't get any signal from the ones that are too far out if all goes well though you should see something like this alright now that we simply could not get any more cool what's next so first of all this GUI is really awesome as a programmer I look at this and I am in total on env but I don't know we can't really do anything from here because one I can't really eat GData I don't know what I'm really looking at but to the hope is to do a brain-computer interface at least to the extent where I want to do control so something like controlling something in a video game for example is the first thing I want to try so to do that we we can only take one or two approaches and that would be either some sort of rule-based approach which is going to require me to have some sort of Industry knowledge which I don't owe or to a deep learning approach which won't require anything of me really except for coding some deep learning stuff so that's the first step I want to take now to take that step I have to decide one how do we get that data to what data do we get so from quick research I'm just gonna bring up a couple of things here just for vague understanding for everybody if you just google like brain wave chart you'll find stuff like these charts okay so this explains what each of like the five major brainwaves are and what they kind of map to I suppose so like Delta waves these are like point 1 to 4 Hertz and if you don't hurt PSA's you can dare I say it's interchangeable with frame rate so FPS and when I show you our code there's going to be things that say FPS mostly because we've copied and pasted the code that because to measure the Hertz or the sampling rate it's the same code you would use to measure FPS it's not the same and we'll talk about some of the various issues hopefully soon but anyways so delta waves this is like like we're not going to control anything with delta waves theta waves um I don't think we're gonna control anything there either so I think you could probably you can tell if someone is like sleeping based on theta and delta waves or and if they're dreaming or not or something like that then with alpha waves we could tell I think and again feel free to correct me I am NOT an expert this is just me learning over time and what I think where I'm at at the time so alpha waves you could tell if someone is relaxed what kind of state they're in blah blah blah beta waves this is your normal waking state so information about that and then gamma waves motor functions higher mental functions okay so just from reading that my very quick understanding is that we really need gamma and beta waves most likely somewhere in this range is what's gonna be most useful to an algorithm like if we're thinking go left go right it's my expectation that we would need information on this level and the way this works is that the Hertz this is like a sampling rate so if you took this information and you had like the raw data so let's assume gamma waves this is the raw data so let's say this was I don't know 60 Hertz you could sample it to 30 Hertz so you could take a 60 Hertz stream and sample it to 30 but you'd probably don't want to do it let's say you can went 60 to 12 Hertz this is what you got then he took 12 Hertz and you and well you wouldn't want to go 7.5 but but basically each of these is like a less granular sampling of that raw data that is whatever the baseline is so we have a couple of issues here one is what it what is the raw sampling rate that we can get from the headset because we need something pretty significant if we're gonna get gamma waves like you probably would want hundreds multiples to get gamma waves so that's our first question and then how to resample I'm pretty sure we can use Syfy for resampling information but anyways that's the question and now how do we get the data actually just in Python so to do that it was actually super simple you just go over to learn down here software and basically there's kind of two major packages one is like Python or it's open sea open BCI - Python or PI open BCI for newbies like me and as I was reading through here I just kind of got eventually to this point where boom print the raw data that's exactly what we want right so so you have to install things where is it yeah the pitman so numpy pi cereal --bowl ah and then you can pip install the PI open BCI but then all it takes is this code right here so let me show you guys a quick example of that so fine take this copy pasta also I need to stop that data stream I'm just gonna close out of that so it's not interfering with our new connection that we want to make and over get to save that and let me run that it'll say PI 3 7 read from dot pi oh we have a couple issues here so first of all it can automatically find the connection you also could go into your device manager to figure out exactly which comport you're using but it can automatically find it also Daisy is true so if you had a different board you would import something different here but I have this site on board and I've got the Daisy so that Daisy's that extra eight channels so I may go ahead and save that in you run that one more time hopefully that works cool looks like it's gonna come in and let me make this big screen boom there's our data beautiful I couldn't believe how simple that was so thank you to the people at open BCI that was really easy so so what you see here is just a bunch of arrays and each element in the array is a separate channel so a separate channel is a separate electrode on the head so this is channel 1 2 3 4 5 and so on all the way to 16 so pretty cool the only thing that you can even see here is it appears that this is not a steady stream of data this is a burst of data which is very problematic I don't know what's causing that if is it just the print like if I wasn't printing would it still be bursting I really don't know so that may or may not be an issue moving forward but anyways it's currently happening so I'm gonna go ahead and break that and then what we need to know is even though it is bursting can we find out what our frame rate is so that's the next thing and I'm just gonna copy and paste this code and explain a couple of things just because I think probably a lot of people are just kind of following along for the interest of what's going on here so I am going to copy and come over here pasta and let me just explain a couple of things here for the FPS counter we are using a deck if you don't know what a deck is it's just a container that you can say hey I want this container to be 50 and as I add new elements beyond the 50th element the new element gets added at the end the oldest element gets popped out really simple but deck is not very fast so instead it with the help of Daniel he wrote a kind of a numpy version of a deck so basically it's this and then these two lines here it just mimics what deck does it just does that way faster so so in this case it's a sequence of 30,000 I'm gonna decrease that just for now I don't actually I think 30,000 was still quite fast I don't think it actually made a difference but anyway I'm trying to think here so sequence we're just populating that and again it's sequence is going to have in this case a hundred samples of the that 16-channel array that you just saw in the console output nothing more to explain other than this that's commented out basically I saved a couple of files because just trying to iterate over the code and like do some like R&D type stuff I needed I wanted to just have a way to mimic the headset without actually needing the headset on because every time you break the connection sometimes it can be hard to reconnect stuff like that I'm sure there's a nice way to like cleanly close but like I said I'm honestly like going through this lot almost live it's not totally new so like some of this stuff I'm just gonna write for you guys before I hop on a video but this has been learning as I go so anyway did I say that I can't remember yeah so this will give us our frame right now so as you can see there's a couple of things going on here I'm just gonna highlight to pause this number here appears to be the legit framerate but actually it's usually more like 105 like sometimes on the initial connection and things are kind of slow and then it and then it finally gets more stable so oops let me yeah if I catch it at the right time there we go so as you can see most of them are rating 106 105 I believe that's the real sampling rate so you would say the raw sampling I think is 105 Hertz but then towards the end we get these much larger numbers I've seen it be as high as 2,000 I don't know if that's because of the bursting or what I'm gonna try to dig a little deeper figure out what's going on maybe ask the people at BC open BC I see if we can figure out like what why is that happening because that's gonna make resampling challenging but still very very possible you just have to resample based on the index of time I think it's totally possible to do it I just need to know for sure that that's what's going on so anyway it's about 105 Hertz now this is a problem for a couple of reasons one is since we're only getting 105 Hertz going to our brainwaves here we're gonna have a really hard time sampling to 30 Hertz like it we can do it it's gonna be a stretch but it's not gonna be very it's gonna be like totally accurate so the only option we could just keep it raw like and I think that's what I would do for like a gamma wave I would just keep the signal raw and then for the beta waves I think you could take it and say you know you can take 105 and downsample to like 20 Hertz that's probably fine and then everything else from there but like I said I'm pretty sure gamma and beta is going to give us the information that we need you know for control like if we're thinking left thinking right I think we're gonna need somewhere in these range okay so yeah if I forgot to say it too I pretty sure I can use sy PI for the actual resampling but it really is going to depend on if this is seamless or not I think it's inverse ting so the next thing is I just want to show an example because as you saw with the GUI you know that the Fourier transform that's nice because it contains all over the Hertz like it contains the 10 20 30 and it went all the way up to 60 which has me curious how are they getting 60 Hertz because I don't think you could accurately resample 105 to 60 Hertz so that's the other thing I'd like to ask is how are they doing that is it because they're probably not using Python they're probably using a much lower level programming language and it might be the case that in this case we don't want to use Python we want to use you know C++ or something like that so for the the actual raw data and then once we convert it to a Fourier transform or convert it to the you know gamma and so on you know reefs and planes then throw it into Python so who knows okay so now to visualize what this data looks like over a much longer time frame as well as in the raw form and why we kind of sample at different rates I'm going to just use the following code so this code just reads from a file that way I don't have to have the headset on all the time and this is sort of mimics what it would do if it was reading from the headset at let's say 105 frames per second or 105 Hertz so cool so what we're gonna do is I'm trying to think if there's anything here I really need to explain it's just gonna iterate over and then sleep for that timeframe unfortunately matplotlib can't plot at 105 frames per second anyways but I just want to visualize the data and then we're just breaking after the first first one anyways but we can say here like how many seconds do we want to slice let's start with one second and then we're going to just do channel eight so this channel eight in theory like swim Python since we index from like zero one word really channel eight would be like Channel nine on the headset but not that it matters at this stage so anyways and then colon comma this just means basically like the sequence of data that we're working with this here this is an array of arrays and those arrays are those channel samples that you saw in the console before so when we say colon comma 8 we're saying we want the eighth index of all of the arrays in whatever array were indexing or we're referencing rather and that would be new data and new data is just a slice of data ok so basically we're just graphing channel 8 so if I run this we should see yeah here so here's one second so you can see there's like these big fluctuations here but then it's kind of hard to see but if we just kind of went through the mid there's even like a little bit of fluctuation going on there so like if we go and let's say we graph 5 seconds you can see it even a little more clearly here we solve all those little fluctuations but in general there's also a more macro fluctuation going on here right that might not be perfect but if you were to you know draw like a moving average or truly resample the data you would have something like what my mouse is doing right it would just be in this middle region so not only do we have like these like really granular waves we've got a little more macro of a wave going on this this much larger motion that's happening then if we go even further out let's say like 20 seconds again there's there it's getting a little messy and this is why we to sample down but even as we go out like if we go based up like kind of up here I guess you can kind of see okay there is some sort of like downward trend it looks like right here right but again if we were to continue to zoom in though we'd see those familiar high granular you know bumps basically whatever you want to call those where it's just these like zigzags and that's as granular as the data gets so now if we let's say we did 180 so what would that be three minutes so this is three minutes of that brainwave data Channel eight now you can see there's even some some even grander thing is going on like we thought it was going down but actually on the grand scale over three minutes is actually kind of going up so this is kind of why like we have like very very macro and micro and why we might want to sample in between because as you've seen there's all cut there's all kinds of different things so inside this gigantic wave we've got these smaller waves doing things so there's there's just a lot of information you know enclosed here so that's kind of why we want a sample at different rates one just to better understand what's actually going on but to because at least science tells us that the more more granular of the granularity I suppose or the sampling rate can inform you on different things so this much larger graph I don't exactly know which one it is but I would say this is probably one of our you know maybe a delta wave or a theta wave right well this information is very grand information you know if someone was really educated they probably could tell if you were sleeping or not you know or dreaming in your sleep and so on so anyway cool so that's just some basic intro to you know open BCI getting this information in Python now to continue I've got to learn about you know resampling this data can we fix the continuous nature of the data what kind of neural network am I even gonna try to use and so on lots of things to look forward to so definitely stay tuned with that I have the best job and it is thanks to Channel members and my new channel members are vegan software cam Thiebaud peffer le jáchym Rama Sri Conte / alum Singha Sens Jose cow Paz you Katya var and Ramesh Raghavan thank you guys very much for your support and of course all of you channel members I really really appreciate the support you guys are amazing and I think that's it if you got questions comments suggestions Corrections all that stuff feel free to leave them below honestly I'm really learning all kinds of new stuff here so yeah so feel free to chime in if you've got anything you want to say okay so I will see you guys in the next video

Channel: sentdex

Views: 125,690

Keywords: brain, computer, interface, eeg, bci, openbci, python, tutorial

Id: Dgo7F-lpyYE

Length: 22min 53sec (1373 seconds)

Published: Mon Oct 28 2019