Mind Reading For Brain-To-Text Communication! 🧠

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments

We're getting closer bois

👍︎︎ 13 👤︎︎ u/philsmock 📅︎︎ Feb 13 2021 🗫︎ replies

Having to imagine writting is way to slow. The best method would be to "listen" to the words you think of.

👍︎︎ 8 👤︎︎ u/MrDreamster 📅︎︎ Feb 13 2021 🗫︎ replies

Let's go! This is so cool.

In 2030 we will live in a different world then we live today.

But I'm still waiting for full immersion vr where i can do/feel everything i want.

👍︎︎ 5 👤︎︎ u/MilkyWahhh 📅︎︎ Feb 14 2021 🗫︎ replies

What a time to be alive!

👍︎︎ 4 👤︎︎ u/attrackip 📅︎︎ Feb 13 2021 🗫︎ replies
Captions
Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today, we are going to read some minds. A few months ago, our first video appeared on brain machine interfaces. It was about a paper from Neuralink, which promised a great deal. For instance, they proposed a brain machine interface that could read this pig’s thoughts as it was running on the treadmill. And what’s more, they not only read is thoughts, but they could also predict what the pig’s brain is about to do. So this was about reading thoughts related to movement. And to be able to use these brain machine interfaces to the fullest, they should be able to enable some sort of communication for people who have lost their ability to move or speak. So about writing or speaking? As impossible as it sounds, can we somehow restore that, or is that still science fiction? Many of you told me that you would like to hear more about this topic, so due to popular request, here it is, a look beyond Neuralink’s project to see what else is out there. This is a collaboration between Stanford University and a selection of other institutions, and it allows brain-to-text transcription, where all the test subject has to do is imagine writing the letters, and they magically appear on the screen. And now, start holding on to your papers, and just look at how quickly it goes! 90 characters per minute with over 94% of accuracy, which can be improved to over 99% of accuracy with an additional autocorrect technique. That is absolutely amazing. A true miracle. 90 characters per minute means that the test subject here, who has a paralyzed hand, can almost think about writing these letters continuously, and most of them are decoded and put on the screen in less than a second. Also, wait a second, 90 characters per minute? That is about 80% as fast as the average typing speed on a smartphone screen for an able-bodied person of this age group. Whoa. It is quite remarkable that even years after paralysis, the motor cortex is still strong enough to be read by a brain-computer interface well enough for such typing speed and accuracy. It truly feels like we are living in a science fiction world. Of course, not even this technique is perfect, it has its own limitations. For instance, we can’t edit or delete text, have no access to capital letters and the method has a calibration step that takes a long time, although it doesn’t get significantly worse if we shorten this calibration time a bit. So how does this work? First, the participant starts thinking of writing one letter at a time. Here you see the recorded neural activity, this is subject to decoding. You can see the decoded signals here. And we can’t just give this to a computer to distinguish between them as is, we project these into a 2D latent space where it is easy to find which letter corresponds to which region. Look, they form relatively tight clusters, therefore it is now easy to decide which of the squiggles corresponds to which letter. The decoding part is done by using a recurrent neural network, which is endowed with memory and can deal with sequences of data. So here, in goes the brain activity, and out comes the decision that says which character these activities correspond to. Of course, our alphabet was not designed to be decoded with neural networks. So here is an almost science fiction-like question - how do we reformulate the alphabet to tailor it to maximize the efficiency of a neural network decoding our thoughts? Or simpler, what would the alphabet look like if neural networks were in charge? Well, this paper has an answer to that too, so let’s have a look. The squiggles indeed look like they came from another planet, so what do we gain from this? Well, look at the distance matrix for the English alphabet. The diagonal is supposed to be very blue, but what is not supposed to be blue at all are the regions that surround it. Look, the blue color here means that in the English alphabet, the letters M and N can be relatively easily confused. Same with the letters O and C, and there are many more similarities. And now, look, here is the same distance matrix for the optimized alphabet. No dark blue in sight outside the diagonal! Much easier to process and decode. If neural networks were in charge, this is what the alphabet would look like. Glorious! Also, the fact that we are talking about squiggles is not a trivial insight at all, traditional methods typically rely on movement in straight lines to select letters and buttons. The other key thought in this paper is that modern neural network-based methods can decode these thoughts of squiggles reliably. That is absolutely amazing. And wait a second…note that there is only one participant in the user study. Why just one participant, why not call in a bunch of people to test this? It is because this method uses a microelectrode array, and this requires surgery to insert, and because of that, these studies are difficult to perform, and are usually done at times when the participant has brain surgery anyways for other reasons. Having more people in the study is usually prohibitively expensive, if at all possible for this kind brain implant. And note that research is a process, and these papers are stepping stones. And now, we are able to help people write 90 characters every minute with a brain machine interface, and I can only imagine how good these techniques will become two more papers down the line. And don’t forget, there are research works on non-invasive devices too! So, what do you think? Let me know in the comments below. What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!
Info
Channel: Two Minute Papers
Views: 166,934
Rating: undefined out of 5
Keywords: two minute papers, deep learning, ai, technology, science, machine learning, mind reading, brain to text, neuralink, braintotext, elon musk, neuralink monkey, neuralink mind pong, neuralink pong, mindpong, Krishna Shenoy, willett, brain machine interface, mindwriting, mind writing
Id: IUg-t609byg
Channel Id: undefined
Length: 7min 36sec (456 seconds)
Published: Sat Feb 13 2021
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.