See in complete darkness with touch

imagine if your phone could see for you this could be really useful in low visibility situations but more than anything it should be useful for people who are blind this should be possible because smartphones and tablets have some pretty incredible sensors in them they know where they are in space and what's around them if you could somehow communicate what the phone is seeing with an interface that doesn't require a screen something you can touch or hear it could actually be a pretty good replacement set of eyes I've been thinking if I could do that it might actually be useful invention in contrast to a lot of stuff I make it took me a few tries but I did eventually get a system working this iPad is running an app that I wrote which is talking to a kind of unusual case for it and what it does is it's communicating what it sees into my hand through a tactile interface right here this allows me to walk around blindfolded and not hit things I think if I refined it a little bit more it could be a Braille display it doesn't have enough resolution to do that right now but there's no reason it couldn't I don't think so inside of this there's a pretty cool neck anism which I'll show you I also spent way too much engineering time trying to prank my wife she's kind of on red alert now so I had to come up with a complicated scheme to get her rate where I wanted her so as usual this project was a lot harder than I was thinking that it was going to be yeah cuz you did it the hard way if you just would have used a bunch of motors like I said it would have been a whole lot easier that way know what we should have done is made an entirely new actuator from scratch I'm telling you nitinol is the future alright alright so I did have a lot of internal debate on the best way to build this thing there's actually several different all good ways to build this I went with this approach because I thought it was the most interesting there's a really cool mechanism inside of here I also want to thank San Juan Steve for nerd sniping with this idea and the comments from my last video also thank you for having a name that is WorkSafe San Juan Steve this totally distracted me from the project I was working on which is making myself better at playing this violin with the help of some robotics it may still be on hold because I have a couple of other really interesting projects that are competing for my attention so I've been I've been thinking a lot about guided missiles and if there might be any kind of constructive uses for them and I finally think I have it so imagine building the Ironman gauntlet that flies like a rocket onto Tony Stark's hand I've been thinking about this problem it definitely is dangerous but I think it might be borderline possible that would be really cool I'm also really interested in the idea of a baseball bat that doubles or triples the power of my swing with the assistance of a very modest explosive charge I think I have a legal an interesting way to do that as well if any of those things sound cool you should subscribe I give you my word that I will hold some all or none of those things I promise oh yeah and I did finally I set up a patreon if you want to help support me making more awesome things and videos this is a great way to do that these videos do typically take more than 100 hours each and also quite a bit of money in material and tooling and other things in exchange I'll be giving more behind-the-scenes info and in-depth project stuff so if that's interesting there's a link in the description alright back to the project I calculate that this thing gives me roughly twenty twelve hundred vision not the best but it's better than nothing what twenty twelve hundred vision means is that if I look at something with this that's 20 feet away it will look as clear as it would look to someone if I took that thing and moved it twelve hundred feet away going into this project I really wasn't sure how I was gonna do the sensing I was looking at using sonar sensors and off-the-shelf lidar unit I was strongly considering using the Kinect but seriously where are you gonna put this thing you gonna ask someone to just wear this on their head and you look like a complete idiot think about how people would treat you if you walk down the street with this on your head the the goal of this project is maybe something that can help humanity not hurt it and I then remembered that the new iPad has a lidar in it which is awesome the only problem is that it's only the latest iPad that has the lidar in it and I don't have one really think about it I don't have a choice here I have to get this it's to help humanity it's gonna help blind people let's think about it sorry lidar is one of those things that if it didn't exist and you proposed it to me as an engineer I would laugh you out of the room it just does not seem like it should exist it's kind of like ball screws where if you look at how they work it still seems like magic lidar is crazy because it measures the time that it takes light to travel from the iPad to an object reflect off it and then come back and this is crazy because light is mind-bogglingly fast it's going around 670 million miles per hour unless it's light in the EU or Canada in which case it's going about a billion kilometers per hour so the time that it takes to go from the iPad to a wall is on the order of five nanoseconds and to actually matter distance you have to measure much more precisely than this the lighter is actually sending out beams of light in all directions it gives you a bunch of measurements on the surface of everything that it sees and how far away it is from the iPad so this is an incredibly powerful technique for perceiving the world it's how most self-driving cars actually figure out what's around them the other big architectural question I had to figure out is how do I communicate what the thing is seeing using touch the proposal in the comments was to use some kind of vest but my limited experience with nerves and humans is that your torso isn't actually very good at sensing so we're gonna do a two-point discrimination test tell me if you feel one or two touches one so funny I'm like jabbing you with two screwdrivers I know it's just how your body works but it's really kind of fun all right now turn around so I can test front that's probably a good idea is it your back isn't so great for tactile input but your hands are incredible not only do they have good resolution so they can distinguish lots of touches they also are extremely sensitive if this device could be a phone case I think that's legitimately more useful a vest you have to wear it and have all these sensors whereas your phone you have your phone and it helps you see so let's imagine that the iPad is seeing this it's a room with a door in a chair what I do is I convert this into the simplest representation possible for navigating the space right here there's something that you don't want to hit that's very close that's the chair I have some stuff that's further that's the walls and then everywhere else is clear I can actually communicate this in your hand because there's not so much information if something is really close I push really hard on your fingers at those corresponding locations if something is there but not as close I can push not as hard where it's clear I don't push at all this gives me basically a course map of what's going on in the world on my hand so I also made a couple different views so if you want to zoom in and get more detail I have the ability to zoom I also have an experimental top-down view so this is the chair this is the wall looking down from above I can say that this is no go this is open and these areas are unknown because I can't see through the walls with the lidar remember I'm looking from this perspective so I can only see what I can see through the doorway I can communicate this to the hand the same way I do the other view it's just a different view of the room this view is kind of like the little mini map in a first-person shooter and the bottom of the screen if you know I'm talking about I think this could be a pretty nice view with more improvement but this problem of things blocking other things happens a lot in practice and so it can be really difficult to interpret what you're looking at to make this work I had to find a way to poke all of my fingers simultaneously with a controllable amount of poke and that's hard because I would like this to be an iPhone case and that means I really this is about as big as I can go kind of a grip shape so if I just try to do the simple thing and cram a bunch of servo motors in here attached each pin there's just no way that's gonna fit especially if I try to scale it up so it can be a Braille display which i think is important and very useful so the big challenge for this project was finding a way to move all those pins with a compact and ideally simple system so what I wanted to do is make something that had a little metal pin that I could push into every joint of every finger and the way that I did this imagine that I have one of the pins that I want to move pressed against a cylinder that I can rotate if I attach a little wedge to this cylinder when it goes underneath this pin it's going to force it up then if I make an even bigger wedge and I move my cylinder over I can spin the cylinder and then push this pin up even higher all that I have to be able to do is can is rotate the cylinder and shift it left and right to control how high up this pin goes and then because it's a cylinder I can pack lots of ramps and lots of pins into it very compactly here's what some well used ramps look like IRL that big bump next to them is used to trigger the pin locking and unlocking mechanism it's basically another ramp and this keeps the pins in place when they're not under a ramp and then you can see that I can control where the cylinder goes with two opposing motors by driving them together it rotates and by driving only one of them I can move it left and right planning the motor moves for this system is a lot harder than I expected mostly because I wanted to move one motor at a continuous speed and then have the other one keeping pace it was a lot of quadratic equations and now the pain and joy of making this and sixty seconds or less [Applause] [Music] [Music] this electronics and the only thing that really mattered for proving the system is that they work so I made zero attempts at miniaturizing them and they're very simple there's just a microcontroller Bluetooth communication module and then two big fat stepper motor drivers if I was trying to make this a real product I'd be trying to fit it onto a little circuit board which there's no reason it couldn't this is just what I had on hand I should also point out that these motors are massively overkill for the application it doesn't take much power to raise a little pin I went with these partially because they're what I had on hand and also because it's convenient to have a lot of extra power when you're prototyping now we don't worry about stalling and other stuff it's been almost a decade since I last wrote an iPad app and it's gotten a lot easier and this is the app it's really quite simple I have all the controls on the side for the different views and for zooming and these are all accessible when you're holding the grip on the side there's a little preview of the depth data and this just helps me see what's going on ideally this would be running on a phone but the lidar is only in the iPad if Apple puts the lighter on the iPhone you could imagine it could just be a phone with a little case on it which would be really cool that's pretty much all there is to this app Apple makes it really easy to work with the lidar data I spent a lot more time on the embedded system trying to get the motor profiles working and it was kind of waste of time but I spent probably more time trying to get Wi-Fi mode working than anything else in this app I really wanted to see my wife walk into something so I set it up to initially tell her correct data then after a bit start giving her bad data probably just a bug it's a complicated system this thing is really weird to use it's a lot less intuitive than I was thinking it would be you can feel all the Pokes but I think about every finger and what it's feeling to try to create a mental image of what's in front of me although I do think that if you used it a lot it it might become second nature I remember learning the type was pretty similar to this it also lets you see things like overhead obstructions so you can get under them although it's since it's only giving two levels of depth it was hard for me to figure out where the obstruction actually was so and you can see me pointing the iPad up to see if there's anything above me I would really like to see more touch points and more control of those touch points because you really can feel the dip it's between these different pin pressures I think you could get a much nicer picture of what's around you if they could push with more more granularity if I was doing another revision or trying to make this as a product I would be trying to find some little tiny actuator that I could buy that would allow me to directly control every pin I think it's a cool proof of concept and hopefully you like it as well so a lot of people would point out that I have a lot of tools and that I may in fact even have a problem which is probably true but I would have an even bigger problem if all my tools got stolen though I do spend a lot less time worrying thanks to this video sponsor simply safe I spent most of my adult life and my adult dollars building out this shop and the idea of someone busting in and stealing my tools makes me want to curl up and die well though let's be real if someone manages to steal this they can have it I'm only joking that's not a challenge please don't rob me although if you do try to rob me you're busted because simply safe detects basically every possible intrusion and cause the police immediately you're gonna kick down my door you just triggered my interest sensor you're gonna break my window you just triggered my glass break sensor you're gonna cut a hole through my wall say hello to my motion sensor then wave to me in the police dispatch alarm at HD security camera you try to break in when the system is disarmed boom they've got a panic button simply safe protects against outsiders but it also protects against me and honestly with all the stuff I get up to I'm probably the biggest danger here I've got a fire alarm a fax and I leave a smoldering heap in the basement that catches fire once I'm gone I'm gonna get notified got a carbon monoxide sensor if I leave my jet engine running and it makes a bunch of carbon dioxide it's gonna let me know before it kills me the number one thing for me though is the automatic deadbolt the number of times I've come back to see if the shop is actually locked as way too high do you get notified if you forget to lock the door you can lock it remotely you can check it super handy so if I was gonna rob your house the first thing I do is I cut your power and your telephone although if you'd simply say I'd be a fool because it has a battery backup and use a cell service it's super reliable usually ordered online or over the phone and they ship it straight to your house you stick the sensors wherever you want them and hit a few buttons and that's about it doesn't even take an hour and it's only fifty cents a day with no contract that's cheaper than my previous security system which had a contract so have a lot more peace of mind with simply safe knowing that you're protected around the clock from basically anything you could think of visit simply safecom slash stuff made here to learn more or click the link in the description

Channel: Stuff Made Here

Views: 4,150,430

Rating: 4.9650998 out of 5

Keywords: awesome inventions, incredible invention, electronics, cool electronics projects, how to, electronics projects, engineering, tactile interface, make, ipad case, tech, ipad pro 2020, computer vision, arduino, 3d printing, 3d printed, lidar, iPad, technology, diy, diy projects, brilliant ideas, diy ideas, prototype, devices, robot, arduino projects, augmented reality, Lidar sensor, build video, project video, engineering project, shop, machining, stuff made here

Id: 8Au47gnXs0w

Length: 14min 12sec (852 seconds)

Published: Sat Jun 20 2020

Reddit Comments

[u/Fendag]

I like the idea of becoming a dedicated inventor just to troll your wife in your test runs.

[u/[deleted]]

This is super cool. Wonder if you could scale it down to the size of glasses, so you could get the tactile feedback on your forehead/nose/ears, that way you don't have to walk around holding something in front of you.

[u/Choice_Garbage]

I admire how smart this guy is but I imagine a cane would be so much more practical even if this were fleshed out into a product.

[u/[deleted]]

https://en.wikipedia.org/wiki/Pin_Art

[u/arseiam]

I have a blind buddy who uses a braille attachment for his iPad. It has a series of 12 braille pads that actuate based on-screen content. It's been life-changing for him.

The concept of haptic feedback is not new and products exist already, what I find interesting in his software that readfs the environment and gives feedback. I'd be keen to see a more commercial version of this that can work with exitsting braille tech.

[u/JSHOWCLIPS]

Nice

[u/Points_To_You]

I feel like audio feedback would be better than touch. It doesn't have to be like a speaking voice, it could just be a system of beeps and tones that gets progressively louder for closer or more danger.

Hard to say without trying it, but I think it could be done in a way that most of the time it is just background noise that you are unconsciously following most of the time.

[u/sex_panther_by_odeon]

We are automating the jobs of guide dogs! They better start a Union.