The billion dollar race for truly smart glasses

the Apple Vision Pro has brought the excitement back into the VR and AR space I tried both it and the quest 3 and I'm pretty convinced that eventually some version of this idea will turn into a mass Market product even if it's going to take a while but what about turning these things smart are we ever going to get to the point where smart glasses are a real mainstream product category well I duck through all the research and I think the answer is yes and also pretty [Music] soon this video was sponsored by ground news while meta and apple are currently busy parading their VR or I guess spatial Computing headsets around we know that both companies have in fact worked on AR devices with see-through Optics as well meta has clearly confirmed that they're working on multiple devices even now and has had their whole device road map leaked but they've also admitted that AR devices are just a way harder challenge for now in a lot of ways augmented reality is even harder not just because we need to invent a complete completely new Optical stack that's not based on screens but because we basically need to fit a supercomputer into a pair of normal looking glasses and meanwhile plenty of leaks and patent filings showed that Apple was working on its own glasses too but that the vision pro team actually won the internal competition and their see-through AR devices were shelved for now so both companies and also basically every analyst that I could find agree that making augmented reality glasses is just significantly harder than making virtual reality glasses and to understand why we have to understand how augmented reality glasses actually work starting with the displays and the Optics which offer the first massive challenge as they come with loads of inherent downsides unlike in a VR headset where we can put really high quality screens and many thick lenses polarizers and a dozen other things directly in front of our eyes in smart glasses we typically can't your first instinct might be to say that hey we can just use transparent screens but the human eye sadly can't focus on anything as close to to your eyes as your classes so even if the displays themselves were transparent we would literally just look through them in VR we solve this by placing Optics between you and the screen that move the focus distance of the screen to a couple of meters away from you but we can't do this with see-through glasses because then the screen itself might be in Focus but actual reality no longer would be there is no known solution to this problem right now and so instead the actual displays in AR glasses are put somewhere on the side then the light from those goes through a complicated set of Optics that can refocus the image and the thing that we end up looking at is actually just a reflection you can do this in relatively straightforward ways like on this Lenovo Legion glass here a simple full color micro olet screen is placed inside this little bulge above your eyes it shines through a bunch of optical elements down into this thing that they call a prism inside of which there's this thing called a free form combiner which is basically a fancy curved semi-transparent mirror that projects the now properly focused light at your eye while still being somewhat see-through so you can still see the world through it another simple solution is what's called a bird bath design which you'll find in glasses from xre roet Etc once again we have a screen on top and that is shining down this time you have a straight mirror called a beam splitter that first shoots the image away from you into a curved mirror on the front then reflects the now properly focused image back into your eye these two are sort of the simplest designs they're pretty common and they're usually branded as kind of portable monitors in a way because you can just plug them into a game console your computer or something else and have a large screen with you all the time then on the more advanced extreme we have something called waveguide systems here a tiny projector sits typically in the stem of your glasses from where it shines into the sheet of glass and onto a set of tiny mirrors these reflect the light waves sideways so that they travel through the glass itself bouncing off the sides as they get guided to the part of the glass that is in front of your eye and there finally another set of mirrors in the glass bounce the image back out and into your eyes the lens in your glass basically acts as a sort of fiber optic cable that guides the light waves hence why we call these wave guides and this is generally the preferred method for thin and light glasses as you just need a single slim surface for the whole thing to work then option number three is what we saw on the focals by North classes which has sadly actually shut down by now they had a tiny laser beam scanner built into the stem of the glasses which they projected onto a holographic film which then reflected the image back onto your eyes so you saw a really simple heads up display and then the last design that I'll show here is something called a mixed wave guide by a company called Ant reality that actually uses two displays one on the bottom and one on the top and then uses wave guides to get those to meet in the middle and to combine into a single really wide 120° field of view image there are of course a million other Solutions but these are the basic ones that you have to know about for now and of course they all have their tradeoffs the really simple designs that we showed in the beginning are inherently bulky since you have to have both the display up top and then also some really clunky Optics Below Wave guide systems are historically hard to make and really expensive too with ENT reality for example quoting $280 per eye and the laser system from focals is limited to a tiny low resolution surface which also has to be perfectly aligned with your eyes for it to work which means custom fittings when you get your glasses and always having to fiddle for the right alignment and regardless of the specific designs transparent displays also have some just fundamental characteristics that are not exactly great first all those Optical Elements by definition can't be completely invisible from all angles because otherwise they wouldn't be reflective this means you always end up with some visible artifact and also often warping on the glasses themselves but perhaps it's even worse that the displays work by by adding light on top of an environment which pretty quickly ends up with your glasses having to outshine other lights and if you go outside also having to outshine the sun Outdoors you would apparently need around 4,000 knits as a minimum which is far brighter than any phone screen and even indoors any bright object will simply outshine your image oh and here we're talking about the brightness of the reflection that you're looking at which first had to go through a bunch of lenses wave guides and polarizers that typically lose 60 70 or even in many cases over 90% of the brightness on the way to your eyes so your actual screen would ideally have to Output some multiple of 4,000 nits that is mind-bogglingly bright especially for a device that has a very small battery and also very limited capabilities for dissipating heat so it's no wonder that most AR glasses come with something like Shades and there's actually a second reason for using Shades that goes beyond just the brightness issue as well see a transparent display can only add light not substract it which means that it can't actually show black or anything dark really so you will never get a proper full color high contrast image unless you completely black out all the light from around you now at that point you're kind of just back to creating VR glasses but with way more steps and so I think that AR glasses will never actually truly become the best way to consume really immersive content now of course some glasses just have permanent Shades attached to them or they offer physical clip on Shades which is a low Tech way of kind of solving this problem but we've also gotten adjustable shades on high-end devices too the first option here is to add a layer of liquid crystals and some polarizers that then allow you to control the dimming basically instantly with the same Tech that LCD TVs use to control their brightness magic leap has even shown a version of this where they don't just dim the whole display but actually just the pixels behind whatever object the the main display is trying to illuminate which means they're trying to create local dimming I find that conceptually kind of funny like it's the opposite of what we're trying to do with TVs and other regular displays right where we're trying to put tiny background LEDs to only illuminate certain pixels where here we're doing the opposite and we're trying to black out those certain pixels sadly the local dimming is just really imprecise and just the Liquid Crystal system alone needs a ton of different layers that adds bulk and it also blocks out around 60% of the light even when it is its most see-through State according to AR expert called gag so these things are always kind of dark and only really have limited use cases and the final and most promising option seems to be electrochromatic dimming using gels these do take a few seconds but actually seem to be working really well and if you've ever been in a Boeing 787 for example you might have experienced basically the same Tech on those windows already so that's cool but of course no matter which shading solution you end up using all of them add complexity cost and thickness while they also take away from some of the inherent promise of having a transparent display in the first place I think you can already tell that AR displays and Optics are just an incredibly difficult thing getting something that is thin efficient looks good in all conditions is just very very hard and this is an Optics problem and Optics as a field is not something that moves very fast so you can't just wait for Moors law to hand you a solution in a couple of years I think it's telling that the first kind of successful pair of consumer smart glasses which is the MEA Ray bands actually don't have a display at all but that said there's also definitely progress that is worth being excited for I've already talked about ENT reality super wide- angle systems which are also super high resolution and fairly compact but another interesting example is a company called lumus now this is a component supplier so the demo device that you're seeing here is not a final product but what they have delivered so far is a system with a tiny projector and the new Ultra efficient wave guide system that gives you a color display with a pretty respectable 50° of field of view a pretty high 1440px 1440p resolution over 3,000 nits of brightness that should approach being bright enough to be worn even outdoors and all of that in a package that is only 20 G this is also reportedly not prohibitively expensive and they even claim that they have successfully integrated prescriptions by partnering with a company called Lux XL which meta AKA Facebook has since acquired that is a lot of boxes checked real experts like call good have seen and more or less confirmed these claims and lumus is reportedly ramping up manufacturing for mainstream consumer products right now so something like this could finally become the big leap that we've been waiting for okay enough about the screens and the Optics we have to talk about well everything else starting with the chips since qualum is the clear leader in the space Here's a list of Snapdragon chips made for headsets by release date chips that are named XR are meant for bigger VR headsets like The Meta Quest series or high-end AR headsets in the class of a Hol lens for example with a modern example of this being the digilens Argo XR series chips were at least originally slightly modified smartphone chips and have similar performance characteristics too meanwhile the newer Edition is the so-called AR series which is obviously aimed at smart glasses like The Meta Rayband series these chips are supposed to draw less than one watt of power but can somehow still support displays up to 1280 x 1280 plus they also have an image signal processor that is good enough for decent cameras and an mpu or a neural processing unit that is good enough for running a basic voice assistant like we get on the meta Ray bands and the spec sheet of the next gen glasses even mentions support for ey tracking cameras which means that we're basically at a point now where we have decent enough chips to support basically all the things that we'd expect simple smart CL glasses to do you can see that the pace of these new releases has increased a lot lately and these chips are also now getting radically better with each generation as Qualcomm takes development more seriously which I think means that the chips are no longer going to be the real bottleneck Now battery science on the other hand is a really slow moving field again and this sadly does create real hard limits on how much you can miniaturize the headsets The Meta Rayband for example have a tiny 0.59 WTH battery for context that is half to a third as much as most SmartWatches 33 times smaller than a meta Quest 3 and 61 times smaller than the Apple Vision Pro clearly this massively influences what you can and cannot do in a given form factor I've made a whole video about various upcoming battery technologies already which you can watch here but for our purposes things are unlikely to fundamentally change in the next few years and finally we have to talk about inputs in virtual reality most companies that are not called Apple use dedicated Hardware controllers but obviously in AR where we're supposed to interact with real life objects and you know the entire world around us that is clearly not an elegant solution so instead we have a couple of options first we could track our hands like we do with division Pro and the quest that is convenient and the company xreal has shown that it is also possible to do in AR glasses but to get really good results here you would need multiple cameras and they would have to point downwards as well I imagine it would be extremely difficult to fit that onto actually elegant smart glasses like imagine putting four cameras somewhere here bulging out from the frame something like this plus also I think your cheeks would kind of block them you know like on VR headsets we kind of come out of the head a little bit more so there's more play it can see you in more angles but in AR glasses that would be blocked so yeah it's kind of awkward So unsurprisingly Meta has said that they are instead betting on neural interface bands basically EMG bands that sit on your wrist that read your nerve impulses that your brain sends to your hands as you move your fingers apparently these work so well that you can swipe and even type full text as if you had a keyboard and the band converts all of that into input like you don't even necessarily have to move your hand but you just have to think about moving it to get the nerve impulses and you apparently teach a system where you think a key is as you press it and it learns to recognize that as input there is extremely sci-fi stuff like you apparently don't even have to like match a physical keyboard or a specific design that the company gives you instead the machine learns where you think the various keys are and turns that into a keyboard so that's crazy people have already tried it and apparently it works MAA acquired a company called control lab in 2019 specializing in these machines for something like 500 million to a billion dollars and they claimed that this Tech is almost ready for consumers in an interview with the morning brew a few days ago Mark Zuckerberg said that these are coming soon and from their leaked timeline we've learned that they're supposed to launch along the Next Generation Rayband glasses in 2025 with support for VR devices expected later too now having to sell consumers both on a new pair of glasses and on a new wristband is going to be hard especially if both are bulky and expensive so you know this could go either way but zck seems really optimistic so let's see how this this goes also a finished startup called double point is already doing a very basic version of this using the sensors of off-the-shelf SmartWatches plus Apple of course has also shown double tap actions on their apple watches so I imagine the core idea could eventually be integrated into regular Sleek SmartWatches somehow too even if in a very simplified form and the second input method that a lot of companies are working on is of course eye tracking of course the problem here is the same as it was with the hand tracking which is that you have to place cameras somewhere on your head and that's going to be really difficult to integrate into cool looking designs that are sleek and modern but still there are two potential promising technologies that I found first a company called Toby has embedded the cameras and the illuminators needed for a full eye tracking system into a really compact lens form factor I'm not sure if this design wouldn't be distracting for both the user and others around them but it sure seems to be Compact and second are so-called event-based cameras which both Stanford University students and a company called Zin Labs have proven to work for eye tracking these are special cameras that don't actually capture a full image but only the change from one frame to the next this radically reduces how much data you need to collect making constant on device processing possible and this also makes the camera systems way more Compact and efficient you can see that even in early prototypes these are quite small and conveniently the leading company that makes these event-based cameras is prophecy which quac invested into not long ago hm and so given all that we know about the research pipeline I can now tell you what I think is realistic to expect to come to headsets and smart glasses on the market in the next 5 years what we definitely won't get anytime soon is a full ER headset like the Hol lens that tracks its environment places virtual objects on top Etc all in a form factor that looks anything like your regular glasses we are nowhere near this as you would need a high-powered Snapdragon XR chip big batteries multiple cameras Etc now the chip chips and displays and whatnot are getting a lot better and meta's leaked road map shows that they expect to bring their first real full AR headset to Market in 2027 but I don't expect any of these to become mainstream consumer products before I guess at least 2030 Palmer lucky the founder of oculus said that you basically can't make these AR headsets really compact the only thing that you can do is design them in a way that they appear cool and then also like socially engineer your way into them becoming acceptable and maybe even fashionable is that possible I guess maybe but it's going to be really hard for the slim and light form factor on the other hand where you basically just show a souped up simple heads up display instead there's a perfect storm building on the horizon right now the new meta Ray bands have proven that you can already do cameras microphones speakers and a voice assistant that can even analyze the things that you're looking at at least when paired with your phone and all of that in a really compact form factor and meanwhile smart glasses like the Mau my view which is an actual product that is available for purchase for about 3 $330 in China right now well those prove that you can also have a proper slim wave guide display in a decently sized pair of glasses that doesn't break the bank either I've linked to a review of this pair in Chinese that is in the description and while it's a bit more bulky than the ray bands I think if this was designed more Tastefully you could probably make these socially acceptable in the US a startup called brilliant Labs has caught a lot of attention already combining many of these ideas into one open AI powered device as well I honestly can't believe that the final product will look this Sleek but the direction is clearly there meta is leaked to Target a 2025 launch for their next gen bands with a basic viewfinder display and an EMG wristband too and I'd be shocked if Google didn't build pixel glasses in the next few years too after all they already tried with Google Glass they bought North the company that made focals plus they of course also have an extremely capable and popular AI assistant the hardware Tech is finally coming together in a way that that it clearly wasn't ready when the original Google Glass came out and also since we finally have kind of really smart voice AI assistants now especially ones that can also look at their surroundings using their cameras this might be a real gold rush that I think a lot of companies going to try their luck at mea has already said that even the current rayb bands are a bigger success than they expected and I think they are going to go all in on this and if in 2 three four 5 years we also get proper neural bands through an actual Smartwatch plus then also halfway decent eye tracking then I think the future of smart classes is suddenly going to look very very exciting like imagine being able to air type concrete messages without taking your phone out of your pocket or being able to look at a specific object or maybe even a specific person and asking a really act smart AI assistant about them I think all of these things are on the horizon of being possible and if they come true then they're going to be real game changers now when I research a topic like this I have to go through a million different news news reports press releases patent filings Etc and much of that ends up being quite frankly being able to get to the bottom of any story is just a tremendous challenge but that is where today's sponsor ground news becomes a real asset ground news is a website and an app designed by a former NASA engineer on a mission to give readers an easy datadriven and objective way to read the news every story comes with a quick visual breakdown of the political bias factuality and ownership of the sources reporting and all of that is backed by ratings from three Independent News monitoring organizations for example take a look at this story about neurer Link's first human implant right away you can see that there's over 400 Outlets that have reported on this story out of these Publications 24% are left leaning and 26% are right leaning which is pretty balanced reporting you can also see the factuality ratings for these Outlets as well as their owners 52% of them for example are owned by media conglomerates you can even easily compare head headlines to see how biases might affect their framing for example many headlines refer to the details as being scanned I also really like the blind spot feed which highlights stories that are being disproportionately covered on one side of the political Spectrum for example if you lean right you might have missed this story on journalists and lawyers being hacked by the Pegasus spyware ground news is one of those tools that just make sense it gives you the necessary context to create balanced and smart media consumption habits you can go to ground. news/ te alter to get 40% off your ground news Vantage plan which includes another great feature called my news bias which is basically a dashboard for your news diet and allows you to track your reading Trends over time so go to ground. news/ tear or follow the link in the description to check out what your reading habits look like especially over time and to support an independent news organization making media more [Music] transparent

Channel: TechAltar

Views: 257,505

Keywords: AR, VR, XR, MR, spatial computing, vision pro, apple vision pro, meta quest 3, oculus, facebook, zuckerberg, zuck, mixed reality, virtual reality, smart glasses, augmented reality, techaltar, tech altar, analysis, friday checkout, the friday checkout, waveguide

Id: VhFKKvKO6sU

Length: 22min 0sec (1320 seconds)

Published: Tue Mar 05 2024