Apple Watch Series 8 : Full Scientific Review

over the last four days I've extensively tested the new series 8 Apple watch for its Sports and Health tracking performance in this video we will take a comprehensive look at the heart rate accuracy sleep tracking performance GPS tracking oxygen saturation measurements and step counting accuracy of this brand new watch from Apple I will also make a few quick remarks about the new temperature sensor that Apple added hello everyone for those of you that are new to the channel my name is Rob and I'm a postdoctoral scientist specializing in biological data analysis as I mentioned in my first video I think there are plenty of resources out there from which you can get the specs of this new series 8 Apple watch like Apple's website for instance so in this video I will focus on Independent performance testing of this new Apple watch so without further Ado let's get to the the test results and I want to start off with the heart rate tracking results I showed you an initial test in the last video in which I did five Training sessions but I now more than tripled that data which means I tested the Apple Watch series 8 during a total of 16 training sessions to test that I'll compare the heart rate measurements of the Apple Watch series 8 against the polar h10 ECG chest strap which can generally record my heart rate very accurately as I showed you in a recent video we'll start by looking at one of the easiest types of exercises for a watch to track cycling indoors and I'll be looking at a total of three interval spinning sessions now this involves very little movement or tension on my arms and will therefore produce less noise and here we can see an overview of the accuracy over all of the rides each dot here is a single heart rate measurement with long horizontal axis the value according to the polar h10 ECG chest strap and on the vertical axis the value according to the watch 8. now the closer the points are to this blue line the better the agreement and the darker black the color the more dots there are as you can see there's a really good agreement between the watch 8 and the ECG Jazz strap as basically all points are super close to the Blue Line the correlation this r value up here is also really good at a rounded value of 1.00 when we allow for two decimal points this correlation cannot be higher than one so a rounded value of 1.00 is more or less perfect we can see why that is if we actually look at the individual training sessions and here you can see my first interval spinning session for instance along the horizontal axis we have the time and my heart rate is along the vertical axis in blue green I upload my heart rate according to the polar h10 ECG Jazz strap and then red is my heart rate according to the watch 8. as you can see the two lines overlap very well to the point where you can basically not see the red line at all so that means there's a good agreement and this is basically what we see for all of the spinning sessions the Apple watch 8 does not show any significant deviation from the reference device in any of my testing so far so this is looking really good however let's put this into perspective by comparing it to roughly 60 other watches I've tested over the last two years that overview is displayed right here the correlation value I was talking about before is the metric I will use for this which is displayed along the horizontal axis right here we want that value to be as close to 1 as possible on the vertical axis I ordered the watches from worst to best so the further to the right and the higher devices the better is its correlation with the reference device now here I Mark the Apple watch 8 in red and as you can see the watch 8 has the best score out of any of the watches I've tested over the last years we can see that even more clearly if we zoom in and get rid of some of the worst performing watches the watch 8 has the highest score but is still very close in performance to the other Apple watches the old Apple watch SE the Apple watch 6 and the Apple watch 7. I'm not sure that it's actually performing better it might just be within the error margins of my measurements I suspect honestly that this is the case one surprising thing is that it performs even a bit better than wearing two ECG chest straps at the same time one explanation could be the error margins that you have in any measurement or the fact that at the place one HD chest strap in a lower position than recommended so it will wasn't completely ideally positioned overall this means that the Apple Watch series 8 is doing really well when it comes to heart rate tracking next let's briefly take a look at another easy type of exercise for a watch to track walking outside for which I did two tests these are the results for the first walk again in blue green are the measurements of the chest strap and then wrap my Apple watch we see more or less a perfect overlap here just with some small deviations and we see something similar for the second walk though there are a few more deviations as you can see for instance right here and right here overall it's still really good but these deviations do show that the Apple watch is not perfect so this is still looking really good but the extra movements associated with walking might just interfere a little bit with the measurements taken by the Apple watch next let's take a look at an even more challenging type of exercise cycling outside while cycling Outdoors watches tend to shift a lot more on the wrist making accurate heart rate readings much more difficult to see if and how this affected the Apple watch I tested it during a total of eight bike rides here we see a similar overview plot to B4 but now for biking outside as you can see there's still a really good agreement between the watch 8 and the HD Jazz trap though some of the points are just a little bit further away from the blue line now the correlation is also just a little bit lower compared to what we saw before for cycling indoors with the correlation now being 0.98 however this is still really good we can see that in even more detail if you look at some of the bike rides themselves as you can see the red line which is representing the watch 8 overlaps almost perfectly with the ECG chest strap in blue green and this is what we see for many of the bike rides like this one right here where we see a really good overlap between both the Apple watch and the chest strap there were a few rides that showed some moments where the Apple watch was not able to detect the heart rate like you can see here for instance why the watch 8 didn't not detect a heart rate for a minute or two and we see the same thing in the middle of this right right here where the Apple watch was more or less spot on when it did detect the heart rate but it did for some moments fail to detect the heart rate at all we can again put this into perspective by looking at many of the watches I've tested over the last years similar to before we use the correlation with the ECG chest strap as the value on the horizontal axis and the better the agreement the more it's operator devices as you can see the Apple Watch series 8 which I marked in red is again amongst the best devices when it comes to Hardware tracking and we can see that even more clearly if we zoom into the best performing watches the watch 8 performs about the same again as the Apple Watch series 7 and the older generation SE also some selected Huawei watches are close in performance for cycling outside however so far these exercises represent easy and medium hard exercises for a watch to track your heart rate so let's now move on to one of the most difficult exercises for a watch to track weightlifting as I've mentioned previously this makes it much more difficult because of the increased tension on my wrist and on my arm making it much harder for the watch to get a clean heart rate signal however first a quick side note if you're interested in the latest updates on the wearables I'm testing I'm planning to start back up with my newsletter and posting more Optical things on my Instagram and my YouTube shorts channel so if you're interested did in any of those those are linked below of course you would also make me really happy and it would help my efforts if you subscribe to this YouTube channel but of course again this is totally up to you now enough self-promotion let's take a look at the performance of the Apple watch during weightlifting here we again see an overview of the accuracy similar to before but now for weightlifting the performance of the watch 8 seems to be similar to the performance it had during cycling outside and still most points are along the blue line so this is really good the correlation is also still really high at 0.98 if we look at the individual training sessions we can indeed see that the watch 8 follows along quite nicely with the chest strap we do see it struggles a bit here in the beginning where we also see it does seem to miss a few of the heart rate measurements at the peak of my exercise each time I do a set of exercises my heart rate increases and the Apple watch in red is generally able to follow along with this though it doesn't follow along perfectly with a chest strap which is displayed in blue green still it looks quite good as is also true for this other training example again the red line is the Apple watch 8 which follows along quite well with the blue green line of the HD chess strap we can again put this into perspective by comparing it to many of the other watches I've tested in the past and we can see those results in this overview right here again the more to the top right the better the consistency with the ECG chest strap and as you can see relative to other watches at least the watch 8 which is marked in red is doing really well which we can again see more clearly by zooming in we see that the Apple Watch series 8 is amongst the best performing watches together with the series 7 and Series 6. so for weightlifting the Apple Watch series 8 appears to be a solo Choice So based on this initial testing I would conclude that the Apple Watch series 8 is a really good heart rate tracker like basically all newer Apple watches it is one of the few smart watches that has decent perform orange during weight lifting and it also does really well during cardio overall I would therefore give the heart rate tracking 5 out of 5 Stars since Apple watches are basically my go-to devices for on the wrist heart rate tracking now before getting to the sleep test results I do have to make a remark about a potential small issue of the Strava exports of the Apple watch in my last video I showed you this plot for the heart rate tracking of the Apple Watch series 8 during cycling and we saw that during this part of the ride right here there was some missing data now to make this plot I use the Strava export from Apple since the normal native export I get from Apple has two mistakes in the file format since the new watch OS was released that prevents me from loading it however I now wrote some custom code to fix this format and if I load that data we get this plot right here so we don't have that missing data right here so before we saw here there was some missing data which is not there if I use the export from Apple which means the plot looks even a bit better and and we see a better overlap between the Apple Watch series 8 and the HG chess trap I just wanted to share this with you since it means there might be small deviations if you use Strava to get your Apple heart rate data instead of the Native Apple health for the Nerds out there like me that want to know how to fix the Apple export issue the XML file you get from Apple Health has a mistake in the header and the workout statistics declared to start dates instead of a start and end date okay enough super nerdy talk let's get back to the normal level of nerdiness let's move on to something that Apple was late to implement but immediately seems to have become the market leader at sleep stage tracking I showed you in a recent video that the new algorithm implemented by Apple outperformed all other watches I tested so far so let's see if this is also true for the implementation of the series 8 sleep tracking I do have a relatively Limited Test sample set for now since I could only track my sleep for three nights so far and I actually had to exclude one the first night because the watch did not report my sleep I think this could have potentially been my own fault since I went to bed after 4am and I might have deactivated the sleep mode now like basically all smart watches the watch 8 will track your sleep throughout the night and let you know when it thinks you were in REM sleep deep sleep light sleep and awake let's take a look at those results to check if the Apple Watch series 8 can detect my sleep stages I'll compare it to an EEG device called to dream 2 that can actually measure my brain waves and has been shown to be relatively reliable at sleep stage tracking now here I show an overview of the sleep test results for getting an overall impression of how well the watch 8 performs The Dream 2 should likely be good enough however the gold standard would be polysomnography which I would also like to try on the watch 8 in the future now on top are the sleep stages as recorded by EEG device and on the left the sleep stages as recorded by the watch 8. I wore both the EEG device and the Apple watch the best for three nights but I only had two nights of usable data and I will see how close the predictions of the Apple watch are to those of the EEG device now each column here sums to 100 meaning that we can see what percentage of each of the sleep stages according to the dream 2 was predicted as a sleep stage by the watch 8. if the two devices perfectly agree all values on the diagonal should be 100 first of all we see that 97.5 percent of what was deep sleep according to the EEG defies was also predicted as deep sleep according to the Apple watch which is some of the best tracking we've seen for any watch on the few occasions that deep sleep according to the EEG device was predicted differently by the Apple watch it was predicted as light sleep at 2.5 percent and we can see that based on the individual nights on top here we have the sleep stages according to the dream to EG headband with the clock time along the horizontal axis and the sleep stages on the vertical axis on the bottom we have a similar plot but now for the Apple Watch series 8 I've highlighted all the EEG recorded deep sleep and purple right here now for this Knight we can indeed see that the Deep Sleep detected by the EEG headband and the deeply detected by the Apple Watch series 8 basically show the same results so they have more or less perfect agreement and this second Knight shows much of the same results with a good match between both devices when it comes to Deep Sleep detection the Apple watch did detect some extra very brief deep sleep segments later in the night though as you can see right here I'm right here light sleep agreement was also really good at about 90 percent which is again showing that Apple watches appeared to be really good at sleep stage tracking and if they did disagree it could be that the Apple watch 8 predicted any of the other sleep stages but mostly as deep sleep and awake time REM sleep agreement was also quite good at about 65 agreement with the EEG device if it was predicted differently it was mostly predicted as being light sleep at about 31 and we can see that clearly based on the individual Knights as well this is a similar plot to B4 but now with the ram sleep as measured by the EEG device marked in red again we see a very good agreement as all of the ram sleep segments detected by the EEG device are also detected by the Apple watch it's just the total duration that varies a bit but not much at all and we see the same thing for the other night where we see a good agreement between both devices when it comes to Ram sleep tracking though again the last segment shows some disagreement when it comes to duration this means that we can also see the sleep cycles based on just the data from the Apple Watch series 8. now you go through roughly four to six sleep cycles each night each one starting with light and deep sleep marker in blue and each one ending in REM sleep marked in red as you can see I likely had one two three four complete sleep cycles this night and were able to see this based on just the data from the Apple watch which is actually something that most Watchers struggle with so that's a really good result a wake detection also agreed very well at about 93 agreement and any disagreement was with live sleep which makes sense as light sleep is the closest Sleep stage to being awake in general the Apple Watch series 8 does tend to detect the awake moments that the EEG device also detected which are marked in green but it generally tends to detect some more awake moments as well as you can see here for instance in the middle of the night and also right here and we see the same thing for this second example night right here where the EEG device detected no significant awake moments whereas the Apple watch did detect several really short Awakenings now I should note here that the Apple watch might be intentionally set to also detect a really short awake moments whereas the dream to EG headband might be set to ignore these to put these result results into context we can compare the performance of the Apple Watch series 8 to that of 38 other watches I've tested in the past this graph shows an overview of the agreement of different watches with the EEG device along the horizontal axis we have the average agreement over the four individual sleep stages and on the vertical axis we have the agreement of the worst sleep stage now the battery agreement the more to the top right the devices and as you can see the device with the best agreement so far was the Apple Watch series 7 which I tested for 80 nights other good devices include different fitbits whoop straps and the wetting sleep analyzer if we now plot the Apple Watch series 8 in the same plot which is marked in red we see that this is again amongst the best agreeing watches of course this is just based on two knights so there will be a significant error margin However the fact that it's so close in results to the series 7 indicates the results are consistently good between apple watches and we can confirm that by also plotting the results for the new 2022 Apple watch SE in the same plot for which I'm actually now making a separate complete video as you can see it's also in the same region of performance really confirming the quality of Apple watches when it comes to sleep stage tracking overall the Sleep stage tracking of the Apple Watch series 8 seems to be really good and I suspect it does about as well as all of the other Apple watches therefore overall I'd also give the Sleep stage tracking of the Apple Watch series 8 5 out of 5 stars now one of the new features that Apple Watch series 8 implemented is that it tracks your skin temperatures throughout the night I already discussed this in my last video but now that I had some more hands-on experience I want to make three new remarks first of all the data that is accessible to me only has a single value for the whole night so it does not seem that Apple Stores the changes of temperature throughout the night second many people ask in the comments if I think that the Sleep stage tracking will be better on the series 8 because of this new sensor now I don't think that's the case for two reasons first of all that would mean that Apple would have to implement two different algorithms for different models which would be a lot of extra work and potentially also make them less robust and two the sleep tracking on the new Apple watch SE also appears to be quite good and that does not have the new temperature sensor but more on that in a different video one more thing I wanted to mention is that you officially cannot see your wrist temperature during your first five nights since the Apple watch still needs to measure your Baseline however if you want you can already see your skin temperature after the first night by creating an export of your Apple Health Data and then searching the file for something called Sleep risk temperature okay enough about the temperature let's now move on to another important feature of the Apple Watch series 8 the location or GPS tracking to test that I track my route during seven bike rides while cycling to and from work and I want to test two things one how long does it take for the watch to get a GPS signal and two how well the GPS signals overlap when starting the same route several times I should mention that I put my phone in flight mode and turned off Bluetooth to make sure that the Apple watch could not piggyback on my phone's signal and would have to completely rely on its own sensors those results are displayed here for four times I cycled two work now I started the activity the moment I was ready to leave and I did not provide the watch with any extra time to acquire the signal the green markers indicate those moments it connected to GPS signals and as you can see it connected almost instantly in all of the cases which is really good if we now look at the consistency between the signals this is also really good the signals mostly overlap really well and they're very close together most of the time though sometimes there are some small deviation however you can also see here for instance that they overlap almost per perfectly and this is true for most of my journey where the overlap is actually very very good I would say that only near the end here do we see some deviation so this is not that bad and also right here do we see a deviation and here there's a more significant deviation but overall this is not looking too bad at all and for cycling back from work we see more or less the same thing this signal is acquired almost instantly as you can see here and once it has the signal it overlaps really well now overall I would say it looks pretty good though there are some moments with a bit of deviation we see one example right here for instance but also here the lines deviate a bit and especially here do we see some more deviation however most of the time the signals are pretty consistent as you can see right here for instance overall I'm happy with the GPS tracking performance of the watch 8. it locks onto my signal really fast and tracks my routes quite consistently therefore overall I get a GPS tracking between four and four and a half out of 5 Stars though I still have to quantify it the quality seems similar to what we've seen for some of the better Garmin watches next let's take a look at the feature of the watch that became more important during the covet pandemic namely measuring your oxygen saturation or in other words spo2 to test the oxygen saturation measurements I wanted to see if the watch 8 ever attacks a low oxygen saturation when it's not supposed to and I did this by taking measurements with the watch when I knew that my spo2 was on normal levels now I also discussed this in the last video but I now more than double the data to a total of more than 80 measurements I systematically swap the watch between both of my wrists and this way I hope to get a good representation of the measurements it can take at the same time I also recorded my oxygen saturation with a dedicated finger pulse exhibitor now at ground level my oxygen saturation should be in my normal range which is generally between 97 and 100 and should not fall below roughly 95 however when the effective oxygen concentration is much lower as it is for instance in a low air pressure environment my oxygen saturation can drop below 90 percent the same can happen for instance when you have certain medical conditions like sleep apnea or respiratory infection now here you can see the measurements I took at ground level on the left are all the measurements taken with the watch 8 and on the right the matching measurements taken with the finger pulse examinter on the vertical axis are the spo2 values with each dot indicating a single measurement as you can see the watch 8 on the left generally recorded lower spo2 values than the fingerpills examiner on the right in general the spread of values is much larger for the Apple Watch series 8 than for the finger pulse oximeter we can see that even more clearly if we display these results as a histogram in this case the spo2 values are along the horizontal axis and the larger the bar the more often this value is recorded as you can see the Apple Watch series 8 which is played in Red mostly recorded lower values than the finger pulls oximeter which is displayed in blue green and sometimes it even recorded values of 94 or lower which is much lower than what it would normally experience at ground level however it also seems that the new measurements I took with the Apple watch compared to those in my previous video are more often in a normal range making it appear a bit better than it did in my last testing so this is looking okay though definitely not great overall based on the testing I've done so far I'd give the spo2 measurements of the Apple Watch series 8 either 3 or 3 and a half out of 5 stars now the next thing I tested on the Apple watch 8 is this step counting accuracy to test the step counting accuracy I went out and took exactly 8 000 steps with the Apple watch series 8. now I do not like counting 8 000 steps in my head which is why I counted each step manually using this telecounter let's take a look at those results here you can see the results for the first 4 000 steps where I wore the Apple Watch series 8 on my left arm along the horizontal axis we have the time and the total number of steps counted by the Apple watch is on the vertical axis the red line represents the actual steps counted by the watch and the blue vertical lines indicate the moments that it should have counted 1 000 more steps so the first Blue Line indicates where it should have counted 1000 steps the second two thousand three thousand and so on as you can see the Apple watch was pretty consistent and it generally counted very closely The Substitute of counted which is good though it might have just counted a few steps to little near the end but as you can see it's still really close to the four thousand steps it should have counted I wore the new 2022 Apple watch SE at the same time but on the other wrist which is display it here in the same plot but now in blue green we actually see that both watches perform more or less similarly both consistently counting more or less the correct number of steps I actually repeated this test but now switching the watches to the opposite wrists and also changing that in the settings as you can see the watch was again pretty close to the actual number of steps it should have counted though there were some deviations also be aware that I took a short break in walking here and at the end which is why this final segment looks a bit longer overall the watch 8 did pretty well doneri ended counted a few steps too many again if we displayed a new 2022 Apple watch SE in blue green we see that this performed more or less the same as the Apple watch 8 I would say that both did quite well DOTA SE might have done slightly better so this is looking quite good however the next question is if the Apple Watch series 8 ever count steps when it's not supposed to for instance when doing activities that did not involve walking those results are just played here for four activities on the vertical axis so that's walking weight lifting spinning and biking on the horizontal axis are the number of steps per minute the Apple watch counted for each of these activities now ideally it would count zero steps per minute during all of the activities except for walking of course which should give a significant number of steps per minute as you can see when I was walking it counted about 80 to 90 steps per minute which is a normal walking pace now when cycling outside which you see here in blue it actually counted close to zero steps per minute most of the time which is really good interestingly though while weightlifting which is purple here and while spinning which is displayed here in green it did Count some steps I would say roughly between 10 and 20 steps per minute though this is much less than while walking it is still surprising it counted quite some steps during these static exercises so overall the step counting performance of the Apple Watch series 8 seems to be quite good when you're walking it comes roughly the correct number of steps however it does sometimes count some steps when it's not supposed to for instance while cycling indoors or weightlifting therefore overall I'd give this step counting accuracy 4 out of 5 stars now looking at the series 8 Apple watch as a whole I'm quite happy with this performance it has some of the best best heart rate tracking and sleep tracking out of all of the devices I've tested and in my cycling test it also had pretty reliable GPS tracking the step counting was quite good and the only thing that was not great were the oxygen saturation measurements I would say that the main downside of this watch is the relatively short battery life of the Apple watch you need to be really consistent with your charging schedule and find some time each morning or evening to top of the charge however as long as you're willing to do that and you own an iPhone the Apple watch is one of the best Sports and Health trackers out there the main thing that is still missing with regards to health wrecking is providing advice based on the health metrics it takes still overall I'd give the Apple Watch series 8 4 and a half out of 5 stars now if you want to buy the Apple watch another device or anything at all on Amazon for that matter even something as small as toilet paper and at the same time support the channel there are different affiliate links in description below they do not cost you any extra and some even provide a discount by the way you might want to actually stick with Android and still have good heart rate tracking in that case some Huawei watches are actually quite good as I discussed in these videos right here for sleep tracking Fitbit devices are much cheaper and also perform quite well as you can see in this video right here now I hope that this video provided you with some value thank you so much for watching and catch you in the next video [Music]

Channel: The Quantified Scientist

Views: 101,678

Keywords: Apple Watch Sleep, Apple Watch Sleep Accuracy, apple watch ultra, apple watch series 8, apple watch sleep tracking, apple watch sleep tracking app, apple watch sleep apnea, apple watch ultra worth it, apple watch ultra workout, apple watch ultra worth buying, apple watch se 2022, apple watch se vs series 8, apple watch se vs 8, the quantified scientist, quantified scientist, spo2, spinning, workout heart rate apple, apple watch vs garmin, apple watch vs garmin fenix 7

Id: u8XbiWIUW4s

Length: 29min 38sec (1778 seconds)

Published: Tue Sep 20 2022