Hey Marco, I found your 2022 deep dive into wearable data comparisons and have been working through your subsequent work. I love it! Where are you at right now in terms of using a wearable for sleep tracking only? Ex. wear an Oura only at night. My main goal is to improve my sleep quality.
I do not use any wearable for sleep tracking. I think 1) they are not accurate enough - as mentioned in this blog, the error is too large even for something like sleep time - and even when it does work, we know we are not really measuring sleep time, just guessing it from movement and other somewhat related variables, which is not something I can trust. For stages, the data is all over the place - it's unfortunately not reliable 2) even if the data was reliable, what is it for? what is the actionability? There isn't any. To me, sleep tracking in wearables is another tool for distraction (https://marcoaltini.substack.com/p/how-wearables-became-tools-of-distraction), a technology-driven application without an actual need. How much deep sleep do you need? How does that quantity change in relation to your behavior (e.g. after a hard workout)? Why? What can you do to impact it? We have no answers to these questions, hence tracking often only leads to unnecessary concerns based on rather inaccurate or completely made up data.
Sleep is the single most important thing for our health and performance, but this doesn't mean that we need technology or 'to track it'. We just need to implement good sleep hygiene, really the basics that everybody at this point knows: consistent sleep time, enough sleep hours, reduced evening stressors (alcohol, late / heavy dinners, hard exercise, etc.), a temperature where you feel comfortable, etc. - and then see if you feel better and perform better in the long term.
If we want to track something, I think at that point it is better to track our physiological response, i.e. our morning heart rate and HRV in response to our interventions, as in that case we are actually measuring something (instead of trying to guess it), and we also know better what to do with that information, at least in the short term. In this framework, sleep is just context, similarly to exercise or all other positive and negative stressors that impact our resting physiology. This is how "i track" sleep, just annotating my subjective sleep quality and sleep time in the HRV4Training app via the questionnaire, no other tracker or "pretending to be accurate" information that eventually never led to any useful insights or changes.
Thanks Marco this is really helpful. It’s so tempting to think you can “do stuff” (Cold plunge! Sauna! Theanine!) to improve deep sleep or recovery _and_ that you could actually observe the improvements. Do you see any value in Oura, Whoop, for tracking trends? Do any of them do morning HRV?
thank you Eric, and indeed so many questions arise at that point (is the acute change a good thing or leading to a chronic change in the opposite direction? the answer is often not obvious). Unfortunately most trackers do not implement standard protocols and therefore cannot send RR intervals (the basic unit of information needed to do a spot check HRV measurement) to third party apps. The whoop does, even though the functionality is undocumented and might go away anyday. Normally I recommend simply to use the phone camera on iPhone with HRV4Training (which has been validated: https://www.researchgate.net/publication/315059917_Comparison_of_Heart_Rate_Variability_Recording_With_Smart_Phone_Photoplethysmographic_Polar_H7_Chest_Strap_and_Electrocardiogram_Methods) or a Polar H10 chest strap, which is, after all, the only way to actually measure HRV.
Hi! This is a really interesting article, and I’d like to share a few thoughts. If sleep detection algorithms are biased in a consistent way, why would they be such a bad option for tracking trends? For example, if I want to see if I’m sleeping better or worse than last week, it doesn’t matter too much if the algorithm mistakenly detects me as sleeping while I’m reading before bed, as long as it happens consistently, right? (Of course, this assumes my habits are mostly the same over time.)
In my case, I’ve found that self-questionnaires can be problematic (though I understand they may not be for everyone). I struggle with sleep issues and try not to look at the time when I have insomnia. Over the last year with my Coros watch, I’ve found that the algorithm is quite accurate. When I have a bad night, the metrics reflect that, and when I sleep well, they show that too.
thanks Victor! I understand what you are saying and certainly agree that there can be cases in which things are tracked better, also depending on the individual, our habits and behavior, as well as our physiology. Regarding the bias, there is no proof that this bias is consistent over time, as studies tend to look at different individuals and trackers for very short periods (one night or a few nights), hence I would not assume that there isn't a random error in there that makes even long term tracking ineffective within individuals (this is the case for me, I use the same device you use, a Coros watch, and it can't even track well my recent reduction in sleep time, which is obvious when looking at the data I manually log - again, not saying this can't work for others, but there are huge limitations and I think it is important to stress them, as many people seem overly confident in these devices, given the great marketing efforts the companies selling them make). I understand also that questionnaires are not for everyone, I don't think there is an ideal way to do this that works for everyone but we need to find tools and approaches that fit with our lifestyle and eventually provide valuable data, while being aware of the limitations. I think we could even question the whole sleep tracking idea... personally I find no value in it, apart from adding context for parameters and outcomes that are more interesting, e.g. my resting physiology, health or performance. I never spent a minute looking at made-up sleep scores or estimated sleep stages :) All the best for your training!
I have worn an Oura since they first came out and it tracks my sleep perfectly since I put it on when I put my Kindle down after reading in bed and take it off when I wake up! Problem solved. Kind of….
(edit as I forgot to reply to this: when tracking time in bed, I am tracking the sleep opportunity I am giving me, which is all about behavior and consistency, and as such, important - I am not claiming I am tracking accuratley time - I also wake up in the night, go to the bathroom, etc. - but the only actionable thing is the sleep opportunity window, i.e. when I go to bed and when I wake up, and I can track that much better myself than with a wearable)
On gamification: similar to the above in essence, i.e., a made-up number is the furthest thing from what I would use to be engaged with my health. Often short-lived, backfires easily, and can lead to the opposite outcome (orthosomnia). Does it work for some? Sure. Does that mean I should be on board with what I consider nonsense? Nope.
no doubt it can work for some, and hopefully you can also accept it doesn’t for others :)
as per the last sentence, it’s incredible what we can convince ourselves of, once we’ve already seen the data. Careful not to let a toy tell you how your day should be
I think we have already established that the tracker is working well for you, hence the score, largely based on sleep time, will also make sense. No surprise there (and good for you!).
For something "to work" it needs to work for everyone. For something "not to work" it just needs not to work for some (the definition of accuracy in an evaluation is often the average error across a number of people/situations). It works for you, great, but the point is that it doesn't work for everyone, hence I cannot comfortably recommend people to go this way.
Hey Marco, I found your 2022 deep dive into wearable data comparisons and have been working through your subsequent work. I love it! Where are you at right now in terms of using a wearable for sleep tracking only? Ex. wear an Oura only at night. My main goal is to improve my sleep quality.
thank you Eric.
I do not use any wearable for sleep tracking. I think 1) they are not accurate enough - as mentioned in this blog, the error is too large even for something like sleep time - and even when it does work, we know we are not really measuring sleep time, just guessing it from movement and other somewhat related variables, which is not something I can trust. For stages, the data is all over the place - it's unfortunately not reliable 2) even if the data was reliable, what is it for? what is the actionability? There isn't any. To me, sleep tracking in wearables is another tool for distraction (https://marcoaltini.substack.com/p/how-wearables-became-tools-of-distraction), a technology-driven application without an actual need. How much deep sleep do you need? How does that quantity change in relation to your behavior (e.g. after a hard workout)? Why? What can you do to impact it? We have no answers to these questions, hence tracking often only leads to unnecessary concerns based on rather inaccurate or completely made up data.
Sleep is the single most important thing for our health and performance, but this doesn't mean that we need technology or 'to track it'. We just need to implement good sleep hygiene, really the basics that everybody at this point knows: consistent sleep time, enough sleep hours, reduced evening stressors (alcohol, late / heavy dinners, hard exercise, etc.), a temperature where you feel comfortable, etc. - and then see if you feel better and perform better in the long term.
If we want to track something, I think at that point it is better to track our physiological response, i.e. our morning heart rate and HRV in response to our interventions, as in that case we are actually measuring something (instead of trying to guess it), and we also know better what to do with that information, at least in the short term. In this framework, sleep is just context, similarly to exercise or all other positive and negative stressors that impact our resting physiology. This is how "i track" sleep, just annotating my subjective sleep quality and sleep time in the HRV4Training app via the questionnaire, no other tracker or "pretending to be accurate" information that eventually never led to any useful insights or changes.
I hope this helps!
Thanks Marco this is really helpful. It’s so tempting to think you can “do stuff” (Cold plunge! Sauna! Theanine!) to improve deep sleep or recovery _and_ that you could actually observe the improvements. Do you see any value in Oura, Whoop, for tracking trends? Do any of them do morning HRV?
thank you Eric, and indeed so many questions arise at that point (is the acute change a good thing or leading to a chronic change in the opposite direction? the answer is often not obvious). Unfortunately most trackers do not implement standard protocols and therefore cannot send RR intervals (the basic unit of information needed to do a spot check HRV measurement) to third party apps. The whoop does, even though the functionality is undocumented and might go away anyday. Normally I recommend simply to use the phone camera on iPhone with HRV4Training (which has been validated: https://www.researchgate.net/publication/315059917_Comparison_of_Heart_Rate_Variability_Recording_With_Smart_Phone_Photoplethysmographic_Polar_H7_Chest_Strap_and_Electrocardiogram_Methods) or a Polar H10 chest strap, which is, after all, the only way to actually measure HRV.
Hi! This is a really interesting article, and I’d like to share a few thoughts. If sleep detection algorithms are biased in a consistent way, why would they be such a bad option for tracking trends? For example, if I want to see if I’m sleeping better or worse than last week, it doesn’t matter too much if the algorithm mistakenly detects me as sleeping while I’m reading before bed, as long as it happens consistently, right? (Of course, this assumes my habits are mostly the same over time.)
In my case, I’ve found that self-questionnaires can be problematic (though I understand they may not be for everyone). I struggle with sleep issues and try not to look at the time when I have insomnia. Over the last year with my Coros watch, I’ve found that the algorithm is quite accurate. When I have a bad night, the metrics reflect that, and when I sleep well, they show that too.
thanks Victor! I understand what you are saying and certainly agree that there can be cases in which things are tracked better, also depending on the individual, our habits and behavior, as well as our physiology. Regarding the bias, there is no proof that this bias is consistent over time, as studies tend to look at different individuals and trackers for very short periods (one night or a few nights), hence I would not assume that there isn't a random error in there that makes even long term tracking ineffective within individuals (this is the case for me, I use the same device you use, a Coros watch, and it can't even track well my recent reduction in sleep time, which is obvious when looking at the data I manually log - again, not saying this can't work for others, but there are huge limitations and I think it is important to stress them, as many people seem overly confident in these devices, given the great marketing efforts the companies selling them make). I understand also that questionnaires are not for everyone, I don't think there is an ideal way to do this that works for everyone but we need to find tools and approaches that fit with our lifestyle and eventually provide valuable data, while being aware of the limitations. I think we could even question the whole sleep tracking idea... personally I find no value in it, apart from adding context for parameters and outcomes that are more interesting, e.g. my resting physiology, health or performance. I never spent a minute looking at made-up sleep scores or estimated sleep stages :) All the best for your training!
I have worn an Oura since they first came out and it tracks my sleep perfectly since I put it on when I put my Kindle down after reading in bed and take it off when I wake up! Problem solved. Kind of….
haha that works indeed!
the night is also the only moment in which wearables data is worth collecting, I used to wear it only then as well
That is very true. And have ordered a Velia Smart Ring and hopefully that will have a more wearable design if they ever come out.
hello Sasha, thank you for your message. I built those models (https://www.mdpi.com/1424-8220/21/13/4302).
The issue for me is simple: what you want to measure cannot be measured without looking at brain waves, hence you are relying on a guess. Can it be decent for some people sometimes? Sure thing. Is that up to my standards? Nope. Feel free to use it of course, but we should be clear that sleep time is not measured and as such, not accurate. For stages, hopefully this speaks for itself: https://substackcdn.com/image/fetch/$s_!xcEK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e41a66b-28b9-4d17-892e-14137d923b96_1886x914.png
(edit as I forgot to reply to this: when tracking time in bed, I am tracking the sleep opportunity I am giving me, which is all about behavior and consistency, and as such, important - I am not claiming I am tracking accuratley time - I also wake up in the night, go to the bathroom, etc. - but the only actionable thing is the sleep opportunity window, i.e. when I go to bed and when I wake up, and I can track that much better myself than with a wearable)
On gamification: similar to the above in essence, i.e., a made-up number is the furthest thing from what I would use to be engaged with my health. Often short-lived, backfires easily, and can lead to the opposite outcome (orthosomnia). Does it work for some? Sure. Does that mean I should be on board with what I consider nonsense? Nope.
As per Johnson, etc. - not my cup of tea.
Take care!
no doubt it can work for some, and hopefully you can also accept it doesn’t for others :)
as per the last sentence, it’s incredible what we can convince ourselves of, once we’ve already seen the data. Careful not to let a toy tell you how your day should be
I think we have already established that the tracker is working well for you, hence the score, largely based on sleep time, will also make sense. No surprise there (and good for you!).
For something "to work" it needs to work for everyone. For something "not to work" it just needs not to work for some (the definition of accuracy in an evaluation is often the average error across a number of people/situations). It works for you, great, but the point is that it doesn't work for everyone, hence I cannot comfortably recommend people to go this way.
I hope this clarifies my view, thanks!