Sorting out metrics confusion

do you know what you are doing?

May 05, 2023

There are often reports of inconsistent outputs from different tools people use to track various metrics, e.g. apps or wearables for heart rate variability (HRV).

Typically, it is not really an inconsistency but just a matter of understanding what is going on: e.g. when and how things are measured, and how the data is turned into messages or scores.

The more you understand the physiology and the technology, the better use you can make of it.

In this blog, I want to show you a simple example of what I mean, and how what seems inconsistent output makes in fact a lot of sense when we understand what we are doing and how the tools work.

Case study

Yesterday I ran 6 hours, it was a bit warm and I didn’t have a great day. At the same time, I am also in the best shape of my life, and I train for this. What I expect is a small acute change in my resting physiology, to be normalized within 48 hours.

I measure my HRV in two (actually three, more on this later) ways:

full night using an Oura ring
first thing in the morning, while sitting, using the iPhone camera in HRV4Training

Remember an important distinction between:

the actual HRV measurement
the interpretation of that measurement into messages or readiness / recovery scores

Let’s look at some data.

Body position and sensitivity to stress

Here is the information I was presented with this morning. On the left, is my morning HRV with respect to my normal range (not good), while on the right is my night HRV incorporated in a readiness score (good).

The first thing to do is to look at what the physiology actually says.

We’ll go back to readiness scores later, but first, let’s also read Oura’s data into another HRV4Training app that I use for this purpose so that I can properly contextualize the physiology with respect to my normal range.

On the left, again is my morning HRV captured with HRV4Training, on the right is my night HRV also read in the app:

We can now see more clearly that the same “issue” remains, not only my readiness was good, but my night HRV is in fact good or within normal range.

Why is this?

The reason is that measuring HRV in the morning while sitting highlights my stress response: the orthostatic stressor challenges the body and amplifies the response.

On the other hand, complete rest (HRV measured during sleep, a highly parasympathetic state), does not capture these more subtle responses well.

This is why in the context of training, it’s an easy choice for me, and I’d always recommend a morning measurement. Especially for athletes, who might have quite high resting parasympathetic activity, sitting is probably the way to go.

Short term vs long term

These situations, cause discrepancies but impact much more the data in the short term, acutely, than in the long term.

If your lifestyle is similar to mine (dinner at 7 pm, bed at 10 pm, no alcohol, no children), then your night and morning data will still look very similar in the long term, as you can see in the past few months of my data:

Above you can see my night data (left) and morning data (right). In February I had some health issues, captured well by both protocols (sustained suppression in HRV with the baseline going below the normal range). Then, I rebounded a bit and went back down with an injury at the beginning of March. The past month and a half have been good, stable, and trending high.

We are looking at resting physiology from different angles, but it is still resting physiology, and as such, if the data and the protocols are good quality, the data will be similar.

However, if we zoom in at times of higher physical stress, night data misses the mark. I had two situations here post hard marathon (April 17th) and post long run (yesterday), in which this is quite clear:

Again, night data is in the left pic, and morning data is in the right pic. Higher stress is captured better when sitting up in the morning, despite the fact that more time has passed since the stressor.

Night data is not incorrect, and is also informative: it tells me that right now I am fit and can recover fast from very hard efforts. When less fit it might take me several days to bounce back from these efforts, a response that is visible also when measuring in a highly parasympathetic state (night data).

Additionally, seeing my heart rate within my normal range for both measurements also tells me that things are looking good, and most likely HRV while sitting will also be back to normal tomorrow.

It all makes sense if we understand the physiology and technology.

Readiness scores

At this point, we have established that morning and night data differ, and that the orthostatic stressor (sitting or standing in the morning) makes the data more useful for the purpose of tracking training responses.

We have also seen how these acute differences are less visible when looking at the data long term, and both methods can track long-term changes in very similar ways.

This is why a device that measures in the night can be a good tool to monitor health or long-term changes while taking a measurement in the morning is probably better for day-to-day actionability. Very large acute stressors will also be visible in night data of course (e.g. excessive alcohol intake, an internet’s favorite - but hopefully, that’s not the only insight you get from your wearable).

Either way, my recommendation would be to always look at the physiology, and not bother much with scores built on top of the metrics, for reasons I discuss in more detail here. Scores make assumptions: you exercise more or sleep less and you are penalized. However, your physiology might be perfectly normal, and that’s what you want to look at, especially in the context of training. Learn to look at the actual data and to use tools that allow you to easily understand if a daily measurement is within your normal range or not.

Yesterday I slept more than normal and without interruptions, which contributes to a higher readiness score. That’s great and might be useful information in general, but when it comes to training, I need to know where my physiology is, without the confounding effect of my behavior.

I often hear from people using e.g. an Oura ring, a Whoop, and a Garmin, all at the same time, that they find the data inconsistent. However, the data is in fact the same, if you know where to look (i.e. the actual physiology) and ignore made-up scores (readiness, recovery, etc.). I have tested all of them against ECG (full blog here), and these devices are now equally good at tracking relative changes in night HRV over time.

It’s fine if you prefer to go that way and not actively measure, but understand the limitations and caveats.

Here is a final example, where the opposite happened: big weeks of training and average sleep give me a 66 readiness, which is as bad as it gets. However, my physiology shows an ideal response: I cannot be any better. As athletes, wearables can be helpful, but you need to understand how to use the data.

Readiness or recovery scores aim at guiding you reactively, in the absence of a plan. While this might make sense for lifestyle, it makes no sense for training. When you do have a plan, look only at the physiology, as the physiology tells you how you are responding, and if changes are needed.

PPG vs ECG

A final word as all listed above is based on PPG technology, which gets some hate from time to time. First, given how similar the data looks long-term between completely different protocols (night data collected from a wearable and morning data collected while sitting using an iPhone camera), it should be fairly obvious that these devices are working well, otherwise, they wouldn’t look anything like this.

However, since I started wearing a Coros and we integrated the morning ECG measurement in HRV4Training, I have also been taking a morning ECG while sitting right after I take the one with the camera. The protocol is not exactly the same of course as I do not measure at the same time: I wake up, I sit, within a few seconds I start measuring with the camera, then I fill in the questionnaire, and then I measure with the watch and the ECG.

The data looks very similar for morning PPG and ECG (show below), with a suppression today:

Morning data with ECG or PPG and night data are all good ways to capture resting physiology when using good tools and protocols. However, there can be differences in particular due to body position and measurement time. The better we understand these differences, the better use we can make of the data.

I hope this was informative, and thank you for reading!

Marco Altini’s Substack

Discussion about this post