Summary
Ultrahuman Ring AIR was compared with a consumer grade wearable (Apple Watch) and an FDA approved medical sleep testing device (SleepImage) to examine overlap of heart rate detection during sleep.
Low mean error and high degree of overlap at the level of individual data points was found.
High degree of overlap in sleep session-wise heart rate data and resting heart rate during sleep was found.
Heart rate sensing at rest is central to precise sleep and recovery prediction. Ultrahuman Ring AIR shows equivalence in this domain to medically approved and popular consumable wearables.
Background and Rationale
Heart rate (HR) detection has been the cornerstone of medical practice and diagnostics for close to 150 years1. Measuring pulse has been intrinsic to almost every traditional school of medicine going back to ancient times2. The cadence of heart pumping and blood flow across the body is tightly controlled by a complex system. Intrinsically, the “heart rhythm” is controlled by its natural pacemaker, the sino-atrial node3. Extrinsic factors include control by the sympathetic and parasympathetic nervous system4. Sympathetic nervous system (SNS) control mediates the heart’s quick response to stress, danger and performance in daily life and does so by increasing heart rate, breathing and even circulating glucose levels. Parasympathetic nervous system (PNS) signals induce a state of relaxation and recovery by downregulating heart rate, breathing and promotes insulin secretion5,6.
Algorithms and sensors for detection of HR and heart rate variability (HRV) are fundamental to wearable technology and devices. Especially given the critical importance of the SNS and PNS in sleep stages, HR serves as a major proxy of brain wave patterns for sleep detection using wearable devices. Motivated by queries from the Ultrahuman user base, we conducted a pilot study (ahead of a structured trial) to measure the real world overlap of heart rate detection during sleep between the Ultrahuman Ring AIR and two different devices. Firstly, the Apple watch (various models) as a representative of consumer grade wearable and also of wrist based HR/HRV tracking. Secondly, we compared the Ultrahuman Ring AIR to SleepImage, an FDA-approved, medical grade device that is used to detect obstructive sleep apnea and is worn on the finger7.
Our objective was to assess the concordance of Ultrahuman AIR with these two wearable devices with distinct profiles in measuring sleep HR that forms a fundamental part of sleep tracking.
Design
Six volunteers wore Apple watch and SleepImage ring alongwith Ultrahuman Ring AIR for a period of six days. All devices were worn on the same arm and hand. They also reported sleep disruption incidents and their qualitative impact on sleep in a log sheet which was used to review eligible nights of sleep data for analyses. In general, sleep duration of less than 4 hours was not considered for the analyses recorded on Apple Watch and Ultrahuman Ring AIR. SleepImage provides signal quality metrics with the sleep report of every night for a user. Nights with signal quality less than 70% were not included in subsequent analyses.
Volunteers did not have any changes in working hours, diet or supplements/medicines taken during the study period. Stimulant intake was also reported to be identical for all participants in the study period versus earlier. Data analysis was performed without any downloads by the participants using permissions to read data by Ultrahuman in the Apple Health application. SleepImage data was obtained with user consent from
Empower Sleep. Statistical analysis was using in-house Matlab Python code, to derive error comparisons, Bland Altman analyses and overlap coefficients using linear models of regression. Datapoints at the level of sleep sessions (aggregate, n=36) and individual HR readings (granular, n=2349) were used for statistical analysis.
Result
We obtained a value of 3.9 and 2.6 for root mean square error (RMSE) and mean absolute error (MAE) respectively for Ultrahuman Ring AIR versus Apple Watch at the granular data level. We found very similar results for RMSE and MAE for Ring Air versus SleepImage ring at 3.9 and 2.4 respectively.
Bland Altman analyses (shown below) indicated a high degree of concordance of data values across users and sleep sessions. Average differences in readings was -1.8 beats per minute for Apple Watch to UH Ring AIR comparison, and -1.2 beats per minute for SleepImage to UH Ring AIR comparison.
Figure 1: Bland Altman plots comparing differences in heart rate values measured by Apple Watch vs UH Ring AIR (left) and SleepImage vs UH Ring AIR (right).
At the level of sleep sessions (aggregated data) we found an extremely strong and statistically significant linear relationship between the mean heart rates measured by the Apple Watch and Ring AIR devices on a user-wise, day-wise basis. Almost all (99.5%) of the variance in the Ring AIR mean HR can be explained by the Apple Watch device's mean HR, indicating excellent predictive power and a very close agreement between the two devices. Granular data sets also yielded statistically significant and high overlaps.
The linear regression analyses at the aggregate level between Ultrahuman Ring AIR and SleepImage Ring indicated an extremely strong and statistically significant relationship between the mean heart rates measured by the SleepImage and Ultrahuman Ring AIR devices, with nearly all the variance in Ultrahuman Ring AIR’s mean heart rate explained by SleepImage's mean heart rate.
Heart rate sleep minima is an important metric to determine rest and recovery and can be calculated at the level of individual sleep sessions. In keeping with the aggregated trends above, we obtained a highly significant degree of overlap between Ultrahuman Ring AIR and Apple Watch and SleepImage pairwise.
Figure 2: Graphical representation of Spearman correlation coefficients for HR data at the granular level (left), mean heart rate at the level of sleep sessions (centre) and heart rate minima at the level of sleep sessions (right). In all cases p<0.0001.
Conclusions and Limitations
This report is a drill-down analysis of a series of in-house pilot studies conducted to build awareness of Ultrahuman Ring AIR metrics that are used to compute sleep index and recovery scores in real life settings. As with all pilot studies, the participant numbers are small which is offset by incorporating greater test-retest reliability included by recording multiple sleep sessions and wearing all three devices simultaneously.
Real world sensing of HR and derived HRV has been central to wearable technology. HRV calculation and absolute overlap was not studied due to differences in sampling rates of the three devices, different methods of calculation used across devices and the limitation of exported data including only HR from Apple Watch and SleepImage8. However, Nelson and Allen, 2019 compared the measurement error of Apple Watch 3 and Fitbit Charge 2 in one individual across 24h and compared it to ambulatory ECG benchmarks9. Both devices reported error ranges of 5-6% from ambulatory ECG. With an MAE of 2.4 to 2.6%, Ultrahuman Ring AIR falls within these ranges. Miller et al., reports ECG difference of one model of Apple Watch to be 0.5 beats per minute in controlled sleep lab testing10. It is expected that home setting sleep metrics will have greater differences. SleepImage, in keeping with its clinical use predominantly, has reports of its performance in context of sleep apnea and breathing related sleep disorders11. The motivation to use SleepImage as a comparator arose from evidence of its use in the real world and equivalence to polysomnographic analyses software with data gathered across approximately 700 adult individuals7.
HR overlap in individual sleep sessions and daily sleep minima across Ring AIR, Apple Watch and SleepImage indicate the high degree of equivalence between photoplethysmography (PPG) sensors (be it on the wrist or on the fingers) which is under tight control of the autonomic nervous systems components SNS and PNS mentioned above. Measurement of HR during sleep is a well established proxy of SNS-PNS activity which also regulates sleep stage12. We hope that the findings enable Ultrahuman Ring AIR users to understand the foundational aspects of the collected data stream better.
We wish to acknowledge the support of all team Ultrahuman volunteers, and Sahil Chopra, Sagar Chopra, Eugene Radlinskas from the Empower Sleep team. Special thanks to Prof. Hans-Peter Kubis, Bangor University on early ideation of these pilot studies.
Reach out to support@ultrahuman.com for commercial queries and science@ultrahuman.com for scientific queries.
