Below is a summary of all the devices/apps I tested. Click on the links, for those that detect sleep interruptions, to see more details.
- B1 by basis (www.mybasis.com): detects sleep interruptions but only about 5% of my sleepwalking.
- Peak by basis (www.mybasis.com): this device replaces B1 but has a comparable detection rate of my sleepwalking.
- UP by Jawbone (jawbone.com): detects sleep interruptions but only about 9% of my sleepwalking.
- Flex by Fitbit (www.fitbit.com): not enough data to be conclusive yet (ongoing) but so far worst than UP or B1.
- Beddit by Beddit (www.beddit.com): does not identify sleep interruptions per say but might be suggesting them, with about 22% of my sleepwalking detected then.
- Sleepcycle by Northcube (www.sleepcycle.com): does not identify sleep interruptions
- Sleepbetter by Runtastic (www.runtastic.com): does not identify sleep interruptions
For the devices that precisely provide the sleep stage at the time of a sleepwalking event, I compared those stages between them. For example if device 1 shows a light sleep stage at the time of a sleepwalking event, what is the stage on the other devices?
Only B1, UP and Beddit could be used for this comparison as the other devices do not provide sleep stages or access to the data.
The comparisons (graphs A to F below) show that B1 and UP have the most similar profiles while Beddit differs the most.
In general the light/deep sleep differentiation does not correlate between devices and is roughly a 60%/30% distribution across all devices and all sleep stages in the other 2 devices. This could be due to one device distinguishing light and deep sleep correctly while the other 2 do not. But since the spread of my sleep stages is relatively similar between devices (see graph G below), this is a strong suggestion that the light/deep sleep distinction made by all those devices is irrelevant.
Moreover, the drug clonazepam, which decreases my sleepwalking about 50%, has almost no impact on the distribution of my sleep stages, as shown in graph G and H below.
Comparison of sleep onset determination between devices
Since determining my sleep onset is a key parameter of my sleepwalking recording I need some reliable data and as such compared the 6 sleep trackers I am using concomitantly.
The graph below shows the sleep onset average deviation for each device out of 81 nights. Interestingly the most accurate and least variable device is a smartphone app (Sleeptracker), while 2 devices from the same company (B1 and Peak) show the most variability between them.
Are sleep tracking devices even accurate?
As part of the general questions around those consumer health devices, knowing their accuracy is essential.
I found surprisingly little on that subject, which is already a problem in my hands: my B1 device (www.mybasis.com) is showing some signs of wear and tear but I cannot replace it as this device has been discontinued, and replaced with Peak (www.mybasis.com). As the sleep onset data above show, I cannot use Peak as a replacement of B1 but have to treat them as different devices. Which puts into question all the other data generated by those devices, and by any sleep tracker in general.
If there is no data showing that those devices are comparable to standard approaches to measure sleep they are as good as assessing sleep through a crystal ball. So as fancy as those devices sound at this point they are only useful to give me my sleep onset time (as long as I use multiple of them at the same time) and my heart rate at precise moments of my sleep.
Last updated: 22 Feb 2016
This comment has been removed by the author.
ReplyDelete