Wearable tech, activity monitors/trackers, health and fitness gadgets – call them what you will, there is a burgeoning market for these types of devices. The concept is simple enough place something on your person and wear it 24/7 while it uses various sensors to capture data. Then sync your data to a smartphone or computer and magically become healthier. So that last part may not be so simple, but helping you be healthier or at least more conscientious of your health is the purpose of this class of devices. While the market for activity monitors (my preferred name for this class of devices) continues to grow this review aims to examine four activity monitors that are available at brick-and-mortar retail stores, the Fitbit One, Jawbone Up, Nike+ FuelBand, and the BodyMedia LINK.
What’s not covered in this review.
I have shied away from reviewing the activity monitors in areas that are too subjective and are thoroughly covered by most major tech blogs. Is it easy to lose the tiny removable cap on the Jawbone Up? Is the Nike+ FuelBand comfortable to wear? These types of issues are personal and based on your own individual preference, needs, and lifestyle so I do not consider them in this review. What I will focus on regardless of the devices form factor is which achieves the goal of accurately tracking your activity, reporting your data back to you, and the potential of making you healthier.
The next area I will not delve into are any features of the activity monitors that are not passively collected. As an example, each device’s application allows you to manually track your calories consumed by logging what you have eaten each day. You can manually log your workout activity and some devices even allow you track how many glasses of water you drink. Tracking by manually entering data into software is in strong juxtaposition to the value proposition of what activity monitors should provide. There are also a host of smartphone applications that track calories and workouts so these features, while allowing you to store all your data in one application, are not unique to the activity monitoring devices.
So that leaves us at the first place to objectively evaluate the Fitbit One, Jawbone UP, Nike+ FuelBand, and BodyMedia LINK – Do they accurately capture data?
Step Count Accuracy.
I conducted two types of “controlled” experiments to evaluate each activity monitor’s ability to track steps. First, I did a treadmill test in which I walked 0.25 miles at 2.5mph at a zero degree incline. I logged the current step count on each device before I began walking and after I had walked the required 0.25 miles. I also manually counted how many steps I took. To ensure the accuracy of manually counting my steps I had someone independently observe me on the treadmill and count my steps as I walked 0.25 miles. The table below shows the results of this test.
As the table above indicates the Fitbit One and the Jawbone UP proved to be the most accurate with the Fitbit One getting a perfect step count and the Jawbone UP only off by one step.
The second test was a walking trail test where I again walked 0.25 miles (the walking trail I used has quarter mile markers) at a natural walking pace. I manually counted my steps and again had someone with me to independently count my steps. The table below shows the results of this test.
Again you can see that the Fitbit One and Jawbone UP came out on top with the Jawbone UP providing a perfect count and the Fitbit One two steps under.
28 Days of Steps.
By logging 28 days of steps with all four devices, the graph below shows a timeline of each device and how many steps it counted each day. Unlike the results of the more “controlled” walking tests here we get a sample of real world usage.
In this graph some clear patterns emerge. The Fitbit One is a clear high outlier consistently providing the highest step count while the Nike+ FuelBand is the low outlier typically providing the lowest step count. The Jawbone UP and the BodyMedia LINK hold together in the middle very close to each other. The clear challenge here is that we don’t know for sure exactly how many steps I took each day so there is no reliable benchmark. Below is a matrix that shows the percent difference in steps for each device by using each device as the benchmark as if it were the accurate step count for all 28 days.
To help you understand the above matrix lets assume that the BodyMedia LINK is 100% accurate. If this were the case you can read across the third row labeled ‘BodyMedia LINK’ and see that the Jawbone UP was 3% over, the Nike+ FuelBand was 24% under, and the Fitbit One was 26% over the BodayMedia LINK’s count of steps over the 28 days.
If we go strictly by the data, the BodyMedia LINK and the Jawbone UP appear to give the most accurate step counts given that we have two independent devices that only have a 3% variance in their total step count for the 28 days. Based on the two “controlled” tests, treadmill and walking trail, the Jawbone UP out performed the BodyMedia LINK in terms of accuracy. Hence, I would pick the Jawbone UP as the most accurate step count, but I have one major issue with this conclusion which is the location where you wear the devices. The Jawbone Up and the BodyMedia LINK were worn on my non-dominant arm (left). I consistently notice (especially given that the data was collected in the winter months) that I often put my left had in my pocket as I walk and that I often carry items in my left hand as I walked, both of which lend to under counting my actual steps.
To more objectively evaluate the issue of left arm placement there are two specific days in the data that highlight this issue. On Day 15 a significant amount of my steps were walking in a mall while carrying shopping bags in my left hand. On Day 28 I spend a significant amount of the day moving boxes and furniture leaving my left arm motionless while walking. The table below shows the difference of steps and the percentage difference from the Fitbit One for Day 15 and Day 28.
It is possible that the Fitbit One is the most accurate step counter. To really isolate any inaccuracy caused by the left arm placement I look forward to comparing the results of the Fitbit One vs. the Fitbit Flex (worn on the wrist) released this spring. They will theoretically use similar algorithms and provide the opportunity to control for placement of the device when comparing step count results.
If step accuracy is extremely important to you I would pick the Jawbone UP or the Fitbit One. If you go with the Jawbone UP just be cognizant of your behaviors with your left hand and the potential for under counting of steps it could produce. The clear conclusion no matter how you look at it is that the Nike+ FuelBand performs poorly at step counting and is significantly under counting steps by at least 24%.
Accessing the activity monitors accuracy for calories burned proves to be much easier to do than steps if I make one key assumption. That assumption is possible due to fact that the BodyMedia LINK can calculate calories burned using sensors not contained in the other devices. Specifically it can measure Skin Temperature, Heat Flux, and Galvanic Skin Response. Given these sensors the BodyMedia LINK can calculate METS and in turn use METS to compute calories burned (Weight in kg * METS = Calories Burned per hour). So in order to evaluate calories burned for accuracy I will assume that the BodyMedia LINK’s calories burned is the benchmark.
A quick side note on the Nike+ FuelBand. In order to be able to compare the Nike+ FuelBand’s calories burned I had to add a “standard” 2000 calories burned per day to the number reported by the Nike+ FuelBand. For example if the Nike+ FuelBand reported that I burned 944 calories in a day I was required to change it to 2944 calories in order for the number to be comparable to the other devices. It appears that the Nike+ FuelBand does not include a resting calorie burn rate in the data it reports back to you. As a comparison the Fitbit One even if you don’t wear the device at all will report calories burned based on an assumed resting calorie burn rate it has computed for you.
Below you can see a graph of the daily calories burned reported by each device.
The first and most noticeable thing is that the Jawbone UP significantly under counts calories burned compared to all other devices while the other devices show very similar results. The table below, again assuming that the BodyMedia LINK is an accurate count of actual calories burned, shows the total number of calories burned over the 28 days and the percent difference in the total number of calories burned.
The takeaway is unexpected; both the Fitbit One and the Nike+ FuelBand despite not having any specific sensors to track calories burned yield marginal errors in counting calories burned compared to the BodyMedia LINK. Ultimately the key differentiator of the BodyMedia LINK, i.e. possessing specific sensors to track calories burned, provides no real added value. This is exacerbated by the fact that the BodyMedia LINK is larger and more difficult to wear (it must be worn on the upper part of your arm using an arm band) than the other devices.
For calories burned the Fitbit One and the Nike+ Fuelband come out on top while the Jawbone UP proves to be quite inaccurate.
Two methods are used to help you track how active you are; time and points. All devices provide a measure of time for activity. The Nike+ FuelBand and the Fitbit One also provide a point system. Since there is no standard of how to measure what it means to be “active” the results from each device are essentially incomparable. Since each device at least uses time as a form of measure of activity I have made an attempt to compare them. Here is what each devices tracks as time measures.
- BodyMedia LINK – Moderate Activity, Vigorous Activity
- Fitbit One – Sedentary, Light Activity, Fairly Active, Very Active
- Jawbone UP – Active Time
- Nike+ FuelBand – Active Time
What each of these exactly means is a bit of a mystery. To attempt a comparison I used the following measures:
- BodyMedia LINK – Moderate Activity
- Fitbit One – Fairly Active
- Jawbone UP – Active Time
- Nike+ FuelBand – Active Time
Note: The Nike+ FuelBand is on a separate axis as it is not clear whether it is reporting hours and minutes (in which case it is just waking hours) or minutes and seconds. All the other devices report hours and minutes.
As you can see there is correlation between all the devices but no ability to determine the accuracy of the data. In practical usage I found the Fitbit One and the BodyMedia LINK the most useful. The Fitbit One has a distinction in that its categories account for a 100% of your time (excluding sleep) giving it an edge over the other devices.
The Nike+ FuelBand’s primary focus is on its activity points called ‘NikeFuel’ and the Fitbit One provides an ‘Activity Score’ which is represented on the device’s display as a flower that grows the more your ‘Activity Score’ increases. I found these measures to be useless. For example if you set a goal of earning 2000 Nike Fuel points per day but are 200 Fuel Points short, what exactly do you have to do to earn 200 more points? Run for 5 minutes, 30 minutes?
By comparing Nike+ Fuel Points to the Fitbit One’s ‘Activity Score’ in the graph, as you can see, they correlate very strongly. However, if you are really seeking helpful and actionable data the BodyMedia LINK and the Fitbit One’s activity time proves to be the most useful in helping you get healthier by allowing you to set specific goals and clearly see if you are achieving them and how to achieve them.
All the devices except for the Nike+ FuelBand provide sleep tracking. I found this feature to be the weakest of the data tracked. It was difficult to make sure I tracked my sleep every night. Both the Jawbone UP and Fitbit One require you to push a button to indicate you are going to sleep and to press a button again when you wake up. The fact that you need to push a button on sleep and wake voids this as real feature based on the fact that it is not passively tracked, one of my key criteria in this review. Furthermore the Fitbit One forces you to wear a wrist band in which to store the devices while you sleep. This proved to be annoying and I would often find the wrist band off my arm when I awoke.
The BodyMedia LINK has a leg up on all other devices in the fact that it does passively track your sleeping time. The only thing I had to provide the BodyMedia LINK at the time of setup was what time I typically go to bed. The down side of the BodyMedia is that if I were to lay down and watch a movie for 2 hours it would assume I was sleeping.
So you can track how many hours you sleep and by using the accelerometer in each device they attempt to help you track sleep quality by using your motion during sleep to determine times of deep sleep, light sleep, how many times you awoke during the night, and how long it took you to fall asleep. Below you can see specifically what each device claims to track.
- Fitbit One – Time to fall asleep, times awakened, time in bed, sleep efficiency, total time slept
- Jawbone UP – Hours Slept, Light Sleep, Deep Sleep, time to fall asleep, time in bed, times awaken, total time awake
- BodyMedia LINK – Sleep Start Time, Sleep End Time, Time Sleeping, Time Lying Down, Sleep Efficiency
How much your motion equates to what stage of sleep you are in on the surface seems like a crude proxy. Overall it feels like this feature is forced and added to expand the number of features that can be marketed. I suspect, as with what happen to me, is that most activity monitor users for the first couple of weeks will track their sleep and then consistent tracking of sleep will begin to become hit or miss since the data doesn’t really provide any meaningful insight or actionable data and is easy to forget to track. For what its worth below is a graph of my total sleep time. I would say that the BodyMedia LINK does a good job of tracking your sleep without the complexity of the Fitbit One and Jawbone Up, leaving the only challenge to be remembering to wear the device to sleep.
Knowledge is the least effective indicator of fitness. While activity monitors enable you to capture data, knowing is only half the battle. The missing key to the equation is that there isn’t enough insight from the data. The Jawbone UP at least attempts to provide you with insight. Here is an example: say every time you eat after 8pm (and are willing to log your food intake) you have a decreased sleep efficiency. The Jawbone UP states that it tries to report back to you this type of insight over time, as it continues to capture data about you. Even thought I wasn’t manually logging all the data possible the Jawbone UP provided some small pieces of insight and tidbits of information over the 28 days, but it still needs to go much farther to be fully engaging.
Currently none of the devices solve the toughest challenge of truly creating behavior modification. The things that motivated me the most to change my behavior the were as follows:
- Fitbit One – Step count, strongly aided by the device having a display I could check it easily and regularly
- Nike+ FuelBand – I often found myself curious as to how many NikeFuel points I had, but constantly felt inept at understanding what I need to do to reach my Fuel point goal for the day.
- Jawbone UP – random insights/information provided
- BodyMedia LINK – Total time of Moderate Physical Activity with a goal of 30 minutes a day
One of the latest entrants into the activity monitor market Basis has a key focus on habit formation and their approach to both data visualization and behavior modification, in what appears to be a very practical solution, takes this key shortcoming of the reviewed devices head on.
It is worth noting that I did lose 15 pounds during the 28 days I was reviewing the devices, but it had little to do with the devices themselves. What the devices did accomplish was keeping my inner gadget and data geek engaged in my goal to be healthier, but not much more than that.
The Real Opportunity.
As mentioned at the beginning of this article the market for activity monitors is still in its infancy so these devices only begin to show whats possible. I think the shortcoming of each company’s approach to activity monitors is their myopic view of focusing on the devices as only “health” devices. If these types of devices are to become a staple to your person like your smart phone they must evolve beyond their limited range. Where I would like to see the market for activity monitors head is into the smart watch category. By combining the functionality of a smart watch such as Pebble with activity monitoring you can really start to envision a must have product that could become as ubiquitous as smartphones. In fact the Pebble has an accelerometer built in so its no farther away from being able to replicate the functionality of the activity monitors than by someone creating the application for it. The Basis really starts to hint at this possibility with their solution. I would love to see the Basis and Pebble combined into one device.
For those fascinated by the idea of the quantified self like me, I highly recommend these devices; as for true mass-market adoption I believe these devices are still in the realm of early adopters. The best-in-class device will be the most passive device. I refer to being passive in a holistic sense. Passive to wear, you don’t have to think or worry about putting it on and/or feel you are even wearing it. Passive to sync data, I shouldn’t have to open an app or plug my device in. Passive in behavior modification, the devices and accompanying software should simply and easily empower and engage me to be healthier in a realistic way.
So after 28 days which device will be the one I continue to use? The Fitbit One.