WOO 3.0 Game over?

The WOO 3.0 is here now for a little while and it’s most likely to change the game. Although were not quite sure if that is in an positive way.  Since 2015, when WOO introduced its device to record your kite sessions -including jump height- the sport got an interesting addition to it. Although it is a fun sport, some sort of competition was added to it; Who can jump -world wide- the highest?

Recently WOO offered a rather affordable way to buy the 3rd generation.  The business model has little other ways to get money in, so it seems a smart way to collect some fresh coins. It does have added value like a better battery, better mount and an instant smart watch read out possibility. But there seems to be a big catch in this WOO 3.0 and in the end it may have ruined its own game. Or should we all get rid of our previous version and buy the 3.0?

WOO 3.0 instant read out on a smart phone
WOO 3.0 instant read out on a smart watch

Soon after the launch of the WOO 3.0’s the big question does it record higher jumps compared to the previous versions?  popped up on several forums. The answer is partly true but also more complex than only that. We took a bunch of different versions on one board and made some statistical comparing on the data.  After looking into it deeper and deeper we think the level playing field between different WOO generations has gone.

We see unexplained differences and structural variations between the  newest WOO and the previous versions. The WOO 3.0 can give much higher maximum jump heights. How is that possible? We tried to contact WOO -without succes- so for the moment we can only guess. Below you can read what efforts we did to properly compare them and our conclusions is quite bitter. But to stay on top of the game you better switch to the 3.0 version.

The good news is that the 3.0 versus 3.0  version seems to be a fair game. But the differences between the WOO versions possibly qualifies the WOO 3.0 to be a -sarcastically ment- ‘true game changer’. 

Testing details

This test isn’t about the accuracy it self. So we didn’t investigate if 10m readings are really 10m. This test is neither on the added value of the latest version of the WOO (better battery, connection to a smart watch or the waterproof mount). The test is about a level playing field. Is each Woo version – to some degree- comparable?

Current records on the WOO leaderbaoard
Current records on the WOO leaderboard

Comparing WOO’s properly isn’t easy. For example WOO is updating it’s firmware from time to time (for the WOO 3.0 for example already has had 3 versions). Do those versions influence the data? We cannot be 100% sure. Since the 1.0 vs 1.0 in this test (with different software versions) is a lot more different compared to our previous tests, it suggests different software versions play an important role. Another problem is the time recordings; the difference in readings of the same jumps (gradually) deviate easily 10-15 seconds in one hour.  Which is much more as we would expect from a measuring device and makes comparing jumps for this test more difficult.

Results

Result of 1.5 hrs of kiting with 4 different WOO's on one board
Result of 1.5 hrs of kiting with 4 different WOO’s on one board

It is not difficult to see there is something strange with our data. The difference in the maximum jump height in our test is up to an immense 4.4m (the 1.0 version is 30% lower compared to the Woo 3.0). Is this an incident or are there structural differences?This specific, highest jump on both 3.0 WOO’s (at about 20 minutes kiting) had 16.1 and 15.3m, where the Woo’s 1.0 only had 8.6 and 5.7m. An even worse difference compared to the complete session results (where the WOO’s 1,0 found up to 12.3m on a completely other jump). Obviously the Woo 1.0 ‘s have had totally different moments for the highest jump, where the 3.0’s had quite similar readings. The WOO game starts to sound a bit like a lottery!

Structural deviations

To see whether this mis-match was an occasional incident or an structural deviation we made several X-Y comparisons. In those X-Y graphs, for those who don’t know them, the R-squared should be close to 1.0. Low R-squared indicates there is no link between the two readings (of the different WOO’s). The “a”  in Y=aX tells you what the structural difference is between X and Y, where 1.0 means they are similar.

The 3.0 to 3.0 data looks decent. With an 0.8 R-squared and a slope of 0.98. This means pretty much a level playing field between WOO 3.0 users. But mind you that a more deeper statistical analysing shows an easy chance of having a couple of meters differences on 2 similar 3.0 WOO’s on the same board, on the same jumps in the top league competition -let’s say above  20m (the 99 % CI is about 10%)-.

The WOO 3.0 to 1.0 data looks pretty bad.  Although it seems the WOO 1.0a is only about 10% lower, the R-squared is way too low.

The same result is found at the other WOO 1.0; about 25% lower readings, but more worrying is again the very low R-squared. Which means these two WOO’s are different devices and incomparable. 

WOO 3.0 to 2.0 data looks slightly better compared to the 3.0-1.0 correlation.

Similar 2.0 vs 3.0 results, but they are deceiving
Similar 2.0 vs 3.0 results, but they are deceiving

The summary results for the session with the 2.0 and 3.0 look very promising. But this is deceiving. The highest jumps are not at the same moment and are coincidentally similar in height. In fact the jumps with 12.5 and 12.4m are paired with jumps 1.4m and 2.0m lower on the other device. The correlation between the individual jumps on the 2.0 and 3.0 is quite weak and we wouldn’t call it a fair and comparable measuring devices.

WOO 1.0 to 1.0 correlation is also quite weak. Much weaker as our previous tests back in 2015.  We did use different software versions in this test but it should not be causing such a huge difference to our opinion.

Explanation

The cause of these differences between the WOO 3.0-2.0 and 1.0 versions are to be guessed; software or hardware differences, calibration issues or maybe the quality assurance person didn’t get enough time (or money) to sort things out properly? Too bad since we had much better results at some point between two WOO’s 1.0, so it should be technically possible to sort things out. Better do it before some one hits the 30m mark on a WOO 3.0 and everybody will discuss the quality of the data instead of the amazing achievement.

Statistical information on our test:

Firmware’s used:

Woo 1.0a : 3.16.70

Woo 1.0b:  3.16.72

Woo 3.0a and 3.0b: (paired with the 1.0) 30.1.15

W00 2.0 ; 3.16.137

WOO 3.0c: (paired with the 2.0) 30.1.9

2 WOO 3.0 (in the new mount) and 2 WOO 1.0 (old mount) on one board for 1.5hr, total of 70 jumps above 5m

1 WOO 3.0 and 1 WOO 2.0 on one different board for 1.5hr, total of 17 jumps above 5m

Only >5m jumps on both devices are analysed. More results on air time and g-force landings will be reviewed separately.

Total Views: 20508 ,
Facebooktwittermail
Facebooktwitter

16 thoughts on “WOO 3.0 Game over?”

  1. Hi Robert thanks for coming back at it.
    I still think the performance could have been improved without getting a consistent bias or deviation from previous versions. You are not afraid the one in charge of marketing insisted on something new that would be more interesting to use than a 1.0 version? Not only performance wise but also to move on tot he 3.0 otherwise you are 30% behind in the game? We know a lot of people that have 1.0 versions that work perfectly fine but don’t want to be behind in the leader boards and are therefor buying the 3.0 version.
    Dave
    We Test Kites

  2. Hey Dave,
    its Robert again, thanks for your feedback and sorry for my late reply.
    I can see your point, but to be honest – I dont share it.
    In summary, your point is mainly about keeping the game fair so that everyone that ones a Woo (regardless of which generation) gets the same records.
    Thats an sacred opinion! But in the same moment, this would be the end of any development to improve the accuracy of the measurement device.
    Just because the WOO 1.0 is reliable when compared to the same generation, it does not mean that it is valid or accurate. And in my view this is much more important.
    Is it a pity when WOO 1.0 Users get lower records? Maybe. But dont you regularly get worse performance when sticking to an older product compared to a newer (cars, laptops..)?

    If you would translate your argument to other fields of daily live, it may be becomes more obvious. If it is all about the fair game instead of approximate the true value, people would have had to stop to improve the resolution of MRI screens because it would have been no fair game for those patients who are screened with an older device.
    And the reliability issue? You can have a perfect reliable, internally valid measurement device, but if it is just not externally valid (it does not measures what it intends to measure) it is of no diagnostic value.

    That the step from WOO 1.0 to 2.0 was just about the introduction of the freestlyemode is not true. Marketingwise, this was the main point. But performancewise, there was a huge improvement in the measurement-technique.

  3. Hi,
    For our non-Dutch speaking readers; he is asking about any reaction from WOO. Well, we did not get any follow ups. We also have seen some statements from WOO we doubt about. Like the WOO 1.0 cannot be upgraded and that the WOO 3.0 can’t be used on a hydrofoil. Just like their request to users to keep ons asking smartwatch company’s to be able connect to the WOO, we ask our readers to point out to WOO we haven’t been contacted by WOO. We would have liked to discussed the results and are open for any reaction or re-test and specific answers on questions like you have.
    Dave
    We Test Kites

  4. Hebben jullie ondertussen reactie van Woo ontvangen? Voor mij is de lol eraf. Gisteren beweerde het apparaat dat ik 10.7 meter had gesprongen, klopt echt niets van. Ik wil mijn geld terug!

  5. Hi Tom,
    Thanks for your suggestions. We have had this question numerous times. There are several options to check if a WOO meter is a real meter. For sure your suggestion is interesting. We also had suggestion to do it with cameras at several heights.
    We will see if we can set up such a test. For the moment we are more focused on the kite equipement itself.
    Dave
    We Test Kites

  6. Hi guys!

    What do you think about a test where the actual distance, height and g force are controlled?

    What’s the reason of comparing unknown jumps in terms of the above metrics?

    I think using some sort of cable pulley system would work. Measure the robe, add a stopper ball, pull it up and let it fall down.

    Let me know your thoughts

  7. Hi Xelas,
    Thanks for asking. The PIQ is/was quite close to the WOO 1.0. https://www.wetestkites.com/2016/07/31/north-piq-vs-woo-sports-jump-height-and-air-time-part-1/
    The only way to see if the software updates did change anything is a (PIQ) one to one test. More realistic (We don’t know if you can/could not update the PIQ)and more interesting is a test between WOO 3.0 and PIQ with the newest software version.
    Which we will intend to do so!
    Keep an eye out on our website (like us on FB, or leave your email for an update by mail or message) for the results.
    Dave
    We Test Kites

  8. Hello,
    What about PIQ?
    Several firmware updates was made.
    Could it be that they are also play this unfair game?
    May be you can test PIQ vs WOO 3.0?
    As I remember WOO 1.0 was showed abot 10% higher results.
    with additional 30% from WOO 3.0 it shold be ~43%
    What do you think about it?

    Alexander

  9. Hi Robert,
    Thanks for adding some thoughts on the review.
    The 2.0 has been introduced mostly because of the freestyle competition. It did not have the massive discount when introduced. The WOO 3.0 did have a solid discount,which meant a huge amount of people that changed from 1.0 to 3.0. As a result PR’s where beaten at an instance.
    It is all about a fair game. Which might have already be ruined introducing the 2.0, but with the 3.0 it is introduced much more (user wise).
    We could have understood that the number of jumps would increase due to better technology, or even a bigger difference when jumping in the 10 range. But instead there is a solid +30% difference. It cannot be explained by better technology as the 1.0 to 1.0 (on the same software versions) did have a really good R-squared.
    If -and we don’t really think this way- the WOO 3.0 would have read -30% a quick fix would have been made instantly. As commercially wise a device that reeds lower can be very disappointing. But if we look into the eyes of Leo from WOO we truly think it more has to do with the (time) pressure than the need for a commercial thought into the algorithm.
    As for “the golden standard”, this is nice to know but isn’t important for a fair competition. In reality a 10m jump can be 5m or 15m, as long as we have a similar reading on a similar jump it is okay.
    Feel free to contribute again your thoughts!
    Dave
    We Test Kites

  10. I cant understand why the WOO 3.0 is compared to the WOO 1.0 instead of comparing it to the WOO 2.0.
    It is well known that 2.0 measured higher jumps than the 1.0 did. The reason for this are improved sensors (higher g-force sensitivity). So why you did not tested the 3.0 against the 2.0? It would be really awkward if they had done a step back with 3.0.
    And the conclusion that questions the validity of the WOO at all is highly doubtable. You cant dispute the validity of a measurement device, if you compare it to a less reliable measurement device instead of comparing it to a gold standard measurement device.

  11. Hi “Downwinder”,
    thanks for adding this theory. We are not sure this may be the reason, but if so it is even more worrying. IF a new theory is chosen per definition a unfair competition is introduced. Leo from WOO is suggesting it may have to do with different/better hardware in the 3.0 (f.e. in the 3.0 version a higher g-force can be measured, which could mean big boosts are too high for the 1.0 version since it may exceed the g-force maximum readings which can lead to differences).
    Dave
    We Test Kites

  12. My understanding is that woo 1.0 measures from take off from water where ever that may be, where 3.0 is measuring total height off sea level. I was confused as I was boosting off big waves 20ft plus in Maui outerreefs and the height didnt make sense as I was way up, but woo was calling it 35feet. I knew I was more than 50ft. I started reading the research and that was takeaway. Have not mounted the 3.0 but if you were boosting off kickers in your study add the difference and see it matches up… just sayin

  13. Hi Ed,
    Yes we thought of that. But think again; if this would be having an effect it would be ridiculous. We placed it right in the middle of the board and not different from the instructions or anybody else (we even put the button from all of them to one side).
    Dave
    We Test Kites

  14. It’s possible that the mount position has an effect. Would be good to control for this by swapping the positions around for half of the session.

  15. Hi Matthieu,
    Thanks! We adjusted it. (although We know the amount of smart phones that can be used in (salty) water is very very limited).
    Dave
    We Test Kites

  16. Great test, you keep saying “smart phone” but i think you meant smartwatch … 😉

Leave a Reply

Your email address will not be published. Required fields are marked *