biobankAccelerometerAnalysis icon indicating copy to clipboard operation
biobankAccelerometerAnalysis copied to clipboard

Impossible to process CWA

Open gianlucaperna opened this issue 2 years ago • 17 comments

Good Morning, I'm trying to analyze some CWA files that I collected using Axivity AX6 (https://axivity.com/product/ax6) sensor. Unfortunately, I keep receiving the error that I attached.

Any possible solution? Thanks a lot!

Screenshot 2022-06-20 at 16 31 34

gianlucaperna avatar Jun 20 '22 14:06 gianlucaperna

Hi @GianlucaPoliTo Unfortunately, we don't support AX6 at the moment -- only AX3. It is in our plans to support it eventually though, so keep an eye!

chanshing avatar Jun 20 '22 15:06 chanshing

Hi @chanshing, thanks a lot. Anyway, it seems a problem with some numerical stuff. Do you know if there is the possibility to easy fix by myself this, or, do you believe that it is an hard task?

Thank you again for your availability!

gianlucaperna avatar Jun 20 '22 16:06 gianlucaperna

@GianlucaPoliTo I see. One thing you could try is using the device parser from the original manufacturers of Axivity: https://github.com/digitalinteraction/openmovement It is open source. This will only parse the device data and give you the raw readings in a long CSV file. Then you could use our tool to read from this CSV file to extract the summary activity metrics of interest.

chanshing avatar Jun 20 '22 17:06 chanshing

Just to let you know that I implemented code to read .cwa AX6 for R https://github.com/wadpac/GGIRread/blob/main/R/readAxivity.R with a bit off c++ code to speed it up. The code automatically recognises whether it is AX3 or AX6.

I know this is not directly useful in Python, but maybe helpful as inspiration/reference point when checking your future code for AX6. I verified with Dan at Axivity that it is correct.

vincentvanhees avatar Aug 15 '22 10:08 vincentvanhees

Hi @vincentvanhees, good to know! Thanks a lot for your time =) No problem with the code in R, but, if I'll have time, probably I'll try to translate the code also in Python, just to have the extension at this point =)

Thanks again!

gianlucaperna avatar Aug 15 '22 20:08 gianlucaperna

Hi @chanshing, I run in the same problem for a research project using the AX6 sensor.
 Are there any concrete plans to implement this feature in the near future or is it still an ‘eventually’? 
 An update would be greatly appreciated.

DanielHieber avatar Nov 10 '22 09:11 DanielHieber

You can load .CWA files (including AX6 data) into Python using openmovement-python - CwaData.

danielgjackson avatar Nov 10 '22 12:11 danielgjackson

You can load .CWA files (including AX6 data) into Python using openmovement-python - CwaData.

@danielgjackson Oh thanks for the plug. It seems sufficiently simple to integrate with the rest of the pipeline with some sort of data transformation.

@DanielHieber For the time being, supporting AX6 sensor is likely a TBD task because none of our studies is using AX6. If someone would like go it a go with a PR for the integration, we would be more than happy review and revise the the PR quickly.

angerhang avatar Nov 10 '22 12:11 angerhang

Thanks for the update @angerhang Can you point me in the direction how to best implement this feature/do you maybe have any ideas where I could start or what I have to take into consideration? Depending on the required work I could give it a try.

DanielHieber avatar Nov 10 '22 15:11 DanielHieber

Dear @angerhang , @chanshing and whoever it may concern,

is there any progress with this issue? All the best

johannesschobel avatar Dec 15 '22 09:12 johannesschobel

Thanks for being patient on this issue @danielgjackson and @johannesschobel

If you would like to give it a go for the ax6 support, you would change the binary parsing mechanism by modifying the file below:

https://github.com/OxWearables/biobankAccelerometerAnalysis/blob/master/accelerometer/java/AxivityReader.java

The reference parsing code can be found in openmovement repo: https://github.com/digitalinteraction/openmovement/commit/570001c28c7fec3d5aba904b3712e0cb9b12ef6c

If you can write something sensible with some tests results. We would be more than happy to troubleshoot and sync it with our python backend migration with https://actipy.readthedocs.io/en/latest/ early next year.

angerhang avatar Dec 19 '22 15:12 angerhang

Hello ,

I'm experiencing the same problem as @GianlucaPoliTo but I'm using the AX3 instead.

The calibration fails.

I have narrowed it down to a java call to JTransforms-3.1-with-dependencies.jar. This call should create a file *-stationaryPoints.csv.gz, which is in fact empty. Therefore, when accelerometer/device.py attempts to read the file, it encounters an empty DataFrame, which then breaks the if (np.max(xyz, axis=0) < CALIB_CUBE).any() on accelerometer/device.py's call around line 277.

Do you know if this is actually a JTransforms issue? Could it be something about the acquisition that's wrong? (Here is my recordSetup.xml.txt file with the acquisition configuration)

Can you imagine a way to overcome this empty data frame?

Thank you in advance !

balpeaux avatar Mar 15 '23 08:03 balpeaux

@chanshing will know better, but do you have a sufficiently long file @balpeaux ? If you have a short file (just a few hours long) with no 10 seconds of stationary time, this may result in a null output.

aidendoherty avatar Mar 16 '23 21:03 aidendoherty

Hello @aidendoherty thanks for your reply Actually it has almost 5 days of acquisition.

Here's the link to download it !

Any suggestions @chanshing ?

Thanks in advance !

balpeaux avatar Mar 21 '23 13:03 balpeaux

@balpeaux

Thanks for sharing the file. Something seems to be off about the measurements in this file. There are values as high as 100s, which should be impossible for AX3 devices (+/- 16).

Like you discovered, bug is due to the calibration routine not finding any available static points (because values are crazy high). Thanks for spotting this. It's an extreme edge case but fixed now (I'll push the patch in a bit).

Thanks for your help!

01-timeSeries-plot

image

chanshing avatar Mar 28 '23 06:03 chanshing

@balpeaux It should be fixed now in v6.2.3. $ pip install -U accelerometer to upgrade.

chanshing avatar Mar 28 '23 06:03 chanshing

@balpeaux It should be fixed now in v6.2.3. $ pip install -U accelerometer to upgrade.

Excellent, thank you very much @chanshing !!

balpeaux avatar Mar 28 '23 07:03 balpeaux