mifit-data-export icon indicating copy to clipboard operation
mifit-data-export copied to clipboard

Problems to extract data in non-root phone

Open vicmanmo opened this issue 8 years ago • 9 comments

Hi,

I need to extract MyBand data to create a virtual sensor in OpenIoT.

I have a non-rooted Xioami RedMi Note 2 and I set ForceBackupMode value to Y in run.sh, but in my phone I don't see backup screen. In the other project of author is @xmxm, I see backup screen, but the program doesn't run.

Can you help me?

The result of running run.sh is this:

extraction started on 02/23/2017 09:36 cannot find database files. non-rooted phone? attempting native backup approach adb server is out of date. killing...

  • daemon started successfully * adb: unable to connect for backup dd: mi.ab: no se puede saltar al desplazamiento especificado 0+0 registros leídos 0+0 registros escritos 0 bytes copied, 5,8166e-05 s, 0,0 kB/s tar: Esto no parece un archivo tar tar: apps/com.xiaomi.hm.health/db/origin_db: No se encuentra en el archivo tar: apps/com.xiaomi.hm.health/db/origin_db-journal: No se encuentra en el archivo tar: Se sale con estado de fallo debido a errores anteriores cp: no se puede efectuar `stat' sobre 'apps/com.xiaomi.hm.health/db/origin_db*': No existe el archivo o el directorio extracted MiBand database deleting temp backup files extraction failed still cannot find new database files - restoring original database backups

Thank you very much

vicmanmo avatar Feb 23 '17 08:02 vicmanmo

I have changed run.sh for using adb of my OS, and now I see backup screen and read registers.

But tar: apps/com.xiaomi.hm.health/db/origin_db: it doesn't find the file.

Can you help me?

The result of running run.sh is this:

extraction started on 02/23/2017 10:27 cannot find database files. non-rooted phone? attempting native backup approach press Backup My Data button on device... adb server is out of date. killing...

  • daemon started successfully * Now unlock your device and confirm the backup operation. 17+0 registros leídos 17+0 registros escritos 17 bytes copied, 3,0043e-05 s, 566 kB/s tar: apps/com.xiaomi.hm.health/db/origin_db: No se encuentra en el archivo tar: apps/com.xiaomi.hm.health/db/origin_db-journal: No se encuentra en el archivo tar: Se sale con estado de fallo debido a errores anteriores cp: no se puede efectuar `stat' sobre 'apps/com.xiaomi.hm.health/db/origin_db*': No existe el archivo o el directorio extracted MiBand database deleting temp backup files extraction failed still cannot find new database files - restoring original database backups

Thank you very much

vicmanmo avatar Feb 23 '17 10:02 vicmanmo

When it executed

adb backup -f mi.ab -noapk -noshared com.xiaomi.hm.health

The file mi.ab doesn't have data.

Where is the problem??

vicmanmo avatar Feb 23 '17 13:02 vicmanmo

Same here - it doesn't matter if I use system adb or recent platform-tools adb from Google. com.xiaomi.hm.health is even not in the full backup.

jreznik avatar Jun 29 '18 11:06 jreznik

Same, tried to figure out why but no luck

ErikBjare avatar Dec 02 '18 20:12 ErikBjare

Hi,

the mi fit app now has an export functionality, most like because of GDPR. Through some google search I came across the URL the in-app browser of Mi Fit would be using for the export: https://user.huami.com/hm_account/2.0.0/index.html?loginPlatform=web&platform_app=com.xiaomi.hm.health&v=s3.1.1#/

You can get your data from there*. I just checked the files and the numbers matched the app so far. The export contained 6 CSV files in my case:

(NAME OF FILE).csv
(CSV HEADER)

ACTIVITY_<ID>.csv
date,lastSyncTime,steps,distance,runDistance,calories

BODY_<ID>.csv
timestamp,weight,height,bmi,fatRate,bodyWaterRate,boneMass,metabolism,muscleRate,visceralFat,impedance

HEARTRATE_<ID>.csv
date,lastSyncTime,heartRate,timestamp

SLEEP_<ID>.csv
date,lastSyncTime,deepSleepTime,shallowSleepTime,wakeTime,start,stop

SPORT_<ID>.csv
type,startTime,sportTime,distance,maxPace,minPace,avgPace,calories

USER_<ID>.csv
userId,gender,height,weight,nickName,avatar,birthday

*I did not test the app export functionality itself as it requested me to login again - with my main Google account in an app-internal browser, despite using Google as my "Mi Fit App login".

ironjan avatar Dec 09 '18 13:12 ironjan

@ironjan That's great news! Trying it out now, thanks a bunch for letting us know :heart:

Edit: Export worked fine, but disappointed to only find day-resolution data and not minute resolution for steps/heartrate :disappointed: Was that available in the exports made with the mifit-data-export tool?

ErikBjare avatar Dec 09 '18 14:12 ErikBjare

TL;DR: GDPR complaint caused export to gain minute-wise data.


@ErikBjare Took a while to come back. After your comment I checked the data and verified that it is not minute-wise. Since then I have requested minute-wise data from xiaomi (my mistake; even though the play store privacy agreement states Xiaomi, the data seems to be at Huami). After there was no reaction, I issued a GDPR complaint to the data protection office of Northrhine-Westphalia and received word a few weeks back that "the app has been updated to export minute-wise data". This was untrue then; I tried again today and the most export contained minute wise data. I did not yet verify if the export is complete.


Extract:

$ ls
ACTIVITY.csv  ACTIVITY_MINUTE.csv  ACTIVITY_STAGE.csv  BODY.csv  HEARTRATE_AUTO.csv  HEARTRATE.csv  SLEEP.csv  SPORT.csv  USER.csv
$ head ACTIVITY_MINUTE.csv
date,time,steps
2018-03-24,18:26,12
2018-03-24,18:29,104
2018-03-24,18:32,117
2018-03-24,18:33,37
2018-03-24,18:34,104
2018-03-24,18:37,127
2018-03-24,18:39,116
2018-03-24,18:40,115
2018-03-24,18:41,112
$ head HEARTRATE.csv
date,lastSyncTime,heartRate,timestamp
1565020753,70
1564605323,76
1554157496,72
1546406148,62
1545737025,81
1545006196,80
1542865300,78
1542788260,72
1542615574,67
$ head HEARTRATE_AUTO.csv
date,time,heartRate
2018-04-12,19:48,47
2018-04-12,19:50,53
2018-04-12,19:52,65
2018-04-12,19:54,53
2018-04-13,01:52,74
2018-04-13,01:54,68
2018-04-13,01:56,71
2018-04-13,01:58,69
2018-04-13,02:00,70

ironjan avatar Sep 07 '19 21:09 ironjan

@ironjan Amazing! Thank you so much for putting in the effort!

ErikBjare avatar Sep 08 '19 08:09 ErikBjare

The new website address is https://mifit.huami.com/t/account_mifit (which, for me, redirects to https://user.huami.com/privacy2/index.html?loginPlatform=web&platform_app=com.xiaomi.hm.health&v=4.0.17#/)

However it still does not include all of the data which is in the app. For example sleep data, it only has total minutes in each zone, whereas the app has exact start times of each zone transition. Does anyone know how to get huami to update their export with this data?

howff avatar Jan 15 '22 21:01 howff