oref0-upload-profile fails to upload

[viq]

Describe the bug oref0-upload-profile either has issue, or is not documented sufficiently - for someone not using a medtronic pump it seems impossible to use.

To Reproduce Steps to reproduce the behavior:

1. Have a fresh OpenAPS setup, with no profiles in place
2. Use simplest profile provided, for example the one listed in step 3 from autotune page with the values filled in, so for example having test-profile.json like this:

{
  "min_5m_carbimpact": 3,
  "dia": 5.5,
  "basalprofile": [
    {
      "start": "00:00:00",
      "minutes": 0,
      "rate": 1
    },
    {
      "start": "08:00:00",
      "minutes": 480,
      "rate": 1
    },
    {
      "start": "13:00:00",
      "minutes": 780,
      "rate": 1
    },
    {
      "start": "21:00:00",
      "minutes": 1260,
      "rate": 1
    }
  ],
  "isfProfile": {
    "sensitivities": [
      {
          "i": 0,
          "start": "00:00:00",
          "sensitivity": 55,
          "offset": 0,
          "x": 0,
          "endOffset": 1440
      }
    ]
  },
  "carb_ratio": 17,
  "autosens_max": 1.2,
  "autosens_min": 0.7
}

3. Try to upload the profile, get error:

$ oref0-upload-profile ./test-profile.json $NS_SITE $API_SECRET -p
Could not parse input data:  TypeError: Cannot read property 'length' of undefined
    at Object.<anonymous> (/home/diabox/src/oref0/bin/oref0-upload-profile.js:82:37)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)

Expected behavior Have steps decribed in How to automatically sync your profile from Autotune work

Smartphone (please complete the following information):

Setup Information (please complete the following information):

• Pump type: - Dana R
• CGM type: Libre+Miao via xdrip+
• Rig type: AndroidAPS
• oref0 version: oref0 dev branch

Additional context I'm trying to set up an "out of band" autotune process, and this is one of steps required for this.

[jrw35]

I have exactly the same problem, although the error isn’t printed in the same level of detail you give. I think it’s because the uploader checks for a section called bg_targets in the profile. If it’s not there then it fails. That section isn’t included in the simple profile in the docs. I don’t really understand how the uploader can work with profiles generated by OpenAPS because they don’t include that section.

I’m trying to construct a minimal profile that works but I haven’t got there yet.

[viq]

@jrw35 I'm working on a script that will convert NightScout profile into something that OpenAPS will understand, you can see my WIP here. It still needs improvements, but the OpenAPS profile generated is accepted both by autotune and oref0-upload-profile.

And indeed updating docs with minimal profile that would be accepted would be awesome.

[jrw35]

I thought I might have to start working on something like that myself, but you're well ahead of me. Thanks for sharing.

I tried your script this morning, and it downloaded my Nightscout profile, but it wasn't accepted by autotune. Unfortunately I had to go to work, and I didn't have time to figure out what was going wrong. I'll try again later. Should I be using an option to get it to output in OpenAPS format?

If you're willing to share an example profile that works with the uploader then I'd be very interested in seeing one so that I can work out what's missing from mine. Thanks again.

[viq]

Try something like

python ./get_profile.sh --nightscout https://your.nightscout.address display --format openaps > profile.json

Below is my current profile as generated by script, though I'm still reworking the script based on AutoTuneWeb's NSProfile

{
    "min_5m_carbimpact": 8.0,
    "dia": 5.5,
    "basalprofile": [
        {
            "i": 0,
            "minutes": 0.0,
            "start": "00:00:00",
            "rate": 0.242
        },
        {
            "i": 1,
            "minutes": 60.0,
            "start": "01:00:00",
            "rate": 0.256
        },
        {
            "i": 2,
            "minutes": 120.0,
            "start": "02:00:00",
            "rate": 0.265
        },
        {
            "i": 3,
            "minutes": 180.0,
            "start": "03:00:00",
            "rate": 0.276
        },
        {
            "i": 4,
            "minutes": 240.0,
            "start": "04:00:00",
            "rate": 0.378
        },
        {
            "i": 5,
            "minutes": 300.0,
            "start": "05:00:00",
            "rate": 0.436
        },
        {
            "i": 6,
            "minutes": 360.0,
            "start": "06:00:00",
            "rate": 0.406
        },
        {
            "i": 7,
            "minutes": 420.0,
            "start": "07:00:00",
            "rate": 0.401
        },
        {
            "i": 8,
            "minutes": 480.0,
            "start": "08:00:00",
            "rate": 0.392
        },
        {
            "i": 9,
            "minutes": 540.0,
            "start": "09:00:00",
            "rate": 0.387
        },
        {
            "i": 10,
            "minutes": 600.0,
            "start": "10:00:00",
            "rate": 0.383
        },
        {
            "i": 11,
            "minutes": 660.0,
            "start": "11:00:00",
            "rate": 0.383
        },
        {
            "i": 12,
            "minutes": 720.0,
            "start": "12:00:00",
            "rate": 0.376
        },
        {
            "i": 13,
            "minutes": 780.0,
            "start": "13:00:00",
            "rate": 0.375
        },
        {
            "i": 14,
            "minutes": 960.0,
            "start": "16:00:00",
            "rate": 0.373
        },
        {
            "i": 15,
            "minutes": 1020.0,
            "start": "17:00:00",
            "rate": 0.372
        },
        {
            "i": 16,
            "minutes": 1080.0,
            "start": "18:00:00",
            "rate": 0.373
        },
        {
            "i": 17,
            "minutes": 1140.0,
            "start": "19:00:00",
            "rate": 0.373
        },
        {
            "i": 18,
            "minutes": 1200.0,
            "start": "20:00:00",
            "rate": 0.372
        },
        {
            "i": 19,
            "minutes": 1260.0,
            "start": "21:00:00",
            "rate": 0.37
        },
        {
            "i": 20,
            "minutes": 1320.0,
            "start": "22:00:00",
            "rate": 0.359
        },
        {
            "i": 21,
            "minutes": 1380.0,
            "start": "23:00:00",
            "rate": 0.351
        }
    ],
    "bg_targets": {
        "units": "mg/dl",
        "user_preferred_units": "mg/dl",
        "targets": [
            {
                "i": 0,
                "start": "00:00:00",
                "offset": 0,
                "low": 100,
                "min_bg": 100,
                "high": 130,
                "max_bg": 130
            },
            {
                "i": 1,
                "start": "08:00:00",
                "offset": 28800,
                "low": 90,
                "min_bg": 90,
                "high": 120,
                "max_bg": 120
            },
            {
                "i": 2,
                "start": "22:00:00",
                "offset": 79200,
                "low": 100,
                "min_bg": 100,
                "high": 130,
                "max_bg": 130
            }
        ]
    },
    "isfProfile": {
        "first": 1,
        "sensitivities": [
            {
                "i": 0,
                "sensitivity": 100.0,
                "offset": 0.0,
                "start": "00:00:00"
            },
            {
                "i": 1,
                "sensitivity": 55.712,
                "offset": 480.0,
                "start": "08:00:00"
            },
            {
                "i": 2,
                "sensitivity": 100.0,
                "offset": 1380.0,
                "start": "23:00:00"
            }
        ]
    },
    "carb_ratios": {
        "first": 1,
        "units": "grams",
        "schedule": [
            {
                "i": 0,
                "start": "00:00:00",
                "offset": 0.0,
                "ratio": 14.0
            }
        ]
    },
    "carb_ratio": 14.0
}

[jrw35]

Excellent. Thanks -- that's very helpful. I think I did use pretty much that command earlier but it didn't like the --format option. I'll try it again when I get home.

[viq]

And I'll see if the version I have now is suitable for showing. --

viq

[jrw35]

Your script crashes when I add the --format openaps option. It works without that. I think I’ve worked out why: it’s because the profile from NS is missing the timeAsSeconds key for the final basal entry of the day. Look at the difference between 22:00 and 23:00:

    {
        "timeAsSeconds": "79200", 
        "value": "0.72", 
        "time": "22:00"
    }, 
    {
        "value": "0.71", 
        "time": "23:00"
    }

It’s not your script’s fault. That’s how NS returns the profile.

Have you seen anything like that? I might modify your script to load it from a file and then I can edit the file first!

[viq]

No, I haven't. I guess I can modify the script to just look at time and generate timeAsSeconds from that.

[viq]

@jrw35 have you tried using https://autotuneweb.azurewebsites.net/ ? Does it accept your profile, or does it choke on it as well?

[jrw35]

Yes, it works, but it seems to fill in the missing timeAsSeconds entry.

The profile generated by AutotuneWeb works as an input to autotune on my Raspberry Pi... but the output doesn’t work with oref0-upload-profile because of the missing targets. When I get back to my computer I can test the uploader with your example profile.

[viq]

Interesting, I'll have a closer look at what it does and how.--

viq

[jrw35]

Thanks, viq. I’ve just tested the profile you gave me above with oref0-upload-profile —preview and it works.

[jrw35]

Progress. I've taken the output from autotuneweb and added the missing sections from your example profile (changing them to match my numbers). Autotune accepts it, and oref0-upload-profile uploads it.

I can't use it yet because some numbers are being incorrectly multiplied by 18, but at least I know it's theoretically possible!

Your tool will make it much easier if I/you can find a way to deal with the missing timeAsSeconds. Thanks for helping me.

[viq]

I've seen how autotuneweb does it, I'll need to spend some time to adapt it, hopefully this weekend.

--

viq

[jrw35]

Hello again. I've got your script working with autotune and have successfully uploaded the results to Nighscout. Three key things:

1. I had to change int to float in lines 102 and 107 of your code. I use mmol/L and so it was crashing when it read my target (6.5).

2. It only works when I have a single profile on Nightscout, not multiple profiles.

3. I also had to edit oref0-upload-profile.js to remove Math.Round() for the targets and ISF, and I had to comment out the lines immediately afterwards that deal with unit conversion.

Thanks for helping me get to this point. If you can improve further then please do!

[viq]

1. Good point, changing to floats.
2. It tries to get profile marked as default, or you can specify the name of the profile with --name

[jrw35]

Yes, I think it was probably oref0-upload-profile that went wrong with multiple profiles, not your downloader. To be more precise I had weird results in Nightscout after using the uploader when there were multiple profiles present, so I went back to a single starting profile and the problems went away.

Not all my problems are solved yet -- the dev branch of oref0 won't install at the moment -- but you've helped me make a lot of progress. Thanks again for working on this tool. I'll be interested to see where you go with it.

[viq]

I guess https://github.com/viq/oref0/blob/profile_from_nightscout/bin/get_profile.py is the better place to keep working on it, it has an updated version.

[scottleibrand]

@viq do we need to keep this issue open, or should we track it in other places (like https://github.com/openaps/oref0/issues/300)?