Resources (fonts & pictures) packaging

[JF002]

Verification

• [X] I searched for similar feature request and found none was relevant.

Pitch us your idea!

Let's specify how resources will be packaged and handled by companion apps

Description

In #321 I did a lot of experiments with the external flash storage and the file system. The goal is to use this external flash memory to store big resources (pictures and fonts) so that we can free quite a lot of space in the internal flash memory.

In this comment, I listed the action points that must be done before we can release this feature. This discussion is about the resources packaging and management by companion app : how will we packaged the resource files so companion app can upload them to the watch using the BLE FS API ?

Please note that this is a technical discussion mostly targeted to InfiniTime and companion apps developers.

During my experiments, I used ITD to manually upload previously generated binary files to the watch. I sometimes needed to retry because the transfer would fail for some reason. I also had no way to check if the data were corrupted during the transfer. The upload of a full picture (24024016b = 115200B) took ~2 minutes, iirc.

Sending random binary files to the watch does not fit well with the end-users. I would like all the file to be automatically generated at build time (this is not the scope of this discussion), packaged into a single file with all the info needed for companion app to automatically upload the files to the watch.

Here's my proposal :

• package all the files in a compressed file format (.tar.gz or .zip). Maybe change the file extension to something else (InfiniTime Resource Format, IRF)
• add a small 'manifest' file that companion app will be able to use to identify this archive as a resource pack for InfiniTime
  • Could be as simple as a JSON file with the target infinitime version :

{ 
  "infinitime" : "1.11"
}

• it could also contain the file mapping (path and name of each file from the archive -> path and name in the in the PineTime FS)

{
  "files": [
    {
      "file1.bin": "/resources/watchface1/background.bin"
    },
    {
      "file2.bin": "/resources/watchface2/font.bin"
    }
  ]
}

• this file could also contain other meta data in the future
• I don't think our BLE API supports any kind of CRC functionality. The companion app will be able to check the data validity by reading back the file it has just transferred.
• I noticed that overwriting a file is very slow. If the file already exists, I recommend to delete it first before writing it again.

Questions:

• Am I missing anything?
• Is there some kind of existing standard that we could use instead of creating a new one?

So... this is a request for comment. Will this simple packaging strategy work well with InfiniTime and companion app to upload fonts and pictures in the external flash storage?

[Elara6331]

Since the filename would likely be hardcoded, we could check for corruption by having a BLE characteristic that generates and sends a crc32 (or some other algorithm) back to the companion. This would probably be a lot faster than reading back the whole file and prevent false positives where the read itself was corrupted.

[Elara6331]

Your proposal seems good. The only problem I foresee is that gzip decompression would have to take place on the watch and I am not sure how expensive that is resource-wise and how long it would take. The .gz is actually unnecessary. If just .tar is used alone, there is no compression. There is also the possibility of using a .zip without compression, but that might just confuse people when they try to modify the assets file and their archiving program generates a compressed zip.

[JF002]

Thanks for your feedback @Arsen6331 :+1: Adding a characteristic, or extending the BLE FS API is indeed a good idea to allow checking data validity with a CRC.

Regarding (de)compression, it would be done by the companion app. I certainly don't want to implement zip/gzip algorithms in InfiniTime. The archive is just a way to pack multiple files into a single one to make the flashing procedure easier for the end user. And yes, you are right, the compression might not be needed and we could just use .tar for this use-case. I mentioned compression because that's how DFU are currently packaged (using zip).

[Elara6331]

Also, the archive (without compression) could possibly be placed into the already existing DFU zip with a new entry for it in the manifest. That will mean there is still just one file for the user to get, rather than two,

[JF002]

Also, the archive (without compression) could possibly be placed into the already existing DFU zip with a new entry for it in the manifest. That will mean there is still just one file for the user to get, rather than two,

That's a good idea! Assuming it won't confuse companion apps, of course :)

[Elara6331]

That's a good idea! Assuming it won't confuse companion apps, of course :)

It shouldn't, since each file has a name in the manifest, like bin_file for the binary and dat_file for the init packet. This could just be asset_file or something, and companions that don't use it just won't get that key from the map.

[JF002]

We might not need to alter the manifest, and just add the resource file and maybe a metadata json file to the dfu archive.

[Elara6331]

As long as the name of the asset file always stays the same. The reason I use the manifest is that the names of the .bin and .dat files contain the version, so it's the only way I can know for sure what the filename is.

[JF002]

I did a bit of progress : files (fonts and pictures) are now converted and packed into a single .zip file using a CMake custom_target. This target

• Converts the fonts into .bin files (using a slightly modified version of src/displayapp/fonts/generate.py)
• Converts the images into .bin files (using a new script inspired from generate.py above)
• Generates a .json file with metadata (see below)
• Zip everything in a single infinitime-resources-1.11.0.zip

The content of the zip file looks like this:

infinitime-resources-1.11.0.zip
|
|---- 7segments_40.bin
|---- 7segments_115.bin
|---- infineat-1bin
...
|---- resources.json

resources.json:

{
    "resources": [
        {
            "filename": "7segments_40.bin",
            "path": "/7segments_40.bin"
        },
        {
            "filename": "7segments_115.bin",
            "path": "/7segments_115.bin"
        },i
        {
            "filename": "infineat-1.bin",
            "path": "/infineat-1.bin"
        }
    ],
    "obsolete_files": [
        {
            "path": "/background_v1.bin",
            "since": "1.11.0"
        },
        {
            "path": "/font_v2.bin",
            "since": "1.11.0"
        }
    ]
}

• The array resources contains the list of files from the .zip file that must be uploaded to the watch. It contains the name of the file in the .zip file and the path (filename) where it must be written to the watch.
• The array obsolete_files contains a list of files that can optionally remove from the watch. This is a hint to the companion app which lists the files that are not used anymore in this version of infinitime and that can be removed. It's up to the companion app to remove them to free some memory space. Each item contain the filename in the watch FS and the version of infinitime that made that file obsolete.

Any feedback regarding this packaging strategy?

EDIT : here is an example package: infinitime-resources-1.10.0.zip

And this is the expected result once the files are uploaded (screenshots using ITD:

./itctl fs ls                            
d 0.0 B  .
d 0.0 B  ..
- 4.9 kB 7segments_115.bin
- 760 B  7segments_40.bin
- 4.4 kB bebas.bin
- 1.4 kB infineat-1.bin
- 1.8 kB lv_font_dots_40.bin
- 115 kB matrix.bin
-  40 B  settings.dat
- 440 B  teko.bin

To support this, companion apps will have to implement our BLE FS API.

[Elara6331]

I'm wondering how changed assets should be handled. If someone decides to upload a font that is different from the one in the update, how would the companion know not to override it? I am thinking that maybe there should be a file full of crc32s for each asset, and then when the companion is doing the update, it leaves the files that have changed since that crc was taken?

[JF002]

I have also been thinking about this. Here's what we could do:

• the developer of an app that use external resource name the resource file something_v1.bin for example.
• in a future version of the app, if the developer updates the resource, name the new resource something_v2.bin' and add something_v1.bin' to the list of obsolete files.

This seems quite simple, allows to easily upgrade/downgrade the resources, and hints the companion app which older files can be removed.

Using filename instead of CRC to detect changes in files will probably be faster (and easier as there's no support for CRC in the FS right now).

[Elara6331]

No, I mean what if the user wants to modify a resource? Like if an app uses an image of some sort and the user wants to use a custom image instead.

[JF002]

Mhm that's a use-case I haven't anticipated... But I have the feeling that this should be handled on the companion app side. It could manage a list of "locked" files that cannot be overwritten, for example. Or yes, maintain a list of file that were overwritten manually by the user and not automatically update them when updating the whole resource package.

[Elara6331]

It probably should be handled by companions, but I think it should be standardized, because some users will use Gadgetbridge to update once, and then ITD the next time, and if ITD uses a different mechanism than Gadgetbridge, it'll just overwrite all their changes.

[JF002]

That won't be easy.. even if companion app developers agree on a single mechanism, you'll have to sync them across all your devices... Unless this "do not overwrite this file" flag is stored in the watch itself?

[Elara6331]

Yeah, that's what I was thinking. Maybe a file on the watch that looks like this

764df7d2 a.bin
b3e0e55f b.bin
...

And then add a characteristic or change to the FS that allows you to get a crc from the watch. It's the only way I can think of, because if it's stored on the companion device, that will mean, as you said, users would have to sync the file, and make sure it never gets deleted if they, for example, reinstall their OS.

[JF002]

Good idea! However, I might work on this on a second step, as I would like to implement a working version of the support for "external resources" soon. This feature could probably be added in a second step.

[NeroBurner]

maybe we could ignore this on the companion side and let the companion install everything which is referenced.

On the InfiniTime side we could have default-images used by the firmware and another set of user-provide-able overrides. For example we have a default background image for the clock, but if a user installs a user-bg.bin InfiniTime loads this instead of the default image

[Elara6331]

That works, but we have to consider that it will take up a lot more space

[JF002]

I've just added an example package and the expected results (via ITD) in the original comment.

[JF002]

Here is the corresponding PR : https://github.com/InfiniTimeOrg/InfiniTime/pull/1299

[Elara6331]

Ok, I have a working implementation of this in my infinitime package now. Next step is to integrate that into ITD, which shouldn't be difficult if I do it before performing DFU.

[Elara6331]

The new branch of ITD with resource loading is now complete: https://gitea.arsenm.dev/Arsen6331/itd/src/branch/resource-loading.

It currently does not do it as part of the DFU as I am not sure yet whether to do it before or after, and I don't know how this resources zip will be distributed (separate file, inside the dfu zip, etc.). Instead, there is now an itctl res load <zip> command.

[JF002]

Thanks @Arsen6331 for integrating this in ITD! On my side, I'm testing the procedure in Amazfish. Here are 2 videos :

• Upload resources using ITD :

https://user-images.githubusercontent.com/2261652/188277370-ea6f0280-fa48-43c7-80d2-be7c621668bc.mp4

• Upload resources using Amazfish (the integration is still WIP, I'll open a PR on the project when it's ready) :

https://user-images.githubusercontent.com/2261652/188277385-ed7fe88e-cbad-4758-954e-7c37c26d7350.mp4

And everything seems to work well in both case ;)

[tucuongbrt]

Hi all, I'm try to do this PR in my side, but I have problems with cmd : ./idctl fs ls or command related to fs, it returns error : "Operation is not supported"

cuong@cuongPC:~/itd$ ./itd 3:53PM INF Connected to InfiniTime version=1.10.0 3:53PM INF Initialized InfiniTime music controls 3:53PM INF Relaying calls to InfiniTime 3:53PM INF Relaying notifications to InfiniTime 3:53PM INF Started control socket path=/tmp/itd/socket

cuong@cuongPC:~/itd$ ./itctl fs ls 4:06PM FTL Error while running app error="Operation is not supported"

I use other command normally, I can get heart rate, motion, steps ..., and the Watch can receive date, time, alarm .... Just only BLE Filesystem can't work for me.

Can you help me about it ?? Thanks

[JF002]

I don't think the resource feature has already been merged into the master branch of ITD. Have you tried the 'resource-loading' branch?

[Elara6331]

"Operation is not supported" is an error coming from BlueZ (specifically org.bluez.Error.NotSupported). It usually means either your BlueZ or your Bluetooth adapter doesn't support something ITD is trying to do.

[tucuongbrt]

Hi @JF002 I have tried the 'resource-loading' branch but still having problems. Hi @Arsen6331 I have read this topic found that I had the same problem with ITCactus and detailed his problems here So I think cli for FS (e.g. "itctl fs ls") doesn't work with some PC or Laptop and I can't find useful info about this issues. I will check some ways to resolve this problems or find alternative app to test this feature. Thanks all.

[JF002]

@tucuongbrt Any update about your issue? Have you tried with amazfish which already implements this feature?

[tucuongbrt]

@JF002 I have fixed my issue. In my computer, I use BlueZ version 5.53 and it dosen't support battery UUID and FS UUID. So I have updated the latest BlueZ version (5.65) and it works normally. Currently, I use ITD and InfinitimeExplorer for uploading resources.