(nucleus): component config file access

[qoomon]

trafficstars

Feature Description

It would be awesome if there would be away to access the whole or even parts of a component config as a json file.

Use Case Why do you need this feature?

I need to have a quite big config (> 1048576 characters as a pretty json) Related SFC issue

Proposed Solution My current approach was to read the recent component config from /greengrass/v2/config/effectiveConfig.yaml (see example below), however it turns out that these component configs are not reflecting the most recent config of a component :-(

        Lifecycle:
            Install:
                RequiresPrivilege: true # due to access /greengrass/v2/config/effectiveConfig.yaml
                Script: |
                    chmod +x '{artifacts:decompressedPath}/greengrass-sfc-playground/bin/yq_linux_arm'
                    alias yq='{artifacts:decompressedPath}/greengrass-sfc-playground/bin/yq_linux_arm'
                   
                    COMPONENT_NAME="$(basename $PWD)"
                
                    cat /greengrass/v2/config/effectiveConfig.yaml \
                        | yq ".services.\"${COMPONENT_NAME}\".configuration.sfc-config" \
                            --output-format=json \
                                > "sfc-config.json"
            Run:
                RequiresPrivilege: true # due to access /greengrass/v2/config/effectiveConfig.yaml
                Script: |
                    export SFC_DEPLOYMENT_DIR='{artifacts:decompressedPath}/greengrass-sfc-playground/sfc-modules'
                    alias sfc-main='{artifacts:decompressedPath}/greengrass-sfc-playground/sfc-main/bin/sfc-main'

                    sfc-main -config sfc-config.json

Other I'm using AWS shopfloor-connectivity as a Greengrass component. Therefore I'd like to configure SFC by a Greengrass config field. Related SFC issue

I'm facing two issues right now.

1. The config needs to be converted in to a json file to use it for SFC e.g. sfc-main -config sfc-config.json

• Currently the common workaround is to set an environment variable to the value of the sfc-config field from greengrass config and then write this environment variable to a file. e.g.

       Lifecycle:
          Install:
              Setenv:
                  SFC_CONFIG: '{configuration:/sfc-config}'
              Script: |
                  echo "$SFC_CONFIG" > sfc-config.json
  

1. This works until the sfc-config gets too big, then nucleus will throw an Exception Argument list too long at java.lang.ProcessBuilder.start because of system limit getconf ARG_MAX

• [ ] :wave: I may be able to implement this feature request
• [ ] :warning: This feature might incur a breaking change

[MikeDombo]

Hi,

Using environment variables is the correct way to do this. Trying to pass it over the command line instead will have a lot of issues with length and shell escaping.

Having a unique feature to write config as a file does not seem necessary given the ability to use environment variables which you may then choose to save to a file.

In greengrass, files are artifacts, so the other way to do this is to have any static content as artifacts and not component configuration.

[MikeDombo]

You may also use Greengrass IPC to read configuration. https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-component-configuration.html#ipc-operation-getconfiguration

This would be my recommendation. Write a python (or language of your choice) component which pulls the config using IPC and then writes to a file and invokes the shop floor functionality.

[qoomon]

Well unfortunately using environment variables does not work for big configs :-(

I make it work now with following setup.

        Lifecycle:
            Install:
                RequiresPrivilege: true # to access /greengrass/v2/bin/greengrass-cli
                Script: |
                    apt install jq -y
                    
                    COMPONENT_NAME="$(basename $PWD)"
                    /greengrass/v2/bin/greengrass-cli component details --name "${COMPONENT_NAME}" \
                        | grep "^ *Configuration: *" | sed 's/ *Configuration: *//' \
                        | jq -r '."sfc-config"' > sfc-config.json
            Run:
                Setenv:
                    JAVA_HOME: /usr/lib/jvm/java-8-openjdk-armhf
                RequiresPrivilege: true # /greengrass/v2/config/effectiveConfig.yaml (Permission denied)
                Script: |
                    export SFC_DEPLOYMENT_DIR='{artifacts:decompressedPath}/greengrass-sfc-playground/sfc-modules'
                    alias sfc-main='{artifacts:decompressedPath}/greengrass-sfc-playground/sfc-main/bin/sfc-main'
                    sfc-main -config sfc-config.json