Serialize YAML and JSON into string -> cannot unmarshal !!seq !!map into string

[1oglop1]

Hi, what's the reason I could not import a simple config like this?

ch_secret: secret
ch_list:
  - https://abc.def.com
ch_dict:
  a: abc
  b: def
ch_dict_nested:
  a:
    b: abc
    c: def

ch_dict_list:
  c: 
    - 1
    - 2
    - 3

ch_mix:
  a:
    b: abc
    c: def
  c: 
    - 1
    - 2
    - 3

chamber import service file_above.yaml results in

Error: Failed to decode input as json: yaml: unmarshal errors:
  line 3: cannot unmarshal !!seq into string
  line 6: cannot unmarshal !!map into string
  line 10: cannot unmarshal !!map into string
  line 16: cannot unmarshal !!map into string
  line 23: cannot unmarshal !!map into string
  

[alecjacobs5401]

This is likely failing because we don't support complex nested yaml files for import. Currently, a flat file structure is expected like so

ch_secret: secret
other_secret: other_secret

[1oglop1]

is there any chance you, consider supporting nested config? I'd like to dump some JSON into SSM. Otherwise, a better error message is also an acceptable solution.

[alecjacobs5401]

I'd like to understand what problem you're trying to solve for?

If you want raw JSON as a value, you could provide a JSON string as the value as a workaround

[1oglop1]

That workaround is exactly what I'm trying to avoid.

So far I've been deploing stuff with pulumi but I am trying to move some values from pulumi config to SSM and this inconvenience is the only thing preventing me. Otherwise I'd have to write my own cli for setting the parameters.

I understand it can get complicated because the nested value can be serialized into string but what should happen during the export or exec.

On Tue, Nov 21, 2023, 22:01 Alec Jacobs @.***> wrote:

I'd like to understand what problem you're trying to solve for?

If you want raw JSON as a value, you could provide a JSON string as the value as a workaround

— Reply to this email directly, view it on GitHub https://github.com/segmentio/chamber/issues/438#issuecomment-1821673517, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZCK5Q2GA6ZCDDN5GEYRJTYFUJDFAVCNFSM6AAAAAA7OFGSOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRRGY3TGNJRG4 . You are receiving this because you authored the thread.Message ID: @.***>

[alecjacobs5401]

Just trying to fully understand what you're looking for - you'd like chamber import to support nested structures and manage serializing it out into a JSON string to write to SSM, correct? Are you expecting that reading the value or exporting it does the same?

[1oglop1]

Yes, that's what I expect.

Here's a more detailed description of my explanation:

input:

ch_secret: secret

ch_list:
  - https://abc.def.com
  
ch_dict:
  a: abc
  b: def
  
ch_dict_nested:
  a:
    b: abc
    c: def

ch_dict_list:
  c: 
    - 1
    - 2
    - 3

ch_mix:
  a:
    b: abc
    c: def
  c: 
    - 1
    - 2
    - 3

chamber import should create params like a so:

param_name = value as plain string

ch_secret = "secret"
ch_list = '["https://abc.def.com"]'
ch_dict = '{"a": "abc","b": "def"}'
ch_dict_nested = '{"a": {"b": "abc","c": "def"}}'
ch_dict_list =  '{"c": [1,2,3]}'
ch_mix = '{ "a": { "b": "abc", "c": "def"}, "c": [ 1, 2, 3 ] }'

and then chamber export

• Default behaviour - export as plain string ( current behaviour )
• extra option --try-parse (or similar) to reconstruct the nested input if the output is JSON or YAML, this could potentially also support tfvar output because since there are also nested objects https://developer.hashicorp.com/terraform/language/values/variables

[alecjacobs5401]

Gotcha - I can see chamber handling attempting to JSON serialize any value from import, but I'm not sure about parsing arbitrary objects/values on export. It's an interesting use case, but outside of the core purpose of chamber.

SSM does support parameter hierarchies, which seem analogous to what you’re asking for but there’s no plan to support them in chamber at this time.

If you’d like to contribute a feature to JSON serialize the import values, we’re happy to review it but it’s not something we can commit to at this time

[1oglop1]

I should be able to contribute to this feature myself. But I was more worried if it would even fit the chamber use case and my effort wasn't wasted in yet another fork.

If this is something you'd be interested in I am happy to help.

Re: parameter hierarchies - I think the idea of hierarchies is slightly in conflict with a chamber or a different use case than mine. ATM. The chamber uses only the last portion of the path eg. params:

/dev/namespace_1/my_app/my_param
/dev/namespace_1/my_other_app/my_param

• chamber exec dev/namespace_1/my_app renders as my_param
• chamber exec dev/namespace_1/my_other_app renders as my_param
• chamber exec dev/namespace_1 render as nothing

with hierarchy, it may be unclear how to render the ENV. chamber exec --recursive dev/namespace_1 may render as

my_app_my_param
my_other_app_my_param

PS. The absence of the leading / often drives me crazy 🤷

[alecjacobs5401]

I agree with your assessment on the parameter hierarchies not fitting within the bounds of chamber as is.

We'd be open to review contributions for the import serialization, but I'm not sure export deserialization is something thats a big focus, currently.