support yaml

[willkg]

JSON is a pain in the ass to edit by hand especially summary and description fields. Plus it's picky about quotes and commas and things.

Maybe YAML is a better option?

[codersquid]

I am tempted to agree with you.

[codersquid]

:beers: if we switch to yaml. I am more opinionated about preferring it these days.

[willkg]

Would we want to switch for yaml for all the things? Including the steve.ini file?

[willkg]

Also, @codersquid and I talked on IRC about this. Instead of switching wholesale to yaml, we want to add support for yaml and additionally change the default to do things in yaml rather than json. We need to support json so that we don't have to go redo all the things we've already done.

[AvdN]

YAML is supposed to be a superset of JSON. So start with the current routines that read in JSON and replace those, making sure the resulting data structures are compatible through testing. There are some caveats with 4 byte unicode encoded files (but you are unlikely to have those), with YAML recognising some more types of data ( "No" for False, and "00:10:02" for 602 ). Also PyYAML has some problems with properly recognising corner cases of floats (I never encountered these myself, but found an item on stackoverflow by someone that ran into them), I fixed those in my ruamel.YAML (I started that because PyYAML is stuck on YAML 1.1 and not 1.2 and development seems stalled).

So start with replacing normal JSON usage, load them with the ``jaml.load(fp, Loader=yaml.SafeLoader) option that should be relatively painless. It is relatively easy to support both .ini and YAML files for reading. I do so in my AppConfig but that is based on using ConfigObj and configparser/ConfigParser in the standard lib is a little different.

You can continue to have the application write JSON files, you should be able to read those back in. Direct advantage of changing to YAML is that:

{
  "a": 1,
  "b": 2,
}

becomes valid input. JSON chokes on the trailing , and I have had that multiple times editing out some stuff, including the last dict item from JSON files. And that you can use # end of line comments to clarify/document your files. At a later stage you could consider making the writing out YAML an configuration/commandline option. If those files are for human perusal, you could consider basing them on the RoundTripParser in ruamel.yaml, which allows you to include explanatory comments in the output.

[AvdN]

I created some routines to sync a directory containing JSON files with a directory containing YAML (with the same basename - extension). You provide a last-synced timestamp to prevent syncing when a file on both sides has been updated.

For that I would like to:

• convert steve.ini to steve.yaml so I can write the last-synced timestamp somewhere without losing comment information
• add a command to steve-cmd that syncs the json subdir with a new yaml subdir

After that the following workflow would be possble

• create project
• fetch json
• sync json and yaml
• repeat
  • update .json or .yaml files (but not the same file on both sides)
  • sync json and yaml
• upload json files to richard.

Once that works I could have the webeditor and all the other commands work with the YAML files, further reducing JSON to the format that the YAML is converted to just before uploading.