Feature request: Use YAML for all CloudFormation files

[rrrix]

Yes. Issue #2914.

Describe the solution you'd like Default support for using YAML in all generated CloudFormation stacks. Currently only JSON is supported today.

The js-yaml package on npmjs.org is well supported, fast, and quite robust, and would work well for this scenario.

I'm happy to submit a Pull Request if supporting YAML CloudFormation Templates is a feature the Amplify team would like to include!

Note: Changing metadata, and local environment JSON files is not part of this request.

Describe alternatives you've considered N/A - YAML is not supported by the Amplify CLI.

Additional context

YAML is:

• Smaller than JSON (in bytes)
• Easier to read
• Easier to edit
• A Superset of JSON, meaning you can parse JSON files with a YAML parser - existing JSON templates are backwards compatible
• YAML CloudFormation templates support short form Intrinsic Functions, such as !Sub, !Ref and !Join - further reducing template size, and improving readability

JSON is:

• Significantly larger on-disk for the same data compared to YAML
• Arguably Harder to read and edit than YAML
• JSON is Valid YAML (but YAML is not valid JSON)

Strictly as matter of preference, YAML is arguably more developer-friendly than, and preferred over JSON.

As an example, using the CloudFormation template generated by this schema, the YAML CloudFormation stack is 271KB versus 820KB. YAML version created using cfn-flip.

-rw-r--r--    1 rbowen  staff   820K Dec  3 01:16 cloudformation-template.json
-rw-r--r--    1 rbowen  staff   271K Dec  3 01:31 cloudformation-template.yaml

[berenddeboer]

See also #1904 I think?

[danieletieghi]

+1

[dtelaroli]

+1

[qtangs]

Please see my comment here: https://github.com/aws-amplify/amplify-cli/issues/1904#issuecomment-603699231

[half2me]

I would really appreciate this, as I’m finding my json cf templates very difficult to navigate :/

[MontoyaAndres]

Any news?

[kylekirkby]

Any updates? Is this supported?

[nnaskov]

Any updates?

[jiahao-c]

Would also love YAML CloudFormation in Functions. Maybe add as an advanced option? (select either json or YAML)

[AlexAsplund]

I worked around this by adding a pre-push hook that converts my cloudformation yaml to json.

Ugly and far from perfect but it works 🤷 I also added some support to include files from the same dir (as string) for Serverless functions with inlinecode by using !Inc:

  RandomFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.8
      Timeout: 60
      Handler: index.handler
      InlineCode: !Inc RandomHandler.py

amplify\hooks\pre-push.js

Just change the cfCustomDirs array to include the dirs that you want to transform. The directory should have a <dir name>.cf.yml file in it that will get converted to a <dir name>-cloudformation-template.json.

/**
 * @param data { { amplify: { environment: string, command: string, subCommand: string, argv: string[] } } }
 * @param error { { message: string, stack: string } }
 */
const hookHandler = async (data, error) => {

    // dirs where you want to convert yaml -> json relative to your amplify folder
    cfCustomDirs = [
        "backend/custom/randomHandlers"
    ]

    await yamlToJsonCfConverter(cfCustomDirs)

};

function getYamlSchema(basedir) {
    var yaml = require('js-yaml')

    function Model() {
        return function () { }
    }

    function CustomYamlType(name, kind) {
        const model = Model();
        return new yaml.Type('!' + name, {
            kind: kind,
            instanceOf: model,
            construct: function (data) {
                const obj = new model();
                // We hide the original data on the `_data` property for the `represent`
                // method to use when dumping the data...
                Object.defineProperty(obj, "_data", {
                    value: data
                });
                // And we make the shape of `obj` match the JSON shape of obj
                const prefix = name === 'Ref' ? '' : 'Fn::';
                switch (kind) {
                    case 'scalar':
                        obj[`${prefix}${name}`] = data;
                        break;
                    case 'sequence':
                        obj[`${prefix}${name}`] = data ? data : [];
                        break;
                    case 'mapping':
                        obj[`${prefix}${name}`] = data ? data : {};
                        break;
                }
                return obj;
            },
            represent: function (obj) {
                return obj._data;
            }
        });
    }

    var localTags = {
        "mapping": [
            "Base64",
            "ImportValue"
        ],
        "scalar": [
            "Ref",
            "Sub",
            "GetAZs",
            "GetAtt",
            "Condition",
            "ImportValue",
            "Cidr"
        ],
        "sequence": [
            "And",
            "Equals",
            "GetAtt",
            "If",
            "FindInMap",
            "Join",
            "Not",
            "Or",
            "Select",
            "Sub",
            "Split",
            "Cidr"
        ]
    }

    function Include(path) {
        var fs = require('fs')
        var _path = require('path');


        try {
            var yamlCF = fs.readFileSync(_path.join(basedir, path), 'utf8');
        } catch (err) {
            console.error(err);
        }

        this.klass = 'Include';
        this.path = path
        this.fileContents = yamlCF



    }

    var IncludeYamlType = new yaml.Type('!Inc', {
        kind: 'scalar',
        construct: function (data) {
            data = data || ''; // in case of empty node
            return new Include(data).fileContents;
        },
        instanceOf: Include
        // `represent` is omitted here. So, Space objects will be dumped as is.
        // That is regular mapping with three key-value pairs but with !space tag.
    });

    var yamlTypes = []
    Object.keys(localTags).map((kind) => localTags[kind].map((tag) => yamlTypes.push(new CustomYamlType(tag, kind))));
    yamlTypes.push(IncludeYamlType)

    return yaml.DEFAULT_SCHEMA.extend(yamlTypes)
}

const yamlToJsonCfConverter = async (customDirList) => {
    var path = require('path');
    var fs = require('fs')
    var yaml = require('js-yaml')

    customDirList.forEach((dirPath) => {
        var customName = dirPath.split("/").slice(-1)[0]
        var basedir = path.join(__dirname, '..', dirPath)
        var inputfile = path.join(basedir, `${customName}.cf.yml`);
        var outputfile = path.join(__dirname, '..', 'backend', 'custom', 'eventHandlers', `${customName}-cloudformation-template.json`);
        console.log("Converting ", inputfile, "to json")

        var obj = yaml.load(fs.readFileSync(inputfile, { encoding: 'utf-8' }), {
            schema: getYamlSchema(basedir),
            skipInvalid: true
        });



        fs.writeFileSync(outputfile, JSON.stringify(obj, null, 2).replace("2010-09-09T00:00:00.000Z", "2010-09-09"));
    })
}

const getParameters = async () => {
    const fs = require("fs");
    return JSON.parse(fs.readFileSync(0, { encoding: "utf8" }));
};

getParameters()
    .then((event) => hookHandler(event.data, event.error))
    .catch((err) => {
        console.error(err);
        process.exitCode = 1;
    });

[hackmajoris]

It will be nice to have YAML over JSON.

[endymion]

Will YAML files ever be supported? They're much better for humans. I'm trying to set up CloudWatch dashboards using JSON templates, and the strings for the dashboard bodies are completely unreadable. In YAML, I could use "!Sub |" and cleanly include the dashboard code in the template in a readable way.

AWS Amplify promised to discuss this and respond -- years ago. https://github.com/aws-amplify/amplify-cli/issues/1904#issuecomment-515547597

Is this on the roadmap at all?

[armenr]

+1 for me -- bringing the tally to 53 upvotes for the request.

Going to try to use rain with some pre/post hooks to see if we can get this done.