SUBMARINE-1110. Add yaml linter

[joshvictor1024]

What is this PR for?

Use yamllint to enforce coding style of YAML files.

Note that since yamllint doesn't support helm syntax, test is NOT run on files under helm-charts directly. Instead, those files are rendered with helm template before being tested. See script dev-support/style-check/lint-yaml.sh for more detail.

What type of PR is it?

Improvement

Todos

What is the Jira issue?

https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-1110

How should this be tested?

A new test script is added: dev-support/style-check/lint-yaml.sh

Screenshots (if appropriate)

Questions:

• Do the license files need updating? No
• Are there breaking changes for older versions? No
• Does this need new documentation? No

[joshvictor1024]

@KUAN-HSUN-LI thanks for the review.

1. Yes. The outputs are now properly formatted and can be seen at Actions > Check Style > View raw logs. Do ctrl+F for .github to see warnings regarding those files.
2. I think I just fixed the script lint-yaml.sh with commit 2fae698.

Right now submarine-cloud-v2/test/e2e/ is failing, but it passes in my previous commit. Should I be concerned?

Also, should I mark this PR as a draft before autoformat and the corresponding docs are added? yamllint doesn't support formatting, and right now I'm looking at yq.

[joshvictor1024]

Actually, I can't quite find a YAML formatter that supports helm syntax. I'm tempted to leave autoformat for a future issue and limit this one to linting.

[joshvictor1024]

Another issue I found is that Prettier, which is used in submarine-cloud-v2/, will always break linting for multiline flow sequences (the [] list). An example would be line 64 of submarine-cloud-v2/artifacts/submarine/submarine-mlflow.yaml

# output of prettier (4 spaces for items inside "[]", 2 spaces anywhere else)
Item:
  Item-1: [
      "item-a",
      "item-b",
    ]
  Item-2: item-a
  Item-3: item-a

# expected: consistent indentation (2 spaces)
Item:
  Item-1: [
    "item-a",
    "item-b",
  ]
  Item-2: item-a
  Item-3: item-a

# work-around: no flow styles, only block styles?
Item:
  Item-1:
    - "item-a",
    - "item-b",
  Item-2: item-a
  Item-3: item-a

We might need to give up Prettier if we want to use linting for indentation...

[KUAN-HSUN-LI]

Can prettier with the command --no-bracket-spacing do well in helm template? If it could, we can change the link by using prettier --check --no-bracket-spacing and write a autoformat script using prettier

[joshvictor1024]

The one thing that prettier (and yamllint) fails to parse is the {{- -}} syntax, so it won't work with helm-charts/submarine/charts/traefik/templates/deployment.yaml for example.

Other than that, we could switch yamllint for prettier.

[KUAN-HSUN-LI]

The one thing that prettier (and yamllint) fails to parse is the {{- -}} syntax, so it won't work with helm-charts/submarine/charts/traefik/templates/deployment.yaml for example.

Other than that, we could switch yamllint for prettier.

Got it. @joshvictor1024 Would you handle the yq or other formatter that can format helm syntax here? Or is this PR just for a YAML linter?

[joshvictor1024]

It seems that yq doesn't and won't support helm syntax either. In its issue #636 it's mentioned that support is impossible.

I'll limit this PR to just a YAML linter. I'll make changes to submarine/submarine-cloud-v2/docs/developer-guide.md accordingly.

[joshvictor1024]

Upon discussion, we concluded that helm syntax support and auto-formatting are still desired. Can we put this PR on hold for now?

[KUAN-HSUN-LI]

@joshvictor1024 OK, I will merge this PR until the helm syntax and auto-formatting are provided.