CLI exits with return code of zero on parse error, even with flag set

[KevOrr]

trafficstars

When running semantic --fail-on-parse-error parse FILE, it is expected that Semantic exit with a nonzero return code if the underlying tree-sitter parser ran into parse error(s) while parsing FILE. However, with any of the flags that turn on JSON output (semantic --fail-on-parse-error parse [--json[-graph]|--[json-|proto-]symbols] FILE), it will always exit with a return code of 0, even if there were parse errors.

MWEs:

test.py

1 *** 2

test.php

<?php
asdf

Tested on semantic versions:

• 0.6.0.0 (commit d147bb8, 2019-07-02)
• 0.6.0.0 (commit 212a37c, 2019-07-26)

[robrix]

Good catch!

[tclem]

Agree this is confusing!

I think I might vote to remove --fail-on-parse-error and --fail-on-warning altogether in a future version. These switches were added to aid in some integration level testing that has since moved to haskell instead of shell scripts. The reason they don't fail hard right now is that json and those other output formats accept multiple inputs and are expected to return partial results. E.g. if you did semantic parse --json test.rb test.py

where test.rb

def foo
  puts "hi"
end

where test.py

1 *** 3

You'll get this (one tree has an error with details about what's wrong and the other is a valid parse tree):

{
  "trees": [
    {
      "tree": {
        "term": "Statements",
        "statements": [
          {
            "term": "Method",
            "methodAccessControl": "Public",
            "methodBody": {
              "children": [
                {
                  "term": "Send",
                  "sendArgs": [
                    {
                      "term": "TextElement",
                      "textElementContent": "\"hi\"",
                      "sourceRange": [
                        15,
                        19
                      ],
                      "sourceSpan": {
                        "start": [
                          2,
                          8
                        ],
                        "end": [
                          2,
                          12
                        ]
                      }
                    }
                  ],
                  "sendBlock": null,
                  "sendReceiver": null,
                  "sendSelector": {
                    "term": "Identifier",
                    "name": "puts",
                    "sourceRange": [
                      10,
                      14
                    ],
                    "sourceSpan": {
                      "start": [
                        2,
                        3
                      ],
                      "end": [
                        2,
                        7
                      ]
                    }
                  },
                  "sourceRange": [
                    10,
                    19
                  ],
                  "sourceSpan": {
                    "start": [
                      2,
                      3
                    ],
                    "end": [
                      2,
                      12
                    ]
                  }
                }
              ],
              "sourceRange": [
                10,
                19
              ],
              "sourceSpan": {
                "start": [
                  2,
                  3
                ],
                "end": [
                  2,
                  12
                ]
              }
            },
            "methodContext": [],
            "methodName": {
              "term": "Identifier",
              "name": "foo",
              "sourceRange": [
                4,
                7
              ],
              "sourceSpan": {
                "start": [
                  1,
                  5
                ],
                "end": [
                  1,
                  8
                ]
              }
            },
            "methodParameters": [],
            "methodReceiver": {
              "term": "Empty",
              "sourceRange": [
                0,
                0
              ],
              "sourceSpan": {
                "start": [
                  1,
                  1
                ],
                "end": [
                  1,
                  1
                ]
              }
            },
            "sourceRange": [
              0,
              23
            ],
            "sourceSpan": {
              "start": [
                1,
                1
              ],
              "end": [
                3,
                4
              ]
            }
          }
        ],
        "sourceRange": [
          0,
          24
        ],
        "sourceSpan": {
          "start": [
            1,
            1
          ],
          "end": [
            4,
            1
          ]
        }
      },
      "path": "test.rb",
      "language": "Ruby"
    },
    {
      "tree": {
        "term": "Error",
        "errorActual": "ParseError",
        "errorCallStack": [
          {
            "startLine": 126,
            "module": "Language.Python.Assignment",
            "package": "semantic-0.7.1.0-inplace",
            "site": "parseError",
            "file": "src/Language/Python/Assignment.hs",
            "startColumn": 119,
            "endColumn": 129
          }
        ],
        "errorChildren": [],
        "errorExpected": [],
        "sourceRange": [
          0,
          4
        ],
        "sourceSpan": {
          "start": [
            1,
            1
          ],
          "end": [
            1,
            5
          ]
        }
      },
      "path": "test.py",
      "language": "Python"
    }
  ]
}