Return robocop status code value when calling it via python

[ferroiz]

If you run Robocop by using python code, this is what it's returned: self.reports["json_report"].issues

Imho it could be useful to report also the return_status (which is returned if you run it with cli) in order to use it in checks/statistics.

[bhirsz]

Yes, there is this weird trick done after running the tests:

  if self.from_cli:
      sys.exit(self.reports["return_status"].return_status)
  else:
      return self.reports["json_report"].issues

Which could lead to unwanted behaviour (for example user configure the tresholds in pyproject.toml, then run robocop both from cli & and from plugin in editor and gets different statuses).

Do you propose just doing:

      return self.reports["return_status"].return_status

Then in run_robocop:

return_status = linter.run()
sys.exit(return_status)

(it would be also less dependent on "from_cli" status)

[ferroiz]

Yes exactly, that's what I thought. Or at least to add the return_status value to the returned array with the all the issues dictionaries.

So the user won't have to iterate on the number of reported issues to calculate the return_status that is already calculated by robocop itself.

[bhirsz]

I took a look into those parts of the code and there is a bit of a mess here :p Hovewer I cannot simply change what's returned from this method. It would be backward incompatible change for those who are using API already (and they're expecting the run method to only return issues). I now see the design should be different - for each run we should create separate "run result" that contains issues, reports, return status. Currently wer'e keeping those results inside the Robocop class (together with static data like config) and we're clearing it between runs.

I think I should split it and try to improve it - and release it as another major Robocop version. Thanks for this it would be also easier for people using API to get results.

In meantime it's possible to just get return status directly - after calling run() you can get return status from robocop_instance.reports["return_status"].return_status. Does that work for you as temporary workaround? I will work on better solution aimed for next major release.

[ferroiz]

Sure, that totally works fine as a workaround :)