docopt-ng icon indicating copy to clipboard operation
docopt-ng copied to clipboard

Assert error message content in tests

Open h4l opened this issue 2 years ago • 3 comments

I'm going to have a crack at making the error messages a little more user-friendly, to resolve #5. Especially the Warning: found unmatched (duplicate?) arguments [Argument(None, '.\\10. PowerShell.ps1')] that occurs in response to an end user passing incorrect or too many arguments when running a program using docopt-ng. I need that to be resolved before I can use docopt-ng in place of docopt really.

Before making changes there, I'd like to ensure the current messages are tested, so I can be sure I don't change messages unexpectedly.

This PR mostly adds exception message assertions to all the places where tests trigger an exception or SystemExit with usage.

The first 3 commits are unrelated to these messages assertions — small improvements to the dev experience using this repo. They could go into a separate PR if you prefer, but I've already opened a few, so I didn't want to swamp with too many!

h4l avatar Aug 28 '22 09:08 h4l

h4l force-pushed the test-error-messages branch from 1f166be to 93ea3f0 35 seconds ago

Removed import re from one of the commits.

h4l avatar Aug 28 '22 09:08 h4l

Also, just to flag that one of the commits (ecc12c2) makes a change to the GitHub Actions config.

h4l avatar Aug 28 '22 14:08 h4l

h4l force-pushed the test-error-messages branch from 93ea3f0 to 9c0f189 1 minute ago

Fixed tests failing on Python 3.7 due to annotation with dict[...]. Also changed "use a stable sys.argv" to use list instead of a tuple for the patched argv — docopt expects it to be a list.

And also just added:

  • fix: detect multi-word options sections in usage
  • fix: handle 3+ word Options: section names

We'd lost a line of coverage after "test: assert the messages of raised errors" as no test was triggering the "options (case-insensitive) was found in usage" error. I added a test to trigger it, and in doing so realised the code checking for the error condition was not handling options sections with extra qualifying words, like "Advanced Options:". And further to that, a separate error occurred with 3+ word Options sections.

h4l avatar Sep 01 '22 18:09 h4l

This looks great @h4l, but let's wait for #36 to merge so we can then rebase on top of that?

NickCrews avatar Sep 07 '22 23:09 NickCrews

Codecov Report

Merging #37 (c817b29) into master (850293b) will increase coverage by 0.20%. The diff coverage is 100.00%.

:exclamation: Current head c817b29 differs from pull request most recent head df66068. Consider uploading reports for the commit df66068 to get more accurate results

@@             Coverage Diff             @@
##           master       #37      +/-   ##
===========================================
+ Coverage   99.79%   100.00%   +0.20%     
===========================================
  Files           1         1              
  Lines         487       492       +5     
===========================================
+ Hits          486       492       +6     
+ Misses          1         0       -1     
Impacted Files Coverage Δ
docopt/__init__.py 100.00% <100.00%> (+0.20%) :arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Sep 07 '22 23:09 codecov[bot]

Yep, I can rebase this up once 36 is sorted.

h4l avatar Sep 08 '22 10:09 h4l

Eh I just tried to fix the merge conflicts in the Web UI and it doesn't look that good, do you think you could actually do a rebase locally and fix the conflicts in their respective commits?

NickCrews avatar Sep 08 '22 21:09 NickCrews

@NickCrews No problem, there were quite a few merge conflicts in there! I've rebased it on master.

Two of the changes/fixes from here were indirectly fixed by #36 so they're gone now:

  • c817b29dc77fa15d64e2126e9eadb88c28bb42f9
  • 57dd340d17571f45c8ef8e5c501f8acf562d796d

Should be good to go!

h4l avatar Sep 09 '22 09:09 h4l

Excellent work @h4l ! I'm going to add the isort-ing now. Didn't want to add before and cause more merge conflicts.

NickCrews avatar Sep 09 '22 17:09 NickCrews

Very sensible, thanks @NickCrews! I guess when you do the isort pr it's probably easier to just take the change adding the isort config and re-run it to update the files rather than using the commit I made.

h4l avatar Sep 09 '22 18:09 h4l