Various fixes related to makePromise() pointing behind EOF

[techee]

For more info, see the individual commit messages and the discussion in https://github.com/geany/geany/pull/4303

To reproduce the crash, use https://github.com/geany/geany/pull/4303/commits/79fb41516f746c588a52b1b01aa805cdf31da852 from https://github.com/geany/geany/pull/4303. Note that to reproduce this issue, it has to use CRLF line endings (might get lost when copy/pasting from the github web interface).

The issue can be reproduced using ctags --extras=+rg -o- wp-guest.php.

[techee]

Note that the MIO patches are more or less a "bonus" and not really essential. I think, regardless of what the exact problem of the offset miscalculation is, pushArea() should detect such situations and if something invalid is detected, discard the promise.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 80.95238% with 4 lines in your changes missing coverage. Please review.

Project coverage is 85.95%. Comparing base (c06d333) to head (f9969f5).

Files with missing lines	Patch %	Lines
main/read.c	75.00%	4 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4266      +/-   ##
==========================================
- Coverage   85.95%   85.95%   -0.01%     
==========================================
  Files         246      246              
  Lines       63434    63445      +11     
==========================================
+ Hits        54525    54532       +7     
- Misses       8909     8913       +4     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[b4n]

See also #4267 that adds a simpler version of the test case mentioned above. It seems that any CRLF that triggers a sub-parser will reveal the issue.

[b4n]

Apart from the remark above, looks reasonable. I'll have to test it properly with a clean head though, it's 1:20am here and I'm not fresh anymore -- but what's left of my brain like the other changes :slightly_smiling_face:

[techee]

I've added a few more checks inside pushArea() - I believe all the mio_setpos() and mio_setpos() return values should be checked because the offsets coming from parsers could be wrong.

[masatake]

I'm sorry for not getting back to you sooner. I'm looking for bulk-free time to work on these pull requests submitted from the Geany project. I had a hard time resolving https://github.com/universal-ctags/ctags/pull/4097, so I will have a hard time again. However, these are at the head of my run queue in my mind. Please wait for a while.

[techee]

@masatake No worries. I think for Geany itself, if we apply just this PR alone without any other ctags patches that were made in the meantime, we'll be fine for the upcoming Geany release.

[masatake]

I started working on this one. Before reading your change, I would like to fix the issue myself. After finding a fix, I would like to compare it with the change in this pull request. Please, give me time.

NOTE: The way to reproduce.

1. Save wp-guest.php in https://github.com/geany/geany/pull/4303/commits/79fb41516f746c588a52b1b01aa805cdf31da852 .
2. Convert the line ending with CRLF with M-x set-buffer-file-coding-system => dos on Emacs.
3. Run ctags --options=NONE --extras=+g -o - wp-guest.php .

What I got:

ctags --options=NONE --extras=+g -o - /tmp/input.php
ctags: Notice: No options will be read from files or environment
ctags: out of memory

[techee]

We did some more things related to invalid promises in Geany in https://github.com/geany/geany/pull/4330. Just let me know if I should port those changes to ctags too.