Compilation server. Wrong error position info. Regression

[djaonline]

Stuff OS: Windows, Linux Haxe: 4.2.5, dev build

Instruction of reproduction

1. Clone repo https://github.com/djaonline/haxe_compile_server_issue
2. Open project in vscode and run compiler(ctrl + shift + B)
3. The compilation server reports correct error position
4. Open src/entry/BaseApp.hx file and add space in the place pointed below:
5. Run compiler and see incorrect error position info:

It works correctly with haxe 4.0.5.

[Simn]

Thank you for the detailed instructions! I can reproduce this on latest Haxe, so this should be quite interesting to figure out.

[Simn]

One thing that's immediately strange to me is that the initial state of the application is already erroneous, which means that the compiler won't cache anything in the first place. I wonder if there's some macro interaction here which causes this problem. We'll likely have to reduce the example further, ideally eliminating these haxelibs in particular.

[Simn]

I added a debug print for the raw position and it consistently stays at src/entry/BaseApp.hx 555 573, which is correct. This must mean that Lexer.get_error_pos changes behavior, which must mean that the auxiliary position information changes for some reason.

This reminds me of an old issue where Context.parseInlineString would cause the lexer information to be modified, but that particular issue was fixed long ago. I'll try to track down where the modification comes from.

[Simn]

I can confirm that the position information gets really messed up upon saving.

Before:

lfile: src/entry/BaseApp.hx
lline: 35
lmaxline: 35
llines: [(577,35);(574,34);(553,33);(550,32);(533,31);(532,30);(529,29);(488,28);(485,27);(468,26);(467,25);(464,24);(448,23);(445,22);(428,21);(427,20);(424,19);(421,18);(398,17);(396,16);(382,15);(381,14);(348,13);(315,12);(282,11);(249,10);(216,9);(183,8);(150,7);(117,6);(84,5);(50,4);(16,3);(15,2);(0,1)]
lalines: [(0,1);(15,2);(16,3);(50,4);(84,5);(117,6);(150,7);(183,8);(216,9);(249,10);(282,11);(315,12);(348,13);(381,14);(382,15);(396,16);(398,17);(421,18);(424,19);(427,20);(428,21);(445,22);(448,23);(464,24);(467,25);(468,26);(485,27);(488,28);(529,29);(532,30);(533,31);(550,32);(553,33);(574,34);(577,35)]
llast: 555
llastindex: 32

After:

lfile: src/entry/BaseApp.hx
lline: 49
lmaxline: 1
llines: [(1453,49);(1418,48);(1390,47);(1364,46);(1330,45);(1298,44);(1272,43);(1227,42);(1332,41);(1311,40);(1281,39);(1227,38);(333,37);(321,36);(577,35);(574,34);(553,33);(550,32);(533,31);(532,30);(529,29);(488,28);(485,27);(468,26);(467,25);(464,24);(448,23);(445,22);(428,21);(427,20);(424,19);(421,18);(398,17);(396,16);(382,15);(381,14);(348,13);(315,12);(282,11);(249,10);(216,9);(183,8);(150,7);(117,6);(84,5);(50,4);(16,3);(15,2);(0,1)]
lalines: [(0,1)]
llast: 4611686018427387903
llastindex: 0

That llast value suggests an int64 underflow. Still don't know what causes this though.

Edit: Never mind, that ones gets initialized to max_int.

[Simn]

After only 4 hours I have finally found the problem. String parsing from Context.parse and such was able to mutate the state of other files, most importantly upon processing newline. While the parser made an effort to save and restore the file information structures themselves, it would not do so for the actual data.

In particular, Context.parse in non-inline mode would not call Lexer.init, which means that it happily worked on whatever was currently stored in Lexer.cur. Any newline in the parsed string would then modify that data, leading to all sorts of funky behavior.

I have fixed this by copying the actual data before parsing strings, and restoring it afterwards. This is done in both inline- and non-inline modes, which I think is correct.

Adding a test for this seems very difficult because it depends on what file is currently stored in Lexer.cur. Perhaps we could just add this entire repository as a party test. I'll leave that to @kLabz though!

[djaonline]

@Simn Thank you so much for solving this really hard issue🙏. @kLabz We are looking forward to see it in the nearest haxe release to try new haxe features in our project:)