AST transfer WIP

[overlookmotel]

Very rough POC for #2409. Nowhere near ready to merge.

Mostly works. JS AST for checker.ts via "raw" transfer matches AST produced via current JSON API exactly. ~~Some errors in AST for pdf.mjs and antd.js due to some of reasons listed below.~~

Edit 8/3/24: Output for pdf.mjs and antd.js is now correct.

Current deficiencies:

• ~~Incorrect output for files containing any non-ASCII characters.~~
• ~~Does not handle strings/identifiers/regexps containing \ escapes.~~
• Statement and a few other types increase in size by 8 bytes (likely the cause of the performance regression seen in benchmarks).
• ~~Requires size of AST to be known in advance.~~
• Hacky method of creating allocations.
• ~~Memory handling may be unsound!~~

[overlookmotel]

• #2457 👈
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @overlookmotel and the rest of your teammates on Graphite

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #2457 will improve performances by 4.52%

_{Comparing 02-20-AST_transfer_WIP (79d2808) with main (88ded0c)}

Summary

⚡ 2 improvements ✅ 28 untouched benchmarks

🆕 5 new benchmarks

Benchmarks breakdown

	Benchmark	main	02-20-AST_transfer_WIP	Change
🆕	parser_napi[RadixUIAdoptionSection.jsx]	N/A	7.5 µs	N/A
🆕	parser_napi[antd.js]	N/A	24.3 ms	N/A
🆕	parser_napi[cal.com.tsx]	N/A	4.1 ms	N/A
🆕	parser_napi[checker.ts]	N/A	8.7 ms	N/A
🆕	parser_napi[pdf.mjs]	N/A	2.6 ms	N/A
⚡	semantic[antd.js]	218.3 ms	208.8 ms	+4.52%
⚡	semantic[pdf.mjs]	35.9 ms	34.6 ms	+3.7%

[andymac4182]

Does this mean we could use oxc_parser for eslint as well as prettier?

[overlookmotel]

Maybe... Just to say this is very early stage work, the concept is not proven, and there is a long way to go yet. Some significant things would need to change within OXC to convert this from an experiment to a solid, dependable reality.

Personally, I do believe there is potential in this approach, but I think it's premature at this stage to talk a great deal about applications.

Sorry if that sounds negative. I'm just trying to give a realistic answer.

[andymac4182]

That's okay 😀 that's a perfectly reasonably answer

[overlookmotel]

I don't believe the Codspeed benchmarks are accurate. Running benchmarks locally, I'm seeing more like x3 - x5 speed-up. Will investigate.

[overlookmotel]

Yes, benchmarks on CodSpeed are completely wrong. Have raised an issue https://github.com/CodSpeedHQ/action/issues/96

[overlookmotel]

Benchmarks are now accurate (though with a lot of variance in them).