ClojureDart icon indicating copy to clipboard operation
ClojureDart copied to clipboard

Reader is slow and synchronous read-string should be supported

Open cgrand opened this issue 2 years ago • 1 comments

The current reader is slow. One reason is that it uses subs (which performs in O(n) in Dart) thus turning linear processing in quadratic ones.

The current reader should be able to act bot in sync and asynchronous manner, so that read-string works.

IMO we can't be content with batching input until complete and then synchronously parsing it (like the JSON codec does). The main reason is the REPL. Otherwise you have to have some pre processing to detect top-level ends. The other reason is being able to read large files.

Plugging into dart:convert would allow better integration with dart libs. I believe we can represent chunked conversions inner state using something inspired by transducers https://api.dart.dev/be/175791/dart-convert/Converter/startChunkedConversion.html

cgrand avatar Jan 19 '23 15:01 cgrand

Another source of slowness is that maps of strings are used in lieu of fixed lists indexed by the character codepoint.

cgrand avatar Jan 19 '23 15:01 cgrand