Feature Request: Support for Tokenizer Interface via strtok3 Module

[Borewit]

Hi @ZJONSSON,

Are you interested in adding support for the Tokenizer interface provided by the strtok3 module and its ecosystem?

This ecosystem offers functionality very similar to the read options you currently provide. It allows for efficient, streaming-based data reading from various sources, such as:

• Local files
• Web streams API
• S3 cloud storage
• HTTP ranged requests

Other projects that successfully utilize this solution include:

• music-metadata
• file-type

Context

I started exploring node-unzipper as part of an effort to improve the detection of zipped file documents, such as Open Office XML (OOXML) files. Incorporating strtok3 could simplify and enhance such functionality.