Not running with Deno

[laem]

Hi, just to point out that node-gtfs does not run with Deno 1.41.2.

It's due at least to this problem :

error: Uncaught (in promise) TypeError: tty.WriteStream.prototype.hasColors is not a function
    at file:///home/sapin/gtfs/node_modules/.deno/[email protected]/node_modules/yoctocolors/index.js:4:45

I'm not saying it has to run with Deno, but this issue will serve for future people trying it like me.

[laem]

I wonder if the problem is fixed by yoctocolors v2.0.0 published last week.

https://github.com/sindresorhus/yoctocolors

[laem]

Then various other problems with better-sqlite3

 Could not locate the bindings file. Tried:
   → /home/xxx/node-gtfs/node_modules/bindings/build/better_sqlite3.node

Downloading the .node results in a new error :

undefined symbol: node_module_register

[brendannee]

Thanks for testing this on Deno.

Working around yoctocolors should be straightforward - but we'll need better-sqlite3 to work (or find a different sqlite3 library). I see https://github.com/denodrivers/sqlite3 as an option - but I think we'd need to find a library that works with both node and deno.

[anbraten]

https://github.com/tursodatabase/libsql or https://db0.unjs.io/ might be an option.

[alessandroamella]

It would also be nice if it worked with Bun, but it gives the same error of Could not locate the bindings file. Tried: → .../node_modules/gtfs/node_modules/better-sqlite3/build/better_sqlite3.node

[laem]

Also, side note, related to those more modern node engines : I'm using both Motis and node-GTFS. Motis has a GTFS routing engine included, Nigiri. Nigiri takes a few seconds to load into memory a GTFS dataset that node-GTFS takes a few minutes to load into its DB.

I wonder why such a difference. That's also why I wanted to try bun and deno. Performance differences can come from multiple factors : features that Nigiri doesn't have ; quicker getters after a longer import ; big bottelneck in the import functions ; node botttelneck vs Nigiri's C++ ; etc.

I'm not expert enough to find what makes the difference, but it could be an inspiration to more knowledgeable people.

[brendannee]

Thanks for these details - I'll check out Nigiri to see if there are ideas on how to speed up import.

[laem]

Deno 1.45 has node compatibility improvements that are told to target sqlite3.

https://deno.com/blog/v1.45#nodejs-compatibility-improvements

[brendannee]

Thanks for letting me know - I'll check into that and try it out.

[laem]

Hi, just an update to say that I almost managed to get it run on Deno's latest. Sorry, nothing clean to propose as a PR, but my code is here https://github.com/laem/gtfs/tree/deno

I had to clone node-gtfs to change the references to libsql, a drop-in replacement do better-sqlite3.

It started to import the GTFS but failed on one record of one of the CSV. I didn't go further though, just a partial update about the state of node-gtfs on deno :)

[brendannee]

Thanks for letting me know about that.

I created a branch that uses libsql to try it out:

https://github.com/BlinkTagInc/node-gtfs/tree/feature%2Flibsql

I had to make one adjustment: the "name" field of the database object wasn't being populated, so I added this line: https://github.com/BlinkTagInc/node-gtfs/blob/feature/libsql/src/lib/db.ts#L10 and all tests pass.

Can you check this out and see what you think and if it works on Deno? Any other ideas or suggestions?