unstorage icon indicating copy to clipboard operation
unstorage copied to clipboard

feat : load driver by uri like file:/// http:/// etc

Open AntoineMontane opened this issue 11 months ago • 11 comments

solve #581

AntoineMontane avatar Jan 25 '25 17:01 AntoineMontane

This is a nice idea however as it has bundle size implications, I would keep it completely separate.

Can you please as first step, deduce diff Generate a new entry in dist that is an object mapping from URI scheme like "file", to driver object. (there should be no changes in driver src files)

(we also need to introduce a new subpath export like unstorage/full to export anything related to this from)

pi0 avatar Jan 27 '25 11:01 pi0

True, i was so displeased by the bundling issue i rewrote from scratch (i'm always amazed how good dynamic imports look from far away ;-) ) Now the driverFromUri takes a list of imported driver which means the bundling is left to the library consumer. More flexibility, more control, less issues ...

import http from "unstorage/drivers/http"
import memory from "unstorage/drivers/memory"

driverFromUri(uri, [http, memory])

We could still provide the "full bundle" if needed one way or another, or profiles: "node", "browser", "azure", etc.

BTW i saw that mongodb was not declared as peer / optional and presented issue with the full bundle in my consumer project. fs was also an issue client side ... that was the last straw for the dynamic import tentative.

AntoineMontane avatar Jan 27 '25 21:01 AntoineMontane

I think i should go back to the previous approach to export the regexp from each driver module. And keep the new one for the loader. What do you think of this ?

AntoineMontane avatar Jan 28 '25 16:01 AntoineMontane

Unstorage is primarily targeting server runtimes. Supporting relative URLs is really an edge case that possible in browser-only environments.

(and we should have one entry exporting dynamic imports inside it / no extra browser.ts)

pi0 avatar Jan 29 '25 23:01 pi0

Is it wrong to support browser runtimes ? In fact i started my development especially for this cross-environment ability. Relative url for file: or db+sqlite: is quite possible & not an edge case either

AntoineMontane avatar Jan 30 '25 07:01 AntoineMontane

js runtime support depends on drivers (if they support broeser they do) — we shouldn’t make an exception entey for browser.

re parsing, it kinda makes sense for some drivers but also incompatible with URL standards. To be honest i really have to locally play with some ideas my self at this point.

Please ping me when your work finished later when i could will try to work on it.

pi0 avatar Jan 30 '25 08:01 pi0

A RFC about the generality of url for reflexion : https://datatracker.ietf.org/doc/html/rfc1738#section-2.1

I'm almost done for the generic loader ( but the part we are discussing on). I will work on some drivers (s3, ...) that i plan to use

AntoineMontane avatar Jan 30 '25 09:01 AntoineMontane

Is it possible someway to mark node:fs external so that it doesn't represent an issue at client bundle build ?

AntoineMontane avatar Jan 30 '25 09:01 AntoineMontane

that's a bundler-specific config people should do.

Also, we need to have exactly syntax below in the generated list:

() => import("unstorage/drivers/name")

this is because when using import(unstorage/drivers/${name}) bundler cannot statically analyze and has to scan all possible patterns.

pi0 avatar Jan 30 '25 09:01 pi0

I've

  • refactored uri to url,
  • renamed protocol to scheme
  • refactored the factoryloader to dynamicaly import with raw string
  • dropped the browser bundle

It would be less deceptive to only include in the all bundle drivers that were verified or reworked to correctly handle the url protocol (+ i had to guess some schemes without completly understanding what it could mean)

AntoineMontane avatar Jan 30 '25 10:01 AntoineMontane

Hey @pi0 Do you have a good advice to handle driver import location in the all bundle ? I don't want to import from unstorage/drivers/ because it's not available until build and adds a step before testing. If i import relatively from ../drivers/, it works in test but fails in production as the path is different (../../drivers/) What was the point to output the build of drivers outside of the ./dist driectory ?

AntoineMontane avatar Jan 30 '25 12:01 AntoineMontane