file-type icon indicating copy to clipboard operation
file-type copied to clipboard

Consider alternate (browser friendly) buffer

Open rolanday opened this issue 2 years ago • 15 comments

Hello! :-) I'd love to be able to use this module in my SvelteKit app, but node:buffer dep is not something I've been able to workaround via Vite polyfill (same issue for other frameworks like Nextjs as well).

Have or can you consider alt buffer impl that is browser friendly? Or, any tips on configuring for use in SvelteKit, Vue, Nextjs, etc?

Thanks!

rolanday avatar Feb 06 '23 08:02 rolanday

Ideally, all code should be using Uint8Array instead of Buffer, but there's just too much Node.js code that depends on Buffer.

If anyone wants to work on this, the first task would be to convert the dependencies of file-type to use Uint8Array instead of Buffer.

sindresorhus avatar Feb 11 '23 18:02 sindresorhus

That work has started to some extend Borewit/token-types#246, but yet there is much more work to do.

Borewit avatar Feb 13 '23 15:02 Borewit

+1 this library really needs a browser version without node and webpack deps something like fromFile(file: File) because file.mimeType doesnt seems to work properly all times

volarname avatar Mar 01 '23 12:03 volarname

+1 this library really needs a browser version without node and webpack deps something like fromFile(file: File) because file.mimeType doesnt seems to work properly all times

That function already exist @volarname : fileTypeFromBlob(blob). Where you can pass you file as blob, since File inherits from Blob.

Borewit avatar Mar 06 '23 14:03 Borewit

+1 this library really needs a browser version without node and webpack deps something like fromFile(file: File) because file.mimeType doesnt seems to work properly all times

That function already exist @volarname : fileTypeFromBlob(blob). Where you can pass you file as blob, since File inherits from Blob.

Yes, but it doesn't work in modern frameworks like SvelteKit as example. It uses vite under the hood and shows the warnings:

Module "node:buffer" has been externalized for browser compatibility. Cannot access "node:buffer.Buffer" in client code.

akkie avatar Mar 09 '23 16:03 akkie

This module has been primary written for Node.js. It does depend on Buffer. There is browser friendly buffer module available, but you depend on your framework how easy that is to fit it in. There is not much more we can do, as these core dependencies are not shared between browser and Node.js.

Borewit avatar Mar 13 '23 08:03 Borewit

https://sindresorhus.com/blog/goodbye-nodejs-buffer

sindresorhus avatar Oct 25 '23 09:10 sindresorhus

https://sindresorhus.com/blog/goodbye-nodejs-buffer

Interesting!, very nice guidance. I will give it try (starting with the dependent modules first).

Borewit avatar Oct 25 '23 10:10 Borewit

It is not only node:buffer file-type depends on, also safe-buffer

graph TD;
    FT-->S
    FT(file-type)-->RWNS(readable-web-to-node-stream)
    S(strtok3)-->P(peek-readable)
    S(strtok3)-->TO("@tokenizer/token")
    TY(token-types)-->TO
    TY-->IE("ieee754")
    FT-->TY
    TY-->NB(node:buffer)
    RWNS-->RS(readable-stream)
    RS-->SD(string_decoder)
    SD-->SB(safe-buffer)
    RS-->UD(util-deprecate)
    RS-->I(inherits)
    style NB fill:#F88,stroke:#A44
    style SB fill:#F88,stroke:#A44
    style SD fill:#CCC,stroke:#888
    style IE fill:#CCC,stroke:#888
    style UD fill:#CCC,stroke:#888
    style I fill:#CCC,stroke:#888

Borewit avatar Nov 01 '23 15:11 Borewit

I still don't see a good alternative for the string encoding node:buffer supports: ascii, utf8, utf-8, utf16le, ucs2, ucs-2, base64, base64url, latin1, binary, hex;

e.g.:

Buffer.from(uint8Array).toString(encoding);

Borewit avatar Nov 01 '23 15:11 Borewit

I still don't see a good alternative for the string encoding node:buffer supports: ascii, utf8, utf-8, utf16le, ucs2, ucs-2, base64, base64url, latin1, binary, hex;

I think this library can help? https://github.com/sindresorhus/uint8array-extras

I can see there are methods to hex, string, base64. We have to construct every encoding method from sketch though.

songkeys avatar Nov 04 '23 10:11 songkeys

i've got a working browser-consumable fork at https://github.com/sgtpooki/file-type, but stream and file support has been disabled for now (since I don't need it for my use-case). All tests for fileTypeFromBuffer are passing.

SgtPooki avatar Feb 05 '24 17:02 SgtPooki

Hi 👋 Facing similar issues after upgrading from a very old fork of this lib. I feel like I've read a 101 different issues/comments/responses & ultimately keeping ending up in the same situation.

So just to be crystal clear - this lib can't be used without polyfiling node:buffer right? (in my case with something like this) There's no "out-of-the-box" way to use this on the client-side?

andrewbrennanfr avatar Feb 21 '24 10:02 andrewbrennanfr

Encountering this while playing with Cloudflare workers, would be nice for fileTypeFromStream to also handle ReadableStream, when I try to pass one (from File.stream()) I get Error: Expected an instance of stream.Readable.

mgcrea avatar Apr 22 '24 08:04 mgcrea

You can use this dependency vite-plugin-node-polyfills to solve the loading problem of node:buffer

Install the package as a dev dependency.

# npm
npm install --save-dev vite-plugin-node-polyfills

# pnpm
pnpm install --save-dev vite-plugin-node-polyfills

# yarn
yarn add --dev vite-plugin-node-polyfills

Add the plugin to your vite.config.ts file.

import { defineConfig } from 'vite'
import { nodePolyfills } from 'vite-plugin-node-polyfills'

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [
    nodePolyfills(),
  ],
})

cbingb666 avatar May 08 '24 18:05 cbingb666