Import files as direct bytearray

[tristan-f-r]

Description

I'm making a tool that allows reading NBT files, and NBT files are directly binary data. In order to provide a clear demonstration, I'm grabbing all files from my local /resources folder with a glob import, but whenever I use ?raw or { as: "raw" }, it corrupts the data when it turns it into a string.

This also seems to be a used use case for other situations (stackoverflow) by other developers as well.

Do note that (if this enhancement is accepted), I am fully willing to implement this myself.

Suggested solution

Provide a new query type, e.g. ?buffer that loads the data as a Uint8Array.

Alternative

I could make a new Vite plugin, but this seems useful enough to not mandate the use of a plugin.

Validations

• [X] Follow our Code of Conduct
• [X] Read the Contributing Guidelines.
• [X] Read the docs.
• [X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[bluwy]

Seems like a nice addition, but we probably can't add this until the next major since it could collide with other Vite plugins that uses ?buffer for something else. A vite plugin would be good to implement this at the meantime though.

[tristan-f-r]

Alright, it seems that the stackoverflow answer offers a pretty good substitute and works for both glob imports { as: "raw-hex" } and regular/dynamic imports in the meantime.

[patak-dev]

@bluwy given that the plugin is straight forward, maybe we should close this one? I also think it is an interesting feature but for completness we may later need to add other types too? ArrayBuffer, SharedArrayBuffer, etc.

[bluwy]

I'm not sure about your point with ArrayBuffer and SharedArrayBuffer since we're reading the file as a buffer itself. But I could see the scope grow with needing Int32Array etc instead, which maybe we can support with ?buffer=Int32Array.

I think it's a nice addition for Vite since it's small, generic, and (kinda) useful when you need it, unlike other feature requests that are specific to their preference and/or non-standard.

[braebo]

Alright, it seems that the stackoverflow answer offers a pretty good substitute and works for both glob imports { as: "raw-hex" } and regular/dynamic imports in the meantime.

Any idea why this plugin works during build, but not during dev? In dev, my import.default is just a string (the path to the file) so I'm a bit stuck on this one 😅

[ryuujo1573]

@LeoDog896 Hi there 👋 I was wondering if import raw from 'some.wasm?raw' would help!

[AlbertMarashi]

God damn this is annoying. There needs to be an arraybuffer import

I used this plugin to get it to work

import { sveltekit } from "@sveltejs/kit/vite"
import { defineConfig } from "vite"
import arraybuffer from "vite-plugin-arraybuffer"

export default defineConfig({
    plugins: [
        sveltekit(),
        // used for importing fonts in satori /api/og/+server.ts
        arraybuffer()
    ],
})

and declared a fonts.d.ts in the folder where I was importing some fonts

declare module "*.ttf?arraybuffer" {
    const content: ArrayBuffer
    export default content
}

[Serator]

Seems like a nice addition, but we probably can't add this until the next major since it could collide with other Vite plugins that uses ?buffer for something else.

@bluwy Any chance of supporting ?buffer in Vite 6?

[pepefeliblu]

I am not sure if ?buffer will be added, but a neat workaround that might work for some is to use fetch in order to obtain the arrayBuffer:

import someResource from '/dir/resource.ext?url';
const arrayBuffer = await fetch(someResource).then(b => b.arrayBuffer());

The only downside with this method is that it relies on a Promise, which in some use cases might be inconvenient. Anyways I hope it helps someone!

[sapphi-red]

Since we now have ?url&inline, you can convert it into ArrayBuffers with a function like this:

function dataUrlToArrayBuffer(dataUrl) {
  const base64StartIndex = dataUrl.indexOf('base64,');
  if (base64StartIndex >= 0) {
    const base64 = dataUrl.slice(base64StartIndex + 'base64,'.length);
    const binaryString = atob(base64);
    const bytes = new Uint8Array(binaryString.length);
    for (var i = 0; i < binaryString.length; i++) {
      bytes[i] = binaryString.charCodeAt(i);
    }
    return bytes.buffer;
  }
  const str = decodeURIComponent(dataUrl.slice(dataUrl.indexOf(',') + 1));
  const enc = new TextEncoder();
  const bytes = enc.encode(str);
  return bytes.buffer;
}

https://stackblitz.com/edit/vitejs-vite-jgmtwvec?file=src%2Fmain.js&terminal=dev

[sapphi-red]

For those who are interested in having this a built-in support, it would be helpful if you described your expected usage of it. We are not sure how the API should be designed. For example,

• whether you need an ArrayBuffer or a TypedArray
• whether you can convert ArrayBuffer to what you want on your own
• why fetching via URL is not an option
• if you are using it on the server side, why fs.readFile is not an option

[Its-Just-Nans]

Hi

My usage was about inlining a svg in astro in a static website. The easiest way would be to have a simple (single) line to import it as binary I've now others solutions

Note that I think your function is almost correct but it should use decodeURIComponent(). If we use decodeURI(), hashtag are not replaced and are still %23

[carlos-dubon]

I want to import an image and then convert it to an ArrayBuffer to get a blurhash from it in my Astro app.

---
import arrayBuffer from "@assets/img/img.jpg?arraybuffer";
import { encode } from "blurhash";
import { getPixels } from "@unpic/pixels";

const jpgData = await getPixels(
  arrayBuffer
);

const data = Uint8ClampedArray.from(jpgData.data);
const blurhash = encode(data, jpgData.width, jpgData.height, 4, 4);
const placeholder = blurhashToCssGradientString(blurhash);
---

[tristan-f-r]

For those who are interested in having this a built-in support, it would be helpful if you described your expected usage of it. We are not sure how the API should be designed. For example,

• whether you need an ArrayBuffer or a TypedArray

• whether you can convert ArrayBuffer to what you want on your own

• why fetching via URL is not an option

• if you are using it on the server side, why fs.readFile is not an option

A typedarray would be nice, but I can't think of a nice API design for it, and converting from ArrayBuffer to any TypedArray worked perfectly fine for my use case. Fetching via URL was an option, but this meant that I would have to deal with lifecycle hooks when I can very much just avoid it: essentially, fetching via URL would be a minor performance penalty on my side. This fetching, regardless, is happening client-side.

[Its-Just-Nans]

A typedarray would be nice, but I can't think of a nice API design for it

Another API design idea https://github.com/vitejs/vite/pull/19346#issuecomment-2669832133

import data from 'path/to/file' with { type: "arrayBuffer" }