Alternative solution

[jimmywarting]

I find vinyl files to be abnormal files. it's not so web-ish. NodeJS now also ships with Blob built in (and Files too, ...in a roundabout way) this blobs have .stream(), .arrayBuffer(), .text(), .slice(), name & type something that can easily be shared with consistently with deno, browsers and Deno.land without needing any custom file wrappers.

if you are using fetch-blob then you can get files/blob's that are just references point to some location on the disc (just like vinyl files) You can also construct own in memory Files. this blobs are immutable, re-readable and also slice:able (and when you slice a blob then you are just creating a new ref point with a new start/end offset

And for the storage you could just use Map, WeakMap, Set, or WeakSet. and maybe just use the path as the key.

so all in all:

import { File, fileFromSync, Blob } from 'fetch-blob/from.js'

var store = new Map();

var coffeeFile = new File(['test = 123'], 'test/file.coffee', { type, lastModified });
var pkg = fileFromSync('./package.json')

// will not allocate much memory, will just be 2 references point to other  files to read from
var both = new File([coffeeFile, pkg], '') 

store.set(coffeeFile.name, coffeeFile)
store.set(pkg.name, pkg)

// Map also has a set/delete/has/forEach method...

// to stream every file in the store: use something like:
new Blob(store.values()).stream()

You could also extend Map to override the get method and fallback to using the fs if the file isn't stored

[SBoudrias]

This would cause issues with plugins expecting to get vinyl files when handling the output stream.

I understand this might be more efficient, but if we break the Yeoman ecosystem, there's small value overall to implement such a change.

[jimmywarting]

yea, you are right. probably not worth it unless vinyl would start implementing some File-like IDL behavior to their vinyl files and then deprecate the other read methods...

the benefit of blob/files are that they work with the now built in FormData and fetch api in NodeJS 18 now. the undici fetch also accepts 3th party blob like my fetch-blob library that creates kind of like a file handle (without reading anything into memory)

it would be possible for vinyl files to also be acceptable in FormData and fetch if they somehow started to add the same functions that exist on web Files. think some requirements are stream() text() arrayBuffer() slice() name and type & also symbol.toStringTag for it to work in fetch & FormData.

any way, you can close this issue if you like.

btw, maybe what you could do is creating a store that extends map? class Store extends Map { ... } they can still be vinyl files. and you would then be able to use for...of with async/await

[SBoudrias]

The store interface can change, I do not think there's any hard requirement downstream for this (it's an internal detail.) Feel free to send a PR!

I'm also happy if we explore a dual interface or a way to get Vinyl compatibility or a way to easily convert between one or the other (like we could internally handle the file, but convert to Vinyl when streaming, and potentially behind a flag? And we could have Yeoman default to Vinyl)

[jimmywarting]

could start by breaking it down to smaller tasks. start for instances with making it extend Map instead of EventEmitter

class Store extends Map {
  constructor () {
    super()
    const EE = new EventEmitter()
    this.on = EE.on.bind(this)
    this.emit = EE.emit.bind(this)
  }
}

but personally i would instead have just used EventTarget instead as it's more cross env friendlier and don't require any dependencies

Feel free to send a PR!

nah, i'm good. I don't like or help out and support Typescript projects with PRs I'm anti typescript developer. Prefer much more plain vanilla javascript without compilers and use JSDoc instead browser are never going to support TypeScript natively It's like trying to fit a square into a circle, there is nothing wrong with a loosed typed language, and that is a beauty in itself i think. it dose not need to be so darn strict. I reject every job offer i get that involves something with TypeScript and refer them to https://jimmywarting.github.io/you-might-not-need-typescript/