Document needed polyfills for webpack 5

[angelo-v]

This package also works out-of-the-box in browsers via tools such as webpack

I think this is not totally true anymore for webpack 5, since many polyfills got removed and need to be added explicitely. It would be helpful if rdf-dereference documented what exact polyfills are needed.

From what I know this is at least stream-browserify, which needs to be installed and added in webpack config via

resolve: {
  fallback: {
      stream: require.resolve('stream-browserify'),
  },
}

After doing so I ran into a runtime error "_stream_readable.js:561 Uncaught ReferenceError: process is not defined", which I could get rid of by installing process and adding a polyfill via ProvidePlugin:

plugins: [
  new ProvidePlugin({
      process: 'process',
    }),
]

But I noticed a problem with dereferencing JSON-LD data then. From a html resource containing both rdf-a and JSON-LD the stream only contained the rdf-a data then. So I am probably missing something else.

[rubensworks]

Sure, documentation on this is definitely welcome.

On the long-term, no polyfills would be needed at all though. But this would require some bigger internal changes. Related to https://github.com/rubensworks/jsonld-streaming-parser.js/issues/78

But I noticed a problem with dereferencing JSON-LD data then. From a html resource containing both rdf-a and JSON-LD the stream only contained the rdf-a data then. So I am probably missing something else.

Works fine for my use cases though. This service uses rdf-dereference internally, and it is able to extract both RDFa and JSON-LD from my personal webpage: https://rdf-play.rubensworks.net/#url=https%3A%2F%2Fwww.rubensworks.net%2F

[angelo-v]

The project you mention uses Webpack 4. Did you try with webpack 5 and the polyfills I mentioned? I can prepare a minimal demo projekt reproducing the problem, but not before friday

[rubensworks]

No, I haven't updated to Webpack 5 yet.

It seems unlikely that your RDFa/JSON-LD issue is caused by this though. Can you parse your document properly via https://rdf-play.rubensworks.net/ ?

[angelo-v]

Yes it worked before, then I upgraded another library (inrupt/solid-client-authn-browser) which implicitly removed some polyfills. I added the above 2 explicitly and then the problem occurs, so it is probably polyfill related

[rubensworks]

Hmm, no idea what could be causing that tbh.

FYI, I've also experienced several weird errors with Webpack 5.x, which is why I haven't updated my packages to it yet (but it has been a couple of months since I last tried).

[angelo-v]

Here is a demo setup were you can reproduce the issue: https://github.com/angelo-v/rdf-dereference-webpack5

RDFa is found there, while JSON-LD is not

[angelo-v]

I found a workarround by using node-polyfill-webpack-plugin which basically restores all the polyfills from webpack 4. You can find it in this branch and see that both RDFa and JSON-LD are working there: https://github.com/angelo-v/rdf-dereference-webpack5/tree/node-polyfill-webpack-plugin

Still it would be nice to know the exact polyfills needed, so that one can specifically add those.

[rubensworks]

The upcoming release will not require any polyfills anymore.