File:/// access has been dropped

[timbl]

ooops ... running rabel the command line client https://github.com/linkeddata/rabel using the current rdflib, it gives a "status 0" error to the console, and debugging reveals that the access to fileL URIs no longer works at all, as the fetcher now uses, in a node.js environment, node-fetch module which includes:

	return new Fetch.Promise(function(resolve, reject) {
		// build request object
		var options = new Request(url, opts);

		if (!options.protocol || !options.hostname) {
			throw new Error('only absolute urls are supported');
		}

		if (options.protocol !== 'http:' && options.protocol !== 'https:') {
			throw new Error('only http(s) protocols are supported');
		}

Maybe this a question of switching to a different one of many fetch modules. File:// access is important, for command line apps, test suites, and browser extensions also can also access files. Surprised we had no tests.

[dmitrizagidulin]

The reason that the node fetch modules dont support file:// urls is because the in-browser native fetch implementations also forbid file urls. (And they do that for security reasons).

So, this is unlikely to be able to be changed.

[ericprud]

I don't know if other programs use node-fetch but Fetcher uses node-fetch only on node:

if (!this._fetch)
  if (typeof window !== 'undefined')
    this._fetch = window.fetch.bind(window);
  else
    this._fetch = require('node-fetch');

This implies that node-fetch could be useful for e.g. file.

[timbl]

The logic that file access isn't allowed in browsers and so it won't happen on the command line I don't follow. No reason to make restrictions from the browser environment constrain the command line. The command line environment is really valuable, for testing, for utility programs, and for batch processing, and so on. It used to work with XHR. The fact that it doesn't now is a serious bug introduced by the refactoring.

[timbl]

Just as we have a need to have a separate fetch for file: URIs, also @thewebalyst was asking about allowing other URI schemes for other protocols such as decentralized storage spaces. Suggests having a top layer that dispatches based on URI scheme.

[timbl]

I note rabel works with [email protected]

[dmitrizagidulin]

The reason that file access was dropped in browser's fetch API is the same reason most Node fetch libs don't support it -- it's a serious security risk (with people being able to potentially request arbitrary files on the server). With Solid servers specifically, the risk is somewhat mitigated with the ACL mechanism, but I still think it should not be in any way encouraged.

[ericprud]

IIRC, the browsers wanted to prevent malicious websites from running JS in your browser that would access e.g. /etc/passwd or .cache/mozilla/…. This is analogous to XSS issues, though more dire because local filesystems typically have more sensitive info.

I have the impression that the vision for Solid is that it offers both filesystem and web access. It's true that in good NFS tradition, filesystem access is unrestricted (because there's no trusted service authenticating users and checking permissions). That said, anyone who's motivated to attack local files with malicious note script invocations could more easily do it with a text editor or require('fs').

[timbl]

@dmitrizagidulin Yes certainly it would be very foolish for the server to allow an HTTP client to allow a HTTP request to ask for a file on its local file system. That is NOT what we are talking about. We are talking about. Now are we talking about JS code accessing local files local to the browser. We are talking about

command line RDF tools

which you use like sed, ark, grep as well as curl. A node command line client must be able to access files!!

[timbl]

Rabel is really useful but only works with rdflib pre-fetch aka 0.15.0 which uses XMLhttpRequest which works with files in its node version.

[timbl]

See gitter https://gitter.im/linkeddata/rdflib.js?at=5c8e8f0bfa5b721a1fa57b17

[jeff-zucker]

Solid-rest provides complete (AFAIK) support for rdlib use of file:/// URIs including fetcher load, putBack, webOperation, and UpdateManager. To use these features in e.g. mashlib, this requires three changes to rdflib (see PR #444 ):

• fetcher.saveResponseMedata - currently refuses to save headers on file:// URIs, change it to save headers if headers exist (solid-rest returns them for file:///)

• UpdateManager.editable - currently refuses to allow editing of file:// URIs unless they have a MachineEditable triple; change this to also allow editing based on wac-allow headers in file:// URIs (which solid-rest provides for file:///)

• UpdateManager.updateLocalFile - currently saves using Firefox-specific code; add to this the ability to use regular fetcher webOperation and putBack (which solid-rest can handle)