fscrawler icon indicating copy to clipboard operation
fscrawler copied to clipboard

Read from any FS Provider using the REST Service

Open dadoonet opened this issue 2 years ago • 1 comments

Is your feature request related to a problem? Please describe.

We want to be able to send commands to FSCrawler which could fetch a file from any provider like the local FS where FSCrawler is running or S3...

Describe the solution you'd like

curl -XPOST http://127.0.0.1:8080/fscrawler/_upload -d '{
  "type": "fs",
  "fs": {
    "url": "file://foo/bar.txt"
  }
}
curl -XPOST http://127.0.0.1:8080/fscrawler/_upload -d '{
  "type": "s3",
  "s3": {
    "url": "s3://foo/bar.txt"
  }
}

dadoonet avatar Sep 06 '21 14:09 dadoonet

This could also solve may be #805

dadoonet avatar Sep 06 '21 14:09 dadoonet