frictionless-js icon indicating copy to clipboard operation
frictionless-js copied to clipboard

Slice stream function for node backend

Open risenW opened this issue 5 years ago • 5 comments

Acceptance Criteria

  • [ ] The method should receive start position, end position
  • [ ] I am able to seek a file to a specific location and then read N bytes from it
  • [ ] I am able to return seek position as a stream

risenW avatar Nov 03 '20 14:11 risenW

If we are using the browser's File API, at least for browser this is not needed because File.slice() is already available and does the job.

I am not sure about Node support, but if we want to have node support, we need to provide some kind of unified API - maybe the best approach would be to use something like this that emulates File API on Node.

shevron avatar Nov 05 '20 07:11 shevron

If we are using the browser's File API, at least for browser this is not needed because File.slice() is already available and does the job.

I am not sure about Node support, but if we want to have node support, we need to provide some kind of unified API - maybe the best approach would be to use something like this that emulates File API on Node.

I don't think this is a priority at the moment, right?

risenW avatar Nov 05 '20 08:11 risenW

Node support should be quite straightforward since we have a simple stream API (right?).

rufuspollock avatar Nov 05 '20 10:11 rufuspollock

@risenW I'm not yet clear why we need slice vs basic stream() method we already had (node streams allow one to read chunk by chunk). Can you give more info on slice vs stream and why we need slice?

rufuspollock avatar Nov 09 '20 16:11 rufuspollock

@risenW I'm not yet clear why we need slice vs basic stream() method we already had (node streams allow one to read chunk by chunk). Can you give more info on slice vs stream and why we need slice?

This is not needed anymore I think. It was a backup in case the streaming upload didn't work. But since it has already been fixed, we do not need this anymore.

And yes, after my research as well I found out that node file stream has a start and end parameter that we can use. If needed, we can just create a simple function on top of it.

risenW avatar Nov 09 '20 16:11 risenW