Preview loading entire file

[dbroadhurst]

I'm finding the preview config still loads the entire file which can lock the browser for 10 - 15 secs with 400mb+ files. Is this expected behavior? I've built the following workaround but not sure if it's safe, although it's working as expected and exits quickly.

export function parsePreview(files: File) {
  return new Promise(function(resolve, reject) {
    Papa.parse(files, {
      header: true,
      preview: 100,
      chunk: data => {
        resolve(data)
      },
      complete: (results: any, file: any) => {
        resolve(results)
      }
    })
  }).catch(function(e) {
    console.log(e)
  })
}

[pokoli]

I will be happy to accept a pull request that makes the preview config do not load the entire file.

Could you provide some example code to reproduce it?

[jdesboeufs]

It's very easy to reproduce with Node.js.

const Papa = require('papaparse')
const inputStream = createReadStream('path/to/some-multi-gigabytes-file.csv')

Papa.parse(inputStream, {
  preview: 10,
  complete() {
    console.log('Preview is available')
  }
})

If you run this script with limited RAM it will crash.

node --max-old-space-size=1024 reproduce.js

Can be easily fixed with inputStream.destroy() once the result is available.

I tried to fix this in PapaParse but the dive into the code was too hard for me.