row.meta.cursor in step always same value when parsing in worker mode

[sandergarretsen]

Not sure if this is a bug or simply a limitation of parsing in worker mode, but when I parse a (local) file in worker mode, the row.meta.cursor always points to the end of the file:

Papa.parse(file, {
  header: true,
  worker: true, // << worker is true
  step: (row, parser) => {
    console.log(row.meta.cursor, file.size);
    // => logs "677403 677403" 8973 times
  },
});

Papa.parse(file, {
  header: true,
  worker: false, // << worker is false
  step: (row, parser) => {
    console.log(row.meta.cursor, file.size);
    // => logs increasisng number upto filesize as expected
  },
});

Is there any way to get the cursor (or anything else that can be used as a progress-indicator) when parsing in worker mode?

[pokoli]

I think is a missing feature of worker mode.

[marabesi]

is there a workaround that?

[marabesi]

I don't know if this is the case BUT, I got the cursor progress + worker enable working enabling the chunk option.

The following example was giving me always the same cursor value as described by @sandergarretsen :

  Papaparse.parse(file, {
    header: true,
    worker: true,
    skipEmptyLines: true,
    step: (result, parser) => {
      console.log(result.meta.cursor) // always the same value
      stepCallback(result.data, parser)
    },
    complete: (result, file) => doneCallback(result, file)
  }

Once I enabled the chunk size, it started to work a expected:

  Papaparse.parse(file, {
    header: true,
    worker: true,
    skipEmptyLines: true,
    chunkSize: 500, // add this line
    step: (result, parser) => {
      console.log(result.meta.cursor) // shows the current cursor pointer value as expected
      stepCallback(result.data, parser)
    },
    complete: (result, file) => doneCallback(result, file)
  })

The version I am using is 5.0.0, can someone confirm that?

[marabesi]

btw I just tried that because I found a test case covering this scenario https://github.com/mholt/PapaParse/blob/master/tests/test-cases.js#L2017

[pokoli]

Maybe it's a matther of setting a default value for chungSize.

A PR adding a test case withouth chundSize and solving it will be very welcome.

[ux-engineer]

Experiencing this issue...am updating width of a custom progress bar. If chunkSize is not set, then I get updates to the bar only a few times towards the end of processing, even though it's update method gets fired on time at even 1% increment, but weirdly DOM doesnt update during processing.

EDIT: Am getting a type error with TypeScript now: Object literal may only specify known properties, and 'chunkSize' does not exist in type 'ParseConfig'.

[billsomen]

OMG, since 2019, no one has address this issue. Its pretty annoying.

[victorlucss]

@billsomen have you tested the @marabesi solution? I've tried and this is working fine, but when I use skipEmptyLines greedy the meta.cursor doesn't complete the all file size. I've opened this issue #825