data-studio-connector icon indicating copy to clipboard operation
data-studio-connector copied to clipboard

Data sampling can produce invalid JSON

Open rflprr opened this issue 7 years ago • 1 comments

For dataset: https://data.world/rflprr/google-ds-connector-performance

And query: SELECT * FROM hdx_org

It seems somewhere in return textLines.slice(0, sampleSize).map(JSON.parse); in rest.gs, one of the values has a starting quote but does not end the quote.

rflprr avatar Oct 09 '17 14:10 rflprr

Google App's FetchUrl service limits response size to 25Mb. Solving this issue would require:

  1. Google to extend that limit for data.world, or
  2. data.world to add support for HTTP range requests, so that the connector could retrieve results in 25Mb chunks

rflprr avatar Apr 10 '18 20:04 rflprr