data does not come as accurate length (sometime large sometime missing)

[lchaudharyPinchin]

hello @Keyang please have a look on this issue.

I am using csvtojson/v2 for fetching data from API under loop.

while on single use it works, but under loop fromStream doesnot work, always give wrong number of rows.

Issue:-

let urls = ["http://example.com/key/1", "http://example.com/key/2",.....];
urls.map(el => {
     csv()
       .fromStream(request.get(el))
       .then(jsonData => {
         console.log(jsonData.length);  
       }).catch(error => res.status(500).json({error}));
   }).then(() =>
     res.send("done")
   )

output:- every time when code is executed jsonData.length is diffrent. which means when i save the data, data also save empty. but when the length is accurate then data is accurate and saved.

Thanks in advance.

[Keyang]

I dont have the answer right in my head. are you saying that if you process those urls in sequence it works but if you process them in parallel it breaks?

[lchaudharyPinchin]

Hey Keyang, I found out the solution, and i also found out the bug. It was Async issue. So the issue was when i ran the 5 urls on same time the first iteration data comes and added into the other ones and the last iteration becomes heavy and corrupted.

So I choose sync for calling url instead of async. By the way your package is awesome. But i had to use some other csv parser for sync url calls.