jupyterlab-s3-browser icon indicating copy to clipboard operation
jupyterlab-s3-browser copied to clipboard

Published 20 hours ago • verified publisher icon IBM

File upload function do not upload complete file

Open mohittalele opened this issue 2 years ago • 4 comments

Describe the bug I am trying to upload csv file which contains million rows. Using the s3-browser functionality I can upload file - However not complete file is uploaded and about 50% of the data is lost. The s3-broswer do not show any logs or error message To Reproduce Steps to reproduce the behavior:

  1. Take fairly large csv file
  2. Click on upload button in s3 browser
  3. s3 browser Uploads the data.
  4. Check the number of rows in the file. Not all the rows are present since whole file is not uploaded

Expected behavior Large file should be uploaded completely to s3 endpoint

Screenshots

Desktop (please complete the following information):

  • OS: Ubuntu 20.04.2 LTS
  • Browser Chrome 99.0.4844.83
  • Version of jupyterlab-s3-browser extension - 0.11.1

I am using standalone Minio version - minio/minio:RELEASE.2020-06-14T18-32-17Z as my s3 endpoint

mohittalele avatar Mar 28 '22 11:03 mohittalele

Thanks for reporting the issue! I have been able to reproduce it. I'm pretty busy at the moment but I should have time to investigate and fix it within the next couple of weeks.

reevejd avatar Apr 04 '22 13:04 reevejd

Hi Reeve, thanks for info.

If it's not that complicated, you can leave some instructions on how to fix it. Maybe I can create draft pr.

On Mon, 4 Apr 2022, 16:00 James Reeve, @.***> wrote:

Thanks for reporting the issue! I have been able to reproduce it. I'm pretty busy at the moment but I should have time to investigate and fix it within the next couple of weeks.

— Reply to this email directly, view it on GitHub https://github.com/IBM/jupyterlab-s3-browser/issues/63#issuecomment-1087595044, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEIVOMX6KPV2ON6TIWNE3H3VDLYV5ANCNFSM5R23JOKQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

mohittalele avatar Oct 11 '22 09:10 mohittalele

Hi @mohittalele, thanks for your interest and sorry for the lack of updates on this issue. I'm guessing the issue is that the serverextension doesn't handle the case where jupyterlab breaks a file up into chunks for uploading. The extension is probably just overwriting the whole file with the latest chunk every time. Are you able to confirm that hypothesis? Specifically I'm thinking you'll see the size of the file stays the same throughout the upload (maybe 1MB? or however large the chunk size is) but the content will continue changing.

reevejd avatar Oct 17 '22 14:10 reevejd

I don't see the upload button guys, where is it supposed to be? I see the small video in the readme, I don't see it on my jupyter installation

inteloid avatar Jan 23 '23 17:01 inteloid