s3proxy icon indicating copy to clipboard operation
s3proxy copied to clipboard

ChunkSignature added to the uploaded file

Open aravinds502 opened this issue 8 months ago • 0 comments

When a file is Uploaded to GCP via s3proxy, "chunk signature is added in the first and last lines

We are using s3client, Upload multipart Request to upload. The uploaded data was as follows for a sample.csv

69e;chunk-signature=e2d4edfb35ad945809e4c08c2ed8c8abf4406b1af8621b3b91d1224a4d24a613
"Year","Month","DayofMonth","DayOfWeek","DepTime","CRSDepTime","ArrTime","CRSArrTime","UniqueCarrier","FlightNum","TailNum","ActualElapsedTime","CRSElapsedTime","AirTime","ArrDelay","DepDelay","Origin","Dest","Distance","TaxiIn","TaxiOut","Cancelled","CancellationCode","Diverted","CarrierDelay","WeatherDelay","NASDelay","SecurityDelay","LateAircraftDelay"
"1993","1","29","5","1055","1055","1228","1212","US","66","","93","77","","16","0","EWR","BUF","282","","","0","","0","","","","",""
"1993","1","30","6","1052","1055","1214","1212","US","66","","82","77","","2","-3","EWR","BUF","282","","","0","","0","","","","",""
"1993","1","31","7","1103","1055","1213","1212","US","66","","70","77","","1","8","EWR","BUF","282","","","0","","0","","","","",""
"1993","1","1","5","1738","1730","1949","1941","US","68","","131","131","","8","8","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","3","7","1737","1730","1946","1941","US","68","","129","131","","5","7","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","4","1","1739","1730","1951","1941","US","68","","132","131","","10","9","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","5","2","1729","1730","1939","1941","US","68","","130","131","","-2","-1","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","6","3","1730","1730","1940","1943","US","68","","130","133","","-3","0","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","7","4","1734","1730","1942","1943","US","68","","128","133","","-1","4","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","8","5","1812","1730","2015","1943","US","68","","123","133","","32","42","MCO","PIT","834","","","0","","0","","","","",""

0;chunk-signature=c300db3aca97bfed581d9c494fcb458faddc6eda7382701a5b08a89bd74340b8

How to avoid adding the chunk-signatures, while doing uploads with GCP via s3proxy? Is there any setting we can add? The same thing works fine if we directly connect to GCP.

Thanks, Aravind

aravinds502 avatar Jun 21 '24 10:06 aravinds502