mcli: Unable to prepare URL for copying. Unable to guess the type of copy operation.

[paramazo]

Expected behavior

The folder in a bucket should be moved into another bucket

Actual behavior

The folder was not moved, mc results with error:

mc mv --recursive <bucket1>/<folder> <bucket2>/<folder> mcli: <ERROR> Unable to prepare URL for copying. Unable to guess the type of copy operation.

Other folders in the same folder structure was moved this way successful.

In an earlier version of mc, the error was

mcli: <ERROR> Unable to validate source

Steps to reproduce the behavior

Unknown why some folders cant be moved

mc --version

mcli version RELEASE.2024-03-30T15-29-52Z (commit-id=9f8147bf0e037730077a1b3baef25e53181099b0) Runtime: go1.21.8 linux/amd64

minio version RELEASE.2023-03-20T20-16-18Z (commit-id=05444a0f6af8389b9bb85280fc31337c556d4300) Runtime: go1.19.7 linux/amd64

System information

Debian 11.5 Bullseye

[paramazo]

Tested mc mirror but it does not copy anything from the affected folder and does not result with Error and ends after 2 minutes. The folder i want to copy is 101GB big with 1.8 million files.

Other smaller folders in the same bucket works with mc mv and with mc mirror.

[paramazo]

Tested to copy the folder from minio server to my local system with mc cp -r, same error. This folder can not be copied, mirrored or moved in any way. Other smaller folders work, so obviously it is a bug in MinIO with very big data. This is a gamebreaker and very critical and if this behaviour is not supported, we need to change our environment and switch to another solution.

[paramazo]

The data in our minio are now in a minio folder structure and as we can not access the contents via mcli/awscli/winscp, we have a huge data loss? Can anyone help here? Where do we find support?

[ds-colligence]

From what I can understand, it seems like when the bucket has hyphens in it, it is failing. At least that seems to be the case for me.

[mmedic]

I can confirm the same issue with a bucket with hypens in it

[jiuker]

@mmedic Could you share a script to let us to reproduce that?

[hypesystem]

I was wracking my brain because I got this error, too. It turns out it appears when the source file does not exist

[mratibb]

I had the same issue, and apparently it was a certificate issue. Either add 'insecure' option to your command, or add the certificate to the host/container you running the command on.

[Xun66]

I had the same issue with two mc commands.

Command 1:

mc alias set myminio $MINIO_ENDPOINT $MINIO_ACCESS_KEY $MINIO_SECRET_KEY
mc cp --recursive myminio/<folder> <local>/<folder>
# mc: <ERROR> Unable to prepare URL for copying. Unable to guess the type of copy operation.

Looks like the MINIO_ACCESS_KEY and MINIO_SECRET_KEY were empty. Once I filled in the correct values, everything worked smoothly.

Command 2:

mc cp --recursive <local>/<folder> myminio/<folder>
# mc: <ERROR> Unable to prepare URL for copying. Unable to guess the type of copy operation.

Turns out, the issue was that <local>/<folder> didn't exist. In earlier versions of mc, this would've triggered an error message saying: mc: <ERROR> Unable to validate source <local>/<folder>: Object does not exist.

mc version

It looks like this change was introduced in PR #4710. From version mc version RELEASE.2023-10-14T01-57-03Z (commit-id=d158b9a478a6a5a74795f01097d069be82edfff6).

my test result:

$ docker run -it --rm minio/mc:RELEASE.2023-10-14T01-57-03Z cp -r /foo/ myio/bar 
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
mc: <ERROR> Unable to prepare URL for copying. Unable to guess the type of copy operation. 

$ docker run -it --rm minio/mc:RELEASE.2023-10-04T06-52-56Z cp -r /foo/ myio/bar   
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
mc: <ERROR> Unable to validate source `/foo/`: Object does not exist

[obarisk]

I have the same issue. With a very small file (28 Bytes).

I can replicate the issue by

• use golang PutObject to create a small file like xxx-1/7f9d85cf-7f48-408c-9649-f4c1ba472886/small.json

• use mc cli (windows) to copy the object to local right after the object created.

  mc: <ERROR> Unable to prepare URL for copying. Unable to guess the type of copy operation.

• mc ls can find the object (small.json)

• mc cat the object with redirect to local/small.json (works)

• mc cp works after a couple hours.

cli version

mc version RELEASE.2024-05-03T11-21-07Z (commit-id=b471de8d1882cae21ca84e98d56c2a5e2c321164)
Runtime: go1.22.2 windows/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

server version

minio version RELEASE.2024-06-13T22-53-53Z (commit-id=20960b6a2ddb9594ee418035b3c7c7fe92ae6a12)
Runtime: go1.22.4 linux/amd64
License: GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html
Copyright: 2015-2024 MinIO, Inc.

[XoseRamon]

I had this very same issue when the source folder (uploading files to a minIO instance) contains hyphens. Removing the hyphens in the source folder fixed the issue. The error message is not very helpful.

[jiuker]

@XoseRamon Like what name?

[XoseRamon]

@jiuker I had something like mc cp test-data/*.gzip minio/test-bucket which threw an error and changing it to mc cp testdata/*.gzip minio/test-bucket worked. I am using:

mc version RELEASE.2024-08-17T11-33-50Z (commit-id=2c94b8d03979ed96a6379ecabc3435b271398efc)
Runtime: go1.22.6 linux/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

[jiuker]

@jiuker I had something like mc cp test-data/*.gzip minio/test-bucket which threw an error and changing it to mc cp testdata/*.gzip minio/test-bucket worked. I am using:

mc version RELEASE.2024-08-17T11-33-50Z (commit-id=2c94b8d03979ed96a6379ecabc3435b271398efc)
Runtime: go1.22.6 linux/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

What's the cp? remote to local?

[XoseRamon]

@jiuker I had something like mc cp test-data/*.gzip minio/test-bucket which threw an error and changing it to mc cp testdata/*.gzip minio/test-bucket worked. I am using:

mc version RELEASE.2024-08-17T11-33-50Z (commit-id=2c94b8d03979ed96a6379ecabc3435b271398efc)
Runtime: go1.22.6 linux/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

What's the cp? remote to local?

local to remote

[jiuker]

@jiuker I had something like mc cp test-data/*.gzip minio/test-bucket which threw an error and changing it to mc cp testdata/*.gzip minio/test-bucket worked. I am using:

mc version RELEASE.2024-08-17T11-33-50Z (commit-id=2c94b8d03979ed96a6379ecabc3435b271398efc)
Runtime: go1.22.6 linux/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

What's the cp? remote to local?

local to remote

Ok, I can't replicate this. Please share the quick bashScript to generate the dir test-data. And then I will follow this.

[keplersj]

I'm seeing this issue locally and in our Continuous Integration environment when using the mc cp --recursive command to copy local files to a bucket whose name contains a hyphen.

We did not see this failure before the October 2nd release of mc: https://github.com/minio/mc/releases/tag/RELEASE.2024-10-02T08-27-28Z.

[vbmithr]

Same issue here just trying to copy a remote file to local.

[mmedic]

I think the issue here is a dash - in a bucket name. Can anyone else confirm?

[vbmithr]

Confirmed ;)