The `cf copy-source` command fails

[XanderStrike]

Describe the bug

When running cf copy-source, an error message is returned and the source is not copied.

I didn't know about this feature until now, so here are the docs if you're like me: https://cli.cloudfoundry.org/en-US/v6/copy-source.html

This is causing cf-acceptance-tests to fail in our pipelines, though I've validated it also doesn't work on handmade clusters.

To Reproduce*

Steps to reproduce the behavior:

1. Push two apps (app1 and app2) that respond with different things
2. Run cf copy-source app2 app1

Observed behavior

Command fails with the following cli error:

$ cf copy-source dora2 dora
Copying source from app dora2 to target app dora in org o / space s as admin...

Package failed to process correctly after upload
FAILED

The HTTP response observed when running cf in verbose mode includes the following reference to a CloudController error:

failed to copy - CloudController::Blobstore::FileNotFound

Though I haven't found where that error occurs in the api server or worker logs.

Expected behavior

app1 is running the same code as app2

Additional context

cf-for-k8s SHA

4c864469f2dd7f86d7a259e985b8a707c6ab1f2b (latest as of writing)

Deploy instructions

https://github.com/cloudfoundry/cf-k8s-networking/blob/develop/hack/cf4k8s/methods.sh#L50-L60

Cluster information

GKE

CLI versions

$ ytt --version
ytt version 0.30.0
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:41:02Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.9-gke.1504", GitCommit:"fb367de18820fe57bac5d97ad60b7a2cef8765e9", GitTreeState:"clean", BuildDate:"2020-09-09T01:00:23Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}
$ kapp --version
kapp version 0.34.0
$ cf version
cf version 7.1.0+4c3168f9a.2020-09-09

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/174933903

The labels on this github issue will be updated when the story is started.

[tcdowney]

I bet this was broken by the switch to using an OCI image registry for packages in https://github.com/cloudfoundry/cf-for-k8s/pull/430.

There's no way this code works: https://github.com/cloudfoundry/cloud_controller_ng/blob/aa19fd5c71af54be7b7fa228926161c563e8a26b/app/jobs/v3/package_bits_copier.rb#L20