Bug with COPY and variables with quotes

[metajiji]

Actual behavior Kaniko does not COPY files to directory when used path with variables in quotes with trailing slash inside qoutes, i.e. COPY Dock* "$PATH2FILE/"

Expected behavior Files should be copied to the directory.

To Reproduce Steps to reproduce the behavior:

1. Craete Dockerfile

2. docker build -t test . - OK, file in directory /path/2/files/Dockerfile

3. docker run --rm -ti -v $(pwd):/srv gcr.io/kaniko-project/executor:debug-v0.17.1 --context /srv --dockerfile /srv/Dockerfile --no-push --destination test:latest --tarPath /srv/test.tar - FAIL, file saved as /path/2/files

Additional Information

• Dockerfile

FROM scratch
ARG PATH2FILE=/path/2/files
COPY Dock* "$PATH2FILE/"

• Build Context docker run --rm -ti -v $(pwd):/srv gcr.io/kaniko-project/executor:debug-v0.17.1 --context /srv --dockerfile /srv/Dockerfile --no-push --destination test:latest --tarPath /srv/test.tar
• Kaniko Image: gcr.io/kaniko-project/executor:debug-v0.17.1
• Current workarounds:
  1. Remove any qoutes: COPY Dock* $PATH2FILE/
  2. Move slash out of the qoutes COPY Dock* "$PATH2FILE"/

Description	Yes/No
Please check if this a new feature you are proposing	- [ ]
Please check if the build works in docker but not in kaniko	- [X]
Please check if this error is seen when you use --cache flag	- [X]
Please check if your dockerfile is a multistage dockerfile	- [X]

[raijinsetsu]

As a work around:

FROM scratch
ARG PATH2FILE=/path/2/files
WORKDIR $PATH2FILE
COPY Dock* ./

[WestonThayer]

This also occurs when the trailing slash is included within the ARG:

FROM scratch
ARG PATH2FILE=/path/2/files/
COPY file.txt ./${PATH2FILE}

cat /path/2/files has the contents of file.txt instead of /path/2/files/file.txt existing.

[GusAntoniassi]

Same thing happened to me, I'd just like to leave the error message for google-comers:

INFO[0050] Resolving srcs [/usr/lib/librdkafka*]...     
error building image: error building stage: failed to execute command: resolving src: when specifying multiple sources in a COPY command, destination must be a directory and end in '/'

The solution in my case was just to remove the quotes from the destination COPY path.

[ed9w2in6]

I faced this issue lately and I can confirm that as of [2023-09-06] this behaviour is still not inline with docker. The same workaround still works.

I would recommend moving slash out of quote (i.e. workaround ii.) for good practice.

Regarding @WestonThayer's comment (https://github.com/GoogleContainerTools/kaniko/issues/1080#issuecomment-763215604) I think ENV expansion should be fixed with this issue: #1682. However yours are on ARG so I am not sure if they share the same logic. And a recent comment seems to indicate that the issue resurfaced.

Investigation

Error message:

error building image: error building stage: failed to get files used from context: when specifying multiple sources in a COPY > command, destination must be a directory and end in '/'

From this we can trace the errors.Wrap(..):

• error building image :: kaniko/cmd/executor/cmd/root.go {RootCmd.Run(..)}
• error building stage :: kaniko/pkg/executor/build.go {DoBuild(..)}
• failed to get files used from context :: kaniko/pkg/executor/build.go {stageBuilder.build(..)}
• final :: kaniko/pkg/commands/copy.go {FilesUsedFromContext(..) -> copyCmdFilesUsedFromContext(..)}
  • called :: kaniko/pkg/util/command_util.go {-> ResolveEnvAndWildcards(..) -> IsSrcsValid(..)}
    • error if totalSrcs > 1 and not IsDestDir(path) (culprit)

In which IsDestDir(path) is also the culprit for issue: #1682.

ref, fix to #1682: pull req #1683

A simple test setup for IsDestDir(path): (ref: https://github.com/GoogleContainerTools/kaniko/blob/main/pkg/util/command_util.go#L166)

package main
import "fmt"
import "os"
import "strings"

const (pathSeparator = "/")
func IsDestDir(path string) bool {
	// try to stat the path
	fileInfo, err := os.Stat(path)
	if err != nil {
		// fall back to string-based determination
		return strings.HasSuffix(path, pathSeparator) || path == "."
	}
	// if it's a real path, check the fs response
	return fileInfo.IsDir()
}

func main() {
    pwd, _ := os.Getwd()
    files, _ := os.ReadDir(pwd)
    fmt.Printf("PWD: '%s' \nFiles:\n", pwd)
    for _, f := range files {
        fmt.Printf("\t'%s'\n", f.Name())
    }
    testCases := [...]string{".bashrc", pwd, pwd+"/", "\""+pwd+"\"", "\""+pwd+"/\"", "\""+pwd+"\"/"}
    fmt.Println("Test Cases:")
    for _, f := range testCases {
        fmt.Printf("\tIsDestDir(\"%s\") = %t\n", f, IsDestDir(f))
    }
}

should produce sth like:

PWD: '/home/compiler' 
Files:
	'.bash_logout'
	'.bashrc'
	'.cache'
	'.profile'
Test Cases:
	IsDestDir(".bashrc") = false
	IsDestDir("/home/compiler") = true
	IsDestDir("/home/compiler/") = true     # workaround i.
	IsDestDir(""/home/compiler"") = false   # <- maybe args to COPY
	IsDestDir(""/home/compiler/"") = false  # <- are like these?
	IsDestDir(""/home/compiler"/") = true   # workaround ii.

I do not have time to look into how the arguments to COPY are read but my guess is that they are probably read with quotes (e.g. "/home/compiler"), which WILL FAIL stat and the string-based fallback.

Anyone who wish to work on this can try to confirm if this is true

[ed9w2in6]

May I ask where can I raise to fix this issue?
Are there any formal procedure that has to be go through? Or is this project only accepting pull request?