Error staging directory from S3 as input

[jeremykap]

Bug report

Expected behavior and actual behavior

I'd like to be able to stage a directory from S3 as input to task, but am getting a java.lang.UnsupportedOperationException error when doing so. I'd expect this to work as it does locally, where the local path is symlinked into the working directory, and works like any other file.

Steps to reproduce the problem

This workflow takes a directory path as param input_dir, and lists it's contents in a file:

nextflow.enable.dsl=2


process ListFolder {
    input: 
        path(directory)
    output:
        path("dir_contents.txt")
    """
    ls ${directory}/ > dir_contents.txt
    """
}

workflow {
    input_ch = Channel.fromPath(params.input_dir)
    ListFolder(input_ch)
}

Program output

When running with local path it succeeds, when running with s3 path, it fails

nextflow run test.nf --input_dir {S3_PATH}
N E X T F L O W  ~  version 22.04.5
Launching `test.nf` [jovial_hugle] DSL2 - revision: 5877c29902
[-        ] process > ListFolder -
Error executing process > 'ListFolder'

Caused by:
  java.lang.UnsupportedOperationException

from .nextflow.log:

Aug-10 21:15:39.290 [Actor Thread 3] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-amazon version: 1.7.2
Aug-10 21:15:39.298 [Actor Thread 3] INFO  org.pf4j.AbstractPluginManager - Plugin '[email protected]' resolved
Aug-10 21:15:39.298 [Actor Thread 3] INFO  org.pf4j.AbstractPluginManager - Start plugin '[email protected]'
Aug-10 21:15:39.316 [Actor Thread 3] DEBUG nextflow.plugin.BasePlugin - Plugin started [email protected]
Aug-10 21:15:39.332 [Actor Thread 3] DEBUG nextflow.file.FileHelper - > Added 'S3FileSystemProvider' to list of installed providers [s3]
Aug-10 21:15:39.333 [Actor Thread 3] DEBUG nextflow.file.FileHelper - Started plugin 'nf-amazon' required to handle file: {S3_PATH}
Aug-10 21:15:39.343 [Actor Thread 3] DEBUG nextflow.file.FileHelper - Creating a file system instance for provider: S3FileSystemProvider
Aug-10 21:15:39.353 [Actor Thread 3] DEBUG nextflow.Global - Using AWS credential defined in `default` section in file: /home/ec2-user/.aws/credentials
Aug-10 21:15:39.354 [Actor Thread 3] DEBUG nextflow.file.FileHelper - AWS S3 config details: {secret_key={SECRET_KEY} region=eu-central-1, max_error_retry=5, access_key={ACCESS_EY}
Aug-10 21:15:39.879 [Actor Thread 3] DEBUG c.u.s3fs.S3FileSystemProvider - Using S3 multi-part downloader
Aug-10 21:15:39.882 [Actor Thread 3] DEBUG c.u.s3fs.ng.S3ParallelDownload - Creating S3 download thread pool: workers=10; chunkSize=10 MB; queueSize=10000; max-mem=1 GB; maxAttempts=5; maxDelay=1m 30s; pool-capacity=103
Aug-10 21:15:40.488 [Actor Thread 3] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=2; maxSize=2; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Aug-10 21:15:40.494 [FileTransfer-1] DEBUG nextflow.file.FilePorter - Copying foreign file {S3_PATH} to work dir: /home/ec2-user/work/stage/9f/1e82631cb4bf2e8225d1d8fdf3a65e/test
Aug-10 21:15:40.552 [Actor Thread 3] ERROR nextflow.processor.TaskProcessor - Error executing process > 'ListFolder'

Caused by:
  java.lang.UnsupportedOperationException

java.lang.UnsupportedOperationException: null
	at com.upplication.s3fs.S3FileSystemProvider.getFileAttributeView(S3FileSystemProvider.java:697)
	at java.base/java.nio.file.Files.getFileAttributeView(Files.java:1776)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:221)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:282)
	at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:328)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2792)
	at nextflow.file.CopyMoveHelper.copyDirectory(CopyMoveHelper.java:174)
	at nextflow.file.CopyMoveHelper.copyToForeignTarget(CopyMoveHelper.java:202)
	at nextflow.file.FileHelper.copyPath(FileHelper.groovy:939)
	at nextflow.file.FilePorter$FileTransfer.stageForeignFile0(FilePorter.groovy:294)
	at nextflow.file.FilePorter$FileTransfer.stageForeignFile(FilePorter.groovy:261)
	at nextflow.file.FilePorter$FileTransfer.run(FilePorter.groovy:247)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Aug-10 21:15:40.564 [Actor Thread 3] DEBUG nextflow.Session - Session aborted -- Cause: java.lang.UnsupportedOperationException

Environment

• Nextflow version: Seen in both 22.04.5 + 21.10.6
• Java version: 4.2.46(2)-release
• Operating system: macOS + Linux
• Bash version: 4.2.46(2)

Additional context

Using this to stage a directory of BWA Indices so I don't have to specify the individual files as s3 inputs. Would like to download them from s3 so can run alignment on AWS Batch.