nextflow icon indicating copy to clipboard operation
nextflow copied to clipboard

Remove 64KB size limit on scripts

Open bentsherman opened this issue 1 month ago • 1 comments

Since DSL2, Nextflow scripts have been limited to 64KB. I can't remember the exact reason but I recall that it has to do with the Groovy engine that is used to evaluate scripts. GroovyShell is used for scripts and Grengine is used for config files. However, I think we should be able to use the Groovy CompilationUnit directly instead, and I think that would remove the limit. Might be something to implement in #4613 and #4744 .

Also, the TaskRun uses Grengine to (1) evaluate the task script and (2) evaluate global variables for the task hash. I think we can address both with the custom parser. For (1) we can just use a closure, and the parser can handle the shell: syntax. For (2), I don't think global variables will be allowed by the strict parser so it won't be needed.

bentsherman avatar May 24 '24 10:05 bentsherman