DirectorySource("..") causes an infinite copy loop, fills HDD

[mortenpi]

trafficstars

I guess ideally it would detect these circular cases and either error or avoid copying build/ into build/ (which is what I think is happening here). MWE:

using BinaryBuilder, Pkg
sources = [DirectorySource("..")]
platforms = supported_platforms()
products = [LibraryProduct("test", :test)]
build_tarballs(ARGS, "foo", v"0.0.1", sources, "", platforms, products, [])

which gets stuck after

[ Info: Building for i686-linux-gnu, x86_64-linux-gnu, aarch64-linux-gnu, armv7l-linux-gnueabihf, powerpc64le-linux-gnu, i686-linux-musl, x86_64-linux-musl, aarch64-linux-musl, armv7l-linux-musleabihf, x86_64-appl
e-darwin, x86_64-unknown-freebsd, i686-w64-mingw32, x86_64-w64-mingw32        
[ Info: Directory ".." found                                        

[giordano]

I'm scared to test this locally after reading "fills HDD" :grimacing:

[mortenpi]

I probably should have added "eventually" :laughing: It's not too catastrophic -- it seems to be recursively copying the files at about a rate of a few tens of MB/s on my system. So if you have at least a few tens of GB free, then it will be a while.

But I think it can be a problem for users of BB, since BB is something you'd start up and then just let it run in the background. So it's likely you won't notice that it's stuck and slowly filling up the disk.

[benhamad]

This also happened to me but it fails before filling the disk with ENAMETOOLONG.

Stacktrace:
 [1] uv_error at ./libuv.jl:97 [inlined]
 [2] open(::String, ::UInt16, ::UInt64) at ./filesystem.jl:87
 [3] sendfile(::String, ::String) at ./file.jl:910
 [4] cptree(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:327
 [5] cptree(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:324 (repeats 410 times)
 [6] cp(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:349
 [7] setup(::BinaryBuilderBase.SetupSource{DirectorySource}, ::String, ::Bool) at /home/chaker/.julia/packages/BinaryBuilderBase/fylAz/src/Prefix.jl:307
 [8] setup_workspace(::String, ::Array{BinaryBuilderBase.SetupSource{DirectorySource},1}; verbose::Bool) at /home/chaker/.julia/packages/BinaryBuilderBase/fylAz/src/Prefix.jl:353
 [9] autobuild(::AbstractString, ::AbstractString, ::VersionNumber, ::Array{var"#s648",1} where var"#s648"<:BinaryBuilderBase.AbstractSource, ::AbstractString, ::Array{T,1} where T, ::Array{var"#s647",1} where var"#s647"<:Product, ::Array{var"#s646",1} where var"#s646"<:BinaryBuilderBase.AbstractDependency; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, kwargs::Any) at /home/chaker/.julia/packages/BinaryBuilder/OMMHj/src/AutoBuild.jl:633
 [10] build_tarballs(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any; kwargs::Any) at /home/chaker/.julia/packages/BinaryBuilder/OMMHj/src/AutoBuild.jl:247
 [11] build_tarballs(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any) at /home/chaker/.julia/packages/BinaryBuilder/OMMHj/src/AutoBuild.jl:111
 [12] top-level scope at /home/chaker/git/tpch-dbgen/build_tarballs.jl:66
 [13] include(::Function, ::Module, ::String) at ./Base.jl:380
 [14] include(::Module, ::String) at ./Base.jl:368
 [15] exec_options(::Base.JLOptions) at ./client.jl:296
 [16] _start() at ./client.jl:506

[andrew-at-rotor]

This seems to be still happening...at first go (not knowing how long to expect things to take, etc.) I did fill about 300g with recursive copies. It seems that if you call build_tarballs from within the directory listed in DirectorySource, you're going to have a bad time. My workaround was moving to a temp folder (out of the "source" directory) before calling build_tarballs and then moving back after.

After learning a bit more about binary builder, I tried to remove this step but still run into recursive copy issues with name too long error.