Support file patterns in the `readonly_files` attribute

[vkarak]

Internal issue: https://madra.cscs.ch/scs/reframe/issues/254

[jgphpc]

I believe that by default hidden dirs are copied too. Would it be possible to have exclude patterns too ?

[jgphpc]

It will help when the filesystem is failing:

  * Failing phase: compile
  * Rerun with '-n SphExa_Atp_Check_10_2_1000_0 -p PrgEnv-gnu --system dom:gpu -r'
  * Reason: pipeline error: copying of files failed: [Errno 13] Permission denied: '/scratch/snx3000tds/piccinal/reframe/stage/dom/gpu/PrgEnv-gnu/SphExa_Atp_Check_10_2_1000_0/.git/objects/pack/pack-88da804f7d25f565007048ff195b1c02d0b861d5.idx'
Traceback (most recent call last):
  File "/users/piccinal/reframe.git/reframe/frontend/executors/__init__.py", line 269, in _safe_call
    return fn(*args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 120, in _fn
    func(obj, *args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 134, in _wrapped
    return fn(*args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1298, in compile_wait
    self._build_job.stderr, self._stagedir)
reframe.core.exceptions.BuildError: stdout: 'rfm_SphExa_Atp_Check_10_2_1000_0_build.out', stderr: 'rfm_SphExa_Atp_Check_10_2_1000_0_build.err'
--- rfm_SphExa_Atp_Check_10_2_1000_0_build.err (first 10 lines) ---
ldd: ./mpi+omp: No such file or directory
--- rfm_SphExa_Atp_Check_10_2_1000_0_build.err ---

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/users/piccinal/reframe.git/reframe/frontend/executors/policies.py", line 408, in runcase
    self._reschedule(task)
  File "/users/piccinal/reframe.git/reframe/frontend/executors/policies.py", line 502, in _reschedule
    task.compile_wait()
  File "/users/piccinal/reframe.git/reframe/frontend/executors/__init__.py", line 291, in compile_wait
    self._safe_call(self.check.compile_wait)
  File "/users/piccinal/reframe.git/reframe/frontend/executors/__init__.py", line 281, in _safe_call
    raise TaskExit from e
reframe.core.exceptions.TaskExit: : stdout: 'rfm_SphExa_Atp_Check_10_2_1000_0_build.out', stderr: 'rfm_SphExa_Atp_Check_10_2_1000_0_build.err'
--- rfm_SphExa_Atp_Check_10_2_1000_0_build.err (first 10 lines) ---
ldd: ./mpi+omp: No such file or directory
--- rfm_SphExa_Atp_Check_10_2_1000_0_build.err ---

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1156, in _copy_to_stagedir
    path, self._stagedir, self.readonly_files, dirs_exist_ok=True
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 250, in copytree_virtual
    copy_function, ignore_dangling_symlinks, dirs_exist_ok)
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 191, in copytree
    ignore_dangling_symlinks, dirs_exist_ok)
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 191, in copytree
    ignore_dangling_symlinks, dirs_exist_ok)
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 191, in copytree
    ignore_dangling_symlinks, dirs_exist_ok)
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 185, in copytree
    follow_symlinks=not symlinks)
  File "/usr/lib64/python3.6/shutil.py", line 263, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.6/shutil.py", line 121, in copyfile
    with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/scratch/snx3000tds/piccinal/reframe/stage/dom/gpu/PrgEnv-gnu/SphExa_Atp_Check_10_2_1000_0/.git/objects/pack/pack-88da804f7d25f565007048ff195b1c02d0b861d5.idx'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/users/piccinal/reframe.git/reframe/frontend/executors/__init__.py", line 269, in _safe_call
    return fn(*args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 120, in _fn
    func(obj, *args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 134, in _wrapped
    return fn(*args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1208, in compile
    self.sourcesdir))
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1159, in _copy_to_stagedir
    raise PipelineError('copying of files failed') from e
reframe.core.exceptions.PipelineError: copying of files failed: [Errno 13] Permission denied: '/scratch/snx3000tds/piccinal/reframe/stage/dom/gpu/PrgEnv-gnu/SphExa_Atp_Check_10_2_1000_0/.git/objects/pack/pack-88da804f7d25f565007048ff195b1c02d0b861d5.idx'

[jgphpc]

It may also be nice to be able to skip dirs with wrong permissions:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1156, in _copy_to_stagedir
    path, self._stagedir, self.readonly_files, dirs_exist_ok=True
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 250, in copytree_virtual
    copy_function, ignore_dangling_symlinks, dirs_exist_ok)
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 191, in copytree
    ignore_dangling_symlinks, dirs_exist_ok)
  File "/users/piccinal/reframe.git/reframe/utility/osext.py", line 164, in copytree
    ignore_dangling_symlinks)
  File "/usr/lib64/python3.6/shutil.py", line 315, in copytree
    names = os.listdir(src)
PermissionError: [Errno 13] Permission denied: '/apps/common/UES/reframe/resources/SPH-EXA_mini-app.git/.git'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/users/piccinal/reframe.git/reframe/frontend/executors/__init__.py", line 269, in _safe_call
    return fn(*args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 120, in _fn
    func(obj, *args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 134, in _wrapped
    return fn(*args, **kwargs)
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1208, in compile
    self.sourcesdir))
  File "/users/piccinal/reframe.git/reframe/core/pipeline.py", line 1159, in _copy_to_stagedir
    raise PipelineError('copying of files failed') from e
reframe.core.exceptions.PipelineError: copying of files failed: [Errno 13] Permission denied: '/apps/common/UES/reframe/resources/SPH-EXA_mini-app.git/.git'