cml icon indicating copy to clipboard operation
cml copied to clipboard

Bitbucket Pipelines — GPU support

Open 0x2b3bfa0 opened this issue 3 years ago • 2 comments

Continuation of https://github.com/iterative/cml/pull/798#pullrequestreview-982578801

As per the official documentation, self-hosted Bitbucket Pipelines runners for Linux don't “provide access to the host file system” although that limitation is “subject to change in the future”

Linux-only

Can’t access the local files on the runner host from a build script: In order to provide isolation and reproducibility, a runner doesn’t provide access directly to the host file system, we recommend using SCP / Rsync, if necessary.

The limits described above are subject to change in the future.

In practice, this means that every single step is containerized, and users don't have any control over container options like docker run --gpus, even when the NVIDIA runtime is installed.

  • https://github.com/iterative/cml/pull/798#issuecomment-1129155399
  • https://github.com/iterative/cml/pull/798#issuecomment-1131728075

0x2b3bfa0 avatar May 24 '22 10:05 0x2b3bfa0

Related

0x2b3bfa0 avatar May 24 '22 15:05 0x2b3bfa0

potential work-around: use (TP)I task (BB Pipelines -> TPI, exporting/forwarding BB env vars -> AWS EC2 -> user script && CML report). ~Likely depends on resolving https://github.com/iterative/terraform-provider-iterative/issues/630.~

moved to https://github.com/iterative/terraform-provider-iterative/issues/633

casperdcl avatar Jul 29 '22 06:07 casperdcl