Can't pull HPOBench container

[Game4Move78]

>>> from hpobench.container.benchmarks.nas.tabular_benchmarks import SliceLocalizationBenchmark
>>> bm = SliceLocalizationBenchmark()
[INFO] BenchmarkClient at 2023-03-15 12:35:03,339 --- Start downloading the container tabular_benchmarks_0.0.5 from oras://gitlab.tf.uni-freiburg.de:5050/muelleph/hpobench-registry. This may take several minutes.
INFO[0000] trying next host - response was http.StatusNotFound  host="gitlab.tf.uni-freiburg.de:5050"
FATAL:   While pulling image from oci registry: error fetching image to cache: failed to get checksum for oras://gitlab.tf.uni-freiburg.de:5050/muelleph/hpobench-registry/tabular_benchmarks/0.0.5:latest: while resolving reference: gitlab.tf.uni-freiburg.de:5050/muelleph/hpobench-registry/tabular_benchmarks/0.0.5:latest: not found
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/patrick/HPOBench/hpobench/container/benchmarks/nas/tabular_benchmarks.py", line 15, in __init__
    super(SliceLocalizationBenchmark, self).__init__(**kwargs)
  File "/home/patrick/HPOBench/hpobench/container/client_abstract_benchmark.py", line 129, in __init__
    self.load_benchmark(benchmark_name=benchmark_name, container_name=container_name,
  File "/home/patrick/HPOBench/hpobench/container/client_abstract_benchmark.py", line 238, in load_benchmark
    download_container(self.container_dir, container_name, self.container_source, container_tag)
  File "/home/patrick/.virtualenvs/multifidelity/lib/python3.8/site-packages/oslo_concurrency/lockutils.py", line 414, in inner
    return f(*args, **kwargs)
  File "/home/patrick/HPOBench/hpobench/container/client_abstract_benchmark.py", line 233, in download_container
    subprocess.run(cmd, shell=True, check=True)
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'singularity pull --dir /home/patrick/.cache/hpobench/hpobench-1003 --name tabular_benchmarks_0.0.5 oras://gitlab.tf.uni-freiburg.de:5050/muelleph/hpobench-registry/tabular_benchmarks/0.0.5:latest' returned non-zero exit status 255.

I run into this error when downloading the nas SliceLocalization container.

• HPOBench version: 0.011dev (commit 236e54262fd11a288b8b501f5e1da923e421574c)
• Singularity version: singularity-ce version 3.10.0

[Neeratyoy]

Hi,

Could you try again now and let us know if it works?

May I ask why the dev version, that is, if it is something specific you need that is not on master?

[Game4Move78]

Now it works, thanks! The very hierarchical YAHPO benchmark suite is not on master. Also I believe pulling containers on the master branch was broken some time in the last month, causing me to make the switch.

[Neeratyoy]

I see, thanks for pointing it out. It does look like the set of containers for masters are not how it should be. We are looking to update them as soon as possible.

The dev of course is volatile and is not recommended for use. If the containers don't work, please report here (like you did) and maybe try the non-container version from the master.

In any case, glad you found a workaround for now. Will update here when the containers are up.