Serialization issue with PickledDB

[bouthilx]

This may be a problem specific to how HEBO's state is saved, but in any case the algorithm's state should be pickled and store as a binary object in the DB. When unpickling the DB the algorithm's state should be unpickled as a binary object, which then needs to be fetch from the DB and unpickled within the storage method.

Traceback (most recent call last):
  File "/home/notoraptor/anaconda3/envs/orion/bin/orion", line 33, in <module>
    sys.exit(load_entry_point('orion', 'console_scripts', 'orion')())
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/cli/__init__.py", line 37, in main
    return orion_parser.execute(argv)
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/cli/base.py", line 93, in execute
    returncode = function(args)
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/cli/serve.py", line 43, in main
    web_api = WebApi(config)
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/serving/webapi.py", line 111, in __init__
    setup_storage(config.get("storage"))
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/storage/base.py", line 579, in setup_storage
    storage_factory.create(of_type=storage_type, **storage)
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/utils/singleton.py", line 139, in create
    self.instance = super(GenericSingletonFactory, self).create(
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/utils/__init__.py", line 130, in create
    return constructor(*args, **kwargs)
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/storage/legacy.py", line 97, in __init__
    self._setup_db()
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/storage/legacy.py", line 101, in _setup_db
    if backward.db_is_outdated(self._db):
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/utils/backward.py", line 87, in db_is_outdated
    index_information = database.index_information("experiments")
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/io/database/pickleddb.py", line 135, in index_information
    with self.locked_database(write=False) as database:
  File "/home/notoraptor/anaconda3/envs/orion/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/io/database/pickleddb.py", line 237, in locked_database
    database = self._get_database()
  File "/media/win/Users/notoraptor/mila/dev/git/mila-iqia/orion/src/orion/core/io/database/pickleddb.py", line 203, in _get_database
    database = pickle.loads(data)
ModuleNotFoundError: No module named 'orion.algo.HEBO'

This happened while attempting to load a PickledDB containing a benchmark that ran HEBO. The loading was done in an environment where HEBO was not installed.

[lebrice]

Was Orion installed in editable mode, in the environment where this is run? This would explain the "no module named orion.algo.hebo" error. Otherwise, I'm not sure I understand what's going on..

[bouthilx]

@notoraptor Do you remember if Oríon was installed in editable mode?