palladium icon indicating copy to clipboard operation
palladium copied to clipboard

Additional persisters

Open ottonemo opened this issue 5 years ago • 2 comments

I have written an S3 persister (FileIO and FileLike) based on s3fs and thought about making a PR (if you are interested) but I assume that you probably don't want a direct dependency to packages such as s3fs and boto3. Two questions:

  • are you interested in a PR at all
  • if so, how to integrate the S3 persister without cluttering the dependencies

ottonemo avatar Jul 23 '19 06:07 ottonemo

I believe it would make sense as completely optional dependency.

BenjaminBossan avatar Jul 27 '19 09:07 BenjaminBossan

We're already using setuptools extras for our optional dependencies to julia, rpy2, and others. I like the idea of adding a S3 persister to Palladium. We can even put it into persistence.py, as long as the case of not being able to import these dependencies is handled gracefully.

dnouri avatar Aug 26 '19 18:08 dnouri