feast icon indicating copy to clipboard operation
feast copied to clipboard

feast support for aws and gcp at the sametime

Open boumelhaa opened this issue 1 year ago • 3 comments

I am encountering errors while attempting to install Feast. The intended configuration involves utilizing BigQuery as the offline store and DynamoDB as the online store. My question is: is it feasible to use BigQuery as the offline store and DynamoDB as the online store, considering that our serving infrastructure is on AWS while our data platform is on GCP? I am curious about the possibility of this setup and would appreciate guidance on how to implement it.

boumelhaa avatar Jan 15 '24 08:01 boumelhaa

do you mind to try "pip install feast[aws, gcp]. and define the feature_store.yaml accordingly as your current setup.

shuchu avatar Jan 15 '24 16:01 shuchu

thanks, I did try it but the problem is, it is taking an infinite time to resolve the deps until timeout. I will try spinning a new environment and test it again.

this is how my py env looks like

python = ">=3.8,<3.12" google-cloud-bigquery = { version = ">=2.16.0" } dbt-bigquery = { version = ">=1.5.6, <1.6.0"} dbt-core = { version = ">=1.5.6, <1.6.0"} click = { version = ">=8.0.0", optional = true } typer = { version = ">=0.6.1", optional = true } PyYAML = { version = ">=5.4.1", optional = true } pydantic = { version = ">=1.8.2", optional = true } python-geohash = { version = ">=0.8.5", optional = true } scipy = { version = ">=1.6.0", optional = true } orjson = "^3.9.1"

boumelhaa avatar Jan 16 '24 07:01 boumelhaa

Looks like now it works fine, you can do pip install feast[aws, gcp] and it works correctly, if not on the first try then on the second

tmihalac avatar Feb 15 '24 15:02 tmihalac

@boumelhaa in attempt to reproduce, I was also able to carry out pip install for the dual providers as suggested so I'd like to consider this issue as resolved. That said, please do let us know here if you continue to encounter an issue so we can assist or re-open as appropriate. Thanks!

jeremyary avatar Feb 27 '24 18:02 jeremyary