feast icon indicating copy to clipboard operation
feast copied to clipboard

Add `remote-http` online store

Open tokoko opened this issue 4 months ago • 10 comments

Is your feature request related to a problem? Please describe. Users can deploy http and grpc feature servers that wrap get_online_features calls, but currently there is no way to interact with feature servers from python sdk.

Describe the solution you'd like Add online store types remote-http and remote-grpc (or maybe just call it a remote and differentiate with a url?) that wrap network calls to the respective feature servers. This follows the same pattern as remote registry implementation that wraps calls to the registry server.

tokoko avatar Apr 19 '24 17:04 tokoko

I am currently working on this and will get back in case if i find any difficulty.

lokeshrangineni avatar May 02 '24 19:05 lokeshrangineni

@franciscojavierarceo FYI

lokeshrangineni avatar May 03 '24 16:05 lokeshrangineni

Some things we'll have to do:

~~1. Handle purely feature_views i.e., features="feature_view"~~ 2. Handle feature references; i.e., ["feature_view:feature_1","feature_view:feature_2",...] etc. 3. Handle feature services

We'll have to serialize the data on the server to match what's in the feature registry from the FeatureView itself and fail loudly when it doesn't match, probably return a 400.

franciscojavierarceo avatar May 03 '24 16:05 franciscojavierarceo

@franciscojavierarceo I think you closed this by accident.

  • Handle purely feature_views i.e., features="feature_view"

Is this something we support? The signature for get_online_features is only feature service or a list of features -> features: Union[List[str], FeatureService]

tokoko avatar May 03 '24 16:05 tokoko

Whoops, nevermind that was based on outdated documentation I think.

franciscojavierarceo avatar May 03 '24 16:05 franciscojavierarceo

@tokoko I'm confused about why we would do this? I think adding a client that makes the behavior intuitive makes sense but that's basically in the Python feature server.

franciscojavierarceo avatar May 03 '24 16:05 franciscojavierarceo

Well, it's more of convenience thing, i think. They can still use the feature server directly of course, but what this additional layer would enable is writing a piece of code using store.get_online_features and then configuring how you want to resolve features with feature_store.yaml. You could change the backend from the actual online store to http feature server or a grpc one without changing anything in the code itself.

tokoko avatar May 03 '24 16:05 tokoko

Few updates after started working on the issue:

  1. There is no grpc implementation of APIs in python feature server. So this issue is going to only focus on remote with http implementation. We can enhance the online store to include grpc implementation when the grpc apis available on the python feature server.
  2. Planning to implement the online store api methods of online_write_batch and online_read. Rest all life cycle methods of online_store are not having associated APIs in the feature server.

lokeshrangineni avatar May 15 '24 15:05 lokeshrangineni

@tokoko can you confirm the above assumptions, in order to guide @lokeshrangineni 's implementation?

dmartinol avatar May 15 '24 19:05 dmartinol

sure, sounds like a good starting point.

tokoko avatar May 15 '24 19:05 tokoko