feast icon indicating copy to clipboard operation
feast copied to clipboard

/push API partial writes

Open rogerwu-qz opened this issue 7 months ago • 3 comments

Is your feature request related to a problem? Please describe. Is it possible to have the /push API accept a payload of a subset of the fields of a defined feature? Or does the need for this imply that the feature is not correctly modeled abstractly?

Describe the solution you'd like Our use case of feast would like to see the ability to group fields under a single feature but allow for partial subsets of fields to be updated at a time.

rogerwu-qz avatar May 19 '25 23:05 rogerwu-qz

Have you tried to split your feature view into multiple feature views to handle this?

franciscojavierarceo avatar May 20 '25 11:05 franciscojavierarceo

Hi @franciscojavierarceo I am having same use case and I tried to split feature view into multiple feature views but getting error as we have a single bigquery table with entity and 5 features, 4 features were populated using batch and 1 feature was populated by streaming. We had two feature views defined one for Batch and other for Push. The streaming feature has the message (entity, featurename, value) in this format. When we update one streaming feature we are seeing that write to offline store is still expecting the whole schema. "'Field "COUNT_X_EVENTS" does not exist in schema'" I tried passing None but values was overridden to null.

Deshmukh1992 avatar May 27 '25 09:05 Deshmukh1992

yeah we do full writes to a single table so you'd have to partition your table to be able to write at the moment.

franciscojavierarceo avatar Sep 10 '25 03:09 franciscojavierarceo