feast icon indicating copy to clipboard operation
feast copied to clipboard

Error when trying to run feast with local provider and duckdb as offline store

Open rsoaresp opened this issue 7 months ago • 1 comments

Expected Behavior

When I try to run the python test_workflow.py of the quickstart with duckdb as the offline store, it should run without issues. The feature_store.yaml is configured as in the following example

project: my_local_fs_with_db
registry: data/registry.db
provider: local
offline_store:
  type: duckdb
online_store:
    type: sqlite
    path: data/online_store.db
entity_key_serialization_version: 2

And I have a virtual environment with

feast[duckdb]
duckdb
pandas 
ibis-framework[duckdb]

Current Behavior

However, I got the following error

Traceback (most recent call last):
  File "/home/carrefour/Desktop/playground/feature-store/feast/my_local_fs_with_db/feature_repo/test_workflow.py", line 130, in <module>
    run_demo()
  File "/home/carrefour/Desktop/playground/feature-store/feast/my_local_fs_with_db/feature_repo/test_workflow.py", line 16, in run_demo
    fetch_historical_features_entity_df(store, for_batch_scoring=False)
  File "/home/carrefour/Desktop/playground/feature-store/feast/my_local_fs_with_db/feature_repo/test_workflow.py", line 86, in fetch_historical_features_entity_df
    training_df = store.get_historical_features(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/feature_store.py", line 1161, in get_historical_features
    job = provider.get_historical_features(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/passthrough_provider.py", line 459, in get_historical_features
    job = self.offline_store.get_historical_features(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/offline_stores/duckdb.py", line 161, in get_historical_features
    return get_historical_features_ibis(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/offline_stores/ibis.py", line 230, in get_historical_features_ibis
    read_fv(feature_view, feature_refs, full_feature_names)
  File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/offline_stores/ibis.py", line 192, in read_fv
    fv_table = fv_table.mutate(
               ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'mutate'

Steps to reproduce

Run the example of the quickstart with duckdb as the offline store, as shown above.

Specifications

  • Version:
  • Platform:
  • Subsystem:

Possible Solution

This seems to happen because the _read_data_source of the offline_stores/duckdb.py tries to determine the file_format by comparing the data_source.file_format but its value is None and therefor the _read_data_source also returns None, causing the AttributeError described earlier. A possible solution would be to certify that the file_format of a data source object always has a valid value.

rsoaresp avatar May 27 '25 00:05 rsoaresp

@rsoaresp file_format needs to be defined explicitly, You can make sure to specify the file_format in FileSource like :

from feast.data_format import ParquetFormat
driver_stats_source = FileSource(
    name="driver_hourly_stats_source",
    path="data/driver_stats.parquet",
    timestamp_field="event_timestamp",
    file_format=ParquetFormat(),
    created_timestamp_column="created",
)

https://docs.feast.dev/reference/data-sources/file

ntkathole avatar May 27 '25 05:05 ntkathole