Error when trying to run feast with local provider and duckdb as offline store
Expected Behavior
When I try to run the python test_workflow.py of the quickstart with duckdb as the offline store, it should run without issues. The feature_store.yaml is configured as in the following example
project: my_local_fs_with_db
registry: data/registry.db
provider: local
offline_store:
type: duckdb
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
And I have a virtual environment with
feast[duckdb]
duckdb
pandas
ibis-framework[duckdb]
Current Behavior
However, I got the following error
Traceback (most recent call last):
File "/home/carrefour/Desktop/playground/feature-store/feast/my_local_fs_with_db/feature_repo/test_workflow.py", line 130, in <module>
run_demo()
File "/home/carrefour/Desktop/playground/feature-store/feast/my_local_fs_with_db/feature_repo/test_workflow.py", line 16, in run_demo
fetch_historical_features_entity_df(store, for_batch_scoring=False)
File "/home/carrefour/Desktop/playground/feature-store/feast/my_local_fs_with_db/feature_repo/test_workflow.py", line 86, in fetch_historical_features_entity_df
training_df = store.get_historical_features(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/feature_store.py", line 1161, in get_historical_features
job = provider.get_historical_features(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/passthrough_provider.py", line 459, in get_historical_features
job = self.offline_store.get_historical_features(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/offline_stores/duckdb.py", line 161, in get_historical_features
return get_historical_features_ibis(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/offline_stores/ibis.py", line 230, in get_historical_features_ibis
read_fv(feature_view, feature_refs, full_feature_names)
File "/home/carrefour/Desktop/playground/feature-store/feast/.venv/lib/python3.12/site-packages/feast/infra/offline_stores/ibis.py", line 192, in read_fv
fv_table = fv_table.mutate(
^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'mutate'
Steps to reproduce
Run the example of the quickstart with duckdb as the offline store, as shown above.
Specifications
- Version:
- Platform:
- Subsystem:
Possible Solution
This seems to happen because the _read_data_source of the offline_stores/duckdb.py tries to determine the file_format by comparing the data_source.file_format but its value is None and therefor the _read_data_source also returns None, causing the AttributeError described earlier. A possible solution would be to certify that the file_format of a data source object always has a valid value.
@rsoaresp file_format needs to be defined explicitly, You can make sure to specify the file_format in FileSource like :
from feast.data_format import ParquetFormat
driver_stats_source = FileSource(
name="driver_hourly_stats_source",
path="data/driver_stats.parquet",
timestamp_field="event_timestamp",
file_format=ParquetFormat(),
created_timestamp_column="created",
)
https://docs.feast.dev/reference/data-sources/file