Snowflake Registry does not work

[qianyao11]

Expected Behavior

when setting registry_type: snowflake.registry, i would expect it creates corresponding tables in specified snowflake database, and add defined entities / views etc. to the tables. https://docs.feast.dev/reference/registries/snowflake registry: registry_type: snowflake.registry account: snowflake_deployment.us-east-1 user: user_login password: user_password role: SYSADMIN warehouse: COMPUTE_WH database: FEAST schema: PUBLIC cache_ttl_seconds: 60

Current Behavior

when run feast apply, i get error as below when creating snowflake registry tables (all tables are created already though, seems like there is an extra ':' at the end of the provided .sql receipt for snowflake registry ):

(test_feast) macuser@Mac test_feast % feast apply

/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/pydantic/_internal/_fields.py:198: UserWarning: Field name "vector_enabled" in "SqliteOnlineStoreConfig" shadows an attribute in parent "VectorStoreConfig"
  warnings.warn(
/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/pydantic/_internal/_fields.py:198: UserWarning: Field name "vector_len" in "SqliteOnlineStoreConfig" shadows an attribute in parent "VectorStoreConfig"
  warnings.warn(
/Users/macuser/Desktop/Pilot/test_feast/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
  driver = Entity(name="driver", join_keys=["driver_id"])
**Traceback** (most recent call last):
  File "/Users/macuser/miniconda3/envs/test_feast/bin/feast", line 8, in <module>
    sys.exit(cli())
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/cli.py", line 778, in apply_total_command
    apply_total(repo_config, repo, skip_source_validation)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/repo_operations.py", line 396, in apply_total
    store, registry = _get_store_and_registry(repo_config)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/repo_operations.py", line 274, in _get_store_and_registry
    store = FeatureStore(config=repo_config)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/feature_store.py", line 160, in __init__
    self._registry = SnowflakeRegistry(
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/infra/registry/snowflake.py", line 146, in __init__
    execute_snowflake_statement(conn, query)
  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/infra/utils/snowflake/snowflake_utils.py", line 128, in execute_snowflake_statement
    raise SnowflakeQueryUnknownError(query)
feast.errors.SnowflakeQueryUnknownError: Snowflake query failed: 

Steps to reproduce

conda create --name testenv
conda install python==3.10
pip install 'feast[snowflake]'
feast init
cd 'project_name'
feast apply

Specifications

• Version:

Python 3.10.16
feast                      0.47.0

• Platform: got same issue on MacBook and windows.
• Subsystem:

Possible Solution

if i manually remove the ':' in feast lib in my local environment for file feast/infra/utils/snowflake/registry/snowflake_table_creation.sql, there will be a different error for feast apply:

  File "/Users/macuser/miniconda3/envs/test_feast/lib/python3.10/site-packages/feast/infra/registry/snowflake.py", line 176, in _sync_feast_metadata_to_projects_table
    feast_metadata_projects.add(row[1]["PROJECT_ID"])
AttributeError: 'list' object has no attribute 'add'

if i change line 166 and 167 in infra.registry.snowflake.py into

    def _sync_feast_metadata_to_projects_table(self):
        feast_metadata_projects: set = set()
        projects_set: set = set()

it will give me a new error:

KeyError: 'project_proto'

if i change line 1360 in infra.registry.snowflake.py to ProjectProto.FromString(row[1]["PROJECT_PROTO"]) instead of ProjectProto.FromString(row[1]["project_proto"]) will get a different error AttributeError: 'SnowflakeRegistry' object has no attribute '_refresh_lock'

Then i stop trying.

Am i using the snowflake registry in the right way? and guidance is appreciated.

[FabioTomaz]

I am also facing the same issue. Any updates regarding this?

[hariupwork]

it is because, the table creation DDLs are from a file which are seperated by ; and in code splitting them by ; will leave you with an empty query in the last that's what giving you error, you either iterate through sql_cmds[:-1] or add a try catch block that should work https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/registry/snowflake.py#L139-L146

[michaelneely]

I'm getting the same sequence of errors. What I tried:

1. add sql_cmds[:-1] to line 144
2. change [] to set() on lines 166, 167
3. change ProjectProto.FromString(row[1]["project_proto"]) to ProjectProto.FromString(row[1]["PROJECT_PROTO"]) on line 1360
4. move the initialisation of _refesh_lock up before the initialisation of cached_registry_proto
5. Set cached_registry_proto to None in the init

Seems to work, but not sure if that's the correct approach.