piicatcher icon indicating copy to clipboard operation
piicatcher copied to clipboard

ZIP Type errors with Snowflake

Open guidefreitas opened this issue 10 months ago • 0 comments

Every time the detect process hit a ZIP code column it fails with this error:

Traceback (most recent call last):
  File "/Users/guilherme/.pyenv/versions/3.8.19/bin/piicatcher", line 8, in <module>
    sys.exit(app())
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/piicatcher/command_line.py", line 199, in detect
    op = scan_database(
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/piicatcher/api.py", line 63, in scan_database
    scan_sources(
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/dbcat/api.py", line 140, in scan_sources
    scanner.scan()
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/dbcat/catalog/db.py", line 173, in scan
    self._catalog.get_column(
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/dbcat/catalog/catalog.py", line 244, in get_column
    self._current_session.query(CatColumn)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3490, in one
    ret = self.one_or_none()
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
    ret = list(self)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
    cursor.close()
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.raise_(
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
    rows = [proc(row) for row in fetch]
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 579, in _instance
    _populate_full(
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
    dict_[key] = getter(row)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1283, in process
    return process_value(value, dialect)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/dbcat/catalog/models.py", line 278, in process_result_value
    value = PiiType.parse_raw(value)
  File "pydantic/main.py", line 549, in pydantic.main.BaseModel.parse_raw
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/dbcat/catalog/pii_types.py", line 17, in parse_obj
    return cls._convert_to_real_type_(obj)
  File "/Users/guilherme/.pyenv/versions/3.8.19/lib/python3.8/site-packages/dbcat/catalog/pii_types.py", line 33, in _convert_to_real_type_
    raise TypeError(f"Unsupported sub-type: {data_type}")
TypeError: Unsupported sub-type: zip_code

guidefreitas avatar Apr 17 '24 13:04 guidefreitas