cachew icon indicating copy to clipboard operation
cachew copied to clipboard

keep full schema in cachew instead of just outer type

Open karlicoss opened this issue 2 years ago • 0 comments

After recent google_takeout_parser update getting this:

...
  File "/home/hpi/.local/lib/python3.12/site-packages/cachew/marshall/cachew.py", line 143, in load
    tidx, val = dct
                └ 62.81359

TypeError: cannot unpack non-iterable float object

Seems that this is because we switched some PlaceVisit fields to Optional -- so now it tries to unpack a float as a Union (which Optional is a special case of). Normally cachew would invalidate the cache in this case (since we keep the previous schema in the database) https://github.com/karlicoss/cachew/blob/2abd77b9ab0b5379809f3e6d5b469873a79d9548/src/cachew/init.py#L605 However seems that it is

 "schema": "typing.Union[google_takeout_parser.models.PlaceVisit, Exception]"`

, so if any of fields of PlaceVisit changed, this wouldn't have any impact.

I think we need to dump the datatype recursively in schema (with all field names etc -- doesn't hurt?) instead.

In the meantime for google_takeout_parser should be solvable by bumping google_takeout_version since it's also included in the cache key. cc @seanbreckenridge just in case you encounter this

karlicoss avatar Nov 08 '23 21:11 karlicoss