unitxt
unitxt copied to clipboard
Use of cache after changing card returns stale results
I changed a card (added a preprocessing step), but the dataset was loaded from cache:
07/16/2023 13:49:32 - WARNING - datasets.arrow_dataset - Loading cached processed dataset at /Users/yoavkatz/cache/huggingface/datasets/unitxt___data/card=cards.sst2_sentiment,template_item=0/1.1.1/161c975966d35694e0db488ca61993c4a4cfb44975f0fa25e6aac6dc3806b97f/cache-d2a30425e116067b.arrow
Need to include a hash of the card text in the cache key.