unitxt icon indicating copy to clipboard operation
unitxt copied to clipboard

Use of cache after changing card returns stale results

Open yoavkatz opened this issue 2 years ago • 0 comments

I changed a card (added a preprocessing step), but the dataset was loaded from cache:

07/16/2023 13:49:32 - WARNING - datasets.arrow_dataset - Loading cached processed dataset at /Users/yoavkatz/cache/huggingface/datasets/unitxt___data/card=cards.sst2_sentiment,template_item=0/1.1.1/161c975966d35694e0db488ca61993c4a4cfb44975f0fa25e6aac6dc3806b97f/cache-d2a30425e116067b.arrow

Need to include a hash of the card text in the cache key.

yoavkatz avatar Jul 16 '23 10:07 yoavkatz