GPTQ-triton icon indicating copy to clipboard operation
GPTQ-triton copied to clipboard

Published 20 hours ago • verified publisher icon fpgaminer

Cache auto-tuning?

Open vedantroy opened this issue 1 year ago • 3 comments

When running the model--especially in a serverless environment where there may be many cold starts--it would be desirable to cache the auto-tuning results. Is this possible?

vedantroy avatar May 03 '23 02:05 vedantroy