dfdx icon indicating copy to clipboard operation
dfdx copied to clipboard

Allow TensorCache to return allocations that are bigger than necessary

Open coreylowman opened this issue 1 year ago • 0 comments

Related to #672

Tensors could be backed by allocations that have more space than necessary. BTreeMap already has a method to return keys within a certain range that would make this trivial: https://doc.rust-lang.org/std/collections/struct.BTreeMap.html#method.range.

The main thing would be:

  1. Adding a physical numel field to cuda storage, because the length of the slice won't necessarily be the size of data actually stored in it.
  2. Checking if this actually improves speed/reduces num allocations needed.

coreylowman avatar Apr 21 '23 20:04 coreylowman