horde-ad icon indicating copy to clipboard operation
horde-ad copied to clipboard

Ban numeric literals that result in ambiguous shape

Open Mikolaj opened this issue 1 year ago • 0 comments

For any rank higher than 0, literals such as 42 have ambiguous shape and in ranked tensors this ambiguity usually can't be resolved. Branch https://github.com/Mikolaj/horde-ad/tree/ban-ambiguous-shape bans such cases (actually rank 1 is there permitted, optimistically), but the checks are triggered by a large portion of our tests, probably rightly so. Tests are more important than correctness, so for now this check can't be enabled.

If we switch to shaped tensors, this is no longer a problem. Alternatively, we may try to extend horde-ad and orthotope with a mechanism similar to what hmatrix describes in its haddocks: "In most operations, single-element vectors and matrices (created from numeric literals or using scalar), and matrices with just one row or column, automatically expand to match the dimensions of the other operand".

It's possible horde-ad has some other ways of resolving the ambiguity that I don't remember, e.g., for orphan instances of ranked tensors. That would explain why the tests make sense. Perhaps these ways can be improved and extended to other layers of the horde-ad pipeline.

Edit: there are currently a couple of workarounds in place, e.g., dScaleByScalar.

Mikolaj avatar Mar 20 '23 19:03 Mikolaj