moose
moose copied to clipboard
Pass values by reference instead of clone
Kernel functions currently take ownership of Value
s, which means we clone these every time they are sent between operations. This is wasteful in several ways:
- if a value is only flowing to a single operation so ownership could simply be moved
- operators do not generally mutate their inputs and would work just as well with borrowed values
- certain operators, such as
HostRorateRightOp
andHostIndexAxisOp
, are inherently just projecting their inputs
A likely solution to all three would be to pass Arc
s around instead, which can already be freely shared between tasks. Note that there are also specialized structs, such as ndarray::ArcArray
, that we could consider using directly. It would also be interesting to consider support for passing around ndarray::ArrayView
for projections.
cc @voronaam just to make you aware about this one; some of the ops in https://github.com/tf-encrypted/runtime/pull/508 would likely benefit non-trivially from this since a lot of projection operations are being used
This was mostly done for AsyncSession
and SyncSession
by switching to ArcArray
, but something similar could be done for SymbolicSession
to potentially reduce Moose compilation time