aici icon indicating copy to clipboard operation
aici copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

implement separate stream for memory copy

Open mmoskal opened this issue 1 year ago • 1 comments

While at it, also measure mem transfer speed and see how many KV entries can be transferred in a single inference round

mmoskal avatar Dec 22 '23 08:12 mmoskal

streams/events are now implemented but not tested

mmoskal avatar Jan 11 '24 23:01 mmoskal