kmath icon indicating copy to clipboard operation
kmath copied to clipboard

Move kmath-memory into a separate project

Open altavir opened this issue 4 years ago • 4 comments

Create a separate project for low-level multiplatform memory management. Let's call it memory.kt. Implement several memory backends:

  • ByteBuffer for JVM
  • WebGL buffers for JS
  • ByteArray for Native
  • JEP 393 foreign memory support for JVM (#83). Additional module with JDK 17 target.

The following features should be present:

  • Random access primitive read
  • Random access primitive write
  • Separate lifecycle handlers for readers and writers (control that only one writer could exist at a given time).
  • [Optional] Freeze access on borrow via coroutines like suspend fun Memory.borrow{ block: Memory.()->Unit} to pass it to other code or external process
  • [Optional] Kotlinx-serialization infrastructure to perform operations directly on memory (could be useful for ktor-io).

altavir avatar Nov 09 '21 10:11 altavir

As I see this is already implemented, is it? Thanks.

NorbertSandor avatar Jul 18 '22 09:07 NorbertSandor

No, it is not. @CommanderTvis started working on this, but It is not finished. I have some ideas, but it requires a lot of research to understand how it should work and what benefits it has. For example I have an idea about using a Buffer/Matrix/Tensor algebra based on manual memory layout with pooling/nogc implementation of the Memory. But sadly I have no time for it now. All ideas and contributions are welcome.

altavir avatar Jul 18 '22 09:07 altavir

@altavir Thanks, but the more I dive into the requirements of the "waiting for a hero" issues, the more I see that these are at another level :D

NorbertSandor avatar Jul 18 '22 09:07 NorbertSandor

Yep, it is a research library. It is requires a lot of thinking. For straight cases I would start with https://github.com/mipt-npm/kmath/issues/405 or https://github.com/mipt-npm/kmath/issues/66

altavir avatar Jul 18 '22 09:07 altavir