mojo icon indicating copy to clipboard operation
mojo copied to clipboard

[Feature Request] add `FileHandle.write(self, data: List[UInt8])` and possibly define a serialization interface

Open martinvuyk opened this issue 9 months ago • 0 comments

Review Mojo's priorities

What is your request?

Besides adding the function mentioned in the title, I've been thinking of maybe defining a serialization and deserialization interface (trait) would make many later implementations easier (Json, XML, etc.). That way:

trait Serializable:
  fn to_bytes(self) -> List[UInt8]:
    ...


trait DeSerializable:
  @staticmethod
  fn from_bytes(data: List[UInt8])  ->  Self:
    ...


struct FileHandle:
  fn write[T: Serializable](self, data: T):
    self.write(data.to_bytes())

  fn write[T: Serializable, size: Int](self, data: List[T]):
    # this could be buffered if size is small and sent all at once though it would use more memory
    for item in data:
      self.write(item[].to_bytes())
      _ = self.seek(size, os.SEEK_CUR)

  fn read[T: DeSerializable, size: Int](self) -> T:
    return T.from_bytes(self.read_bytes(size))

  fn read[T: DeSerializable, size: Int, amnt_items: Int](self) -> List[T]:
    var data = self.read_bytes(amnt_items * size)
    var items = List[T](capacity=amnt_items)
    # could this be vectorized/parallelized according to size and or amnt_items?
    for i in range(amnt_items):
      items[i] = T.from_bytes(data[i * size:(i + 1) * size])
    return items

What is your motivation for this change?

Having a clearly defined serialization interface and adding it to all mayor stdlib types will make everything easier later on. And if List[UInt8] will represent every type for exchange with external systems it also helps to define interfaces for it, and there would be no need to make something like pickle.

Any other details?

No response

martinvuyk avatar May 11 '24 02:05 martinvuyk