picos icon indicating copy to clipboard operation
picos copied to clipboard

Use an integer has an identifier for fiber instead of `==`

Open dinosaure opened this issue 9 months ago • 3 comments

Concerning some parts of Picos, it's required to know which is our current fiber to compare with some others (like for mutex and condition). It requires that Fiber must be used to be able to use Lazy, Mutex or Condition. However, only a comparison is needed. To give an opportunity for someone else to implement its own fiber but be able to use these modules, it can be interesting to define an effect such as type _ Effect.t += Current : int Effect.t which returns an unique identifier of the current fiber. By this way, the scheduler implementor just need to provide such unique identifier instead of a Fiber.t.

dinosaure avatar Apr 25 '24 08:04 dinosaure

Would the use of FLS be sufficient?

module Picos_fiber_id : sig
  val fiber_id : unit -> int
  (** [fiber_id ()] returns a unique integer id for the current fiber. *)
end = struct
  let fiber_id_key =
    let next = Atomic.make 0 in
    Fiber.FLS.new_key
    @@ Computed (fun () ->
        Atomic.fetch_and_add next 1)

  let fiber_id () =
    Fiber.FLS.get (Fiber.current ()) fiber_id_key
end

We could put this into a library for Picos.

polytypic avatar Apr 25 '24 20:04 polytypic

Would the use of FLS be sufficient?

My idea is precisely to have an independent Current effect of the Fiber.t type. As it happens, as far as Miou is concerned, I define my own Current effect and, from what I've been able to see in picos.sync, the interest of the Current effect lies solely in its ability to discriminate the given fiber from others and modify the forbid value.

This could be broken down into:

  1. an effect that gives a unique identifier that picos or another scheduler can transmit (@fabbing's idea of using the unique identifier of fibers present in the stack is a good idea)
  2. an effect to modify the forbid of the fiber

This would allow picos.sync to depend on certain elements of Fiber.t but to leave it to the discretion of the implementer to propose his own Fiber.t (as Miou does) while offering a certain compatibility with picos.sync.

In this case, I'd like picos.sync to be able to work with Miou without depending on Fiber.t.

dinosaure avatar Apr 26 '24 08:04 dinosaure

If I understand correctly what you are after then this is something I also thought about a lot while designing the Fiber concept. Yes, I agree, it would be nice to be able to leave the representation of the "per fiber state" abstract. I recall talking about this in some meetings. Unfortunately I am sceptical that any of the ways to do that (e.g. objects, modules, ... effects) would be better (faster, taking less memory, simpler, easier, ...) in practice than the current design (I suspect most other ways will be the opposite: slower, take more memory, more complicated, more difficult to work with or more difficult for a scheduler to provide).

The idea with the current design is to define a "fiber" concept that should not be too costly for a scheduler to maintain as part of their per fiber state.

Nothing prevents a scheduler from having more per fiber state. For example, Picos_threaded has this record type for per fiber state: https://github.com/ocaml-multicore/picos/blob/534944eea85f7e7845122b44a291c218522447d4/lib/picos_threaded/picos_threaded.ml#L3

and the current operation returns the fiber field from the per fiber state record: https://github.com/ocaml-multicore/picos/blob/534944eea85f7e7845122b44a291c218522447d4/lib/picos_threaded/picos_threaded.ml#L52-L56

Would this not work in your case (i.e. define your own per fiber state record with Picos.Fiber.t as one element of it)?

Note that a Picos compatible scheduler should not only allow one to perform Current to get the Fiber.t, but the scheduler should also respect whatever state changes are then then done through the Fiber.t object.

An advantage of having a concrete Fiber.t type is that it can be manipulated efficiently once obtained. Another advantage is that Current is just one effect rather than multiple effects Forbid, Get_computation, ...

Integer ids are problematic. For example, I would much rather have Thread.TLS than Thread.id (I'm referring to the systhreads of OCaml), because I could easily and efficiently implement Thread.id with Thread.TLS, but implementing Thread.TLS with Thread.id is far more involved. You need some sort of mapping from ids to state. That will likely mean one or two orders of magnitude slower accesses to such state. And then you will need to manage such mappings (create entries, remove entries). I would rather avoid that and do not wish that upon anyone.

Note that Picos_sync is just one tiny library implemented in Picos. I'm currently working on finishing #49 where the Bundle concept will need to manipulate both the forbid flag and the computations associated with a fiber. Note that PR #49 also makes it possible to change the computation associated with a fiber (i.e Fiber.set_computation).

polytypic avatar Apr 26 '24 17:04 polytypic