golem icon indicating copy to clipboard operation
golem copied to clipboard

Rib Robustness: Make sure the type building using AllOf and OneOf isn't giving memory pressure.

Open afsalthaj opened this issue 5 months ago • 1 comments

After multiple scanning for type inference going on, we keep building the types using AllOf/OneOf. For example: it looks AllOf(all_possibilities_of_record) is looking pretty huge!

Recently (first week of Sept), we made sure we are not storing duplicate types, and there are no logical errors in this side. Yet I believe we still have a considerable memory footprint that we need to be cautious about. I will be doing a benchmark before we plan the solution for this ticket. May be this is not an immediate issue, yet placing a ticket to track it down later.

There are various ways to handle this building of large data structures representing types.

  1. Maintaining a table instead of attaching the inferred type to every identifier as an optimisation. Expressions may not repeat, but identifiers do!
  2. Pre-unification strategies instead of accumulating types that could have gone through an obvious unification. Example: No need to keep this towards the end AllOf( Result { ok: Some(U64), error: Some(Unknown) }, Result { ok: Some(U64), error: Some(Str) }). We could come up with a progressive type elimination to keep the memory footprint low
  3. And a lot more...

Or may be there is a known theoretical way of doing this, and need discussions with @jdegoes .

afsalthaj avatar Sep 12 '24 03:09 afsalthaj