datascript icon indicating copy to clipboard operation
datascript copied to clipboard

Support for Heterogenous tuples with :db/tupleTypes supporting :db.type/refs

Open arkRedM opened this issue 5 years ago • 8 comments

In datomic there is a way to combine data as

{:db/ident :x/y  
 :db/valueType :db.type/tuple
 :db/tupleTypes [:db.type/ref :db.type/string], 
 :db/cardinality :db.cardinality/one}` (or many).

I understand the types are more friendly in js but is there a way to support ref resolution? hacking around for this will make code less clean.

arkRedM avatar Aug 28 '20 09:08 arkRedM

Interesting! I haven’t thought about it. Yes this makes sense, thank you!

tonsky avatar Aug 28 '20 16:08 tonsky

Would be interested in this exact use case: storing a tuple [:db.type/ref :db.type/string]. Ideally, it would be :db.cardinality/many – hence, storing multiple tuples per attribute. Any plans to support this?

zeitstein avatar Oct 02 '21 12:10 zeitstein

I wouldn’t mind it. Care to provide a PR?

tonsky avatar Oct 04 '21 14:10 tonsky

Hey, I'm back to thinking about this. I'm no longer sure tuples are the best approach, so I would appreciate if you could provide some clarity.

My use case can be described in Clojure data as

{ ...
 :attr [{:ref1 -1, :ref2 -2, :val "string 1"} {:ref1 -3, :ref2 nil, :val "string 2"} ...]
 ...
}

My queries would target all combinations, e.g. querying for entities matching ref2 value, or both ref1 and val, etc.

Currently I use an intermediary entity, e.g:

:attr {:db/cardinality :db.cardinality/many, :db/valueType :db.type/ref, :db/isComponent true}

{:inter/ref1 -1, :inter/ref2 -2, :inter/val "string 1", :inter/order 0}

Which has the obvious downside of polluting indexes, but is nicely used in queries.

Alternative would be to store many 4-tuples [ref1 ref2 val order] on :attr. (So its heterogenous, not composite.) Given my query requirements above, would you say it would be worth pursuing the tuple idea (and the PR)? Thanks!

zeitstein avatar Feb 16 '22 19:02 zeitstein

Well it’s either all possible tuples or hash-join using queries. I’d probably prefer tuples

tonsky avatar Feb 17 '22 13:02 tonsky

I'd love to take a crack at this feature. Do you have any guidance as to how you would go about it, and/or any tips on understanding and navigating the code base?

seepel avatar Sep 10 '22 00:09 seepel

@seepel that would be fantastic! I don’t have many tips, but I’d probably start by listing all the places where having refs in tuples would make a difference (transact, query, pull, entity), then writing tests for each and finally finding the best place to implement the logic. Happy to accept partial PRs!

tonsky avatar Sep 14 '22 19:09 tonsky

That sounds like some solid advice, thank you!

I’ll admit, I started by greping for tupleAttrs and found myself a bit overwhelmed 😅

seepel avatar Sep 16 '22 05:09 seepel