xtdb icon indicating copy to clipboard operation
xtdb copied to clipboard

`open-tx-log` and bus tx events only contain `#xtdb/id`s

Open jarohen opened this issue 5 years ago • 2 comments

for example, [:crux.tx/delete #crux/id "b23c3d7b78bbe109a2b151d027423bf563124ef7"].

These aren't very helpful to users, who have no way of converting them back to entity IDs.

Caused by being a fairly unedited copy of the tx-log, which doesn't contain any entity IDs - these are sent in the doc-store, and only referenced from the tx-log, because they could potentially be PII.

If an entity is later evicted, then we may still have no choice but to return these hashes, but maybe we can do something more helpful in the meantime?

jarohen avatar Dec 23 '20 17:12 jarohen

Just for clarification @jarohen?

As far as I understand, the tx event #crux/id is the entity ID, not the content hash.

~1. Because it's :crux.db/id we have no way of converting it to a specific version of an entity 2. If instead it was crux.db/content-hash we're in other sorts of trouble, since :crux.db/content-hash is nil as output from crux.api/entity-tx.~

~Or am I missing something?~

~I need to be able to convert this #crux/id to a specific version of an entity, but for now I'll handle it elsewhere. My thoughts right now is to traverse the entity history (with-docs set) for the specific #crux/id, and then schedule the deletion at the times where the :crux.db/doc is nil. Quite tedious, but doable for now. Any better suggestions?~

[UPDATE]: I see now that the tx delete event, in addition to the #crux/id, also have the instant as the second param (just as the put event), in cases where the delete is in the future. I'm prettys sure I observed otherwise in an earlier release, hence the need for a custom way of getting the deleted version. But I don't need that now; all is good from my side.

luposlip avatar Mar 05 '21 21:03 luposlip

Given the changes coming in https://github.com/juxt/crux/pull/1535 it could be worth re-thinking what open-tx-log should offer. E.g. providing more information for hooking in subscription logic, as per https://github.com/jacobobryant/biff/issues/102

[UPDATE 2021-10-29] workaround function mentioned in the biff issue is now here https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/modules/lucene/src/xtdb/lucene.clj#L261-L273

refset avatar Jun 21 '21 13:06 refset