The server sends and the client stores incorrect `observed_transaction_data`

[alco]

When the sync service sends SatOpAdditionalBegin and SatOpAdditionalCommit ops, it includes the so-called move_in_ref in their payload. The client stores additional data refs into its local seenAdditionalData meta row, as a comma-separated list of integeres encoded into a string:

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/clients/typescript/src/satellite/process.ts#L1409-L1416

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/clients/typescript/src/satellite/process.ts#L1460-L1470

This seenAdditionalData is later fetched, parsed, and included in the SatInStartReplicationReq message's observed_transaction_data field:

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/clients/typescript/src/satellite/process.ts#L879-L889

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/clients/typescript/src/satellite/client.ts#L375-L379

When the sync service processes that message, it expects the observed_transaction_data field to contain a list of transaction IDs, though:

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/components/electric/lib/electric/satellite/protocol.ex#L1141

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/components/electric/lib/electric/satellite/client_reconnection_info.ex#L320

[linear[bot]]

VAX-1818 The server sends and the client stores incorrect `observed_transaction_data`

[alco]

The same problem is present in the SatOpLogAck message. The protocol definition for this message defines the additional_data_source_ids field:

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/protocol/satellite.proto#L245-L246

However, on the client, move_in_refs from SatOpAdditionalCommit messages are used to popoulate this field:

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/clients/typescript/src/satellite/client.ts#L1057-L1070

https://github.com/electric-sql/electric/blob/0115a0a345c86b1677b997a79f4325ab1ed32b80/clients/typescript/src/satellite/client.ts#L1216-L1217

[KyleAMathews]

👋 we've been working the last month on a rebuild of the Electric server over at a temporary repo https://github.com/electric-sql/electric-next/

You can read more about why we made the decision at https://next.electric-sql.com/about

We're really excited about all the new possibilities the new server brings and we hope you'll check it out soon and give us your feedback.

We're now moving the temporary repo back here. As part of that migration we're closing all the old issues and PRs. We really appreciate you taking the time to investigate and report this issue!