electric icon indicating copy to clipboard operation
electric copied to clipboard

Elixir client `Message.from_message/2` eats change info.

Open thruflo opened this issue 7 months ago • 1 comments

When consuming a shape in an Elixir application using the Electric.Client, the parsing of raw messages into Electric.Client.Message.ChangeMessage structs loses the information about which fields have changed.

To reproduce, make a surgical update, e.g.: just updating the name on a schema with additional fields:

Threads.change_thread(thread, %{name: "some other name"})

This results in a raw message like this:

%{
  "headers" => %{
    "last" => true,
    "lsn" => "786675360",
    "op_position" => 0,
    "operation" => "update",
    "relation" => ["public", "threads"],
    "txids" => [39380]
  },
  "key" => "\"public\".\"threads\"/\"c946e731-ddd7-4331-8147-6a159fede21f\"",
  "value" => %{
    "id" => "c946e731-ddd7-4331-8147-6a159fede21f",
    "name" => "some other name"
  }
}

This is processed by ChangeMessage.from_message (using Electric.Client.EctoAdapter.for_schema/2) into:

%Electric.Client.Message.ChangeMessage{
  key: "\"public\".\"threads\"/\"c946e731-ddd7-4331-8147-6a159fede21f\"",
  value: %MyApp.Threads.Thread{
    __meta__: #Ecto.Schema.Metadata<:built, "threads">,
    id: "c946e731-ddd7-4331-8147-6a159fede21f",
    name: "some other name",
    status: nil,
    inserted_at: nil,
    updated_at: nil
  },
  old_value: nil,
  headers: %Electric.Client.Message.Headers{
    operation: :update,
    relation: ["public", "threads"],
    handle: "8728556-1749895640933410"
  },
  request_timestamp: ~U[2025-06-14 10:07:20.967291Z]
}

Note the addition of status: nil, inserted_at: nil, updated_at: nil. It's impossible to know whether these values were set to nil by the change or were not set.

It is possible to workaround this by setting ?replica=full to get the old_value and then figuring out the changes. However, this seems unnecessary given that the information is present in the raw message already.

thruflo avatar Jun 14 '25 10:06 thruflo

@magnetised Any advice on how ti address this?

alco avatar Aug 18 '25 10:08 alco