include-pk can't work with identity full?

[yichao-figma]

Context:

• We need replica identity = full because we need to get the full record value (all columns) even for partial update
• We also would like to use include-pk option to get the primary key

However we noticed that these two does NOT work together starting from wal2json v2.5 (It was working on v2.3). Looking at the history, I noticed this commit made the behavioral change: https://github.com/eulerto/wal2json/commit/8139a6dbf1d78e62d44ab8ef0cddf186ffcc2c4e From this change it seems that include-pk only works when identity = default. Though it's not very clear from the commit description regarding why this limitation was added.

cc @eulerto please let me know if my understanding is incorrect and / or if there are ways to have both "identity = full" and primary key.

[eulerto]

@yichao-figma Sorry about the late reaction. Are you using format 1, right? I can reproduce it with the following test case.

 INSERT INTO w2j_rename_ri (g, b, c, f, e) VALUES(790, '2020-04-04 10:34:55', 'Myrmecophaga tridactyla', false, 1.8);
 UPDATE w2j_rename_ri SET e = 3.1415 WHERE g = 456;
 COMMIT;
+ALTER TABLE w2j_rename_pk REPLICA IDENTITY FULL;
+INSERT INTO w2j_rename_pk (g, b, c, f, e) VALUES(890, '2023-10-31 03:06:00', 'Crypturellus parvirostris', true, 8.90);
+UPDATE w2j_rename_pk SET e = 8.91 WHERE g = 890;
+DELETE FROM w2j_rename_pk WHERE g = 890;
 SELECT data FROM pg_logical_slot_peek_changes('regression_slot', NULL, NULL, 'format-version', '1', 'pretty-print', '1', 'include-typmod', '0', 'include-pk', '1');

[yichao-figma]

Sorry for the slow reply.

Yes the original format.

I looked into the implementation by searching include_pk in the code, almost all of them come with && relation->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT (example), which aligns with our observation (that include_pk only works with RI=Default)

In the repro you had above, it doesn't seem there's evidence that it's producing the pk info, but just whether the option is enabled? Correct me if I misread.

Also I saw you mentioned a potential fix for this thread - do you mind elaborate what fix are you thinking?