babelfish_extensions icon indicating copy to clipboard operation
babelfish_extensions copied to clipboard

[Feature Request]: Support for INDEX with IGNORE_DUP_KEY=ON

Open PauloHMattos opened this issue 1 year ago • 5 comments

The applications of my company rely heavily on this option and without it, we are effectively unable to move to babelfish.

Is there any plan to support in the near future?

PauloHMattos avatar Sep 11 '24 18:09 PauloHMattos

@PauloHMattos We are already aware about this limitation and is under assessment. Just to get more clarity, how is it absolutely important for your use case, have you tried any alternatives, and (if so) why don't workaround works for your use case? Thank you!

Deepesh125 avatar Sep 12 '24 08:09 Deepesh125

@Deepesh125 right now we are evaluating performing this inserts directly in the PG INSERT ON CONFLICT DO NOTHING, but we are not sure if there's a recommended way to handle this limitation. Do you know how other users of Babelfish handle this?

To us seems a viable option, but our team is not very knowledgeable of PG and Babelfish in general so there are a lot of unknows. This being supported directly by Babelfish would reduce considerably the risk

Thanks for the help!

PauloHMattos avatar Sep 12 '24 15:09 PauloHMattos

I think you could try creating a rule from postgres endpoint to workaround this

-- Babelfish endpoint
1> CREATE TABLE t (id INT PRIMARY KEY)
2> GO
-- Postgres endpoint
-- CREATE RULE from postgres endpoint to ignore an insert based on some conditions

babelfish_db=# CREATE OR REPLACE RULE ignore_insert_when_dup_pk AS ON INSERT
TO master_dbo.t WHERE EXISTS (SELECT 1 FROM master_dbo.t WHERE id = NEW.id)
DO INSTEAD NOTHING;
CREATE RULE
-- Babelfish endpoint
1> INSERT INTO t VALUES (1), (3)
2> GO

(2 rows affected)

1> INSERT INTO t VALUES (1), (2), (3), (4)
2> GO

(2 rows affected)

1> SELECT * FROM t
2> GO
id         
-----------
          1
          3
          2
          4

(4 rows affected)

This is still not same as IGNORE_DUP_KEY since the rule is defined at table level and not index level.

tanscorpio7 avatar Mar 19 '25 00:03 tanscorpio7

Warning: There is a big difference in transactional behaviour between this workaround and IGNORE_DUP_KEY, especially when multiple rows are inserted in the transaction. In addition, there is no guarantee that the rule will be correctly processed when upgrading to a future version of Babelfish (it may work, and you'd be lucky).

robverschoor avatar Mar 19 '25 21:03 robverschoor

Right, if the insert statement has the same key occurring more than once and the table did not have the key before the insert statement, we will still hit the duplicate key error. So in the same example some thing like this would still fail

1> INSERT INTO t VALUES (5), (5)
2> GO
Msg 2627, Level 14, State 1, Server BABELFISH, Line 1
duplicate key value violates unique constraint "t_pkey"

because id = 5 did not exists before the insert statement.

tanscorpio7 avatar Mar 20 '25 01:03 tanscorpio7