firebird icon indicating copy to clipboard operation
firebird copied to clipboard

Generators and replication

Open lwennekers opened this issue 2 years ago • 16 comments

I'm testing the replication function and I noticed that the generators aren't replicated. I wonder if I'm missing a setting or could it be a bug?

lwennekers avatar Aug 15 '22 12:08 lwennekers

This is by design. Generators are replicated only if the generated values were utilized by some DML command that was committed. I'd be happy to know about the real-world case where the generated values must be replicated regardless whether they were stored inside the database or not.

dyemanov avatar Aug 15 '22 12:08 dyemanov

When the replicated database become master database, it is not possible to work without real generator values. Also consider that generators can be used as settings flag.

livius2 avatar Aug 15 '22 16:08 livius2

When the replicated database become master database, it is not possible to work without real generator values.

Why? Who cares if last N values are lost if they were not stored anywhere? The max stored value will still match the replicated generator value, so nothing gonna be broken.

dyemanov avatar Aug 15 '22 16:08 dyemanov

Also consider that generators can be used as settings flag.

This is a problem. But replicating all generator changes unconditionally is a performance killer. So perhaps we need a configuration setting specifying which generators are "weirdly used" and thus should be replicated always.

dyemanov avatar Aug 15 '22 16:08 dyemanov

Such setting will be realy good, someone if like, can even setup all generators to replicate. You can add worning about some potetntial performane problems, but i do not see such risk. As how replicating generator value can be more performance problem then new record with data which need to be replicated? As normally generator is incremented when new record will be available soon to replication, and the record itself is much bigger (mor bytes) then generator.

livius2 avatar Aug 15 '22 18:08 livius2

When the replicated database become master database, it is not possible to work without real generator values.

Why? Who cares if last N values are lost if they were not stored anywhere? The max stored value will still match the replicated generator value, so nothing gonna be broken.

This could become a problem if the generated value is a primary key for a table. By the way does it matter if I use triggers to retrieve the generator value?

lwennekers avatar Aug 16 '22 12:08 lwennekers

This could become a problem if the generated value is a primary key for a table.

Nope. If the new PK value was stored and committed, the generator is always replicated as well. If the PK was stored but rolled back, or not stored at all (e.g. select gen_id() from rdb$database), then the replica will contain neither PK nor the new generator value, but it does not break anything.

By the way does it matter if I use triggers to retrieve the generator value?

No, it does not matter.

dyemanov avatar Aug 16 '22 12:08 dyemanov

Except a strange case when "select gen_id" and "insert" are in different attachments but this is crazy anyway.

aafemt avatar Aug 16 '22 13:08 aafemt

BTW, what exactly Firebird version do you test? Because there was a bug #6848 which was fixed in v4.0.1.

dyemanov avatar Aug 16 '22 13:08 dyemanov

"Except a strange case when "select gen_id" and "insert" are in different transactions but this is crazy anyway." I can imagine this at stateless applications.

omachtandras avatar Aug 16 '22 13:08 omachtandras

I wrote "attachments" and that's exactly what I meant. Different transactions in single attachment are fine.

aafemt avatar Aug 16 '22 13:08 aafemt

Nope. If the new PK value was stored and committed, the generator is always replicated as well. If the PK was stored but rolled back, or not stored at all (e.g. select gen_id() from rdb$database), then the replica will contain neither PK nor the new generator value, but it does not break anything.

How it is replicated, and which generator? Do you mean generator for identity field? I talk only about generatrors where we generate values by GEN_ID explicitly. E.g. we run in application select gen_id() from rdb$database and we store id in new records. We can also use select gen_id(, 10) from rdb$database to reserve 10 sequential records. But how engine know that it must replicate such generator?

livius2 avatar Aug 16 '22 13:08 livius2

How it is replicated, and which generator?

Replication of ALL generators are delayed to the first DML in the attachment. That's all.

aafemt avatar Aug 16 '22 13:08 aafemt

BTW, what exactly Firebird version do you test? Because there was a bug #6848 which was fixed in v4.0.1.

I used v4.0.1. Going to test 4.0.2

lwennekers avatar Aug 16 '22 16:08 lwennekers

Replication of ALL generators are delayed to the first DML in the attachment. That's all.

It looks like good solution, then probably problem was with above post about v4.0.1 instead of v4.0.2.

livius2 avatar Aug 17 '22 09:08 livius2

The test is done and the generators are replicated also :) Thanks for your support

lwennekers avatar Aug 29 '22 13:08 lwennekers