readyset icon indicating copy to clipboard operation
readyset copied to clipboard

Panic when replicating table with large numeric

Open jscheid opened this issue 1 year ago • 0 comments

Describe the problem

Replicating large numeric values will cause Readyset to panic with the (counter-intuitive) error message Number less than minimum value that can be represented. (full backtrace)

To Reproduce

CREATE TABLE foo (a NUMERIC);
INSERT INTO foo (a) VALUES (154000200562066000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000::NUMERIC);

Then arrange for the table to be replicated.

Expected behavior

Your documentation states:

Readyset does not support DECIMAL/NUMERIC values with a scale greater than 28, i.e., with more than 28 digits to the right of the decimal point. 'NaN', 'Infinity', and '-Infinity' are also not supported.

This value doesn't have any digits to the right of the decimal point, but maybe your documentation needs to be more precise and say "does not support values with more than 28 digits"?

Even then, the replication shouldn't panic. Perhaps it should do one of the following instead:

  • Print an error and skip replication of the record.
  • Print an error and pause replication of the table.
  • Replace the value with a user-defined substitute.

Ideally you would also provide guidance on how to ensure that the replicated table cannot contain unsupported values in the first place, such as a CHECK constraint. (I think even more ideal would be if Readyset refuses to replicate a table without that constraint, just as it refuses to replicate one with scale out of bounds.)

Environment

  • ReadySet version [e.g., beta-2022-11-02]: stable-240919
  • ReadySet deployment method [e.g, local orchestrator, Helm chart, binaries]: binary built from source
  • Upstream database and version [e.g., Postgres 14]: Postgres 14
  • Client app [e.g., psql, mysql, SQLAlchemy, TypeORM]: psql

jscheid avatar Oct 16 '24 09:10 jscheid