YADAMU---Yet-Another-DAta-Migration-Utility
YADAMU---Yet-Another-DAta-Migration-Utility copied to clipboard
Published
20 hours ago •
markddrake
markddrake
Yugabyte: Max and Min Money values loose decimal precision.
C:\Development\YADAMU>node C:\Development\YADAMU\src\qa\cli\test.js CONFIG=qa\YDB2\PGDT.json
2022-07-30T16:05:24.187Z [QA][Environemnt][x64][win32][v17.7.2]: Running tests
2022-07-30T16:05:24.374Z [INFO][Postgres][140004][Configuration]: PostGIS Version: 3.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1.
2022-07-30T16:05:24.433Z [INFO][Postgres][140004][Configuration]: PostGIS Version: 3.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1.
2022-07-30T16:05:24.434Z [INFO][READER][Postgres][140004][Manager]: Ready.
2022-07-30T16:05:24.451Z [INFO][WRITER][Postgres][140004][DDL_ONLY][Manager]: Ready.
2022-07-30T16:05:24.800Z [INFO][Postgres]: Operations restricted to the following tables: ["numeric_types"].
2022-07-30T16:05:24.800Z [DDL][Postgres]: Loaded metadata for 1 tables. Elapsed time: 00:00:00.326s.
2022-07-30T16:05:24.858Z [DDL][Postgres]: Executed 1 DDL operations. Elapsed time: 00:00:00.026s.
2022-07-30T16:05:24.952Z [DDL][Postgres]: Generated 1 "Create Table" statements and 1 DML statements. Elapsed time: 00:00:00.014s.
2022-07-30T16:05:24.953Z [INFO][Postgres]: DDL only operation. No data written.
2022-07-30T16:05:24.979Z [INFO][YADAMU][TEST]: Operation completed successfully. Elapsed time: 00:00:00.681.
2022-07-30T16:05:24.981Z [QA][DBROUNDTRIP][COPY][postgres#1://"t_postgres"][postgres#1://"t_postgres1"]: Errors: 0. Warnings: 0. Failed: 0. Elapsed Time: 00:00:00.683s.
2022-07-30T16:05:25.079Z [INFO][Postgres][140004][Configuration]: PostGIS Version: 3.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1.
2022-07-30T16:05:25.264Z [INFO][Yugabyte][110002][Configuration]: PostGIS Version: Not Installed.
2022-07-30T16:05:25.265Z [INFO][READER][Postgres][140004][Manager]: Ready.
2022-07-30T16:05:25.282Z [INFO][WRITER][Yugabyte][110002][DATA_ONLY][Manager]: Ready.
2022-07-30T16:05:25.816Z [INFO][Postgres]: Operations restricted to the following tables: ["numeric_types"].
2022-07-30T16:05:25.817Z [DDL][Postgres]: Loaded metadata for 1 tables. Elapsed time: 00:00:00.325s.
2022-07-30T16:05:28.901Z [INFO][PIPELINE][PARALLEL][1][Postgres][Yugabyte]: Processing 1 Tables
2022-07-30T16:05:28.901Z [DDL][Yugabyte]: Generated 1 "Create Table" statements and 1 DML statements. Elapsed time: 00:00:00.146s.
2022-07-30T16:05:29.165Z [DDL][Yugabyte]: Executed 1 DDL operations. Elapsed time: 00:00:00.236s.
2022-07-30T16:05:29.193Z [INFO][numeric_types][Batch]: Rows 35. Reader Elapsed Time: 00:00:00.003s. Throughput 10083 rows/s. Writer Elapsed Time: 00:00:00.093s. Idle Time: 00:00:00.000s. SQL Exection Time: 00:00:00.000s. Throughput: 374 rows/s.
2022-07-30T16:05:29.197Z [INFO][YADAMU][TEST]: Operation completed successfully. Elapsed time: 00:00:04.167.
2022-07-30T16:05:29.198Z [QA][DBROUNDTRIP][COPY][postgres#1://"t_postgres"][yugabyte#1://"t_postgres1"]: Errors: 0. Warnings: 0. Failed: 0. Elapsed Time: 00:00:04.166s.
2022-07-30T16:05:29.239Z [INFO][Yugabyte][110002][Configuration]: PostGIS Version: Not Installed.
2022-07-30T16:05:29.289Z [INFO][Postgres][140004][Configuration]: PostGIS Version: 3.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1.
2022-07-30T16:05:29.289Z [INFO][READER][Yugabyte][110002][Manager]: Ready.
2022-07-30T16:05:29.323Z [INFO][WRITER][Postgres][140004][DATA_ONLY][Manager]: Ready.
2022-07-30T16:05:41.953Z [INFO][Yugabyte]: Operations restricted to the following tables: ["numeric_types"].
2022-07-30T16:05:41.953Z [DDL][Yugabyte]: Loaded metadata for 1 tables. Elapsed time: 00:00:12.620s.
2022-07-30T16:05:42.065Z [INFO][PIPELINE][PARALLEL][1][Yugabyte][Postgres]: Processing 1 Tables
2022-07-30T16:05:42.065Z [DDL][Postgres]: Generated 1 "Create Table" statements and 1 DML statements. Elapsed time: 00:00:00.020s.
2022-07-30T16:05:42.091Z [DDL][Postgres]: Executed 1 DDL operations. Elapsed time: 00:00:00.002s.
2022-07-30T16:05:42.224Z [INFO][numeric_types][Batch]: Rows 35. Reader Elapsed Time: 00:00:00.001s. Throughput 31500 rows/s. Writer Elapsed Time: 00:00:00.025s. Idle Time: 00:00:00.000s. SQL Exection Time: 00:00:00.000s. Throughput: 1371 rows/s.
2022-07-30T16:05:42.228Z [INFO][YADAMU][TEST]: Operation completed successfully. Elapsed time: 00:00:13.025.
2022-07-30T16:05:42.230Z [QA][DBROUNDTRIP][COPY][yugabyte#1://"t_postgres1"][postgres#1://"t_postgres"]: Errors: 0. Warnings: 0. Failed: 0. Elapsed Time: 00:00:13.027s.
2022-07-30T16:05:42.310Z [INFO][Postgres][140004][Configuration]: PostGIS Version: 3.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1.
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| RESULT | SOURCE SCHEMA | TARGET SCHEMA | TABLE_NAME | SOURCE ROWS | TARGET ROWS | MISSING ROWS | EXTRA ROWS | NOTES |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| FAILED | t_postgres | t_postgres1 | numeric_types | 35 | 35 | 3 | 3 | |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2022-07-30T16:05:42.431Z [QA][DBROUNDTRIP][STEP][postgres#1://"t_postgres"][yugabyte#1://"t_postgres1"][postgres#1://"t_postgres"]: Errors: 0. Warnings: 0. Failed: 0. Elapsed Time: 00:00:18.133s.
+----------------------------------------------------------------------------------+
| Data Set | Step | Mode | Source | Target | Elapsed Time |
+----------------------------------------------------------------------------------+
| postgresDataTypes | COPY | DDL_ONLY | postgres#1 | postgres#1 | 00:00:00.683 |
| postgresDataTypes | COPY | DATA_ONLY | postgres#1 | yugabyte#1 | 00:00:04.166 |
| postgresDataTypes | COPY | DATA_ONLY | yugabyte#1 | postgres#1 | 00:00:13.027 |
| postgresDataTypes | TASK | | postgres#1 | yugabyte#1 | 00:00:17.956 |
| postgresDataTypes | COMPARE | | postgres#1 | | 00:00:00.026 |
| postgresDataTypes | TOTAL | | postgres#1 | yugabyte#1 | 00:00:18.133 |
+----------------------------------------------------------------------------------+
2022-07-30T16:05:42.563Z [QA][DBROUNDTRIP][TASK][postgres#1][yugabyte#1][postgresDataTypes]: Errors: 0. Warnings: 0. Failed: 1. Elapsed Time: 00:00:18.374s
2022-07-30T16:05:42.589Z [QA][DBROUNDTRIP][TARGET][postgres#1][yugabyte#1]: Errors: 0. Warnings: 0. Failed: 1. Elapsed Time: 00:00:18.400s
2022-07-30T16:05:42.593Z [QA][DBROUNDTRIP][TEST][postgres#1]: Errors: 0. Warnings: 0. Failed: 1. Elapsed Time: 00:00:18.404s
+-----------------------------------------------------------------------------------------------------------------------------+
| Source | Target | Data Set | Table | Source Rows | Target Rows | Missing Rows | Extra Rows | Cause |
+-----------------------------------------------------------------------------------------------------------------------------+
| postgres#1 | yugabyte#1 | postgresDataTypes | numeric_types | 35 | 35 | 3 | 3 | |
+-----------------------------------------------------------------------------------------------------------------------------+
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| End Time | Operation | Source | Target | Task | Results | Memory Usage | Elapsed Time |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2022-07-30T16:05:42.620Z | DBROUNDTRIP | postgres#1 | | | 0 | 0 | 1 | 41,984,000 | 12,132,352 | 11,122,432 | 1,126,140 | 136,227 | 00:00:18.404 |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| End Time | Operation | Source | Target | Task | Results | Memory Usage | Elapsed Time |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2022-07-30T16:05:42.588Z | DBROUNDTRIP | postgres#1 | yugabyte#1 | postgresDataTypes | 0 | 0 | 1 | 41,975,808 | 12,132,352 | 11,113,584 | 1,126,100 | 136,227 | 00:00:18.374 |
| 2022-07-30T16:05:42.593Z | DBROUNDTRIP | postgres#1 | yugabyte#1 | | 0 | 0 | 1 | 41,979,904 | 12,132,352 | 11,118,896 | 1,126,140 | 136,227 | 00:00:18.400 |
| 2022-07-30T16:05:42.620Z | DBROUNDTRIP | postgres#1 | | | 0 | 0 | 1 | 41,984,000 | 12,132,352 | 11,122,432 | 1,126,140 | 136,227 | 00:00:18.404 |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2022-07-30T16:05:42.621Z | | | | | 0 | 0 | 1 | 41,984,000 | 12,132,352 | 11,123,824 | 1,126,140 | 136,227 | 00:00:18.432 |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2022-07-30T16:05:42.905Z [QA][YADAMU][REGRESSION][qa\YDB2\PGDT.json]: Errors: 0. Warnings: 0. Failed: 1. Elapsed Time: 00:00:18.718s.
C:\Development\YADAMU>psql -Upostgres -hyadamu-db1 -dyadamu
Password for user postgres:
psql (13.3, server 14.4 (Debian 14.4-1.pgdg110+1))
WARNING: psql major version 13, server major version 14.
Some psql features might not work.
WARNING: Console code page (437) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
Type "help" for help.
yadamu=# select * from t_postgres.numeric_types except select * from t_postgres1.numeric_types;
smallint_col | integer_col | bigint_col | decimal_col | numeric_col | real_col | double_precision_col | money_col | small_serial_col | serial_col | bigserial_col
--------------+-------------+----------------------+---------------------+---------------------+----------------+--------------------------+-----------------------------+------------------+------------+---------------
-32768 | -2147483648 | -9223372036854775808 | -12356789.123456789 | -12356789.123456789 | -3.4028235e+38 | -1.7976931348623157e+308 | -$92,233,720,368,547,758.08 | 1 | 1 | 1
32767 | 2147483647 | 9223372036854775807 | 12356789.123456789 | 12356789.123456789 | 3.4028235e+38 | 1.7976931348623157e+308 | $92,233,720,368,547,758.07 | 2 | 2 | 2
| | | | | | | -$92,233,720,368,547,758.08 | 30 | 30 | 30
(3 rows)
yadamu=# select * from t_postgres1.numeric_types except select * from t_postgres.numeric_types;
smallint_col | integer_col | bigint_col | decimal_col | numeric_col | real_col | double_precision_col | money_col | small_serial_col | serial_col | bigserial_col
--------------+-------------+----------------------+---------------------+---------------------+----------------+--------------------------+-----------------------------+------------------+------------+---------------
-32768 | -2147483648 | -9223372036854775808 | -12356789.123456789 | -12356789.123456789 | -3.4028235e+38 | -1.7976931348623157e+308 | -$92,233,720,368,547,758.00 | 1 | 1 | 1
32767 | 2147483647 | 9223372036854775807 | 12356789.123456789 | 12356789.123456789 | 3.4028235e+38 | 1.7976931348623157e+308 | $92,233,720,368,547,758.00 | 2 | 2 | 2
| | | | | | | -$92,233,720,368,547,758.00 | 30 | 30 | 30
(3 rows)
yadamu=#
