hop icon indicating copy to clipboard operation
hop copied to clipboard

[Bug]: Hop serializes numeric values into scientific notation, causing precision loss in downstream databases

Open magiclen opened this issue 3 weeks ago • 7 comments

Apache Hop version?

2.16.0

Java version?

21.0.8

Operating system

Linux

What happened?

Description

When reading a float(53) or numeric value such as: 55487400.0

Apache Hop serializes it as: 5.54874E+7

Even though the value can be represented exactly as a decimal integer.

This causes downstream databases (e.g., ClickHouse Float64) to parse and store a different floating-point value such as: 55487400.00000001

This rounding drift happens because scientific notation forces additional floating-point parsing and cannot preserve exact integer precision.

Expected

  • Hop should not convert integer-like floats into scientific notation.
  • Or provide an option to disable scientific-notation formatting for numeric fields.
  • Or preserve original decimal representation when no precision is gained.

Steps to Reproduce

  1. Read a SQL Server float(53) value 55487400.0 in Hop.
  2. Pass it through a Number/BigNumber field.
  3. Output using "Table Output".
  4. Observe Hop sending 5.54874E+7 instead of 55487400.

A Quick Test On ClickHouse

SELECT
toFloat64('5.54874E+7') AS a,
reinterpretAsUInt64(a) AS bits_a,
toFloat64('55487400') AS b,
reinterpretAsUInt64(b) AS bits_b
toFloat64('55487400.0') AS c,
reinterpretAsUInt64(c) AS bits_c;

a and b represent different values. b and c are the same.

The following INSERT statements cause different results:

INSERT INTO TAG_VALUE VALUES (fromUnixTimestamp64Nano(173566111000000), 5.54874E+7);
INSERT INTO TAG_VALUE VALUES (fromUnixTimestamp64Nano(173566111000000), 55487400);

But if we SELECT them directly, they represent the same value.

SELECT
5.54874E+7 AS aa,
reinterpretAsUInt64(aa) AS bits_aa,
55487400 AS bb,
reinterpretAsUInt64(bb) AS bits_bb,
55487400.0 AS cc,
reinterpretAsUInt64(cc) AS bits_cc;

Issue Priority

Priority: 2

Issue Component

Component: Hop Gui, Component: Database, Component: Transforms

magiclen avatar Dec 03 '25 13:12 magiclen