Failing tests with lower max_error

[kylebarron]

When trying to add more tests to compare against Martini output, I'm getting slightly smaller vertices and triangles arrays (on the order of 25 fewer elements for vertices) for smaller max_error values. Specifically, running on the fuji.png tile with max_error=20, I get 25 fewer vertices elements in the array.

I tested the pre-cython code and got the same results.

I think isolated to the issue to how many times the upper branch of an if block is called. Specifically, here https://github.com/kylebarron/pymartini/blob/7d5bdf7bddf51bcb861f9b0f91e1573f8a8231e4/pymartini/martini.pyx#L217-L231

For get_mesh(50) (which is equivalent to the Martini.js output) the number of upper branches is the same, but for get_mesh(20), there are 25 fewer times when the upper branch is taken.

            if (Math.abs(ax - cx) + Math.abs(ay - cy) > 1 && errors[my * size + mx] > maxError) {
                numUpperLoop++
                countElements(cx, cy, ax, ay, mx, my);
                countElements(bx, by, cx, cy, mx, my);

  var out = tile.getMesh(20);
  out.numUpperLoop // 35107
  var out = tile.getMesh(50);
  out.numUpperLoop // 8255

        if (abs(ax - cx) + abs(ay - cy) > 1) and (self.errors_view[my * size + mx] > self.max_error):
            self.num_upper_loop += 1
            self.countElements(cx, cy, ax, ay, mx, my)
            self.countElements(bx, by, cx, cy, mx, my)

    test = tile.get_mesh(20)
    tile.num_upper_loop # 35082
    test = tile.get_mesh(50)
    tile.num_upper_loop # 8255

[kylebarron]

I wonder if the difference is due to platform differences between python and node…

I calculated the same array of floats on both platforms, and my array of floats has up to a 0.0001 variation

nobs=263169, 
minmax=(0.0, 0.0001220703125), 
mean=1.928869350157027e-05,

That’s the absolute value of the python array minus the js array. (The JS array is exported to json and then loaded into python)

But the length of the final output, which relies on a < using the above intermediate data, is off by a factor of 1.000732 compared to the JS data.

[kylebarron]

As explained in the README, I think this is due to some internal float/bitwise operations differences between Python and JS, and these differences are magnified because errors are summed recursively for the larger triangles.