f2e-spec icon indicating copy to clipboard operation
f2e-spec copied to clipboard

Screen coordinate precision

Open Lord-Kamina opened this issue 5 years ago • 7 comments

I remembered, while working on the text wrapping issue, that we use doubleto represent some values related to animations, as well as screen and possibly texture coordinates. At the same time, the Dimensions class is built around float; and the OpenGL shaders are, I'm fairly sure, also written with float.

So...

  1. I'm thinking this mismatch could potentially introduce errors owing to loss of precision.
  2. I think there is absolutely no sane reason to keep using double instead of float; so I intend to change that as part of the aforementioned PR. Of course, I remain open to be convinced partially or totally otherwise; but I think the following video pretty much demonstrates point 2...

https://www.dropbox.com/s/vvxeuup64fmlzpr/performous_coordinate_demo.mp4?dl=1

For testing, I replaced the actual text with std::to_string(dimensions.x1()), and set up an increment for said value in the middle of the draw loop in screen_intro; this is a screen recording from a display running at 60Hz. As you can see, even around the limit of float precision, the movement is barely perceptible.

Lord-Kamina avatar Jul 01 '20 21:07 Lord-Kamina

I suggest keeping everything except OpenGL and large arrays in doubles because you will get conversion to higher precision every time the CPU touches it. Processors don't support single precision calculation, except for some vector operations. Arrays may need to be float to save memory if they are otherwise too large, and to avoid conversion with OpenGL. Both float and double are far more precise than what Performous needs.

Tronic avatar Jul 02 '20 04:07 Tronic

Floats have about 7 significant digits of precision, while screen pixel coordinates only need four. Doubles have 15.

Tronic avatar Jul 02 '20 04:07 Tronic

I suggest keeping everything except OpenGL and large arrays in doubles because you will get conversion to higher precision every time the CPU touches it. Processors don't support single precision calculation, except for some vector operations. Arrays may need to be float to save memory if they are otherwise too large, and to avoid conversion with OpenGL. Both float and double are far more precise than what Performous needs.

Most of our current use of doubles is stuff that interacts with OpenGL at some point or another; chiefly, I think there is potential for occasional strange behavior considering we're using doubles for text but floats for screen and textures; I guess we could also go for all doubles but that would be VERY overkill IMO.

Of course, I'm keeping time as doubles, as well as all audio/pitch/etc processing.

Lord-Kamina avatar Jul 02 '20 04:07 Lord-Kamina

Audio processing actually uses floats already. There is no way that using double would cause problems with text rendering but OpenGL must use float.

Tronic avatar Jul 02 '20 04:07 Tronic

P.S. I said I was going to do it in that same PR with the text stuff but I've already done it all and limited to a single commit so I can later excise it from that PR because else it's going to be unreviewable.

Lord-Kamina avatar Jul 02 '20 04:07 Lord-Kamina

Audio processing actually uses floats already. There is no way that using double would cause problems with text rendering but OpenGL must use float.

I meant your pitch algorithms, I think those are using doubles.

Lord-Kamina avatar Jul 02 '20 04:07 Lord-Kamina

As long as you don't go fsck~ng up perfectly good code. You may change Dimensions into doubles, that would be a good change if they are currently float.

When using floats, you also need f suffix on each literal value like 1.2f, otherwise the result is double even if the variables were floats.

Tronic avatar Jul 02 '20 04:07 Tronic