EXP approximate equation fault?

[Lanjiejie]

I found the exp function code in MNN is that:

    float alpha = offset[0];
    float beta = offset[1];
    int remain = countC8 * 8;
    auto param = logf(2.0f);
    float xLimit = 87;
    for (int i = remain; i < dataSize; i++) {
        //Origin Function
        //dst[i] = expf(src[i] * alpha) + beta;
        //Approciate Function

        auto x         = alpha * src[i];
        x = ALIMAX(x, -xLimit);
        x = ALIMIN(x, xLimit);

        int div        = (x / param);
        int div2       = (div + 127) << 23;
        auto xReamin   = x - div * param;
        float expBasic = *(float*)(&div2);

        auto t         = xReamin;
        auto expRemain = ((((1.0f / 120 * t + 1.0f / 24) * t + 1.0f / 6) * t + 0.5f) * t + 1.0f) * t + 1.0f;
        dst[i]  = expBasic * expRemain + beta;
    }

Thus the approximate equation used in code is:

exp(x) = expBasic * expRemain (1) expBasic = (x/ln2 + 127)* (2^23); (2) expRemain = 1+t+t^2/2 + t^3/6 + t^4/24 + t^5/120; where t=x- int(x/ln2)*ln2; (3)

But I found this approximate equation didn't work correctly. The fault is in step (2). We can found that step (3) is the approximate equation of e^(x-int(x/ln2)*ln2), thus the correct approximate equation of step (2) should be e^(int(x/ln2)*ln2) = 2^(int(x/ln2)).

I tested the MNN's approximation equation, the standard exp output, and my modified exp approximation equation, and it turns out that the MNN's equation is wrong.

Moreover, the exp approximation equation is also used in functions such as softmax and sigmoid. I do a derivation test for sigmoid, and it can be found that the derivative obtained by using the MNN's exp approximate equation is always close to 0:

exp( -10.0 )  MNN: 7.050874E+08  standard: 4.539993E-05  my: 4.539988E-05
exp( -2.0 )  MNN: 5.675658E+08  standard: 1.353353E-01  my: 1.353182E-01
exp( -1.0 )  MNN: 7.776699E+08  standard: 3.678794E-01  my: 3.678789E-01
exp( -0.5 )  MNN: 6.461478E+08  standard: 6.065307E-01  my: 6.065104E-01
exp( 0.0 )  MNN: 1.065353E+09  standard: 1.000000E+00  my: 1.000000E+00
exp( 0.5 )  MNN: 1.756446E+09  standard: 1.648721E+00  my: 1.648698E+00
exp( 1.0 )  MNN: 1.459365E+09  standard: 2.718282E+00  my: 2.718279E+00
exp( 2.0 )  MNN: 1.998893E+09  standard: 7.389056E+00  my: 7.388731E+00
exp( 10.0 )  MNN: 1.590133E+09  standard: 2.202647E+04  my: 2.202645E+04
exp( 20.0 )  MNN: 2.349934E+09  standard: 4.851652E+08  my: 4.851477E+08
---------------------------------------------------------------
sigmoid_derivatice( -10.0 )  MNN: 1.418264E-09  standard: 4.539581E-05  my: 4.539575E-05
sigmoid_derivatice( -2.0 )  MNN: 1.761910E-09  standard: 1.049936E-01  my: 1.049835E-01
sigmoid_derivatice( -1.0 )  MNN: 1.285893E-09  standard: 1.966119E-01  my: 1.966118E-01
sigmoid_derivatice( -0.5 )  MNN: 1.547633E-09  standard: 2.350037E-01  my: 2.350018E-01
sigmoid_derivatice( 0.0 )  MNN: 9.386558E-10  standard: 2.500000E-01  my: 2.500000E-01
sigmoid_derivatice( 0.5 )  MNN: 5.693316E-10  standard: 2.350037E-01  my: 2.350045E-01
sigmoid_derivatice( 1.0 )  MNN: 6.852295E-10  standard: 1.966119E-01  my: 1.966120E-01
sigmoid_derivatice( 2.0 )  MNN: 5.002770E-10  standard: 1.049936E-01  my: 1.049971E-01
sigmoid_derivatice( 10.0 )  MNN: 6.288780E-10  standard: 4.539581E-05  my: 4.539584E-05
sigmoid_derivatice( 20.0 )  MNN: 4.255438E-10  standard: 2.061154E-09  my: 2.061228E-09

So is this function wrong, or is there a special purpose?