ta-lib-python icon indicating copy to clipboard operation
ta-lib-python copied to clipboard

Error when creating new indicator

Open hhashim1 opened this issue 3 years ago • 87 comments

I am creating a new indicator and have copied parts from BBANDS. I am getting an error on one of the variables and I cannot seem to figure out why I am getting this error. Can you @mrjbq7 or @trufanov-nok can either of you help plz?

tempBuffer2 is the one that is causing the issue.

I am trying to have an equivalent of the following line of code:

def rawRelVol = (volume - Average(volume, length)) / StDev(volume, length);

Here is my code:


   /* Insert TA function code here. */
    ENUM_DECLARATION(RetCode) retCode;
    double periodTotal, tempReal;
    int i, outIdx, trailingIdx, lookbackTotal;
    ARRAY_REF(tempBuffer1);
    ARRAY_REF(tempBuffer2);
    //double rawRelVol;

    /* Calculate moving average. */
    retCode = FUNCTION_CALL(SMA)(startIdx, endIdx, inVolume,
        optInTimePeriod, outBegIdx, outNBElement, tempBuffer1);

    if ((retCode != ENUM_VALUE(RetCode, TA_SUCCESS, Success)) || ((int)VALUE_HANDLE_DEREF(outNBElement) == 0))
    {
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }


    /* Calculate the Standard Deviation */
    retCode = FUNCTION_CALL(STDDEV)((int)VALUE_HANDLE_DEREF(outBegIdx), endIdx, inVolume,
        optInTimePeriod, 1.0,
        outBegIdx, outNBElement, tempBuffer2);

    if (retCode != ENUM_VALUE(RetCode, TA_SUCCESS, Success))
    {
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }

    tempReal = (inVolume - tempBuffer1);
    tempReal = tempReal / tempBuffer2;

Here is the screenshot: image

hhashim1 avatar Jun 23 '22 16:06 hhashim1

Are you trying to divide a double by a double* ?

mrjbq7 avatar Jun 24 '22 04:06 mrjbq7

What is a double*?

This is what I am trying to do. rawRelVol = (volume - Average(volume, length)) / StDev(volume, length)

tempBuffer2 is meant to be the StDev value that is returned.

I have copied the Stdev code from BBANDS.

hhashim1 avatar Jun 24 '22 12:06 hhashim1

You need to learn how arrays in C are designed. double* is a pointer in memory to a first double element of an array. And you can't just divide array by array or make any other operations over arrays - it won't work, they are not objects. You need to iterate both arrays in a loop and devide each element. It's not a TA-Lib related issue. it's a C knowlage related issue. You may better to ask on stackoverflow or something.

trufanov-nok avatar Jun 24 '22 12:06 trufanov-nok

Ok, so I made some changes and added a loop to iterate however I have a different issue now. I don't know how else to initialize the variable.

The error is Using uninitialized memory "tempBuffer1" and Using uninitialized memory "tempBuffer2"

Here is the code:


   /* insert local variable here */
    ENUM_DECLARATION(RetCode) retCode;
    //double periodTotal;
    double tempReal, tempReal2;
    int i;// , outIdx, trailingIdx, lookbackTotal;
    ARRAY_REF(tempBuffer1);
    ARRAY_REF(tempBuffer2);

   /* Insert TA function code here. */
   #if defined(USE_SINGLE_PRECISION_INPUT) || defined( USE_SUBARRAY )
      tempBuffer1 = outRealMiddleBand;
      tempBuffer2 = outReal;
      /* Check that the caller is not doing tricky things. 
       * (like using the input buffer in two output!)
       */
      if( (tempBuffer1 == inVolume) || (tempBuffer2 == inVolume) )
         return ENUM_VALUE(RetCode,TA_BAD_PARAM,BadParam);
   #endif

    /* Calculate moving average. */
    retCode = FUNCTION_CALL(SMA)(startIdx, endIdx, inVolume,
        optInTimePeriod, outBegIdx, outNBElement, tempBuffer1);

    if ((retCode != ENUM_VALUE(RetCode, TA_SUCCESS, Success)) || ((int)VALUE_HANDLE_DEREF(outNBElement) == 0))
    {
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }


    /* Calculate the Standard Deviation */
    retCode = FUNCTION_CALL(STDDEV)((int)VALUE_HANDLE_DEREF(outBegIdx), endIdx, inVolume,
        optInTimePeriod, 1.0,
        outBegIdx, outNBElement, tempBuffer2);

    if (retCode != ENUM_VALUE(RetCode, TA_SUCCESS, Success))
    {
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }



    /* No standard deviation multiplier needed. */
    for (i = 0; i < (int)VALUE_HANDLE_DEREF(outNBElement); i++)
    {
        tempReal = (inVolume - tempBuffer1);
        tempReal2 = tempBuffer2[i];
        outReal[i] = tempReal / tempReal2;
    }


   return ENUM_VALUE(RetCode,TA_SUCCESS,Success);
}

Here is the screenshot: image

hhashim1 avatar Jun 24 '22 18:06 hhashim1

You are probably using ARRAY_REF. That's a template which is declared in ta_memory.h. And it's value is type *name. Which means ARRAY_REF(tempBuffer1); will be substituted by ARRAY_VTYPE_REF(double, name) and its turn by a double* tempBuffer1; during compilation by a C preprocessor. That only a pointer to a first element in array. Pointer to memory. But memory isn't allocated. So you need to use ARRAY_ALLOC(name,size) and ARRAY_FREE(name) templates instead to not only declare a pointer, but also allocate and deallocate a memory for an array at the end. name is tempBuffer1. The question is: what is the size? For ex., for tempBuffer1 I would try endIdx-startIdx-LOOKBACK_CALL(SMA)(optInTimePeriod);. Similar for a second buffer.

trufanov-nok avatar Jun 24 '22 19:06 trufanov-nok

For lookback, here is what I have. return LOOKBACK_CALL(SMA)(optInTimePeriod);

Here are my inputs:

/* Generated */ enum class Core::RetCode Core::RvStdev( int    startIdx,
/* Generated */                                         int    endIdx,
/* Generated */                                         SubArray<double>^ inVolume,
/* Generated */                                         int           optInTimePeriod, /* From 2 to 100000 */
/* Generated */                                         [Out]int%    outBegIdx,
/* Generated */                                         [Out]int%    outNBElement,
/* Generated */                                         SubArray<double>^  outReal )

hhashim1 avatar Jun 24 '22 19:06 hhashim1

@trufanov-nok given the below psuedo code, how would you code it in talib?

input length = 50;
rawRelVol = (volume - Average(volume, length)) / StDev(volume, length);

hhashim1 avatar Jun 24 '22 19:06 hhashim1

I'll answer tomorrow.

trufanov-nok avatar Jun 24 '22 20:06 trufanov-nok

Thank you

hhashim1 avatar Jun 24 '22 20:06 hhashim1

@trufanov-nok given the below psuedo code, how would you code it in talib?

input length = 50;
rawRelVol = (volume - Average(volume, length)) / StDev(volume, length);

I would start with something like that (didn't test it):

/* insert local variable here */
ENUM_DECLARATION(RetCode) retCode;
int outBegIdx1, outBegIdx2;
int currentIdx, currentIdx1, currentIdx2;
ARRAY_ALLOC(tempBuffer1, 50);
ARRAY_ALLOC(tempBuffer2, 50);
 
   
       /* Calculate moving average. */
    retCode = FUNCTION_CALL(SMA)(startIdx, endIdx, inVolume,
        optInTimePeriod, &outBegIdx1, outNBElement, tempBuffer1);

    if ((retCode != ENUM_VALUE(RetCode, TA_SUCCESS, Success)) || ((int)VALUE_HANDLE_DEREF(outNBElement) == 0))
    {
        ARRAY_FREE(tempBuffer1);
        ARRAY_FREE(tempBuffer2);
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }
    
    /* Calculate the Standard Deviation */
    retCode = FUNCTION_CALL(STDDEV)(startIdx, endIdx, inVolume,
        optInTimePeriod, 1.0,
        &outBegIdx2, outNBElement, tempBuffer2);
        
    if ((retCode != ENUM_VALUE(RetCode, TA_SUCCESS, Success)) || ((int)VALUE_HANDLE_DEREF(outNBElement) == 0))
    {
        ARRAY_FREE(tempBuffer1);
        ARRAY_FREE(tempBuffer2);
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }
    
        
    // startIdx is a MAX(startIdx, outBegIdx1, outBegIdx2)
    if (startIdx < outBegIdx1)
       startIdx = outBegIdx1;
    if (startIdx < outBegIdx2)
       startIdx = outBegIdx2;
       
    VALUE_HANDLE_DEREF(outBegIdx) = startIdx;
    
    if (startIdx > endIdx)
    {
        ARRAY_FREE(tempBuffer1);
        ARRAY_FREE(tempBuffer2);
        VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
        return retCode;
    }


    currentIdx = 0;
    currentIdx1 = startIdx - outBegIdx1;
    currentIdx2 = startIdx - outBegIdx2;
    
    for (i = startIdx; i < endIdx; i++)
    {
        outReal[currentIdx++] = (inVolume[i] - tempBuffer1[currentIdx1++]) / tempBuffer2[currentIdx2++];
    }
    
    
    ARRAY_FREE(tempBuffer1);
    ARRAY_FREE(tempBuffer2);
    (int)VALUE_HANDLE_DEREF(outNBElement) = currentIdx;

return ENUM_VALUE(RetCode,TA_SUCCESS,Success);

trufanov-nok avatar Jun 25 '22 09:06 trufanov-nok

@trufanov-nok thank you. I was getting an error saying tempBuffer1 and 2 are uninitialized or something like that. I had to add the following code and it got rid of the error however I wanted to ask if this was the right thing to do or not.

    ARRAY_REF(tempBuffer1);
    ARRAY_REF(tempBuffer2);

Here is my full variable declaration:

   /* insert local variable here */
    ENUM_DECLARATION(RetCode) retCode;
    int outBegIdx1, outBegIdx2;
    int currentIdx, currentIdx1, currentIdx2;
    int i;
    ARRAY_REF(tempBuffer1);
    ARRAY_REF(tempBuffer2);
    ARRAY_ALLOC(tempBuffer1, optInTimePeriod);
    ARRAY_ALLOC(tempBuffer2, optInTimePeriod);

hhashim1 avatar Jun 27 '22 13:06 hhashim1

You're rignt. ARRAY_REF is a type *name. The ARRAY_ALLOC is only a name = (type *)TA_Malloc( sizeof(type)*(size)). So both are required.

trufanov-nok avatar Jun 27 '22 18:06 trufanov-nok

Ok, thanks @trufanov-nok I do have another error that I dont know how to fix.

image

Also, while running make after running ./configure --prefix=/usr, I get the following error. I dont know if the two are related.

image

Thoughts?

hhashim1 avatar Jun 27 '22 19:06 hhashim1

could you upload a whole code at pastebin?

trufanov-nok avatar Jun 27 '22 19:06 trufanov-nok

Here you go...

https://pastebin.com/daPXyTJ1

hhashim1 avatar Jun 27 '22 20:06 hhashim1

you have two declarations of after section 3 and 4

    ARRAY_REF(tempBuffer1);
    ARRAY_REF(tempBuffer2);

try to leave only one of them

trufanov-nok avatar Jun 27 '22 20:06 trufanov-nok

I still have the same error. I think I had copied the section together to share it with you but forgot to undo it. The compiled code had the correct declaration which was only once.

image

hhashim1 avatar Jun 27 '22 20:06 hhashim1

Here is the updated pastebin https://pastebin.com/q41gRu85

hhashim1 avatar Jun 27 '22 20:06 hhashim1

instead of (int)VALUE_HANDLE_DEREF(outNBElement) = currentIdx; try VALUE_HANDLE_DEREF(outNBElement) = currentIdx;

trufanov-nok avatar Jun 27 '22 20:06 trufanov-nok

I still have the same error. image

in my ta_func.h file, all of my function calls show an error and say that the definition is missing. I don't recall having this issue before. I am not sure if I accidentally deleted something. image

Here is the error for 'make testforta-lib-wrapper`

_______________________________________ ERROR collecting talib/test_abstract.py ________________________________________
ImportError while importing test module '/ta-lib-wrapper/talib/test_abstract.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
talib/__init__.py:93: in <module>
    from ._ta_lib import (
E   ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV
_________________________________________ ERROR collecting talib/test_data.py __________________________________________
ImportError while importing test module '/ta-lib-wrapper/talib/test_data.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
talib/__init__.py:93: in <module>
    from ._ta_lib import (
E   ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV
_________________________________________ ERROR collecting talib/test_func.py __________________________________________
ImportError while importing test module '/ta-lib-wrapper/talib/test_func.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
talib/__init__.py:93: in <module>
    from ._ta_lib import (
E   ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV
________________________________________ ERROR collecting talib/test_pandas.py _________________________________________
ImportError while importing test module '/ta-lib-wrapper/talib/test_pandas.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
talib/__init__.py:93: in <module>
    from ._ta_lib import (
E   ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV
________________________________________ ERROR collecting talib/test_polars.py _________________________________________
ImportError while importing test module '/ta-lib-wrapper/talib/test_polars.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
talib/__init__.py:93: in <module>
    from ._ta_lib import (
E   ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV
________________________________________ ERROR collecting talib/test_stream.py _________________________________________
ImportError while importing test module '/ta-lib-wrapper/talib/test_stream.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
talib/__init__.py:93: in <module>
    from ._ta_lib import (
E   ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV
=============================================== short test summary info ================================================
ERROR talib/test_abstract.py
ERROR talib/test_data.py
ERROR talib/test_func.py
ERROR talib/test_pandas.py
ERROR talib/test_polars.py
ERROR talib/test_stream.py

hhashim1 avatar Jun 28 '22 00:06 hhashim1

C6011 is just a warning that indicates that in theory malloc may fail to alloc a required piece of memory and return NULL. I guess it may be ignored or suppressed with additional checks if you replace:

    ARRAY_ALLOC(tempBuffer1, optInTimePeriod);
    ARRAY_ALLOC(tempBuffer2, optInTimePeriod);

with

    ARRAY_ALLOC(tempBuffer1, optInTimePeriod);
    if ( !tempBuffer1 )
        return ENUM_VALUE(RetCode,TA_ALLOC_ERR,AllocErr);

    ARRAY_ALLOC(tempBuffer2, optInTimePeriod);
    if ( !tempBuffer2 ) 
    {
        ARRAY_FREE(tempBuffer1);
        return ENUM_VALUE(RetCode,TA_ALLOC_ERR,AllocErr);
    }

trufanov-nok avatar Jun 28 '22 10:06 trufanov-nok

How you are building your C library and how you're rebuilding the wrapper? Could you show just a sequence of commands without output?

trufanov-nok avatar Jun 28 '22 10:06 trufanov-nok

Ok so I no longer have the warning/error of dereferencing NULL pointer and the build goes fine except I have an error when I import talib in python.

ImportError: /ta-lib-wrapper/talib/_ta_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: TA_RVSTDEV

image

Here are my steps for building the library and the wrapper, which are what we went through last month (or was it the month before?). Talib:

  • Rebuild gen_code
  • Run gen_code
  • Rebuild Solution
  • In Linux, navigate to ta-lib/ta-lib/c and run ./configure --prefix=/usr.
  • Run make (creates the .lo and .o files).
  • Next, run make install

Ta-lib-wrapper:

  • Navigate to ta-lib-wrapper directory in Linux
  • Run the following commands
  • make generate
  • make cython
  • make build
  • make test
  • make install
  • make sdist
  • python3 setup.py install

hhashim1 avatar Jun 28 '22 17:06 hhashim1

What readelf -a /usr/lib/libta_lib.so.0.0.0 | grep TA_RVSTDEV outputs?

trufanov-nok avatar Jun 28 '22 17:06 trufanov-nok

Which directory should I run this in? ta-lib-wrapper or talib/c?

hhashim1 avatar Jun 28 '22 17:06 hhashim1

any of them

trufanov-nok avatar Jun 28 '22 17:06 trufanov-nok

Also in file ./c/src/ta_abstract/tables/table_r.c what is the content of const TA_FuncDef *TA_DEF_TableR[] = array?

trufanov-nok avatar Jun 28 '22 17:06 trufanov-nok

No output if I run from the ta-lb-wrapper directory but if I run in ta-lib/c, here is what I get.

image

hhashim1 avatar Jun 28 '22 17:06 hhashim1


const TA_FuncDef *TA_DEF_TableR[] =
{
   ADD_TO_TABLE(ROC),
   ADD_TO_TABLE(ROCP),
   ADD_TO_TABLE(ROCR),
   ADD_TO_TABLE(ROCR100),
   ADD_TO_TABLE(RSI),
   ADD_TO_TABLE(RVSTDEV),
   NULL
};

hhashim1 avatar Jun 28 '22 17:06 hhashim1

This is from table_r.c

/* RVSTDEV BEGIN */
static const TA_InputParameterInfo * TA_RVSTDEV_Inputs[] =
{
  &TA_DEF_UI_Input_Price_V,
  NULL
};

static const TA_OutputParameterInfo* TA_RVSTDEV_Outputs[] =
{
  &TA_DEF_UI_Output_Real,
  NULL
};

static const TA_OptInputParameterInfo* TA_RVSTDEV_OptInputs[] =
{   
    &TA_DEF_UI_TimePeriod_50_MINIMUM2,
    NULL 
};

static const TA_InputParameterInfo* TA_RVSTDEV_StructParams[] = { NULL };

DEF_FUNCTION(RVSTDEV,                     /* name */
    TA_GroupId_VolumeIndicators, /* groupId */
    "Relative Volume St Dev", /* hint */
    "RvStdev",                         /* CamelCase name */
    0                              /* flags */
);
/* RVSTDEV END */

hhashim1 avatar Jun 28 '22 17:06 hhashim1

Hm, it seems C library exports TA_RVSTDEV correctly. Perhaps somethng is forgotten to be added into the wrapper. Could you upload the output of make cython and make build from wrapper folder to pastebin?

trufanov-nok avatar Jun 28 '22 17:06 trufanov-nok

https://pastebin.com/VEsiVQiZ

hhashim1 avatar Jun 28 '22 17:06 hhashim1

what cat /usr/include/ta-lib/ta_func.h | grep TA_RVSTDEV outputs?

trufanov-nok avatar Jun 28 '22 18:06 trufanov-nok

and

ls -hs /usr/local/include/ta-lib/ta_func.h
ls -hs /opt/include/ta-lib/ta_func.h
ls -hs /opt/local/include/ta-lib/ta_func.h
ls -hs /opt/homebrew/include/ta-lib/ta_func.h
ls -hs /opt/homebrew/opt/ta-lib/include/ta-lib/ta_func.h
ls -hs /opt/deephaven-venv/lib/python3.7/site-packages/numpy/core/include/ta-lib/ta_func.h
ls -hs /opt/deephaven-venv/include/ta-lib/ta_func.h
ls -hs /usr/include/python3.7m/ta-lib/ta_func.h

?

trufanov-nok avatar Jun 28 '22 18:06 trufanov-nok

image

Which directory do I need to be in to run the above commands? All of them are saying No such file or directory.

hhashim1 avatar Jun 28 '22 18:06 hhashim1

Ok, in wrappers folder do cat talib/_ta_lib.pxd | grep TA_RVSTDEV

trufanov-nok avatar Jun 28 '22 18:06 trufanov-nok

    TA_RetCode TA_RVSTDEV(int startIdx, int endIdx, const double inVolume[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outReal[])
    int TA_RVSTDEV_Lookback(int optInTimePeriod)

hhashim1 avatar Jun 28 '22 18:06 hhashim1

Hm, I'm out of options. @mrjbq7 do you have any ideas?

trufanov-nok avatar Jun 28 '22 18:06 trufanov-nok

I would suggest uninstalling the python ta-lib:

$ python3 -m pip uninstall ta-lib

And then build yours in-place for testing:

$ cd ta-lib
$ python3 setup.py build_ext --inplace

Then do all your testing / importing from the dev directory to make sure you're importing and using the code that builds and uses the new indicator, and not accidentally using the previous site-packages version.

The other thing is make sure that you added it to _func.pxi and included it at the end in __ALL__... you could do this by either re-running tools/generate_func.py or writing it manually.

mrjbq7 avatar Jun 28 '22 18:06 mrjbq7

I have it included in two places in _func.pxi like you have suggested. I will uninstall and test again and will report to you.

hhashim1 avatar Jun 28 '22 18:06 hhashim1

Check out the error I get when I run make test for the wrapper. Anything stands out?

https://pastebin.com/cZvwvUSN

hhashim1 avatar Jun 28 '22 19:06 hhashim1

Hmm, and you're sure the library installed in /usr/local/lib has your new function exported?

$ nm /usr/local/lib/libta_lib.dylib | grep TA_RVSTDEV

mrjbq7 avatar Jun 28 '22 19:06 mrjbq7

Says No such file

hhashim1 avatar Jun 28 '22 19:06 hhashim1

Well, whatever path your ta-lib is installed to

mrjbq7 avatar Jun 28 '22 19:06 mrjbq7

$ nm $(brew --prefix ta-lib)/lib/libta_lib.dylib | grep TA_RVSTDEV

mrjbq7 avatar Jun 28 '22 19:06 mrjbq7

I assume your issue is having a homebrew ta-lib, and then also your dev version which you want to override homebrew.

Maybe you should brew uninstall ta-lib so you don't have the one without TA_RVSTDEV in the path.

mrjbq7 avatar Jun 28 '22 19:06 mrjbq7

I dont have brew on the machine and cannot get it installed either. At this point, I dont know how to fix this thing 😭

hhashim1 avatar Jun 28 '22 20:06 hhashim1

Well, where is your libta_lib.dylib installed? It has to be somewhere...

Run that command on it:

$ nm /path/to/libta_lib.dylib | grep TA_RVSTDEV

mrjbq7 avatar Jun 28 '22 20:06 mrjbq7

This is what I got when searching for the file


find: ‘./mnt/i/$RECYCLE.BIN/S-1-5-21-1915536511-2278406390-2236943863-1001’: Permission denied```

hhashim1 avatar Jun 28 '22 20:06 hhashim1

But didn’t you edit and build a ta-lib with your new function?

Where does that install to?

On Tue, Jun 28, 2022 at 1:57 PM hhashim1 @.***> wrote:

This is what I got when searching for the file

find: ‘./mnt/i/$RECYCLE.BIN/S-1-5-21-1915536511-2278406390-2236943863-1001’: Permission denied```

— Reply to this email directly, view it on GitHub https://github.com/mrjbq7/ta-lib/issues/524#issuecomment-1169235438, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAF5AZHG7P6MVXCA3UBJJLVRNRLRANCNFSM5ZVADI7Q . You are receiving this because you were mentioned.Message ID: @.***>

mrjbq7 avatar Jun 28 '22 21:06 mrjbq7

I guess he's on linux and the lib is in the /usr/lib/libta_lib.so.0.0.0

trufanov-nok avatar Jun 28 '22 21:06 trufanov-nok

Here are the directories that I have in the usr/lib folder

image

hhashim1 avatar Jun 28 '22 21:06 hhashim1

$ nm /usr/lib/libta_lib.so | grep TA_RVSTDEV

This may be getting past the scope of debugging we should be doing on this project's issues...

mrjbq7 avatar Jun 28 '22 21:06 mrjbq7

Ok, so I deleted my RVSTDEV.c file and removed all the references from the talib library and also from ta-lib-wrapper. No errors anymore.

image

I need to add everything again but I want yall to look over my code to see if I missed anything. Here is RVSTDEV.c code - https://pastebin.com/LZ6iESy9

Here are all of the references in the talib library


############table_r.c



/* RVSTDEV BEGIN */
static const TA_InputParameterInfo * TA_RVSTDEV_Inputs[] =
{
  &TA_DEF_UI_Input_Price_V,
  NULL
};

static const TA_OutputParameterInfo* TA_RVSTDEV_Outputs[] =
{
  &TA_DEF_UI_Output_Real,
  NULL
};

static const TA_OptInputParameterInfo* TA_RVSTDEV_OptInputs[] =
{   
    &TA_DEF_UI_TimePeriod_50_MINIMUM2,
    NULL 
};

static const TA_InputParameterInfo* TA_RVSTDEV_StructParams[] = { NULL };

DEF_FUNCTION(RVSTDEV,                     /* name */
    TA_GroupId_VolumeIndicators, /* groupId */
    "Relative Volume St Dev", /* hint */
    "RvStdev",                         /* CamelCase name */
    0                              /* flags */
);
/* RVSTDEV END */



####### ta_frame.c


/* Generated */ TA_RetCode TA_RVSTDEV_FramePP( const TA_ParamHolderPriv *params,
/* Generated */                           int            startIdx,
/* Generated */                           int            endIdx,
/* Generated */                           int           *outBegIdx,
/* Generated */                           int           *outNBElement )
/* Generated */ {
/* Generated */    return TA_RVSTDEV(
/* Generated */                 startIdx,
/* Generated */                 endIdx,
/* Generated */                 params->in[0].data.inPrice.volume, /* inVolume */
/* Generated */                 params->optIn[0].data.optInInteger, /* optInTimePeriod*/
/* Generated */                 outBegIdx, 
/* Generated */                 outNBElement, 
/* Generated */                 params->out[0].data.outReal /*  outReal */ );
/* Generated */ }
/* Generated */ unsigned int TA_RVSTDEV_FramePPLB( const TA_ParamHolderPriv *params )
/* Generated */ {
/* Generated */    return TA_RVSTDEV_Lookback(params->optIn[0].data.optInInteger /* optInTimePeriod*/ );
/* Generated */ }




####### ta_frame.h



/* Generated */ TA_RetCode TA_RVSTDEV_FramePP( const TA_ParamHolderPriv *params,
/* Generated */                           int            startIdx,
/* Generated */                           int            endIdx,
/* Generated */                           int           *outBegIdx,
/* Generated */                           int           *outNBElement )
;
/* Generated */ unsigned int TA_RVSTDEV_FramePPLB( const TA_ParamHolderPriv *params )
;



######## ta_func.h



/*
 * TA_RVSTDEV - Relative Volume St Dev
 * 
 * Input  = Volume
 * Output = double
 * 
 * Optional Parameters
 * -------------------
 * optInTimePeriod:(From 2 to 100000)
 *    Number of period
 * 
 * 
 */
TA_LIB_API TA_RetCode TA_RVSTDEV( int    startIdx,
                                  int    endIdx,
                                             const double inVolume[],
                                             int           optInTimePeriod, /* From 2 to 100000 */
                                             int          *outBegIdx,
                                             int          *outNBElement,
                                             double        outReal[] );

TA_LIB_API TA_RetCode TA_S_RVSTDEV( int    startIdx,
                                    int    endIdx,
                                               const float  inVolume[],
                                               int           optInTimePeriod, /* From 2 to 100000 */
                                               int          *outBegIdx,
                                               int          *outNBElement,
                                               double        outReal[] );

TA_LIB_API int TA_RVSTDEV_Lookback( int           optInTimePeriod );  /* From 2 to 100000 */

Here are the references in ta-lib-wrapper


####### _func.pxi


@wraparound(False)  # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def RVSTDEV( np.ndarray volume not None , int timeperiod=2**31 ):
    """ RVSTDEV(volume[, timeperiod=?])

    Relative Volume Standard Deviation (Volume Functions)

    Inputs:
        prices: ['volume']
    Parameters:
        timeperiod: 50
    Outputs:
        rvstdev
    """
    cdef:
        np.npy_intp length
        int begidx, endidx, lookback
        TA_RetCode retCode
        int outbegidx
        int outnbelement
        np.ndarray outReal
    volume = check_array(volume)
    length = volume.shape[0]
    begidx = check_begidx1(length, <double*>(volume.data))
    endidx = <int>length - begidx - 1
    lookback = begidx + lib.TA_RVSTDEV_Lookback( timeperiod )
    outReal = make_double_array(length, lookback)
    retCode = lib.TA_RVSTDEV( 0 , endidx , <double *>(volume.data)+begidx, timeperiod , &outbegidx , &outnbelement , <double *>(outReal.data)+lookback )
    _ta_check_success("TA_RVSTDEV", retCode)
    return outReal
	



######## _ta_lib.pxd



    TA_RetCode TA_RVSTDEV(int startIdx, int endIdx, const double inVolume[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outReal[])
    int TA_RVSTDEV_Lookback(int optInTimePeriod)







####### _stream.pxi



@wraparound(False)  # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def stream_RVSTDEV( np.ndarray volume not None , int timeperiod=2**31 ):
    """ RVSTDEV(volume[, timeperiod=?])

    Relative Volume Standard Deviation (Volume Functions)

    Inputs:
        prices: ['volume']
    Parameters:
        timeperiod: 50
    Outputs:
        rvstdev
    """
    cdef:
        np.npy_intp length
        TA_RetCode retCode
        double* volume_data
        int outbegidx
        int outnbelement
        double outReal
    volume = check_array(volume)
    volume_data = <double*>volume.data
    length = volume.shape[0]
    outReal = NaN
    retCode = lib.TA_RVSTDEV( <int>(length) - 1 , <int>(length) - 1 , volume_data, timeperiod , &outbegidx , &outnbelement , &outReal )
    _ta_check_success("TA_RVSTDEV", retCode)
    return outReal 


hhashim1 avatar Jun 28 '22 21:06 hhashim1

$ nm /usr/lib/libta_lib.so | grep TA_RVSTDEV

This may be getting past the scope of debugging we should be doing on this project's issues...

Since I dont have RVSTDEV anymore in the library, I dont get any output.

hhashim1 avatar Jun 28 '22 21:06 hhashim1

Since I dont have RVSTDEV anymore in the library, I dont get any output.

Yes, but you wanted it in the library so that we could fix the "unknown symbol" error in the python build? Reverting it of course will make it still not work?

mrjbq7 avatar Jun 28 '22 21:06 mrjbq7

Ok, so I deleted my RVSTDEV.c file and removed all the references from the talib library and also from ta-lib-wrapper. No errors anymore.

image

I need to add everything again but I want yall to look over my code to see if I missed anything. Here is RVSTDEV.c code - https://pastebin.com/LZ6iESy9

Here are all of the references in the talib library


############table_r.c



/* RVSTDEV BEGIN */
static const TA_InputParameterInfo * TA_RVSTDEV_Inputs[] =
{
  &TA_DEF_UI_Input_Price_V,
  NULL
};

static const TA_OutputParameterInfo* TA_RVSTDEV_Outputs[] =
{
  &TA_DEF_UI_Output_Real,
  NULL
};

static const TA_OptInputParameterInfo* TA_RVSTDEV_OptInputs[] =
{   
    &TA_DEF_UI_TimePeriod_50_MINIMUM2,
    NULL 
};

static const TA_InputParameterInfo* TA_RVSTDEV_StructParams[] = { NULL };

DEF_FUNCTION(RVSTDEV,                     /* name */
    TA_GroupId_VolumeIndicators, /* groupId */
    "Relative Volume St Dev", /* hint */
    "RvStdev",                         /* CamelCase name */
    0                              /* flags */
);
/* RVSTDEV END */



####### ta_frame.c


/* Generated */ TA_RetCode TA_RVSTDEV_FramePP( const TA_ParamHolderPriv *params,
/* Generated */                           int            startIdx,
/* Generated */                           int            endIdx,
/* Generated */                           int           *outBegIdx,
/* Generated */                           int           *outNBElement )
/* Generated */ {
/* Generated */    return TA_RVSTDEV(
/* Generated */                 startIdx,
/* Generated */                 endIdx,
/* Generated */                 params->in[0].data.inPrice.volume, /* inVolume */
/* Generated */                 params->optIn[0].data.optInInteger, /* optInTimePeriod*/
/* Generated */                 outBegIdx, 
/* Generated */                 outNBElement, 
/* Generated */                 params->out[0].data.outReal /*  outReal */ );
/* Generated */ }
/* Generated */ unsigned int TA_RVSTDEV_FramePPLB( const TA_ParamHolderPriv *params )
/* Generated */ {
/* Generated */    return TA_RVSTDEV_Lookback(params->optIn[0].data.optInInteger /* optInTimePeriod*/ );
/* Generated */ }




####### ta_frame.h



/* Generated */ TA_RetCode TA_RVSTDEV_FramePP( const TA_ParamHolderPriv *params,
/* Generated */                           int            startIdx,
/* Generated */                           int            endIdx,
/* Generated */                           int           *outBegIdx,
/* Generated */                           int           *outNBElement )
;
/* Generated */ unsigned int TA_RVSTDEV_FramePPLB( const TA_ParamHolderPriv *params )
;



######## ta_func.h



/*
 * TA_RVSTDEV - Relative Volume St Dev
 * 
 * Input  = Volume
 * Output = double
 * 
 * Optional Parameters
 * -------------------
 * optInTimePeriod:(From 2 to 100000)
 *    Number of period
 * 
 * 
 */
TA_LIB_API TA_RetCode TA_RVSTDEV( int    startIdx,
                                  int    endIdx,
                                             const double inVolume[],
                                             int           optInTimePeriod, /* From 2 to 100000 */
                                             int          *outBegIdx,
                                             int          *outNBElement,
                                             double        outReal[] );

TA_LIB_API TA_RetCode TA_S_RVSTDEV( int    startIdx,
                                    int    endIdx,
                                               const float  inVolume[],
                                               int           optInTimePeriod, /* From 2 to 100000 */
                                               int          *outBegIdx,
                                               int          *outNBElement,
                                               double        outReal[] );

TA_LIB_API int TA_RVSTDEV_Lookback( int           optInTimePeriod );  /* From 2 to 100000 */

Here are the references in ta-lib-wrapper


####### _func.pxi


@wraparound(False)  # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def RVSTDEV( np.ndarray volume not None , int timeperiod=2**31 ):
    """ RVSTDEV(volume[, timeperiod=?])

    Relative Volume Standard Deviation (Volume Functions)

    Inputs:
        prices: ['volume']
    Parameters:
        timeperiod: 50
    Outputs:
        rvstdev
    """
    cdef:
        np.npy_intp length
        int begidx, endidx, lookback
        TA_RetCode retCode
        int outbegidx
        int outnbelement
        np.ndarray outReal
    volume = check_array(volume)
    length = volume.shape[0]
    begidx = check_begidx1(length, <double*>(volume.data))
    endidx = <int>length - begidx - 1
    lookback = begidx + lib.TA_RVSTDEV_Lookback( timeperiod )
    outReal = make_double_array(length, lookback)
    retCode = lib.TA_RVSTDEV( 0 , endidx , <double *>(volume.data)+begidx, timeperiod , &outbegidx , &outnbelement , <double *>(outReal.data)+lookback )
    _ta_check_success("TA_RVSTDEV", retCode)
    return outReal
	



######## _ta_lib.pxd



    TA_RetCode TA_RVSTDEV(int startIdx, int endIdx, const double inVolume[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outReal[])
    int TA_RVSTDEV_Lookback(int optInTimePeriod)







####### _stream.pxi



@wraparound(False)  # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def stream_RVSTDEV( np.ndarray volume not None , int timeperiod=2**31 ):
    """ RVSTDEV(volume[, timeperiod=?])

    Relative Volume Standard Deviation (Volume Functions)

    Inputs:
        prices: ['volume']
    Parameters:
        timeperiod: 50
    Outputs:
        rvstdev
    """
    cdef:
        np.npy_intp length
        TA_RetCode retCode
        double* volume_data
        int outbegidx
        int outnbelement
        double outReal
    volume = check_array(volume)
    volume_data = <double*>volume.data
    length = volume.shape[0]
    outReal = NaN
    retCode = lib.TA_RVSTDEV( <int>(length) - 1 , <int>(length) - 1 , volume_data, timeperiod , &outbegidx , &outnbelement , &outReal )
    _ta_check_success("TA_RVSTDEV", retCode)
    return outReal 

@mrjbq7 @trufanov-nok any thoughts on this?

hhashim1 avatar Jun 29 '22 03:06 hhashim1

Since I dont have RVSTDEV anymore in the library, I dont get any output.

Yes, but you wanted it in the library so that we could fix the "unknown symbol" error in the python build? Reverting it of course will make it still not work?

I added everything again and I have the error of undefined symbol: TA_RVSTDEV

Here is the screenshot for the command you asked me to run. image

hhashim1 avatar Jun 29 '22 04:06 hhashim1

Hmm, question - we package the "compiled Cython" as C files, perhaps we need to make sure you regenerate those with the new definitions?

Can you try this in the ta-lib-wrapper?

$ make cython

# build in-place

Another idea, is perhaps there needs to be a _stream.pxi version generated using generate_stream.py, since when this initializes, it looks for all the defined funcs and binds stream versions of them.

mrjbq7 avatar Jun 29 '22 04:06 mrjbq7

Let's also verify that the compiled python module is linking against the expected TA-Lib C library:

$ cd ta-lib-wrapper

# on a mac
$ otool -L talib/_ta_lib*.so

# on a linux
$ ldd talib/_ta_lib*.so

mrjbq7 avatar Jun 29 '22 04:06 mrjbq7

Let's also verify that the compiled python module is linking against the expected TA-Lib C library:

$ cd ta-lib-wrapper

# on a mac
$ otool -L talib/_ta_lib*.so

# on a linux
$ ldd talib/_ta_lib*.so

image

hhashim1 avatar Jun 29 '22 12:06 hhashim1

Let's also verify that the compiled python module is linking against the expected TA-Lib C library:

$ cd ta-lib-wrapper

# on a mac
$ otool -L talib/_ta_lib*.so

# on a linux
$ ldd talib/_ta_lib*.so

image

@mrjbq7 @trufanov-nok any thoughts?

hhashim1 avatar Jun 29 '22 18:06 hhashim1

Could you share whole your code for ta-lib and wraper? I'll try to build it on my PC.

trufanov-nok avatar Jun 29 '22 18:06 trufanov-nok

Also, could you try to launch python3 and import ta-lib from other folder than wrapper's sorce code folder?

trufanov-nok avatar Jun 29 '22 18:06 trufanov-nok

I tried from root and its still the same. Do you want me to send you the RVSTDEV code only or all of the accomping code?

image

hhashim1 avatar Jun 29 '22 18:06 hhashim1

ta_RVSTDEV.zip

table_r.c

/* RVSTDEV BEGIN */
static const TA_InputParameterInfo* TA_RVSTDEV_Inputs[] =
{
  &TA_DEF_UI_Input_Price_V,
  NULL
};

static const TA_OutputParameterInfo* TA_RVSTDEV_Outputs[] =
{
  &TA_DEF_UI_Output_Real,
  NULL
};

static const TA_OptInputParameterInfo* TA_RVSTDEV_OptInputs[] =
{
  &TA_DEF_UI_TimePeriod_50_MINIMUM2,
  NULL
};

DEF_FUNCTION(RVSTDEV,                        /* name */
    TA_GroupId_VolumeIndicators,  /* groupId */
    "Relative Volume Standard Deviation",  /* hint */
    "Rvstdev",                      /* CamelCase name */
    TA_FUNC_FLG_UNST_PER        /* flags */
);
/* RVSTDEV END */

ta_frames.c


/* Generated */ {
/* Generated */    return TA_RVSTDEV(
/* Generated */                 startIdx,
/* Generated */                 endIdx,
/* Generated */                 params->in[0].data.inPrice.volume, /* inVolume */
/* Generated */                 params->optIn[0].data.optInInteger, /* optInTimePeriod*/
/* Generated */                 outBegIdx, 
/* Generated */                 outNBElement, 
/* Generated */                 params->out[0].data.outReal /*  outReal */ );
/* Generated */ }
/* Generated */ unsigned int TA_RVSTDEV_FramePPLB( const TA_ParamHolderPriv *params )
/* Generated */ {
/* Generated */    return TA_RVSTDEV_Lookback(params->optIn[0].data.optInInteger /* optInTimePeriod*/ );
/* Generated */ }

ta_frames.h

/* Generated */ TA_RetCode TA_RVSTDEV_FramePP( const TA_ParamHolderPriv *params,
/* Generated */                           int            startIdx,
/* Generated */                           int            endIdx,
/* Generated */                           int           *outBegIdx,
/* Generated */                           int           *outNBElement )
;
/* Generated */ unsigned int TA_RVSTDEV_FramePPLB( const TA_ParamHolderPriv *params )
;

_func.pxi


@wraparound(False)  # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def RVSTDEV( np.ndarray volume not None , int timeperiod=-2**31 ):
    """ RVSTDEV(volume[, timeperiod=?])

    Relative Volume Standard Deviation (Volume Indicators)

    Inputs:
        volume: (any ndarray)
    Parameters:
        timeperiod: 50
    Outputs:
        real
    """
    cdef:
        np.npy_intp length
        int begidx, endidx, lookback
        TA_RetCode retCode
        int outbegidx
        int outnbelement
        np.ndarray outreal
    volume = check_array(volume)
    length = volume.shape[0]
    begidx = check_begidx1(length, <double*>(volume.data))
    endidx = <int>length - begidx - 1
    lookback = begidx + lib.TA_RVSTDEV_Lookback( timeperiod )
    outreal = make_double_array(length, lookback)
    retCode = lib.TA_RVSTDEV( 0 , endidx , <double *>(volume.data)+begidx , timeperiod , &outbegidx , &outnbelement , <double *>(outreal.data)+lookback )
    _ta_check_success("TA_RVSTDEV", retCode)
    return outreal 

_ta_lib.pxd


    TA_RetCode TA_RVSTDEV(int startIdx, int endIdx, const double inVolume[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outReal[])
    int TA_RVSTDEV_Lookback(int optInTimePeriod)

_stream.pxi


@wraparound(False)  # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def stream_RVSTDEV( np.ndarray volume not None , int timeperiod=-2**31 ):
    """ RVSTDEV(volume[, timeperiod=?])

    Relative Strength Index (Momentum Indicators)

    Inputs:
        volume: (any ndarray)
    Parameters:
        timeperiod: 50
    Outputs:
        real
    """
    cdef:
        np.npy_intp length
        TA_RetCode retCode
        double* volume_data
        int outbegidx
        int outnbelement
        double outreal
    volume = check_array(volume)
    volume_data = <double*>volume.data
    length = volume.shape[0]
    outreal = NaN
    retCode = lib.TA_RVSTDEV( <int>(length) - 1 , <int>(length) - 1 , volume_data , timeperiod , &outbegidx , &outnbelement , &outreal )
    _ta_check_success("TA_RVSTDEV", retCode)
    return outreal 

@trufanov-nok here is all of the code

hhashim1 avatar Jun 29 '22 18:06 hhashim1

i meant the zips with whole folders of code

trufanov-nok avatar Jun 29 '22 19:06 trufanov-nok

oh all of talib and talibwrapper?

hhashim1 avatar Jun 29 '22 19:06 hhashim1

yes

trufanov-nok avatar Jun 29 '22 19:06 trufanov-nok

Wrapper - https://file.io/J55R6ed0CUQc LIbrary - https://file.io/ZAgye5WpLQsn

hhashim1 avatar Jun 29 '22 19:06 hhashim1

I've downloaded the Library but the Wrapper zip is seems to already expire. Could you reupload it?

trufanov-nok avatar Jun 29 '22 19:06 trufanov-nok

Try this https://file.io/YfpvtmIb1Tyi

hhashim1 avatar Jun 29 '22 20:06 hhashim1

I was able to build both and access to RVSTDEV from python, so the problem must be in environment.

Screenshot_20220630_005836

trufanov-nok avatar Jun 29 '22 21:06 trufanov-nok

I was able to build both and access to RVSTDEV from python, so the problem must be in environment.

Screenshot_20220630_005836

The name of the function in your copy is different. Your screenshot says RCSTDEV. My function was called RVSTDEV. Do you have one with a similar name or did you change the name?

hhashim1 avatar Jun 29 '22 22:06 hhashim1

I just made a mistake when call it. Then I call it with a right name.

trufanov-nok avatar Jun 29 '22 22:06 trufanov-nok

Oh sorry...I didnt see that.

hhashim1 avatar Jun 29 '22 22:06 hhashim1

What do you suggest for how to fix this?

hhashim1 avatar Jun 29 '22 22:06 hhashim1

find all libta_lib* files on your linux machine with sudo find / -iname "libta_lib*" and list the results

trufanov-nok avatar Jun 29 '22 22:06 trufanov-nok

image


root@aa3bed537114:/# sudo find / -iname "libta_lib*"
/usr/lib/libta_lib.a
/usr/lib/libta_lib.so
/usr/lib/libta_lib.so.0
/usr/lib/libta_lib.so.0.0.0
/usr/lib/libta_lib.la
find: ‘/proc/21033/map_files’: Permission denied
/ta-lib/src/.libs/libta_lib.a
/ta-lib/src/.libs/libta_lib.la
/ta-lib/src/.libs/libta_lib.lai
/ta-lib/src/.libs/libta_lib.so
/ta-lib/src/.libs/libta_lib.so.0
/ta-lib/src/.libs/libta_lib.so.0.0.0
/ta-lib/src/libta_lib.la

hhashim1 avatar Jun 29 '22 22:06 hhashim1

What says sudo find / -iname "_ta_lib.cpython*.so" ?

trufanov-nok avatar Jun 29 '22 22:06 trufanov-nok

image

hhashim1 avatar Jun 29 '22 22:06 hhashim1

you copied it wrong or something..

trufanov-nok avatar Jun 29 '22 22:06 trufanov-nok

Sorry I had a quote at the end. This list goes on though. Rest of the list is permission denied because its on the Windows drive.

image

hhashim1 avatar Jun 30 '22 03:06 hhashim1

@trufanov-nok I got it to work. I renamed my function and compiled it again and it seems to be working. I think it was just getting confused with STDDEV. I renamed my function to RVSTDDEV instead RVSTDEV

hhashim1 avatar Jun 30 '22 03:06 hhashim1

@trufanov-nok I need to get a couple of indicators coded and its over my head. Are you able to help me get these indicators done in talib? I will pay you for it.

hhashim1 avatar Aug 20 '22 17:08 hhashim1

@trufanov-nok thats great. Thank you. I will email you. You can delete your email address from the comment above now. I have copied it.

hhashim1 avatar Aug 21 '22 18:08 hhashim1

I've just passed the whole way from making a new indicator to adding it to the python's wrapper so I can write a little guide about that:

  1. Regarding addition of a new indicator to the TA-Lib library written in C language pls refer to this description: https://github.com/trufanov-nok/ta-lib-rt/wiki/Adding-new-TA-function
    and this thread. It's pretty relevant to original TA-Lib sourcecode.

Just note that the archive of ta-lib sources which are automatically downloaded by the wrapper when it's installed via pip isn't suitable for a new indicator addition bcs of lacking some files. So you'll need a dump of a whole latest sourcecode trunk from sourceforge. This have some consequences which will be shown later.

  1. After successful creation of a new C indicator and compilation of a C library, one need to install it with make install or something like that. Read in console and remember there include/ta-lib/ta_func.h is stored.
  2. Doublecheck that your new indicator is declared in this ta_func.h header. If it's not - you did something wrong in the C code.
  3. Find any other ta_func.h files in default folders of your system and delete them, bcs TA-lib's wrapper will automatically search this header in your system using default paths. If it find an old one before reach up-to-date one - it won't get any info about your new indicator. The default folders are '/usr/include', '/usr/local/include', '/opt/include', '/opt/local/include', '/opt/homebrew/include', 'c:\ta-lib\c\include'.
  4. Get the latest sources of TA-Lib's wrapper for python from this github project. The package in system repositories won't work and you'll need exactly the latest sources as they contain some fixes.
  5. Uninstall all other python wrappers of TA-lib with pip uninstall ta-lib. Execute it a few times until it report that no any TA-Lib packages are installed.
  6. In python's wrapper folder edit files ./tools/generate_func.py and ./tools/generate_stream.py. In both comment out (with #) lines from talib import abstract then comment out:
    func_info = abstract.Function(shortname).info
    defaults, documentation = abstract._get_defaults_and_docs(func_info)

then add after them:

    func_info = []
    defaults = []
    documentation = []

This will disable usage of TA-Lib's python package in these scripts for generation of documentation for indicators. It won't work anyway as scripts will parse your new ta_func.h, and will try to get a documentation for a new indicator which is unknown yet for the wrapper installed in the system. And according to the 6 I advised to uninstall all wrappers anyway, so it won't work.

  1. Launch python3 tools/generate_func.py. This will print a lot of text (which is actually a new content of a _func.pxi) to the console. Check the last lines. If there are any errors - most probably you failed to perform step 7. If __TA_FUNCTION_NAMES__ is empty (__TA_FUNCTION_NAMES__ = []) - script can't find any ta_func.h file. If it contains a list of indicators names, but there is no your new one - then script catch ups some old ta_func.h and you failed step 4.
  2. If everything is okay, overwrite files talib/_func.pxi and talib/_stream.pxi with a new content generated by these scripts, with:
python3 tools/generate_func.py > talib/_func.pxi
python3 tools/generate_stream.py > talib/_stream.pxi
  1. Unfortunately there are few other files that need to be patched to add info about your new indicator. And there is no script to autopatch them. These are declarations in _ta_lib.pxd and (optionally) groups description in __init__.py.
    Also, TA-Lib's wrapper is designed to be used with an old version of TA-Lib 0.4.24, while sourceforge's trunk contain 0.4.25. It has 3 more indicators than previous one. Which means that along with your new indicator you'll need to add the info about these 3. Indicators are: ACCBANDS, AVGDEV, IMI. Scripts will grab their description from ta_func.h and update talib/_func.pxi and talib/_stream.pxi, but following is still need to be added into _ta_lib.pxd manually:
    TA_RetCode TA_ACCBANDS(int startIdx, int endIdx, const double inHigh[], const double inLow[], const double inClose[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outRealUpperBand[], double outRealMiddleBand[], double outRealLowerBand[])
    int TA_ACCBANDS_Lookback(int optInTimePeriod)
    TA_RetCode TA_AVGDEV(int startIdx, int endIdx, const double inReal[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outReal[])
    int TA_AVGDEV_Lookback(int optInTimePeriod)
    TA_RetCode TA_IMI(int startIdx, int endIdx, const double inOpen[], const double inClose[], int optInTimePeriod, int *outBegIdx, int *outNBElement, double outReal[])
    int TA_IMI_Lookback(int optInTimePeriod)

Do not forget to add definitions of the function for your new indicator there too.

  1. Perform python3 setup.py build. In case of any errors check the messages. If it's about your new indicator - the problem is on step 10.
  2. Perform sudo python3 setup.py install to install the wrapper binary. Launch python3 and execute import talib to check if everything is ok. DO NOT DO THIS FROM WRAPPERS SOURCECODE FOLDER - it won't work as python will try to load a local copy and fail. You may print a list of talib functions available from python side with talib.__TA_FUNCTION_NAMES__ and get its version with talib.__version__ to make sure wrapper loads the right wrapper and library.
  3. You may then revert the changes made at step 7 and repeat step 9 to generate .pxi files with additional documentation. but this is optional.

trufanov-nok avatar Aug 25 '22 22:08 trufanov-nok