ta-lib-python
ta-lib-python copied to clipboard
TA-Lib
How to add new indicators to ta-lib?
I am trying to figure out how to add new indicators to ta-lib. I have copied the .c file in the ta_func folder and also added a header for the function in the ta_func.h file in the include folder. I have then compiled the library. I see the .c file in the right directory however I do not see the .o or .lo file. Additionally, python is unable to see my function.
Any help would be much appreciated.
I wrote an article about that for my fork which is also valid for original TA-Lib: https://github.com/trufanov-nok/ta-lib-rt/wiki/Adding-new-TA-function
@trufanov-nok this is great. Thank you for sharing this info. I am stuck though. I followed your instructions and added the file and also to the header in ta_func.c and then did a Build in VS but I do not see any changes in the indicator file which was created from the template. In other words, %%%GENCODE%%% has not been changed in the .c file.
Any suggestions on what could've gone wrong?
gencode.exe isn't launched in scope of a regular build process as it usually not needed. You need to execute it manually.
gencode is a tool which sources is a part of ta-lib project. It's not executed, but it's compiled during a build. If sourcecode was successfully compiled, gencode's binary should appear in ta-lib/c/bin/ folder. Launch it from there.
Can you please give me instructions on how to do that as I am not too familiar with this process? I am on a Windows machine.
If you're able to build ta-lib with VS, I suppose you have opened a VS project file supplied with sourcecode? Like "ta-lib/c/ide/vs2012/lib_proj/ta_lib.sln"?
I tried with VS 2022. I got some errors during the Build process. But in the end, the .c was not changed like it was supposed to. Here are some of the errors that I just noticed.
`Severity Code Description Project File Line Suppression State Error C2065 'TA_TSV_FramePP': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPLB': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPSI': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPS': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPBS': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPSF': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPSS': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390 Error C2065 'TA_TSV_FramePPSL': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\src\ta_abstract\tables\table_t.c 390
Severity Code Description Project File Line Suppression State Error LNK2019 unresolved external symbol _cnvtToUpperCase referenced in function _printDefines gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _cnvtChar referenced in function _printDefines gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _trimWhitespace referenced in function _gen_retcode gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _doubleToStr referenced in function _printOptInputValidation gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _printStateTestFunc referenced in function _doForEachFunctionPhase2 gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _print_xml referenced in function _doForEachFunctionXml gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1`
It's been a long time ago I didn't add any new indicators to ta-lib, so I need to recap with my own notes. It says:
... Ok, now we can build gencode. Is whole TA-Lib project can't be build (some code isn't generated yet), we must build gencode only. It could be done via IDE with help of project files from ta-lib/ta-lib/c/ide/ or ...
In VS you shall have a subprojects inside TA-Lib solution in a projects' tree. There should be a gencode item. Right click on it and choose "Bould this project only", or something like that. That will update the gencode executable. Let me know if there will be any issues in output. Then launch gencode from the folder to update a new .c file. Then build whole solution to include it into libraries' binary.
Which subproject do you want me to right-click on and then choose Build? Or do I need to do it for each of them?
Here is what I have...
Ok I had to download the right build tools and all and then do the Build. Here are the errors I have now.
Severity Code Description Project File Line Suppression State Error LNK2019 unresolved external symbol _cnvtToUpperCase referenced in function _printDefines gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _cnvtChar referenced in function _printDefines gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _trimWhitespace referenced in function _gen_retcode gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _doubleToStr referenced in function _printOptInputValidation gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _printStateTestFunc referenced in function _doForEachFunctionPhase2 gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1 Error LNK2019 unresolved external symbol _print_xml referenced in function _doForEachFunctionXml gen_code E:\Dropbox\Documents\Repos\ta-lib-rt\ta-lib-rt\c\ide\vs2022\lib_proj\gen_code\gen_code.obj 1
What do you mean by "the right build tools"? Please use VS 2022 you have.
Also I see "ta-lib-rt" in the paths. That's my fork of ta-lib - you don't need it. Use original sourcebase, like https://github.com/TA-Lib/ta-lib My fork has additional functions and won't work with wrapper in this project page.
Just open the VS 2022, right click on "gen_code" in a tree. And select "build this project only", or something like that.
I did use VS 2022 but it did not have build toolkit v143 and I had to install that. I will download the original repo and test.
Download it, build it. Check for errors. If there is no - then add a new file, apply the changes desrcibed in my notes, then build gen_code only, launch gen_code - it will update new file, then build the whole solution again.
Ok the build was successful. When I run gen_code.exe from the terminal, here is the error I get.
Yes. But the old repo does not have dotnet code in it so I don't know why it is looking for dotnet.
I don't know too. It's something from VS, I guess we can ignore it. Now you can build the whole solution and in terms of C library you will add a new empty function. Then, I guess if we are discussing this function under the python wrapper project page - you'll need to access it from python? We better to make sure the function is accessible from python side before filling it with a real code
Well, I cannot build the whole solution either. Here are the errors I am getting. TSV is my new function.
Severity Code Description Project File Line Suppression State Error C2065 'TA_TSV_FramePP': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib\src\ta_abstract\tables\table_t.c 271 Error C2065 'TA_TSV_FramePPLB': undeclared identifier ta_abstract E:\Dropbox\Documents\Repos\ta-lib\src\ta_abstract\tables\table_t.c 271 Error C2099 initializer is not a constant ta_abstract E:\Dropbox\Documents\Repos\ta-lib\src\ta_abstract\tables\table_t.c 266 Error LNK1181 cannot open input file '.\..\..\..\..\lib\ta_abstract_cdd.lib' ta_libc E:\Dropbox\Documents\Repos\ta-lib\ide\vs2022\lib_proj\ta_libc\LINK 1 Error MSB3073 The command "lib /OUT:.\..\..\..\..\lib\ta_libc_cdd.lib .\..\..\..\..\lib\ta_abstract_cdd.lib .\..\..\..\..\lib\ta_common_cdd.lib .\..\..\..\..\lib\ta_func_cdd.lib" exited with code 1181. ta_libc D:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.MakeFile.Targets 45 Error LNK1181 cannot open input file 'ta_libc_cdd.lib' ta_regtest E:\Dropbox\Documents\Repos\ta-lib\ide\vs2022\lib_proj\ta_regtest\LINK 1
Do you have a new file listed inside ta_func sources in VS? I guess you need to right click on ta_func project under ta_lib solution in VS and add existing file to this prject. Then rebuild.
The file was missing so I went ahead and added it but I am still getting the same error.
*_FramePP functions are supposed to be automatically generated by gencode.exe in src/ta_abstract/frames/ta_frame.c
You may check if there is a TA_TSV_FramePP declaration.
But I'm more interested in errors in ta_TSV.c which are next to this error. Is it related to your own code or the syntax error is in generated code? If first - remove your own code untill build problems would be resolved. If it's a syntax error in autogenerated code you might made some mistake in function declaration and gencode generated a wrong header. Please check what's wrong in ta_TSV.c?
Another interesting issue is "initializer is not constant" at line 266 in this file. I see "TA_G...." there and last is hidden by pop up hint (it's useless). What is it? There supposed to be one of constants from ta_def_ui.h:
extern const char TA_GroupId_MathOperatorsString[];
extern const char TA_GroupId_MathTransformString[];
extern const char TA_GroupId_OverlapStudiesString[];
extern const char TA_GroupId_VolatilityIndicatorsString[];
extern const char TA_GroupId_MomentumIndicatorsString[];
extern const char TA_GroupId_CycleIndicatorsString[];
extern const char TA_GroupId_VolumeIndicatorsString[];
extern const char TA_GroupId_PatternRecognitionString[];
extern const char TA_GroupId_StatisticString[];
extern const char TA_GroupId_PriceTransformString[];
which are
const char TA_GroupId_MathOperatorsString[] = "Math Operators";
const char TA_GroupId_MathTransformString[] = "Math Transform";
const char TA_GroupId_OverlapStudiesString[] = "Overlap Studies";
const char TA_GroupId_VolatilityIndicatorsString[] = "Volatility Indicators";
const char TA_GroupId_MomentumIndicatorsString[] = "Momentum Indicators";
const char TA_GroupId_CycleIndicatorsString[] = "Cycle Indicators";
const char TA_GroupId_VolumeIndicatorsString[] = "Volume Indicators";
const char TA_GroupId_PatternRecognitionString[] = "Pattern Recognition";
const char TA_GroupId_StatisticString[] = "Statistic Functions";
const char TA_GroupId_PriceTransformString[] = "Price Transform";
Have you tried to add a new group?
There is no mention of TA_TSV_FramePP in table_t.c file. I dont have any code in ta_TSV.c file. I copied over the template and renamed it. I have searched for that word and I cannot find it so I don't know where it is coming from. Here is the content of that file.
`/* TA-LIB Copyright (c) 1999-2008, Mario Fortier
- All rights reserved.
- Redistribution and use in source and binary forms, with or
- without modification, are permitted provided that the following
- conditions are met:
-
- Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
-
- Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
-
- Neither name of author nor the names of its contributors
- may be used to endorse or promote products derived from this
- software without specific prior written permission.
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
- FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
- REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
- INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
- WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
- OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
- EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */
/* List of contributors: *
- Initial Name/description
-
- MF Mario Fortier (Replace when you are the original author)
- Change history:
- MMDDYY BY Description
-
- XXXXXX MF Initial Version */
/**** START GENCODE SECTION 1 - DO NOT DELETE THIS LINE / %%%GENCODE%%% / END GENCODE SECTION 1 - DO NOT DELETE THIS LINE ***/ { / insert local variable here */
/**** START GENCODE SECTION 2 - DO NOT DELETE THIS LINE / %%%GENCODE%%% / END GENCODE SECTION 2 - DO NOT DELETE THIS LINE ****/
/* insert lookback code here. */
return 0; }
/**** START GENCODE SECTION 3 - DO NOT DELETE THIS LINE / %%%GENCODE%%% / END GENCODE SECTION 3 - DO NOT DELETE THIS LINE ***/ { / insert local variable here */
/**** START GENCODE SECTION 4 - DO NOT DELETE THIS LINE / %%%GENCODE%%% / END GENCODE SECTION 4 - DO NOT DELETE THIS LINE ****/
/* Insert TA function code here. */
/* Default return values */ VALUE_HANDLE_DEREF_TO_ZERO(outBegIdx); VALUE_HANDLE_DEREF_TO_ZERO(outNBElement);
return ENUM_VALUE(RetCode,TA_SUCCESS,Success); }
/**** START GENCODE SECTION 5 - DO NOT DELETE THIS LINE / %%%GENCODE%%% / END GENCODE SECTION 5 - DO NOT DELETE THIS LINE ****/`
I don't know anything about groups so no, I have not added a new group.
Would you be open to and available to do a google hangout session and work on this together so that we can troubleshoot it and fix the issue quickly?
I've just booted a Win machine and tried to add a new indicator with VS. And reproduced your problem. gen-code execution results in
./gen_code.exe
gen_code V0.6.0-dev (Apr 29 2022 19:22:13)
Now updating source code...
Cannot access [..\..\dotnet\src\Core\TA-Lib-Core.vcproj]
I never tried to use gencode on Win machine. It seems it's really looking for TA-Lib-Core.vcproj if was build with VS. There is a code for that: https://github.com/TA-Lib/ta-lib/blob/master/src/tools/gen_code/gen_code.c#L709
Give me some time I'll clear up with it under Win.
Ok, I figured out what was the problem. Apparently there are two versions of original TA-Lib sources. One is a complete sourcebase (with scripts to make a java, dotnet version of the lib etc.) and another is a subfolder of it. This shorten version of the code is enough to build the library from sources and seems to widely used for ex. if python wrapper need to build the lib on user machine before installation. But this minimal codebase isn't enough for gen_code to properly work and maintain the library.
So all you need is a proper sourcecode. I've made its copy long time ago and saved under original branch of my fork. To get it use: git clone -b original https://github.com/trufanov-nok/ta-lib-rt.git ta-lib. They will contain a dotnet\src\Core\TA-Lib-Core.vcproj which is required for gen_code.exe under Windows. Just do all the steps again. I tried - everything works. Except the fact that after the first launch of gen_code I found in ta_TSV.c a single %%%GENCODE%%% at the end of the file. That seems to be a gencode bug, just launch it again and it will replace this line with code properly.
I've also created an example project where last 5 commits demonstrate a steps of addition of a new function: https://github.com/trufanov-nok/ta-lib-new-indicator-example
I guess you may even copy ta_TSV.c from it into your project instead of invoking gen_code. As gen_code only purpose is to fill template with code and it's done only once. And if you need to add more indicators you can skkip gen_code step by copying this ta_TSV.c and renaming file and all TSV text in it.
Ok I was able to use gen_code by cloning your backup repo and everything worked. It generated the code and all that.
I am here at this point and not sure what I need to do with Visual Studio. What option should I choose from the Build menu or when I right-click on the Solution?
and rebuild whole project:
ta-lib/ta-lib/build$ cmake ..
ta-lib/ta-lib/build$ make
You either should right click on "Solution 'ta-lib'" on the very top of the tree and click Build or just press "Ctrl+Shift+B".
Ok I did a build and got an error for %%%GENCODE%%% I commented out that line and did the Build again and it was successful. How can I easily test the function to make sure it is available before I proceed further?
Ok I did a build and got an error for %%%GENCODE%%% I commented out that line and did the Build again and it was successful.
It's better not comment the %%%GENCODE%%% line. Uncomment it and launch gen_code again - it replace it with proper code. It seems there is a bug in gen_code of original ta-lib which doesn't allow it to replace all %%%GENCODE%%% at once, but after second launch it completes the replacement. But if you commented it out - it's ok too as this last piece of code is quite useless.
How can I easily test the function to make sure it is available before I proceed further?
The easiest way to check if dll contains the function (TA_TSV) is to list its export symbols and try to find it in them. I would use the old tool dependancy walker Or there is a suggestion to use a command line utility from VS command prompt: https://stackoverflow.com/a/6634722/841424
Or you can just open the dll or lib file in notepad and search for "TA_TSV" if you find something you may be pretty sure the code is in dll.
Oh, I recall that I actually made a tool for listing all the functions inside ta-lib rt. I guess it could be easely modified to list original TA-Lib functions: https://github.com/trufanov-nok/ta-gen
I'll check it.
UPD: No, it can't be compilled for Win so easy as it uses boost library...
so now the question is how can I take this library and install/compile it on a linux machine? I am using WSL on my windows box for docker. I am running Kafka and other tools on docker/Linux and need to get ta-lib there.
Here is the setup.sh file that I have which downloads, extracts and compiles the ta-lib from the internet. I need to replace it so that instead of downloading from the internet, the process would use my file.
Would you happen to have any idea on how to do this?
apt-get upgrade -y
apt-get update -y
apt install build-essential wget -y
wget https://artiya4u.keybase.pub/TA-lib/ta-lib-0.4.0-src.tar.gz
tar -xvf ta-lib-0.4.0-src.tar.gz
cd ta-lib/
./configure --prefix=/usr
/* this copy statement does not work */
docker cp E:/Dropbox/Downloads/ta-lib/c/include/ta_func.h deephaven-server-1:/ta-lib/include/ta_func.h
docker cp E:/Dropbox/Downloads/ta-lib/c/src/ta_func/ta_TSV.c deephaven-server-1:/ta-lib/src/ta_func/ta_TSV.c
/* ignore above copy statements */
make
make install
apt-get install python3-dev -y
pip install ta-lib
Well, I know a little about docker. Perhaps: docker run -v C:\mypath\ta-lib\c\:/etc/ta-lib
And in script replace lines
wget https://artiya4u.keybase.pub/TA-lib/ta-lib-0.4.0-src.tar.gz
tar -xvf ta-lib-0.4.0-src.tar.gz
cd ta-lib/
with just cd /etc/ta-lib/
Ok thanks. So, I need to copy the whole ta-lib directory and not just a single file, right?
You 're not copying anything. This supposed to create a virtual directory /etc/ta-lib inside docker, that in real will be pointing to C:\mypath\ta-lib\c\ filder. You should change C:\mypath\ta-lib\c\ to the talib\c\ folder on your windows machine. Then inside docker we are skipping the lines that downloading archieve with a sources and unpacking it. We just go to /etc/ta-lib/ with cd /etc/ta-lib/ and launching ta-lib's build commands.
So, I need to copy the whole ta-lib directory and not just a single file, right?
Ok, I got you now. Yes, it'll be easier to pass a whole library. Actually, I've realized that you need to do one more thing. Remember we add a link to ta_TSV.c to VS project? Thats for Windows, and now you're building in Linux-like environment.
The
./configure --prefix=/usr
make
is another buildchain and it doesn't know about ta_TSV.c like VS buildchain didn't know before. Now you need to add this file to make buildchain too. For that edit the ta-lib\c\CMakeLists.txt: find there a line
${ta-lib-rt_SOURCE_DIR}/c/src/ta_func/ta_TSF.c
and insert line
${ta-lib-rt_SOURCE_DIR}/c/src/ta_func/ta_TSV.c
after it.
Ok...here is the latest update. I got all this stuff working. I couldn't find ta_TSF.c listing so I copied ta_SMA.c and renamed it.
I was also able to modify the docker file to create the mount. Now I can see the files in my docker container however I am still unable to use my function in my IDE. It says cannot find method TSV.
When I try to manually compile in the container, I get an error for this command.
root@d2eb1d7b4e5e:/ta-lib# ./configure --prefix=/usr
bash: ./configure: No such file or directory
Any idea?
When I try to manually compile in the container, I get an error for this command.
That's a result of a difference between full sourcebase and the shorten version of it which is installed from https://artiya4u.keybase.pub/TA-lib/ta-lib-0.4.0-src.tar.gz
you need to call autoreconf --install before ./configure --prefix=/usr. This command will create a configure script in this folder.
I got this error now.
root@d2eb1d7b4e5e:/ta-lib# autoreconf --install
autoreconf: 'configure.ac' or 'configure.in' is required
which command did you used to launch a docker? You need to be in ./c/ folder. You one directory above it...
Here is the command I used to get into my container
docker exec -ti deephaven_server_1 /bin/bash
it need to be c in ta-lib/c as original build commands certanly need CMakeLists.txt in the folder they're executed,
Ok, let's try ./autogen.sh instead of autoreconf --install. Then ./configure --prefix=/usr
It's need to be said that if C sources are compiled without any problem the new function won't be accesible via python as python wrapper doesn't know about it. I mentioned it in https://github.com/mrjbq7/ta-lib/issues/515#issuecomment-1112517213 So if you reach the compilable code and import talib is working in python - let me know.
So, ./autogen.sh did not work. Here is the error. I changed the directory mount to be c
I also tried ./configure --prefix=/usr and I got the following error:
configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."
google says the problem seems to be in ^M characters displayed in errors in output: The issue is Git converted line endings to Windows format, and now you have extra carriage returns (^M).. So linux machine is looking not for /bin/sh but for /bin/sh^M where ^M is a character with code 0x0D. It seems git on Windows does automatically convert Linux line endings to Windows line endings when you cloned my repository. And would convert them back if you would push the changes to the git server. But now you're accessing the local folder from linux and line endings are wrong for linux.
Try following on Windows machine:
- Backup ta-lib sources, bcs we may damage them.
- Execute in ta-lib's folder
git config --global core.eol lf
git config --global core.autocrlf input
This will instruct git to use linux line endings even on windows and conver windows line endings to linux on checkout. 3. Most dangerous - execute:
git rm -rf --cached .
git reset --hard HEAD
This will delete (
rm) recursively (r) without prompt (-f), all files except those that you have edited (--cached), from the current directory (.). The reset then returns all of those files to a state where they have their true line endings (matching what's in the repo).
repo has files with linux line endings.
Our aim is to get proper line endings in scripts like autogen.sh. I guess line endings in .c files won't have any metter for linux compiler.
Didnt work. I tried all of the commands that we have been trying but nothing worked.
After
apt install build-essential wget -y
add
apt install automake -y
or replace apt install build-essential wget -y with apt install build-essential wget automake -y
automake package should install aclocal tool.
ok check this out. I tried ./configure --prefix=/usr after running the command you suggested ....wget automate -y and I got this. It started creating the Makefile but then it failed. I don't know if you will be able to see the image. I wanted to capture the whole thing. If you cannot then let me know and I will chop up the image to make the text bigger.
Now libtoolize is missing. It's a tool from package libtool. YOu need to add it to the installation list:
apt install build-essential wget -y with apt install build-essential wget automake libtool -y
Ok that worked. I ran apt install build-essential wget automake libtool -y followed by ./autogen.sh I then ran make which started the build process and then I ran 'make install' I uninstalled the old talib library and then reinstall using 'pip install ta-lib'.
I wanted to test in python if TSV is accessible but seems like it is not. Take a look at this screenshot. I created a dummy TSV function so whatever the default is from the template. I have NOT modified the function logic yet. What to you think I should do now?
Great, now you have an own TA-Lib library C code with a new function, and a process that builds and installs it in docker.
pip install ta-lib installs a python wrapper for this library from repositories that compiles in docker and allow you to access the C library from python environment. The problem is that this wrapper doesn't know about your new function. It's not discovering them in C code in compilation time - they are hardcoded. Thus you'll need to modify the python wrapper for ta-lib too. On its turn that means you need to do the same thing you did with ta-lib c codebase - you need to get your own wrapper copy and replace pip install ta-lib with building wrapper from your own local copy. That shouldn't be difficult to do.
First of all download wrapper sources (it's this github project) with git clone https://github.com/mrjbq7/ta-lib.git ta-lib-wrapper. Let's say they will be in C:\ta-lib-wrapper\ folder. Then share this folder to your docker, let's say it be in /etc/ta-lib-wrapper. Replace pip install ta-lib with
cd /etc/ta-lib-wrapper/
python setup.py install
python setup.py install is enough to compile and install ta-lib wrapper. There shouldn't be any problems with ^M endings as no .sh scripts are used.
All of this will do the same as pip install ta-lib does, but with your own local copy.
Note: that python setup.py install will autodetect a ta-lib library you installed with make install in /usr/lib, but in case it fails to autodetect paths to ta-lib one may explicitly point to them by executing:
export TA_LIBRARY_PATH=/usr/lib
export TA_INCLUDE_PATH=/usr/include
before python setup.py install. That's not mandatory in your docker case, but come in handy on Windows machine where autodetection isn't working. Would be useful to learn how to build the ta-lib wrapper on Windows machine too, so you can debug your functions without a docker.
To build wrapper on windows, open its folder in command line: cd c:\ta-lib-wrapper and then set up terminal environment variables with paths to your C library:
TA_LIBRARY_PATH="C:\mypath\ta-lib\c\include\"
TA_INCLUDE_PATH="C:\mypath\ta-lib\c\lib\"
python setup.py install
The Windows build process is most risky one. In case it wails with "Cannot find ta-lib library, installation may fail." - let me know.
Once you're able to build your own local copy of the wrapper under Windows and in docker environment the next step will be Addition of wrapper of TSV function to the wrapper's sourcecode.
Ok so I ran setup.py and eventually it was successful however when I run python and import talib, I get this error.
I'm not sure, we already verified taht c sources are buildable.
May it be bcs python is launched from ta-lib-wrapper folder? I'm not a python develper so not sure...
How about to cd /etc/ then python then import talib ?
It also sort of looks like you're building the pip install ta-lib version not the python3 setup.py install version you downloaded...
I am in the wrapper folder but I will redownload the repo just in case I made a mistake
I've just tried to git clone wrapper successfully python setup.py install then launched python and import talib didn't work with
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/ta-lib/talib/__init__.py", line 93, in <module>
from ._ta_lib import (
ModuleNotFoundError: No module named 'talib._ta_lib'
Then I closed python, cd out of wrapper folder and did the same - it worked. I guess that's just python trying to locate module in local folders if its name matches.
You can build it in-place so it doesn't install to your system python, and use from your ta-lib-wrapper directory
$ python3 setup.py build_ext --inplace
It makes sense that it's trying to load the un-built files when you're in that directory...
Ok I was able to make it work using the suggestion that @mrjbq7 shared and import talib worked however I still cannot access TSV.
Right, the wrapper doesn't "automatically" wrap all the functions, it has definitions for them in talib/_func.pxi. You can either manually add your own, or use the tools/generate_func.py script to have it process the TA-Lib header files.
I'd suggest manually adding it to start?
I found the file. Are there instructions on how to add my function to _func.pxi?
@mrjbq7 can I copy one of the other functions and just rename it or do I need to do more?
Well, I just copied the function from the template file and renamed it. I don't have any logic in it yet.
The inputs to be used in the real function are close, volume, length, avglength however for the dummy function I believe its only close.
I hope that makes sense.
Ok, great. Now we have a local copy of TA-Lib C sources with a newly added function and a local copy of TA-Lib python wrapper, both are propagated to docker and successfully buildable. Now we need to add a wrap function for TSV indicator to the python wrapper (to its local copy on Win machine). @mrjbq7 is a best expert in that as he's a creator of this wrapper.
I would note that wrapping C function would depend on this function arguments. How many optional arguments, how many input arguments (candles?) etc. This is so called function prorotype, it's generated by gen_code based on whatever you defined in ta_abstract/tables/table_#.c and declared in ta-lib/include/ta_func.h. You may open this file on Win machine (as the declaration is crossplatform) and tell us how TA_TSV declaration is looks like. I think that will help for sure to define a right wrapper function
Well, if it only takes close then this is what you'd need:
In talib/_ta_lib.pxd, add the function references to this block:
cdef extern from "ta-lib/ta_func.h":
...
TA_RetCode TA_TSV(int startIdx, int endIdx, const double inClose[], int *outBegIdx, int *outNBElement, double outReal[])
int TA_TSV_Lookback()
And then in talib/_func.pxi:
@wraparound(False) # turn off relative indexing from end of lists
@boundscheck(False) # turn off bounds-checking for entire function
def TSV( np.ndarray close not None ):
""" TSV(close)
TSV
Inputs:
prices: ['close']
Outputs:
real
"""
cdef:
np.npy_intp length
int begidx, endidx, lookback
TA_RetCode retCode
int outbegidx
int outnbelement
np.ndarray outreal
close = check_array(close)
length = close.shape[0]
begidx = check_begidx1(length, <double*>(close.data))
endidx = <int>length - begidx - 1
lookback = begidx + lib.TA_TSV_Lookback( )
outreal = make_double_array(length, lookback)
retCode = lib.TA_TSV( 0 , endidx , <double *>(close.data)+begidx, &outbegidx , &outnbelement , <double *>(outreal.data)+lookback )
_ta_check_success("TA_TSV", retCode)
return outreal
I generate all of these by looking through the ta_func.h and making them automatically, so if it makes sense that you'd be adding more indicators, that might be an easier way to go.
Do I add this block to the end of the file or somewhere in the middle?
cdef extern from "ta-lib/ta_func.h": ... TA_RetCode TA_TSV(int startIdx, int endIdx, const double inClose[], int *outBegIdx, int *outNBElement, double outReal[]) int TA_TSV_Lookback()
Line 195 starts that extern block, so anywhere in that section ... it's alphabetic right now so maybe slot it in after TA_TSF?
Ok done with both. Should I run setup.py again now or is there anything else I need to do?
@trufanov-nok I just realized that this time around I did not have an entry in ta_func.h for TSV. I have copied from TSF Does this look right?
* TA_TSV - Time Segmented Volume
*
* Input = double
* Output = double
*
* Optional Parameters
* -------------------
* optInTimePeriod:(From 2 to 100000)
* Number of period
*
*
*/
TA_LIB_API TA_RetCode TA_TSV( int startIdx,
int endIdx,
const double inReal[],
int optInTimePeriod, /* From 2 to 100000 */
int *outBegIdx,
int *outNBElement,
double outReal[] );
TA_LIB_API TA_RetCode TA_S_TSV( int startIdx,
int endIdx,
const float inReal[],
int optInTimePeriod, /* From 2 to 100000 */
int *outBegIdx,
int *outNBElement,
double outReal[] );
TA_LIB_API int TA_TSV_Lookback( int optInTimePeriod ); /* From 2 to 100000 */```
This is where I start using the Makefile...
# generate func and stream wrappers from the talib header files
$ make generate
# regenerate the talib/_ta_lib.c file with the new functions
$ make cython
# build it in-place for testing
$ make build
# test it
$ make test
# install it to be available system wide
$ make install
# prepare for new release
$ make sdist
I didn't have a timeperiod argument in my example code above, but you could add it...
I did not have an entry in ta_func.h for TSV
Why? You should not copy anything anywhere in C code - gen_code must do it for you. ta_func.h is a file generated by gen_code. You better not modify it on your own.
I'll doublecheck on Win machine if something wrong with ta_func.h generation. Meanwhile you may continue with a TSF copy, provided that this declaration matches ta_TSV.c
Ok I am going back and fixing the logic in my function. Is there an easy way to debug and test my logic from ta-lib or do I need to copy the code in a C Project in Visual Studio and then debug?
@trufanov-nok Well then for some reason gen_code did not do it. :-(
I've checked. gen_code does update ta_func.h. You also may see it in my sample: https://github.com/trufanov-nok/ta-lib-new-indicator-example/blob/master/ta-lib/c/include/ta_func.h#L5072
Perhaps you forgot to commit your changes before launching docker, bcs we've inserted a git reset HEAD in docker commands and this will reset all changes that were not commited. Have you commited the changes?
Upon make I am getting this error. What does this mean and how do I fix this?
This is what I have in table_t.c file
static const TA_InputParameterInfo *TA_TSV_Inputs[] =
{
&TA_DEF_UI_Input_Real,
NULL
};
static const TA_OutputParameterInfo *TA_TSV_Outputs[] =
{
&TA_DEF_UI_Output_Real,
NULL
};
static const TA_OptInputParameterInfo *TA_TSV_OptInputs[] = { NULL };
static const TA_InputParameterInfo *TA_TSV_StructParams[] = { NULL };
DEF_FUNCTION( TSV, /* name */
TA_GroupId_VolumeIndicators, /* groupId */
"Price Volume Trend", /* hint */
"Tsv", /* CamelCase name */
0 /* flags */
);
/* TSV END */```
Perhaps you forgot to commit your changes before launching docker, bcs we've inserted a git reset HEAD in docker commands and this will reset all changes that were not commited. Have you commited the changes?
After I saved the file, I ran ./configure --prefix=/usr and then make How do I commit the code, and at what point?
Then you need to launch gen_code again bcs you revert back most of changes it did when get reset .. was executed from the docker.
You need to commit all changes made since the latest commit of sourcebase.
To commit:
- If you created a new files - add them to the list of files tracked by
gitwith commandgit add path_to_file. In your case this is ta_TSV.c. - Commit all changed tracked files with `git commit -a -m "My changes".
git commands shall be executed from sourcecode root folder or its subfolders.