PyStrategies: Deep Learning Framework for HFT Trading Strategy Development

PyStrategies is a collection of Python tools for implementing, testing, and optimizing algorithmic trading strategies. This framework is built to be used with depth-of-book US equities data. Tools are included for deep learning, parameter optimization, and high-fidelity single-stock backtesting.

This project relies on another one of my libraries: PyLimitBook

The bottom of this README includes important setup instructions.

Toolset

Writing Strategies

Strategies are implemented by extending the BaseStrategy base class and adding logic for event callbacks. Refer to the Sample Strategy for an (unprofitable) example.

Backtesting a Strategy

A single-stock backtester is included that accurately simulates the limit order book.

To run a backtest:

    ./bin/backtest.sh quotes/XOM_BATS_2010-06-23.csv strategy/pyStrategy.py

Graphing Results

A sample Jupyter notebook is included to analyze signals generated from your strategy.

To generate data and interactively graph it:

    # Generate book snapshots - make sure CREATE_ON_NBBO_CHANGE_ONLY variable in the script is set appropriately
    python PyLimitBook/create_graphing_data.py ./quotes/XOM_BATS_2010-06-23.csv ./analyze/data/book.csv

    # Move signals generated by backtest
    mv ./strategy/signals_log.csv ./analyze/data/signals.csv

    jupyter notebook --notebook-dir=./analyze

Strategy Parameter Optimization

Optunity is used for multi-threaded parameter optimization. Parameter ranges must be defined in the parameters.csv file.

To run the parameter optimizer for 1000 iterations:

    # Specify parameter ranges in optimize/parameters.csv
    python optimize/optimize.py quotes/XOM_BATS_2010-06-23.csv strategy/pyStrategy.py 1000

Machine Learning

Wrapper scripts are included to generate features and do deep learning using Theano and Keras:

    # Generate features
    python machine_learning/generate_features.py quotes/XOM_BATS_2010-06-23.csv

    # Generate labels to predict
    python machine_learning/generate_labels.py quotes/XOM_BATS_2010-06-23.csv

    # Train the model
    python machine_learning/train_nn.py quotes/XOM_BATS_2010-06-23.csv

    # Test the model
    python machine_learning/test_nn.py quotes/XOM_BATS_2010-06-24.csv

Setup Instructions

Install Python libraries

1. Setup virtualenv: virtualenv --no-site-packages venv
2. Activate virtualenv: source venv/bin/activate
3. Install libraries: pip install -r requirements.txt

Install jpy for bridging Python and Java code

1. git clone https://github.com/bcdev/jpy.git
2. cd jpy
3. export JDK_HOME=`/usr/libexec/java_home`
4. export JAVA_HOME=$JDK_HOME
5. python setup.py --maven build
6. Copy lib/jpy-0.8.jar to the Java project's lib directory
7. Copy properties file from build/lib.*/jpyconfig.properties into the Java project's strategy directory

Install CuDNN for Speeding up Deep Learning

If your GPU supports CUDA, the machine learning with be much, much faster if you install CuDNN from NVIDIA.

1. Download CuDNN: https://developer.nvidia.com/rdp/cudnn-download

• The following configuration works on Mac OS X:
  • nvcc version: release 7.5, V7.5.26
  • cuda driver: 7.5.27
  • clang version: clang-703.0.29 (XCODE 7.3.0)
  • osx version: 10.11.4

2. Install libraries:

• cd to where you want the libraries to live

• Linux: export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH

• Mac: export DYLD_LIBRARY_PATH=`pwd`:$DYLD_LIBRARY_PATH

  • Copied *.h files to CUDA_ROOT/include and .so files to CUDA_ROOT/lib64

    • By default, CUDA_ROOT is /usr/local/cuda on Linux and /Developer/NVIDIA/CUDA-* on mac

  • Add the following to the end of ~/.profile:

    export PATH="/Developer/NVIDIA/CUDA-7.5/bin:$PATH"
    export DYLD_LIBRARY_PATH="/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH"
    

• Add the install path to your build and link process by adding -Iinstallpath to your compile line and -Linstallpath -lcudnn to your link line.

3. Create a ~/.theanorc file: (adjust cnmem as necessary)

    [global]
    device=gpu
    floatX=float32
    allow_gc=False
    warn_float64=warn
   
    [lib]
    cnmem=0.50
   
    [nvcc]
    fastmath=True