zipline-reloaded
zipline-reloaded copied to clipboard
stefan-jansen
ValueError: Parameter `start` received with timezone defined as 'UTC' although a Date must be timezone naive.
Dear Zipline Maintainers,
Before I tell you about my issue, let me describe my environment:
Environment
- Operating System: Linux inspiron 6.5.0-1-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.3-1 (2023-09-13) x86_64 GNU/Linux
- Python Version: Python 3.10.12
- Python Bitness: 64
- How did you install Zipline: conda
- Python packages: # packages in environment at /opt/conda3/envs/zipline:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
alembic 1.12.0 pyhd8ed1ab_0 conda-forge
alsa-lib 1.2.10 hd590300_0 conda-forge
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
asttokens 2.4.0 pyhd8ed1ab_0 conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge
bcolz-zipline 1.2.6 py310h1f7b6fc_1 conda-forge
beautifulsoup4 4.12.2 pyha770c72_0 conda-forge
blosc 1.21.5 h0f2a231_0 conda-forge
bottleneck 1.3.7 py310h1f7b6fc_1 conda-forge
brotli 1.1.0 hd590300_1 conda-forge
brotli-bin 1.1.0 hd590300_1 conda-forge
brotli-python 1.1.0 py310hc6cd4ac_1 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.19.1 hd590300_0 conda-forge
c-blosc2 2.10.5 hb4ffafa_0 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cairo 1.16.0 h0c91306_1017 conda-forge
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
cffi 1.16.0 py310h2fee648_0 conda-forge
charset-normalizer 3.3.0 pyhd8ed1ab_0 conda-forge
click 8.1.7 unix_pyh707e725_0 conda-forge
comm 0.1.4 pyhd8ed1ab_0 conda-forge
contourpy 1.1.1 py310hd41b1e2_1 conda-forge
cryptography 41.0.4 py310h75e40e8_0 conda-forge
cycler 0.12.0 pyhd8ed1ab_0 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
debugpy 1.8.0 py310hc6cd4ac_1 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
empyrical-reloaded 0.5.9 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.1.3 pyhd8ed1ab_0 conda-forge
exchange-calendars 4.2.8 pyhd8ed1ab_1 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.43.0 py310h2372a71_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
frozendict 2.3.8 py310h2372a71_1 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
glib 2.78.0 hfc55251_0 conda-forge
glib-tools 2.78.0 hfc55251_0 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
greenlet 2.0.2 py310hc6cd4ac_1 conda-forge
gst-plugins-base 1.22.6 h8e1006c_2 conda-forge
gstreamer 1.22.6 h98fc4e7_2 conda-forge
h5py 3.9.0 nompi_py310ha2ad45a_103 conda-forge
harfbuzz 8.2.1 h3d44ed6_0 conda-forge
hdf5 1.14.2 nompi_h4f84152_100 conda-forge
html5lib 1.1 pyh9f0ad1d_0 conda-forge
icu 73.2 h59595ed_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.8.0 pyha770c72_0 conda-forge
importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge
importlib_resources 6.1.0 pyhd8ed1ab_0 conda-forge
intervaltree 3.1.0 pyhd8ed1ab_1 conda-forge
ipykernel 6.25.2 pyh2140261_0 conda-forge
ipython 8.16.1 pyh0d859eb_0 conda-forge
iso3166 2.1.1 pyhd8ed1ab_0 conda-forge
iso4217 1.9.20220401 pyhd8ed1ab_0 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jupyter_client 8.3.1 pyhd8ed1ab_0 conda-forge
jupyter_core 5.3.2 py310hff52083_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.5 py310hd41b1e2_1 conda-forge
korean_lunar_calendar 0.3.1 pyhd8ed1ab_0 conda-forge
krb5 1.21.2 h659d440_0 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
lcms2 2.15 h7f713cb_2 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libaec 1.1.2 h59595ed_1 conda-forge
libblas 3.9.0 18_linux64_openblas conda-forge
libbrotlicommon 1.1.0 hd590300_1 conda-forge
libbrotlidec 1.1.0 hd590300_1 conda-forge
libbrotlienc 1.1.0 hd590300_1 conda-forge
libcap 2.69 h0f662aa_0 conda-forge
libcblas 3.9.0 18_linux64_openblas conda-forge
libclang 15.0.7 default_h7634d5b_3 conda-forge
libclang13 15.0.7 default_h9986a30_3 conda-forge
libcups 2.3.3 h4637d8d_4 conda-forge
libcurl 8.3.0 hca28451_0 conda-forge
libdeflate 1.19 hd590300_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libevent 2.1.12 hf998b51_1 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libflac 1.4.3 h59595ed_0 conda-forge
libgcc-ng 13.2.0 h807b86a_2 conda-forge
libgcrypt 1.10.1 h166bdaf_0 conda-forge
libgfortran-ng 13.2.0 h69a702a_2 conda-forge
libgfortran5 13.2.0 ha4646dd_2 conda-forge
libglib 2.78.0 hebfc3b9_0 conda-forge
libgomp 13.2.0 h807b86a_2 conda-forge
libgpg-error 1.47 h71f35ed_0 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libjpeg-turbo 2.1.5.1 hd590300_1 conda-forge
liblapack 3.9.0 18_linux64_openblas conda-forge
libllvm15 15.0.7 h5cf9203_3 conda-forge
libnghttp2 1.52.0 h61bc06f_0 conda-forge
libnsl 2.0.0 hd590300_1 conda-forge
libogg 1.3.4 h7f98852_1 conda-forge
libopenblas 0.3.24 pthreads_h413a1c8_0 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libpq 15.4 hfc447b1_2 conda-forge
libsndfile 1.2.2 hc60ed4a_1 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.43.0 h2797004_0 conda-forge
libssh2 1.11.0 h0841786_0 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_2 conda-forge
libsystemd0 254 h3516f8a_0 conda-forge
libta-lib 0.4.0 hd590300_2 conda-forge
libtiff 4.6.0 h29866fb_1 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp-base 1.3.2 hd590300_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxkbcommon 1.5.0 h5d7e998_3 conda-forge
libxml2 2.11.5 h232c23b_1 conda-forge
libxslt 1.1.37 h0054252_1 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
lru-dict 1.2.0 py310h2372a71_1 conda-forge
lxml 4.9.3 py310h9b7343a_1 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
lzo 2.10 h516909a_1000 conda-forge
mako 1.2.4 pyhd8ed1ab_0 conda-forge
markupsafe 2.1.3 py310h2372a71_1 conda-forge
matplotlib 3.8.0 py310hff52083_1 conda-forge
matplotlib-base 3.8.0 py310h62c0568_1 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mpg123 1.32.3 h59595ed_0 conda-forge
multipledispatch 0.6.0 py_0 conda-forge
multitasking 0.0.9 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mysql-common 8.0.33 hf1915f5_4 conda-forge
mysql-libs 8.0.33 hca2cd23_4 conda-forge
ncurses 6.4 hcb278e6_0 conda-forge
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
networkx 3.1 pyhd8ed1ab_0 conda-forge
nomkl 1.0 h5ca1d4c_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.94 h1d7d5a4_0 conda-forge
numexpr 2.8.7 py310hcc13569_102 conda-forge
numpy 1.26.0 py310hb13e2d6_0 conda-forge
openjpeg 2.5.0 h488ebb8_3 conda-forge
openssl 3.1.3 hd590300_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.1.1 py310hcc13569_1 conda-forge
pandas-datareader 0.10.0 pyh6c4a22f_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
patsy 0.5.3 pyhd8ed1ab_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
peewee 3.16.3 py310hbc90443_1 conda-forge
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.0.1 py310h29da1c1_1 conda-forge
pip 23.2.1 pyhd8ed1ab_0 conda-forge
pixman 0.42.2 h59595ed_0 conda-forge
platformdirs 3.11.0 pyhd8ed1ab_0 conda-forge
ply 3.11 py_1 conda-forge
prompt-toolkit 3.0.39 pyha770c72_0 conda-forge
prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge
psutil 5.9.5 py310h2372a71_1 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pulseaudio-client 16.1 hb77b528_5 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
py-cpuinfo 9.0.0 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.16.1 pyhd8ed1ab_0 conda-forge
pyluach 2.2.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pyqt 5.15.9 py310h04931ad_5 conda-forge
pyqt5-sip 12.12.2 py310hc6cd4ac_5 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
pytables 3.8.0 py310h374b01c_4 conda-forge
python 3.10.12 hd12c33a_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-interface 1.6.0 py_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.10 4_cp310 conda-forge
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
pyzmq 25.1.1 py310h5bbb5d0_1 conda-forge
qt-main 5.15.8 hc47bfe8_16 conda-forge
readline 8.2 h8228510_1 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
scipy 1.11.3 py310hb13e2d6_1 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
sip 6.7.11 py310hc6cd4ac_1 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.10 h9fff704_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
sqlalchemy 2.0.21 py310h2372a71_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
statsmodels 0.14.0 py310h1f7b6fc_2 conda-forge
ta-lib 0.4.28 py310h1f7b6fc_0 conda-forge
tk 8.6.13 h2797004_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.0 pyhd8ed1ab_0 conda-forge
tornado 6.3.3 py310h2372a71_1 conda-forge
traitlets 5.11.2 pyhd8ed1ab_0 conda-forge
typing-extensions 4.8.0 hd8ed1ab_0 conda-forge
typing_extensions 4.8.0 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
unicodedata2 15.1.0 py310h2372a71_0 conda-forge
urllib3 2.0.6 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.8 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 pyhd8ed1ab_2 conda-forge
wheel 0.41.2 pyhd8ed1ab_0 conda-forge
xcb-util 0.4.0 hd590300_1 conda-forge
xcb-util-image 0.4.0 h8ee46fc_1 conda-forge
xcb-util-keysyms 0.4.0 h8ee46fc_1 conda-forge
xcb-util-renderutil 0.3.9 hd590300_1 conda-forge
xcb-util-wm 0.4.1 h8ee46fc_1 conda-forge
xkeyboard-config 2.39 hd590300_0 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.6 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xf86vidmodeproto 2.3.1 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yfinance 0.2.31 pyhd8ed1ab_0 conda-forge
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zipline-reloaded 3.0.3 py310h278f3c1_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zlib-ng 2.0.7 h0b41bf4_0 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge
Description of Issue
I am trying to run a simple backtest based on the book "Trading Evolved" by Andreas Clenow. It is a slighty modified version of the BuyApple program from the tutorial:
# This ensures that our graphs will be shown properly in the notebook.
%matplotlib inline
# Import Zipline functions that we need
from zipline import run_algorithm
from zipline.api import order_target_percent, symbol
# Import date and time zone libraries
from datetime import datetime
import pytz
import pandas as pd
# Import visualization
import matplotlib.pyplot as plt
def initialize(context):
# Which stock to trade
context.stock = symbol('AAPL')
# Moving average window
context.index_average_window = 100
def handle_data(context, data):
# Request history for the stock
equities_hist = data.history(context.stock, 'close',
context.index_average_window, '1d')
# Check if price is above moving average
if equities_hist[-1] > equities_hist.mean():
stock_weight = 1.0
else:
stock_weight = 0.0
# Place order
order_target_percent(context.stock, stock_weight)
def analyze(context, perf):
fig = plt.figure(figsize=(12, 8))
# First chart
ax = fig.add_subplot(311)
ax.set_title('Strategy Results')
ax.semilogy(perf['portfolio_value'], linestyle='-',
label='Equity Curve', linewidth=3.0)
ax.legend()
ax.grid(False)
# Second chart
ax = fig.add_subplot(312)
ax.plot(perf['gross_leverage'],
label='Exposure', linestyle='-', linewidth=1.0)
ax.legend()
ax.grid(True)
# Third chart
ax = fig.add_subplot(313)
ax.plot(perf['returns'], label='Returns', linestyle='-.', linewidth=1.0)
ax.legend()
ax.grid(True)
# Set start and end date
start_date = datetime(1996, 1, 1, tzinfo=pytz.UTC)
end_date = datetime(2018, 12, 31, tzinfo=pytz.UTC)
# Fire off the backtest
results = run_algorithm(
start=start_date,
end=end_date,
initialize=initialize,
analyze=analyze,
handle_data=handle_data,
capital_base=10000,
data_frequency = 'daily', bundle='quandl'
)
-
What did you expect to happen? Backtesting results.
-
What happened instead? Error message
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[1], line 67
64 end_date = datetime(2018, 12, 31, tzinfo=pytz.UTC)
66 # Fire off the backtest
---> 67 results = run_algorithm(
68 start=start_date,
69 end=end_date,
70 initialize=initialize,
71 analyze=analyze,
72 handle_data=handle_data,
73 capital_base=10000,
74 data_frequency = 'daily', bundle='quandl'
75 )
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/zipline/utils/run_algo.py:397, in run_algorithm(start, end, initialize, capital_base, handle_data, before_trading_start, analyze, data_frequency, bundle, bundle_timestamp, trading_calendar, metrics_set, benchmark_returns, default_extension, extensions, strict_extensions, environ, custom_loader, blotter)
393 load_extensions(default_extension, extensions, strict_extensions, environ)
395 benchmark_spec = BenchmarkSpec.from_returns(benchmark_returns)
--> 397 return _run(
398 handle_data=handle_data,
399 initialize=initialize,
400 before_trading_start=before_trading_start,
401 analyze=analyze,
402 algofile=None,
403 algotext=None,
404 defines=(),
405 data_frequency=data_frequency,
406 capital_base=capital_base,
407 bundle=bundle,
408 bundle_timestamp=bundle_timestamp,
409 start=start,
410 end=end,
411 output=os.devnull,
412 trading_calendar=trading_calendar,
413 print_algo=False,
414 metrics_set=metrics_set,
415 local_namespace=False,
416 environ=environ,
417 blotter=blotter,
418 custom_loader=custom_loader,
419 benchmark_spec=benchmark_spec,
420 )
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/zipline/utils/run_algo.py:103, in _run(handle_data, initialize, before_trading_start, analyze, algofile, algotext, defines, data_frequency, capital_base, bundle, bundle_timestamp, start, end, output, trading_calendar, print_algo, metrics_set, local_namespace, environ, blotter, custom_loader, benchmark_spec)
100 trading_calendar = get_calendar("XNYS")
102 # date parameter validation
--> 103 if trading_calendar.sessions_distance(start, end) < 1:
104 raise _RunAlgoError(
105 "There are no trading days between %s and %s"
106 % (
(...)
109 ),
110 )
112 benchmark_sid, benchmark_returns = benchmark_spec.resolve(
113 asset_finder=bundle_data.asset_finder,
114 start_date=start,
115 end_date=end,
116 )
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/exchange_calendar.py:2275, in ExchangeCalendar.sessions_distance(self, start, end, _parse)
2258 def sessions_distance(self, start: Date, end: Date, _parse: bool = True) -> int:
2259 """Return the number of sessions in a range.
2260
2261 Parameters
(...)
2273 `end` then return will be negated.
2274 """
-> 2275 start, end = self._parse_start_end_dates(start, end, _parse)
2276 negate = end < start
2277 if negate:
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/exchange_calendar.py:2172, in ExchangeCalendar._parse_start_end_dates(self, start, end, _parse)
2170 if not _parse:
2171 return start, end
-> 2172 return parse_date(start, "start", self), parse_date(end, "end", self)
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/calendar_helpers.py:378, in parse_date(date, param_name, calendar, raise_oob)
375 ts = parse_timestamp(date, param_name, raise_oob=False, side="left", utc=False)
377 if ts.tz is not None:
--> 378 raise ValueError(
379 f"Parameter `{param_name}` received with timezone defined as '{ts.tz.zone}'"
380 f" although a Date must be timezone naive."
381 )
383 if not ts == ts.normalize():
384 raise ValueError(
385 f"Parameter `{param_name}` parsed as '{ts}' although a Date must have"
386 f" a time component of 00:00."
387 )
ValueError: Parameter `start` received with timezone defined as 'UTC' although a Date must be timezone naive.
Here is how you can reproduce this issue on your machine:
Reproduction Steps
- Ingest quandl bundle
- Run above program in Jupyter Lab or Notebook/the "BuyApple" from the tutorial has the same error message
- Error message ...
What steps have you taken to resolve this already?
Tried the solution outlined in https://stackoverflow.com/questions/61513097/how-to-solve-assertionerror-when-running-backtest-on-zipline without success
Anything else?
Could you kindly clarify the best way to provide start and end times for backtests? Given that I want to backtest with European and US markets, time zones are relevant to me. ...
Sincerely, Alex
The start and end dates use the time-zone of the trading_calendar that you pass in to run_algorithm(). In your example you don't specify the trading_calendar parameter so it defaults to the XNYS calendar which is New York time, but you can pass in a different trading_calendar e.g. XMAD. To use 24/7 which is UTC:
from zipline.utils.calendar_utils import get_calendar
run_algorithm(
...
trading_calendar=get_calendar("24/7")
)
@phelps-sg Thank you so much for your fast reply. Really appreciated! Unfortunately I still get the same error message when I use the "24/7" calendar:
ValueError: Parameter start received with timezone defined as 'UTC' although a Date must be timezone naive.
What am I missing? Below the complete program including your suggested changes. Did I misunderstood your suggestions? Alternatively, I am happy to use the XNYS calendar. How would I then have to modify the settings for start_date and end_date to use the XNYS calendar?
# This ensures that our graphs will be shown properly in the notebook.
%matplotlib inline
# Import Zipline functions that we need
from zipline import run_algorithm
from zipline.api import order_target_percent, symbol
from zipline.utils.calendar_utils import get_calendar
# Import date and time zone libraries
from datetime import datetime
import pytz
import pandas as pd
# Import visualization
import matplotlib.pyplot as plt
def initialize(context):
# Which stock to trade
context.stock = symbol('AAPL')
# Moving average window
context.index_average_window = 100
def handle_data(context, data):
# Request history for the stock
equities_hist = data.history(context.stock, 'close',
context.index_average_window, '1d')
# Check if price is above moving average
if equities_hist[-1] > equities_hist.mean():
stock_weight = 1.0
else:
stock_weight = 0.0
# Place order
order_target_percent(context.stock, stock_weight)
def analyze(context, perf):
fig = plt.figure(figsize=(12, 8))
# First chart
ax = fig.add_subplot(311)
ax.set_title('Strategy Results')
ax.semilogy(perf['portfolio_value'], linestyle='-',
label='Equity Curve', linewidth=3.0)
ax.legend()
ax.grid(False)
# Second chart
ax = fig.add_subplot(312)
ax.plot(perf['gross_leverage'],
label='Exposure', linestyle='-', linewidth=1.0)
ax.legend()
ax.grid(True)
# Third chart
ax = fig.add_subplot(313)
ax.plot(perf['returns'], label='Returns', linestyle='-.', linewidth=1.0)
ax.legend()
ax.grid(True)
# Set start and end date
start_date = datetime(1996, 1, 1, tzinfo=pytz.UTC)
end_date = datetime(2018, 12, 31, tzinfo=pytz.UTC)
# Fire off the backtest
results = run_algorithm(
trading_calendar=get_calendar("24/7"),
start=start_date,
end=end_date,
initialize=initialize,
analyze=analyze,
handle_data=handle_data,
capital_base=10000,
data_frequency = 'daily', bundle='quandl'
)
Zipline is complaining because you are trying to override the time-zone of the trading_calendar. When you create start_date and end_date you need to ensure they do not have any time-zone info, e.g.:
start_date = pd.Timestamp("1996-01-01")
end_date = pd.Timestamp("2018-12-31")
These dates now have an implicit time-zone of UTC because the trading_calendar is "24/7".
@phelps-sg I did some more experiement. For
trading_calendar=get_calendar("XNYS")
I can specify the times as follow:
start_date = pd.Timestamp('2004-1-1')
end_date = pd.Timestamp('2017-12-31')
How would I specify start_date and end_date for
trading_calendar=get_calendar("24/7")
Thank you so much for your help and pointing me in the right direction!
How would I specify start_date and end_date for trading_calendar=get_calendar("24/7")
What happens when you use the 24/7 calendar with the non-tz start and end dates- are you getting a different error?
I get the following error:
NotSessionError: Parameter session takes a session although received input that parsed to '1990-01-02 00:00:00' which is earlier than the first session of calendar '24/7' ('2003-10-10 00:00:00').
I have used the following time settings:
# Set start and end date
start_date = pd.Timestamp('2004-1-1')
end_date = pd.Timestamp('2017-12-31')
It seems like the dates get interpreted wrongly by the calendar.
I think it is complaining that some of the data in the bundle is outside of the range of the calendar. So there are three date ranges involved here:
- the date range of the back-test
- the date range of the data-set in the bundle
- date range that the trading calendar is defined over
I believe zipline is complaining that the range of 2 is not inside of the range of 3. The calendar logic is provided by this library:
https://github.com/gerrymanoim/exchange_calendars/blob/master/exchange_calendars/exchange_calendar.py
If you look at the code you will notice that the default earliest trading day is 20 years before the current date, which is where the 2003-10-10 date is coming from.
To fix this I would suggest overriding the start date when creating the calendar, something like:
get_calendar("24/7", start=pd.Timestamp("1990-01-01"))
If this doesn't work and generates another error, please include the full-stack trace so we can find the relevant code that generates the error.
Unfortunately I get the same error message even if I use a much shorter time period.
...
# Set start and end date
start_date = pd.Timestamp('2005-1-1')
end_date = pd.Timestamp('2015-12-31')
# Fire off the backtest
results = run_algorithm(
trading_calendar=get_calendar("24/7"),
...
gives the error message:
NotSessionError: Parameter session takes a session although received input that parsed to '1990-01-02 00:00:00' which is earlier than the first session of calendar '24/7' ('2003-10-12 00:00:00').
If I add your additional suggestion
..
trading_calendar=get_calendar("24/7", start=pd.Timestamp("2005-01-01")),
...
I still get
NotSessionError: Parameter session takes a session although received input that parsed to '1990-01-02 00:00:00' which is earlier than the first session of calendar '24/7' ('2003-10-12 00:00:00').
Here is the full message:
---------------------------------------------------------------------------
NotSessionError Traceback (most recent call last)
Cell In[6], line 68
65 end_date = pd.Timestamp('2015-12-31')
67 # Fire off the backtest
---> 68 results = run_algorithm(
69 trading_calendar=get_calendar("24/7",start=pd.Timestamp("2005-01-01")),
70 start=start_date,
71 end=end_date,
72 initialize=initialize,
73 analyze=analyze,
74 handle_data=handle_data,
75 capital_base=10000,
76 data_frequency = 'daily', bundle='quandl'
77 )
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/zipline/utils/run_algo.py:397, in run_algorithm(start, end, initialize, capital_base, handle_data, before_trading_start, analyze, data_frequency, bundle, bundle_timestamp, trading_calendar, metrics_set, benchmark_returns, default_extension, extensions, strict_extensions, environ, custom_loader, blotter)
393 load_extensions(default_extension, extensions, strict_extensions, environ)
395 benchmark_spec = BenchmarkSpec.from_returns(benchmark_returns)
--> 397 return _run(
398 handle_data=handle_data,
399 initialize=initialize,
400 before_trading_start=before_trading_start,
401 analyze=analyze,
402 algofile=None,
403 algotext=None,
404 defines=(),
405 data_frequency=data_frequency,
406 capital_base=capital_base,
407 bundle=bundle,
408 bundle_timestamp=bundle_timestamp,
409 start=start,
410 end=end,
411 output=os.devnull,
412 trading_calendar=trading_calendar,
413 print_algo=False,
414 metrics_set=metrics_set,
415 local_namespace=False,
416 environ=environ,
417 blotter=blotter,
418 custom_loader=custom_loader,
419 benchmark_spec=benchmark_spec,
420 )
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/zipline/utils/run_algo.py:163, in _run(handle_data, initialize, before_trading_start, analyze, algofile, algotext, defines, data_frequency, capital_base, bundle, bundle_timestamp, start, end, output, trading_calendar, print_algo, metrics_set, local_namespace, environ, blotter, custom_loader, benchmark_spec)
159 click.echo(algotext)
161 first_trading_day = bundle_data.equity_minute_bar_reader.first_trading_day
--> 163 data = DataPortal(
164 bundle_data.asset_finder,
165 trading_calendar=trading_calendar,
166 first_trading_day=first_trading_day,
167 equity_minute_reader=bundle_data.equity_minute_bar_reader,
168 equity_daily_reader=bundle_data.equity_daily_bar_reader,
169 adjustment_reader=bundle_data.adjustment_reader,
170 future_minute_reader=bundle_data.equity_minute_bar_reader,
171 future_daily_reader=bundle_data.equity_daily_bar_reader,
172 )
174 pipeline_loader = USEquityPricingLoader.without_fx(
175 bundle_data.equity_daily_bar_reader,
176 bundle_data.adjustment_reader,
177 )
179 def choose_loader(column):
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/zipline/data/data_portal.py:274, in DataPortal.__init__(self, asset_finder, trading_calendar, first_trading_day, equity_daily_reader, equity_minute_reader, future_daily_reader, future_minute_reader, adjustment_reader, last_available_session, last_available_minute, minute_history_prefetch_length, daily_history_prefetch_length)
270 self._first_trading_day = first_trading_day
272 # Get the first trading minute
273 self._first_trading_minute = (
--> 274 self.trading_calendar.session_first_minute(self._first_trading_day)
275 if self._first_trading_day is not None
276 else (None, None)
277 )
279 # Store the locs of the first day and first minute
280 self._first_trading_day_loc = (
281 self.trading_calendar.sessions.get_loc(self._first_trading_day)
282 if self._first_trading_day is not None
283 else None
284 )
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/exchange_calendar.py:1065, in ExchangeCalendar.session_first_minute(self, session, _parse)
1063 """Return first trading minute of a given session."""
1064 nanos = self.first_minutes_nanos
-> 1065 return self._get_session_minute_from_nanos(session, nanos, _parse)
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/exchange_calendar.py:1057, in ExchangeCalendar._get_session_minute_from_nanos(self, session, nanos, _parse)
1054 def _get_session_minute_from_nanos(
1055 self, session: Session, nanos: np.ndarray, _parse: bool
1056 ) -> pd.Timestamp:
-> 1057 idx = self._get_session_idx(session, _parse=_parse)
1058 return pd.Timestamp(nanos[idx], tz=UTC)
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/exchange_calendar.py:973, in ExchangeCalendar._get_session_idx(self, session, _parse)
971 def _get_session_idx(self, session: Date, _parse=True) -> int:
972 """Index position of a session."""
--> 973 session_ = parse_session(self, session) if _parse else session
974 if TYPE_CHECKING:
975 assert isinstance(session_, pd.Timestamp)
File /opt/conda3/envs/zipline/lib/python3.10/site-packages/exchange_calendars/calendar_helpers.py:433, in parse_session(calendar, session, param_name)
431 ts = parse_date(session, param_name, raise_oob=False)
432 if calendar._date_oob(ts) or not calendar.is_session(ts, _parse=False):
--> 433 raise errors.NotSessionError(calendar, ts, param_name)
434 return ts
NotSessionError: Parameter `session` takes a session although received input that parsed to '1990-01-02 00:00:00' which is earlier than the first session of calendar '24/7' ('2003-10-12 00:00:00').
As discussed, there are three different periods: 1. The period of the back-test, 2. the period over which the data is defined, 3. the period over which the calendar is defined. If you change the period of the back-test that will not change the period over which the data defined. I believe the problem occurs because the period over which the data is defined is not within the period over which the calendar is defined.
You can see how the calendar is constructed by looking at src.zipline.utils.calendar_utils.get_calendar. The function is defined as follows:
@wrap_with_signature(inspect.signature(ec_get_calendar))
def get_calendar(*args, **kwargs):
if args[0] in ["us_futures", "CMES", "XNYS", "NYSE"]:
return ec_get_calendar(*args, side="right", start=pd.Timestamp("1990-01-01"))
return ec_get_calendar(*args, side="right")
So zipline provides its own wrapper around the exchange calendars library, but unfortunately it looks like it doesn't pass in kwargs, so the start parameter is being ignored.
If you call exchange calendars directly you should be able to construct a calendar with a much earlier start date. It is important to make this earlier than the start of your trading session. If my hypothesis is correct then you need to make the start date earlier than the earliest data point in the ingested data.
Therefore try the following:
from exchange_calendars import get_calendar as ec_get_calendar
cal = ec_get_calendar("24/7", start=pd.Timestamp("1990-01-01"))
print(cal.schedule)
If the calendar is constructed correctly you should see that the schedule starts from the specified date, e.g.:
open break_start break_end close
1990-01-01 1990-01-01 00:00:00+00:00 NaT NaT 1990-01-02 00:00:00+00:00
1990-01-02 1990-01-02 00:00:00+00:00 NaT NaT 1990-01-03 00:00:00+00:00
1990-01-03 1990-01-03 00:00:00+00:00 NaT NaT 1990-01-04 00:00:00+00:00
1990-01-04 1990-01-04 00:00:00+00:00 NaT NaT 1990-01-05 00:00:00+00:00
1990-01-05 1990-01-05 00:00:00+00:00 NaT NaT 1990-01-06 00:00:00+00:00
... ... ... ... ...
2024-10-09 2024-10-09 00:00:00+00:00 NaT NaT 2024-10-10 00:00:00+00:00
2024-10-10 2024-10-10 00:00:00+00:00 NaT NaT 2024-10-11 00:00:00+00:00
2024-10-11 2024-10-11 00:00:00+00:00 NaT NaT 2024-10-12 00:00:00+00:00
2024-10-12 2024-10-12 00:00:00+00:00 NaT NaT 2024-10-13 00:00:00+00:00
2024-10-13 2024-10-13 00:00:00+00:00 NaT NaT 2024-10-14 00:00:00+00:00
If this works then in your backtest do something like:
from exchange_calendars import get_calendar as ec_get_calendar
cal = ec_get_calendar("24/7", start=pd.Timestamp("1990-01-01"))
run_algorithm(
...
trading_calendar=cal
)
Do not change the start date in the code above to match the date of your back-test- it is important that the start date above is earlier than any date that occurs in the ingested data.
Apologies for the late reply, I was traveling. Unfortunately it still is not working and it is getting more mysterious to me.
Starting with the initial version:
from zipline.utils.calendar_utils import get_calendar
# Set start and end date
start_date = pd.Timestamp('2005-1-1')
end_date = pd.Timestamp('2015-12-31')
# Fire off the backtest
results = run_algorithm(
trading_calendar=get_calendar("XNYS"),
...
)
I get a correct looking equity curve:
BTW: I get the same curve for the calendar "NYSE". What is the specific difference between these two calendars?
from exchange_calendars import get_calendar as ec_get_calendar
cal = ec_get_calendar("24/7", start=pd.Timestamp("1990-01-01"))
run_algorithm(
...
trading_calendar=cal
)
I get the following result:
I also cannot mix and match the two versions. if I try:
from exchange_calendars import get_calendar as ec_get_calendar
cal = ec_get_calendar("XNYS", start=pd.Timestamp("1990-01-01"))
run_algorithm(
...
trading_calendar=cal
)
I get the following error:
AssertionError: All readers must share target trading_calendar. Reader=<zipline.data.bcolz_minute_bars.BcolzMinuteBarReader object at 0x7f36f89316f0> for type=<class 'zipline.assets._assets.Equity'> uses calendar=<exchange_calendars.exchange_calendar_xnys.XNYSExchangeCalendar object at 0x7f36f8762110> which does not match the desired shared calendar=<exchange_calendars.exchange_calendar_xnys.XNYSExchangeCalendar object at 0x7f36f82680d0>
Printing the calendar 24/7 gives the following result:
open break_start break_end \
1990-01-01 1990-01-01 00:00:00+00:00 NaT NaT
1990-01-02 1990-01-02 00:00:00+00:00 NaT NaT
1990-01-03 1990-01-03 00:00:00+00:00 NaT NaT
1990-01-04 1990-01-04 00:00:00+00:00 NaT NaT
1990-01-05 1990-01-05 00:00:00+00:00 NaT NaT
... ... ... ...
2024-10-14 2024-10-14 00:00:00+00:00 NaT NaT
2024-10-15 2024-10-15 00:00:00+00:00 NaT NaT
2024-10-16 2024-10-16 00:00:00+00:00 NaT NaT
2024-10-17 2024-10-17 00:00:00+00:00 NaT NaT
2024-10-18 2024-10-18 00:00:00+00:00 NaT NaT
close
1990-01-01 1990-01-02 00:00:00+00:00
1990-01-02 1990-01-03 00:00:00+00:00
1990-01-03 1990-01-04 00:00:00+00:00
1990-01-04 1990-01-05 00:00:00+00:00
1990-01-05 1990-01-06 00:00:00+00:00
... ...
2024-10-14 2024-10-15 00:00:00+00:00
2024-10-15 2024-10-16 00:00:00+00:00
2024-10-16 2024-10-17 00:00:00+00:00
2024-10-17 2024-10-18 00:00:00+00:00
2024-10-18 2024-10-19 00:00:00+00:00
I am happy that at least one version is working now but I am not sure I understand why and what is wrong with the other versions.
BTW: I get the same curve for the calendar "NYSE". What is the specific difference between these two calendars?
There is no difference. NYSE is just an alias for XNYS. You can see this in the code for the exchange_calendars library at line 125 here: https://github.com/gerrymanoim/exchange_calendars/blob/master/exchange_calendars/calendar_utils.py
nderstand why and what is wrong with the other versions.
I suspect that the issue is that at daily frequency zipline is trying to trade at 0:00 UTC each day (as specified by the trading calendar) but when it looks at the calendar for the asset it is seeing the exchange for the asset is closed at that time.
If your strategy is trading across different exchanges in different time-zones, then you may need to configure it to trade at minute frequency, and then code the strategy so that it only place trades when each exchange opens. I haven't done this myself though, so this is entirely speculative, and you would need to do further testing and experimentation to see whether this is the case.
Alternatively, if your strategy is always trading within the same time-zone and holidays, but you need to take into account that your local time is different from the exchange time, then you can simply manually adjust the start and end dates by +/- 1 day as required and then use a trading calendar with the same zone as the exchange.
I get the following error:
AssertionError: All readers must share target trading_calendar. Reader=<zipline.data.bcolz_minute_bars.BcolzMinuteBarReader object at 0x7f36f89316f0> for type=<class 'zipline.assets._assets.Equity'> uses calendar=<exchange_calendars.exchange_calendar_xnys.XNYSExchangeCalendar object at 0x7f36f8762110> which does not match the desired shared calendar=<exchange_calendars.exchange_calendar_xnys.XNYSExchangeCalendar object at 0x7f36f82680d0>
This may or may not help but I think the AssertionError is a known issue where the calendar used to create the bundle is not the exact same object as the calendar you create when simulating. See section entitled 'Patch to allow calendars other than US calendars for backtesting' from https://pypi.org/project/zipline-norgatedata/#patch-to-allow-calendars-other-than-us-calendars-for-backtesting for a fix
Closing for lack of activity. Feel free to reopen if the issue persists.
