DataFrame constructor not properly called!

[pabx06]

Problem Description

%debug results = pd.read_pickle('results.pickle') returns, positions, transactions = pf.utils.extract_rets_pos_txn_from_zipline(results)

ipdb> w
  <ipython-input-20-b101fe95867c>(2)<module>()
      1 results = pd.read_pickle('results.pickle')
----> 2 returns, positions, transactions = pf.utils.extract_rets_pos_txn_from_zipline(results)

> /home/user/.local/lib/python3.6/site-packages/pyfolio/utils.py(156)extract_rets_pos_txn_from_zipline()
    154     raw_positions = []
    155     for dt, pos_row in backtest.positions.iteritems():
--> 156         df = pd.DataFrame(pos_row)
    157         df.index = [dt] * len(df)
    158         raw_positions.append(df)

  /usr/lib/python3/dist-packages/pandas/core/frame.py(404)__init__()
    402                                          dtype=values.dtype, copy=False)
    403             else:
--> 404                 raise ValueError('DataFrame constructor not properly called!')
    405 
    406         NDFrame.__init__(self, mgr, fastpath=True)

ipdb> pos_row
0

Please provide the full traceback:

[Paste traceback here]

Please provide any additional information below:

Versions

• Pyfolio version:
• Python version:
• Pandas version:
• Matplotlib version:

[zaherweb]

i think you should take a look for 'pos_row' variable as pd.Dataframe() argument should be first transformed and be in array . for example : doc_array = (count_vector.transform(documents)).toarray() frequency_matrix = pd.DataFrame(data=doc_array)

[FrankPSch]

Hi Laadhari, hi Mamoud,

I have got the same error with 'pf.utils.extract_rets_pos_txn_from_zipline(results)'. It works with one of my algos, but a copied version with modifications does not work, so it must be in the 'results' dataframe. Visually, I can't see any structural differences. Is there a solution on how to enhance the 'results' datafram so the error does not occur?

Thank you, Frank

The zipline code is:

`def extract_rets_pos_txn_from_zipline(backtest): """ Extract returns, positions, transactions and leverage from the backtest data structure returned by zipline.TradingAlgorithm.run(). The returned data structures are in a format compatible with the rest of pyfolio and can be directly passed to e.g. tears.create_full_tear_sheet().

Parameters
----------
backtest : pd.DataFrame
    DataFrame returned by zipline.TradingAlgorithm.run()

Returns
-------
returns : pd.Series
    Daily returns of strategy.
     - See full explanation in tears.create_full_tear_sheet.
positions : pd.DataFrame
    Daily net position values.
     - See full explanation in tears.create_full_tear_sheet.
transactions : pd.DataFrame
    Prices and amounts of executed trades. One row per trade.
     - See full explanation in tears.create_full_tear_sheet.


Example (on the Quantopian research platform)
---------------------------------------------
>>> backtest = my_algo.run()
>>> returns, positions, transactions =
>>>     pyfolio.utils.extract_rets_pos_txn_from_zipline(backtest)
>>> pyfolio.tears.create_full_tear_sheet(returns,
>>>     positions, transactions)
"""
backtest.index = backtest.index.normalize()
if backtest.index.tzinfo is None:
    backtest.index = backtest.index.tz_localize('UTC')
returns = backtest.returns
raw_positions = []
for dt, pos_row in backtest.positions.iteritems():
    df = pd.DataFrame(pos_row)
    df.index = [dt] * len(df)
    raw_positions.append(df)
if not raw_positions:
    raise ValueError("The backtest does not have any positions.")
positions = pd.concat(raw_positions)
positions = pos.extract_pos(positions, backtest.ending_cash)
transactions = txn.make_transaction_frame(backtest.transactions)
if transactions.index.tzinfo is None:
    transactions.index = transactions.index.tz_localize('utc')

return returns, positions, transactions

`

[FrankPSch]

Workaround in my case - skip evaluation positions and transactions, focus on returns for the moment ;)

# returns, positions, transactions = pf.utils.extract_rets_pos_txn_from_zipline(results) returns = results.returns

[linehammer]

It seems a string representation isn't satisfying enough for the DataFrame constructor. This means that you are providing a string representation of a dict to DataFrame constructor, and not a dict itself. So this is the reason you get that error. Just simply adding the list() around the dictionary items,:

df=pd.DataFrame(list(test_dict.items()),columns=['col1','col2'])

which will solve the error.