Problems when portfolio value approaches negative

[joseortiz3]

In my experience, when a portfolio becomes infeasible, Policy.get_trades "defaults to no-trades" (usually because it has become close to zero value, and the single leverage limit constraint turns infeasible), the logger built into the Result classes calls

    def log_policy(self, t, exec_time):
        self.log_data("policy_time", t, exec_time)
        # TODO mpo policy requires changes in the optimization_log methods
        if not isinstance(self.policy, MultiPeriodOpt):
            for cost in self.policy.costs:
                self.log_data("policy_" + cost.__class__.__name__,
                                  t, cost.optimization_log(t))

However, the various Costs (HcostModel at least), if the problem is infeasible, will have a self.expression that is None. Thus the following throws a NoneType has no attribute 'A1' error.

    def optimization_log(self, t):
        # BAD: A1 MAY NOT EXIST IF PROBLEM INFEASIBLE
        return self.expression.value.A1

I have modified log_policy so that if this occurs, the value that is logged is simply np.NaN. Otherwise, the entire simulation halts.

What is A1, and what value should be logged if Cost.expression.value is None?

---

Also, sometimes the portfolio value drops below zero. For instance, losses on short positions are not bounded. In such cases, this assertion in MultiPeriodOpt fails:

class MultiPeriodOpt(SinglePeriodOpt):
        def get_trades(self, portfolio, t=pd.datetime.today()):
            value = sum(portfolio)
            assert (value > 0.) # This assumption can actually break.
            #...

What should occur here?

---

Also, when the value becomes negative, the nonlinear "second_term" produces a complex number and breaks the optimization.

class TcostModel(BaseCost):

    def _estimate(self, t, w_plus, z, value):
        # omitting lots of code
        second_term = time_locator(self.nonlin_coeff, t) * \
            time_locator(self.sigma, t) * \
            (value / time_locator(self.volume, t))**(self.power - 1) 
        # If value becomes negative, this produces a complex number.

And then when the portfolio becomes negative, it continues trading.

Maybe there is a simpler solution that retrofitting all the code to accept negative value?

Thanks, Joey

P.S. I've been a long-time lurker to this project

[ghost]

Thanks! These are all valid observations.

1. Yup, I should change that. self.expression.value.A1 converts to numpy array (by default, cvxpy uses numpy matrices). It's kind of annoying, but unfortunately numpy has some inconsistencies there (i.e., there should be no difference between a n x 1 matrix and a n column vector, but there is).

2. If the portfolio value drops below zero, you've gone bankrupt. Stopping the simulation is the correct thing to do. However, it would be more correct to throw an appropriate exception, e.g.,

class SimulationBankruptcy(Exception): pass

3. I never really encountered negative values in my tests, so I haven't implemented guards against all possible breaking cases. Complex numbers in the Tcost model are clearly an absurdity. Negative portfolio values should be caught earlier and throw the same exception of point 2.

If you have implemented these changes, please go forward with a pull request.

Thanks! Enzo

[joseortiz3]

Thanks for your ideas.

1. So do changes need to be made in all the Costs.optimization_logs? Or just Result.log_policy?

2. Throwing an exception sounds interesting. I guess it would be thrown in Policy.get_trades? And then handled in Simulation.run_backtest? How would this handling work? Don't want the other 39 threads to get halted just because one went bankrupt.

The only changes I've made are to allow the simulation to continue, which sometimes maintains zero portfolio value for the rest of the time (at the cost of lots of unnecessary optimization), other times going further negative with further bankrupt trades (I guess, the leverage constraint unbinds when the portfolio becomes negative enough).

I think ideally, trades and portfolio values would get set to zero, and get_trades would return zeros without attempting optimization or throwing exceptions, and logging would record zero for everything. This way, the Result of a simulation that hits zero value has attributes with the same dimensions (DatetimeIndexs) as the other Results, and the simulation finishes very quickly. Then post-analysis doesn't have to worry about different results having different simulation lengths.

Hopefully I'll have time to try it out soon.

[enzbus]

Hello, I know this is very old, but it may still be relevant. Full support for back-tests ending in bankruptcy is now there, you get a BackTestResult object for the full path up to the first period in which the value becomes negative. Sadly it is not as clean as I'd like, since for example the Sharpe Ratio of the result can even be positive, due to how it's defined (but of course the profit is negative and the growth rate is minus infinity). I don't have bandwidth to test it as well as I'd like so if you're still interested please report!