Some questions about model debugging

[WuSiren]

Hello, Prof. @odow ! I would like to ask some questions about model debugging.

1. Here in the doc introduces a method that obtains the deterministic equivalent of the model. I'd like to know more information on what the deterministic equivalent exactly means or how it obtains this deterministic equivalent. Does the deterministic equivalent refer to a model obtained by fixing the random variable in the original model to some certain scenario?

2. I'm trying to convert my model to a deterministic one for debugging but it tells

ERROR: Unable to formulate deterministic equivalent: Time limit exceeded!

Does this mean my model is to big to do so?

The deterministic equivalent scales poorly with problem size. Only use this on small problems!

3. I would like to ask for your valuable advice on model debugging. My model seems not quite robust that it usually fails to solve once if I add some constraints or introduce more scenarios to the random variable. And now, it can be solved if the graph is set to be a LinearGraph but can't be solved once I change the graph to a UnicyclicGraph for infinite horizon practice. According to your experience, where do you think the problem lies? How should I debug this model? The report is usually:

-------------------------------------------------------------------
         SDDP.jl (c) Oscar Dowson and contributors, 2017-23        
-------------------------------------------------------------------
problem
  nodes           : 12 
  state variables : 131
  scenarios       : Inf
  existing cuts   : false
options
  solver          : serial mode
  risk measure    : SDDP.Expectation()
  sampling scheme : SDDP.InSampleMonteCarlo
subproblem structure
  VariableRef                             : [658, 658]
  AffExpr in MOI.EqualTo{Float64}         : [218, 218]
  AffExpr in MOI.GreaterThan{Float64}     : [272, 272]
  AffExpr in MOI.LessThan{Float64}        : [427, 427]
  VariableRef in MOI.GreaterThan{Float64} : [360, 360]
  VariableRef in MOI.ZeroOne              : [152, 152]
numerical stability report
  matrix range     [5e-05, 3e+03]
  objective range  [1e+00, 3e+04]
  bounds range     [0e+00, 0e+00]
  rhs range        [1e+00, 1e+04]
WARNING: numerical stability issues detected
  - matrix range contains small coefficients
Very large or small absolute values of coefficients
can cause numerical stability issues. Consider
reformulating the model.
-------------------------------------------------------------------
 iteration    simulation      bound        time (s)     solves  pid
-------------------------------------------------------------------
†        1   1.343504e+08  5.974647e+05  5.534900e+01     11010   1
         2   2.344191e+08  6.196139e+05  1.177230e+02     29664   1
[ Info: Writing cuts to the file `model.cuts.json`
ERROR: Unable to retrieve solution from node 12.

  Termination status : OTHER_ERROR
  Primal status      : NO_SOLUTION
  Dual status        : NO_SOLUTION.

The current subproblem was written to `subproblem_12.mof.json`.

There are two common causes of this error:
  1) you have a mistake in your formulation, or you violated
     the assumption of relatively complete recourse
  2) the solver encountered numerical issues

See https://odow.github.io/SDDP.jl/stable/tutorial/warnings/ for more information.

Will there be a situation where the LinearGraph version of a model obeys the assumption of relatively complete recourse while its UnicyclicGraph version doesn't?

Thanks!

[WuSiren]

where do you think the problem lies? How should I debug this model?

According to limited observations so far, it seems that the model can be solved normally after I replace the solver HiGHS with a commercial one.

I'm still looking forward to your opinion on the other questions. Thanks!

[WuSiren]

According to limited observations so far, it seems that the model can be solved normally after I replace the solver HiGHS with a commercial one.

I found sometimes it can be solved normally but sometimes it can't, under the same setting. Why? Is it because the algorithm is stochastic?

[odow]

1. You should probably ignore the deterministic equivalent. It builds a single JuMP model of the entire tree. this is useful only for tiny toy problems.
2. Your model is likely too large to build as a single JuMP model
3. What solver are you using? You have a relatively large number of state variables, and the matrix contains some very small coefficients. What discount factor are you using?

it seems that the model can be solved normally after I replace the solver HiGHS with a commercial one.

Yes, for larger models you almost always need to use Gurobi. It is much more numerically robust.

Is it because the algorithm is stochastic?

Yes. You can see the seed before solving with import Random; Random.seed!(1234) to make the code reproducible.

[WuSiren]

Oh I see! Thank you @odow !

I'm using COPT.jl though it can't always work well, either. The discount factor I'm using is 0.9.

You can see the seed before solving with import Random; Random.seed!(1234) to make the code reproducible.

Oh! This looks very useful! But, can I (I know probably I can't) retrieve the random seed after I trained the model, since I have collected all the results?

[odow]

But, can I (I know probably I can't) retrieve the random seed after I trained the model

No

[WuSiren]

OK, thanks a lot! 🤝