Cyclic Markov Policy Graph

[Remmy195]

Hello Oscar,

I'm trying to model an inifinite/cyclic horizon markov policy graph just like in the pastoral farming example here this paper, and I have done that by adding edges from the last node to the first node

graph = SDDP.MarkovianGraph(sim; budget=100, scenarios=1000)
typeof(graph)

# Function to extract nodes by stage
function extract_nodes_by_stage(graph, stage)
    nodes_at_stage = []
    for node in keys(graph.nodes)
        if node[1] == stage 
            push!(nodes_at_stage, node)
        end
    end
    return nodes_at_stage
end

# Extract initial and final nodes
initial_nodes = extract_nodes_by_stage(graph, 1)
final_stage = maximum([node[1] for node in keys(graph.nodes)])  # Find the last stage
final_nodes = extract_nodes_by_stage(graph, final_stage)

# cycle_probability for the transition from the last to first nodes
cycle_probability = 0.95

# For each last node, distribute the cycle_probability evenly across all first nodes
for last_node in final_nodes
    for first_node in initial_nodes
        # Calculate the transition probability from each last node to each first node
        transition_probability = cycle_probability / length(initial_nodes)
        # Add the edge from the last node to the first node with the calculated probability
        SDDP.add_edge(graph, last_node => first_node, transition_probability)
    end
end

It works with the nodal transition shared evenly amongst the nodes. Is this the optimal approach? How can I add weights to nodal transition based on my simulated stochastic process?

[odow]

Is this the optimal approach?

There is no single "optimal" approach. It depends on your model

How can I add weights to nodal transition based on my simulated stochastic process?

Change the transition probability in:

SDDP.add_edge(graph, last_node => first_node, transition_probability)

At the moment, it looks like you are using a uniform distribution.

[Remmy195]

Thanks for your response. I have an idea to use the stochastic process distribution parameters to estimate the transition probabilities.

[odow]

Sure. SDDP.jl doesn't provide tools to help with this. It is up to you to design the graph that is most appropriate for your problem.

[odow]

Closing because I don't think there is anything left to do here. Please comment if you have further questions and I will re-open.