Optimizer Fails to Find Best Values

[AllardJM]

Version: Dev (9/13/24).

I'm looking for some advice on what seems odd to me in terms of the "optimal" channel budget being returned. My data has only 4 channels. The fitted model gives the following median values for the beta coefficients

and it appears that for the "csi" channel, which has a substantially larger effect than the others, is far from completely saturated:

However, if a scenario is ran - I'm using the optimizer to make sure outputs are comparable by fixing the budget constraints- to the actual spends from the last 8 weeks of the training data:

n_new = 8

# Get the last 8 weeks of training data spend amounts
ctv_last_8 = round(X["ctv_cost"].iloc[-n_new:].values.sum(),0)
xdevcombo_last_8 = round(X["xdevcombo_cost"].iloc[-n_new:].values.sum(),0)
dynmob_last_8 = round(X["dynmob_cost"].iloc[-n_new:].values.sum(),0)
csi_last_8 = round(X["csi_cost"].iloc[-n_new:].values.sum(),0)

# total of these spend amounts
total_budget = ctv_last_8 + xdevcombo_last_8 + dynmob_last_8 + csi_last_8



def find_optimal_channel_spend(model, min_prop, max_prop):
    
    # budget bounds are some increase or decrease from the actuals in the training data
    budget_bounds= {'ctv_cost' : [ctv_last_8 * min_prop, ctv_last_8 * max_prop], 
                'xdevcombo_cost' : [xdevcombo_last_8 * min_prop, xdevcombo_last_8 * max_prop], 
                'dynmob_cost' : [dynmob_last_8 * min_prop, dynmob_last_8 * max_prop], 
                'csi_cost' : [csi_last_8 * min_prop, csi_last_8 * max_prop]}


    response = model.allocate_budget_to_maximize_response(
        budget=total_budget,
        num_periods=8,
        time_granularity="weekly",
        budget_bounds=budget_bounds,
        noise_level = 0.0
    )
    
    return(response['y'].mean(axis=1).sum(),
           model.optimal_allocation_dict)

we get an estimated outcome of the following with requiring exact values matching the last 8 weeks:

find_optimal_channel_spend(mmm, min_prop=1, max_prop = 1)

Now, if we allow for wider ranges, where the assumption would be that more "CSI" would be included and revenue expected to be increased. This is not the case though...

find_optimal_channel_spend(mmm, min_prop=0.5, max_prop = 1.5)

we get a suboptimal outcome:

The optimizer decreased the top performing channel and decreased expected revenue.

I see this playing around with various ranges of min_prop and max_prop and see the same inability to select better choices of media spend.