robyn_allocator scenario- max_response_expected_spend giving issues

[NoufaMukhtar]

Project Robyn

I successfully ran the robyn allocator for scenario max_historical_response but when I am doing the same for max_response_expected_spend, my initial and optimized spends are the same for all media channels,

Please let me know what is happening here.

Thanks.

[kyletgoldberg]

Hi - this is usually due to the optimizer not finding any place to actually optimize the spend. Could you try broadening the ranges for the budget allocation for each channel? The channel_constr_low and channel_constr_up params are what you're looking for.

[tgtod002]

I too am having problems.

I want to create a scenario changing 1 year's worth of spending. 2021 spending is 26,166,517 while my expected spend for 2022 would be 34,000,000 .

The allocator calculates this to have an expected spend delta of -0.1354.

channels	date_min	date_max	periods	constr_low	constr_up	histSpend	histSpendTotal	initSpendUnitTotal	initSpendUnit	initSpendShare	initResponseUnit	initResponseUnitTotal	initRoiUnit	expSpendTotal	expSpendUnitTotal	expSpendUnitDelta	optmSpendUnit	optmSpendUnitDelta	optmSpendUnitTotal	optmSpendUnitTotalDelta	optmSpendShareUnit	optmResponseUnit	optmResponseUnitTotal	optmRoiUnit	optmResponseUnitLift	optmResponseUnitTotalLift
channel 1	1/3/2021	12/26/2021	52 weeks	0.7	1.5	1792374	$  26,166,517.24	754171.6	$    52,716.88	0.0699	186.349	487972.8	0.003535	$  34,000,000.00	652054.7945	-0.135402608	36901.81	-0.3	652054.8	-0.1354	0.056593	64.40688	415760.2	0.001745	-0.65437	-0.14798
channel 2	1/3/2021	12/26/2021	52 weeks	0.7	1.5	15258415	$  26,166,517.24	754171.6	$  526,152.26	0.697656	449646.8	487972.8	0.854594	$  34,000,000.00	652054.7945	-0.135402608	492441.3	-0.064070807	652054.8	-0.1354	0.755215	402476.2	415760.2	0.817308	-0.10491	-0.14798
channel 3	1/3/2021	12/26/2021	52 weeks	0.7	1.5	9115728	$  26,166,517.24	754171.6	$  175,302.46	0.232444	38139.65	487972.8	0.217565	$  34,000,000.00	652054.7945	-0.135402608	122711.7	-0.3	652054.8	-0.1354	0.188192	13219.58	415760.2	0.107729	-0.65339	-0.14798

[NoufaMukhtar]

Hi - this is usually due to the optimizer not finding any place to actually optimize the spend. Could you try broadening the ranges for the budget allocation for each channel? The channel_constr_low and channel_constr_up params are what you're looking for.

I did try that as well but it only worked when one of the variables is set to have a different range for budget allocation. I think it might be set as such, Whenever I used different range for even one variable , it works and if all have the same range to move then it doesn't work as expected.

[kyletgoldberg]

@tgtod002 could you make sure you are on the latest version of Robyn? I believe this was fixed in a previous update, but please let me know if that is not the case.

[kyletgoldberg]

@NoufaMukhtar could you share screenshots of your response curves as well as the graph outputs for the two scenarios of budget allocation? Feel free to blind any numbers that you do not want to share.

[tgtod002]

@kyletgoldberg I updated to the latest version of Robyn 3.6.4. I am still getting negative values.

I used 2021 spend which was 26,266,517, I am trying to do a scenaior of bringing up the spend 34M . The allocator comes up with a negative expected spend of -0.1345.

channels	date_min	date_max	periods	constr_low	constr_up	histSpend	histSpendTotal	initSpendUnitTotal	initSpendUnit	initSpendShare	initResponseUnit	initResponseUnitTotal	initRoiUnit	expSpendTotal	expSpendUnitTotal	expSpendUnitDelta
MC1	1/3/2021	12/26/2021	52 weeks	0.7	1.5	1792374	26166517	754171.6	52716.88	0.0699	241.9327	4289190	0.004589	34000000.00	652054.8	-0.1354
mc2	1/3/2021	12/26/2021	52 weeks	0.7	1.5	15258415	26166517	754171.6	526152.3	0.697656	1383827	4289190	2.630089	34000000.00	652054.8	-0.1354
mc3	1/3/2021	12/26/2021	52 weeks	0.7	1.5	9115728	26166517	754171.6	175302.5	0.232444	2905121	4289190	16.57205	34000000.00	652054.8	-0.1354

[kyletgoldberg]

@tgtod002 are there periods where the spend across channels is 0 in the historical data? the hist/init spend data only looks at non-zero spend, so if there are zeroes it may be causing the behavior.

[tgtod002]

@kyletgoldberg , No there isn't. Historically, I have less spend in 2020 vs. 2021. What I am trying to show is what the allocation should be if you spend back to close to 2020 levels ($34M).

The allocator spreadsheet produced shows that historical spend is (2021 spend)26M and expected spend is 34M.

[kyletgoldberg]

@tgtod002 hmm... would it be possible to send me the .RDS file with the saved model results to debug further? i think i know where the problem is but it's a bit hard to debug this way. my email is [email protected]

[tgtod002]

@kyletgoldberg Just checking in. I sent you the .RDS file last week. Were you able to receive it?

[kyletgoldberg]

@tgtod002 yes thank you - I'm working on it. Will get back to you asap

[kyletgoldberg]

Hi @tgtod002 we have identified what is causing this issue. The good news is that it is not a bug, just a relic of the way we calculate the unit spend for the historical period (i.e. only non-zero days). In the expected period, we assume that the spend is going to be equal each period, so the unit spend for the expected period will potentially decrease even with an overall increase in spend. This does have some impact on the budget allocator only in the sense of calculating the ranges of possible unit spends we want to look at for each channel. So while we work on adding some adjustments to the code, I would recommend reducing the channel_constr_low for each channel so that it can investigate more lower values. Thanks for flagging this.

[tgtod002]

@kyletgoldberg Thanks for the update. Will try playing with lower channel_contr_low.

[tgtod002]

@kyletgoldberg . Some suggestion for a possible enhancement...Currently the spreadsheets for the allocator provide response data at unit level (unit response, optimized response). What would help is to be able to provide response data at the total spend time - For example, if I am allocating $50M over 52 weeks. What I would like to also see is what I expect my response levels to be over the whole time period (52 weeks)...Right now, because of the way the unit levels are calculated (on non zeroes). it is difficult for me to see what I expect to see as responses for each media channels.

Hope that makes sense

[kyletgoldberg]

noted - thanks for the feedback

[Tomobay]

Is there any enhancement plan for this? If not, I can create a function. I already created the function and am using it, but need to do linting and adding some error messages.

[tgtod002]

@Tomobay That's great! Do you mind sharing the function you created?

[tgtod002]

Hi, Is there an update on resolving this?

[Tomobay]

You can use https://github.com/Tomobay/Robyn

robyn_allocator_historical( new_budget_ratio = 1, # new budget channels, # channels to optimize channel_constr_low = 1, # lower band channel_constr_up = 1, # upper band InputCollect, # Input Collect OutputCollect, # OutputCollect select_model, # selected_model date_min = NULL, # starting date to optimize date_max = NULL # end date to optimize )

I will try to test and document it to merge it to orginal

[gufengzhou]

Please reopen if this issue reoccurs.