Calculate OnCall schedule quality

[Matvey-Kuk]

Alice, Bob, Sam are on call. I think it would be nice to show some "schedule quality" to help users keep schedules in a good shape.

100% = ((50% * balance) + (50% * working_hours_balance)) * ( 1 / gaps_rate)

We're trying to balance on call hours between engineers on call. Balance = 1 is good. Balance = 0 is bad.

balance = abs (min(Alice, Bob) hours / max(Alice, Bob) hours) * .... all pairs

We're trying to avoid people being on call outside of working hours. At least their on call time outside of working hours should be equal.

working_hours_balance = abs (min(Alice, Bob) hours outside of working hours / max (Alice, Bob) hours outside of working hours) * .... all pairs

We want to avoid gaps in on call schedule. Shift with no users is equal to gap:

gaps_rate = gaps hours / total observed period hours 

[Konstantinov-Innokentii]

@Matvey-Kuk you want to calculate schedule's score or per-user score?

[Matvey-Kuk]

@Konstantinov-Innokentii schedule's score

[raphael-batte]

@Konstantinov-Innokentii We want to provide overall schedule quality. But if we have overloaded users in the schedule, we also focus attention on each of them.

[vadimkerr]

@Matvey-Kuk I propose to use a slightly different formula for balance score calculation:

balance = (min(Alice, Bob) hours / max(Alice, Bob) hours + .... all pairs) / number of pairs

It should eliminate the issue with hours distributions like [7, 8, 8, 8, 8, 8], where the formula you provided returns around 50% score, but [7, 8, 8, 8, 8, 8] distribution is pretty balanced and should score more than 50% (returns around 95% with the formula I proposed).

[maskin25]

@teodosii design here

[vadimkerr]

The backend part is now merged in https://github.com/grafana/oncall/pull/338, @teodosii will be working on the frontend part in a separate PR.

[teodosii]

Frontend is implemented but waiting for a minor backend change to differentiate good remarks from warnings within the api response. Once that is done we can unblock the frontend, because right now on frontend we cannot differentiate info messages (like 'Good schedule') from warnings ('Gaps') as described in the mockups.

PR for it https://github.com/grafana/oncall/pull/1324