food-inspections-evaluation
food-inspections-evaluation copied to clipboard
Refactor `eval_model` and integrate evaluation function more deeply with `30_glmnet_model.R`
This is in reference to pull request @cash
When we look at the evaluation we look primarily at the "days difference", which is an attempt to quantify how much faster we would find a critical violation over the test period.
There is a secondary component to the evaluation, which is the basis for many of the graphs shown in the white paper. The second component is a summary of day by day inspection counts, and the number of critical violations found shown side-by-side with the simulated violations.
An example of the second summary is shown below, where N is the inspection count, POS indicates the critical violations found, and SIM indicates the "simulated" results from the experiment. In all cases TOT indicates a running cumulative sum.
date N NTOT POS POSTOT POS_SIM POSTOT_SIM
1: 2014-09-02 1 1 0 0 1 1
2: 2014-09-03 44 45 10 10 22 23
3: 2014-09-04 50 95 8 18 24 47
4: 2014-09-05 46 141 12 30 8 55
5: 2014-09-08 37 178 6 36 8 63
6: 2014-09-09 40 218 7 43 12 75
7: 2014-09-10 33 251 3 46 4 79
8: 2014-09-11 41 292 10 56 11 90
9: 2014-09-12 42 334 5 61 11 101
10: 2014-09-15 35 369 6 67 5 106
11: 2014-09-16 51 420 9 76 5 111
12: 2014-09-17 24 444 6 82 5 116
13: 2014-09-18 41 485 5 87 7 123
14: 2014-09-19 45 530 8 95 10 133
15: 2014-09-22 34 564 7 102 4 137
16: 2014-09-23 33 597 10 112 2 139
17: 2014-09-24 29 626 6 118 6 145
18: 2014-09-25 48 674 5 123 7 152
19: 2014-09-26 49 723 8 131 11 163
20: 2014-09-29 55 778 9 140 5 168
21: 2014-09-30 32 810 1 141 8 176
22: 2014-10-01 29 839 4 145 3 179
23: 2014-10-02 39 878 6 151 7 186
24: 2014-10-03 46 924 5 156 7 193
25: 2014-10-06 8 932 1 157 0 193
26: 2014-10-07 35 967 5 162 7 200
27: 2014-10-08 48 1015 6 168 9 209
28: 2014-10-09 40 1055 2 170 5 214
29: 2014-10-10 36 1091 3 173 2 216
30: 2014-10-14 30 1121 6 179 3 219
31: 2014-10-15 31 1152 5 184 4 223
32: 2014-10-16 40 1192 3 187 4 227
33: 2014-10-17 43 1235 4 191 5 232
34: 2014-10-20 28 1263 3 194 3 235
35: 2014-10-21 37 1300 8 202 5 240
36: 2014-10-22 37 1337 7 209 4 244
37: 2014-10-23 45 1382 5 214 3 247
38: 2014-10-24 49 1431 11 225 1 248
39: 2014-10-27 33 1464 4 229 3 251
40: 2014-10-28 51 1515 11 240 2 253
41: 2014-10-29 37 1552 6 246 3 256
42: 2014-10-30 42 1594 5 251 1 257
43: 2014-10-31 43 1637 7 258 1 258
It is possible to calculate the "days saved" (the primary evaluation metric) from the table shown above. However, it will be useful to be able to quickly calculate the "days saved" metric directly, because then it's available if someone wants to use it in an objective function or optimization.
So, the point of this issue is to break out the eavl_model
into the first and second components, and remove a lot of unclear and unnecessary code in the 30 model script.