touchstone
touchstone copied to clipboard
Make indicator boundary customisable
I just looked at https://github.com/kgoldfeld/simstudy/pull/129 And I wondered if the users could be a able to specify a threshold himself for the icon. Does it always make sense to check if 0 is contained? One could argue that a ci that contains a 1% performance drop is still not enough to justify a 🐌 icon, so if the user specifies a custom value for null_boundary
(Better name needed I think ), we check if the confidence interval overlaps with it.
Originally posted by @lorenzwalthert in https://github.com/lorenzwalthert/touchstone/issues/71#issuecomment-1015322809
I agree, that would be good, maybe something like max_performance_loss
or something.