Issues with `check_ordering` option and `nofit_hypo`?

[thehrh]

While overhauling the Analysis class I noticed the following issues which I hoped someone would have some insights in:

• What is the motivation for letting nofit_hypo reset free parameters of the DistributionMaker passed in in the case of blindness? Probably not harmful depending on usage, but is there some deeper reason?
• Is anyone (or does anyone know of someone else) successfully using the check_ordering option in conjunction with fit_hypo? I don't even understand how it is intended to work (return the fits in both orderings, or minimize over the ordering?), and it looks to me like the method currently just returns the ih best fit information (best_fit_info created in the loop over the ordering hypos in https://github.com/jllanfranchi/pisa/blob/dev/pisa/analysis/analysis.py#L378 just gets overwritten for each loop iteration), though I might have missed something from staring at the code for too long.

Just as a side note, I'm currently generalising the concept of the "ordering check" to any arbitrary set of discrete hypotheses: loop over any additional parameter selections in a wrapper function which then calls fit_hypo in each iteration and in the end ensures that the resulting fit info is the global optimum across all the discrete hypotheses.

[jllanfranchi]

1. Yeah, we don't want objects carrying fit values around if in interactive use and the user requests for a blind fit; obviously there will always be ways for a user to find values if they want, but we just build as much of a barrier as is practical to do.
2. I thought that @steven-j-wren had tested that code out, but I agree that the loop over extra_param_selections with initialization of best_fit_info inside of it looks like a bug.

[jllanfranchi]

... and as a side note, it's awesome that you're creating generic discrete hypo code!

[steven-j-wren]

I used the check_ordering flag successfully in the limited work I did on vacuum vs. matter oscillations. It seemed to work then. :smile: I just copied the logic for the discrete octant test but for the mass ordering i.e. find the minimum in both mass orderings and just take the global minimum.

[thehrh]

Interesting. It doesn't look like it works that way (anymore?), but it's about to get replaced anyway :stuck_out_tongue:

[thehrh]

Ordering check nowadays just adds extra parameter selections to fit.