ietoolkit icon indicating copy to clipboard operation
ietoolkit copied to clipboard

[iebaltab]: Return regression estimate of difference as default comparison

Open bbdaniels opened this issue 4 years ago • 5 comments

Line 1763 should be adjusted to always show the regression result as the difference between groups by default. The user should be able to request the unadjusted mean differences but in general this will be confusing since the direction or magnitude of the tested difference can be entirely different from the raw difference.

Edit: this should also return the SE of the estimate in the space below the point estimate

https://github.com/worldbank/ietoolkit/blob/ea8f1e2d308d4b99c28f0d178c716b9cdb2002b9/src/ado_files/iebaltab.ado#L1763

bbdaniels avatar Jul 27 '20 20:07 bbdaniels

I am convinced that this is better. How should we deal with backward compatibility?

kbjarkefur avatar Jul 27 '20 20:07 kbjarkefur

Another related thing that I often have received as feedback was that is should be second_group - first_group instead of first_group - second_group as control is often the first group and in second_group - first_group it means that a positive value means a positive effect of the treatment in the second group.

kbjarkefur avatar Jul 27 '20 20:07 kbjarkefur

Agree with that. Whatever group is specified as the base (typically the control) should be reflected so that the regression estimate is added to their outcome to get the estimate for the other groups.

I don't think there's any way to make this backward-compatible since both are substantive changes, but an "old" option could always make it run the original way (although maintaining past versions within the code itself strikes me as insane, compared to just letting people know of such major changes and keeping the old version posted). It's not a syntax change anyway so code will still run.

bbdaniels avatar Jul 27 '20 21:07 bbdaniels

code will still run, but results are no longer replicable...

kbjarkefur avatar Jul 27 '20 21:07 kbjarkefur

we could deprecate iebaltab in favor of something like iebalance? but you really only get to do that like once, as opposed to just saying, it's a major version, for code written before DATE you need to install from a specific release

bbdaniels avatar Aug 14 '20 22:08 bbdaniels