add counterfactual code

[shahashka]

[rajeeja]

@jmohdyusof @j-woz please comment for changes on this.

[rajeeja]

maybe we can get rid of the notebooks.

[jmohdyusof]

I am not sure why this is structured like this, with an entirely separate subdirectory within NT3? Either make it a separate benchmark (like e.g. Uno_UQ, UnoMT) or incorporate it into the NT3 directly (it seems like some files are duplicated from the original NT3?)

[shahashka]

Yes, this was just to get all the code up for everyone to view. Things will have to change. A lot of the reason for the duplicated files was because all my files are in pickle format, whereas the original files read csv's. I can add a pickle reader to the existing code and restructure things around.

[jmohdyusof]

There is a lot of cleanup needed in that repo anyway, like consolidating all the noise stuff into a single add_noise call that ingests all the gParams, so that all the logic is at the 'common' level. I have that in another branch and will be doing some changes during hte hackathon this week. If you can add the pickle reader in a nice modular way that would be good.

[shahashka]

I'm thinking of adding some gParameters: a flag for saving to pickle for the baseline, and a string for input cf noise file. Would this be reasonable? To do this, I need to edit the parse_utils.py file I think

[jmohdyusof]

Yeah, I would start a noise_conf list, and I can add in the new params in there later (noise_type, noise_features, noise_samples etc). We want to enforce that all the names start with noise_ so that they all get grouped together and are easy to find.

[shahashka]

ok sounds good. i will also run all the cf stuff end to end on a smaller data size to make sure its all working and remove some explicit file paths to my directory