Fix growing spheres

[kmariuszk]

[kmariuszk]

@pat-alt @VincentPikand @RaunoArike would really appreciate if you could give this PR a look and tell me what you think about the direction of changes within growing spheres. I'm very skeptical about adding this new fields to the generator itself to keep the state of it but couldn't find any better solution yet. Thanks!

[izagorac]

Hi @kmariuszk, what is the current status of this PR? Could I possibly help you with anything? I'm currently doing my thesis with @pat-alt and I want to use the GrowingSpheresGenerator for my research

[kmariuszk]

Hi @izagorac, I'll get back to you in a couple of hours

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (1504e0b) 77.57% compared to head (0ff567e) 25.98%.

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #396       +/-   ##
===========================================
- Coverage   77.57%   25.98%   -51.59%     
===========================================
  Files          74       73        -1     
  Lines        1565     1474       -91     
===========================================
- Hits         1214      383      -831     
- Misses        351     1091      +740     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[kmariuszk]

Hey @izagorac, @pat-alt, I cleaned up the PR and here's a quick summary:

✅ Growing spheres generation has been updated, so now it is actually using the proper convergence logic (i.e., it actually uses the decision_threshold provided by the user. ❌ Growing spheres is still not properly dispatched on generate_perturbations() (currently, it is 'dispatched' on update()). My main concern which blocked me from implementing that is addressed in the next point. ❌ While generating counterfactuals using growing spheres the path cannot be easily updated using current logic. Currently, the update happens in the following line: https://github.com/JuliaTrustworthyAI/CounterfactualExplanations.jl/blob/1504e0b4e00910845070a141973f9bd9b639f5b4/src/counterfactuals/search.jl#L22 the problematic part about that is that GrowingSpheresGenerator is generating multiple points at the same time. I noticed @pat-alt suggested to use similar methodology as in DiCE but unfortunately, I didn't look into it. There might be a possible fix for that. ❔ I'm not sure if currently GrowingSpheresGenerator is capable of generating multiple counterfactuals at once. To be fair, I'm not even sure if the growing spheres algorithm is ever able to generate more counterfactuals than just one.

Currently, I don't have anymore time to look into the mentioned issues. I should have some time I could spend on it on Saturday, @pat-alt do you have time then? We could have a pair-programming session which could speed-up the whole process. Otherwise, @izagorac you can try to fix the mentioned issues by yourself.

[izagorac]

Thank you for the elaborate summary @kmariuszk !

First of all, you don't have to worry about fixing this issue ASAP because I still have loads to work on for my thesis and if these issues take a while to fix that is no problem. Second, I see you mentioned the amount of counterfactuals that the generator can generate at once. Do you refer to the number of counterfactuals that it will generate per factual, which can be specified with the num_counterfactuals parameter in the generate_counterfactual function, or the amount of factuals that counterfactuals can be generated for (hopefully this sentence still makes sense). If you mean the first one, then I don't see any issue for my work. I will namely only be generating one counterfactual per factual.

Anyways, thanks for your time and hopefully we can solve these issues together!

[pat-alt]

Thanks @kmariuszk for looking into this. I'm at AAAI in Canada right now and we have a pretty packed schedule, even on the weekend. I could probably skip a session on Saturday, but then I'm still in a very different time zone 😅 perhaps easier when I'm back in early March.

In the meantime, might it be worth already merging what we have here? I'm puzzled by the errors on 1.7 and 1.8, but that should be fixable. @izagorac would that fix your current issue? If not, could you post the error your getting here?

[izagorac]

@pat-alt @kmariuszk I'm not sure whether the updates fix my current issues but I can always use the updated generator if it gets merged. Nonetheless, I'll post the exact error I get here:

From looking at the error message, it seems to be a very generic error. However, if I run the exact same code but with the FeatureTweakGenerator, I have no problems at all. Hopefully, this stacktrace gives you some additional information @kmariuszk.