Inconsistence in attacking returns

[yitao416]

The very example at your official website is taking the first return as advs.

Well, the latest attack method suggests using the second return as the better option (clipped).

I think it is better to make this more clear for users.

Thanks.

[jangop]

@yitao416 Which “latest attack method” are you referring to?

I agree that returning adversarials and clipped_adversarials can be confusing, especially seeing that the differences aren't documented, and one might assume that the clipping refers to the bounds specified when initializing fmodel.

Furthermore, when epsilons=None, maybe clipped_adversarials (the second return value) should be None as well, since no clipping takes place. I do not like mixing return types (even Optional[Tensor]), but I suppose that's the price to pay for always returning both variants.

[yitao416]

@yitao416 Which “latest attack method” are you referring to?

I agree that returning adversarials and clipped_adversarials can be confusing, especially seeing that the differences aren't documented, and one might assume that the clipping refers to the bounds specified when initializing fmodel.

Furthermore, when epsilons=None, maybe clipped_adversarials (the second return value) should be None as well, since no clipping takes place. I do not like mixing return types (even Optional[Tensor]), but I suppose that's the price to pay for always returning both variants.

The latest attack method I refer to is version 3.3.1.

The second picture is from the Example section from the Github page: https://github.com/bethgelab/foolbox

[zimmerrol]

You are right that we should put this in the documentation. @jangop If you have the time and already an idea how to implement your suggested changes please feel free to open a PR for it:)

[jangop]

You are right that we should put this in the documentation. @jangop If you have the time and already an idea how to implement your suggested changes please feel free to open a PR for it:)

Sure. However, what would be the desired outcome? What was/is the rationale behind always returning both unclipped and clipped adversarials? If a user knows which one they want, they could probably take care of the clipping themselves.

I still believe it useful to return None if epsilons=None, because in that case, clipping is not just the identity but meaningless. That part is simple to implement, but it might break around 50 % of existing code.

[zimmerrol]

I would just suggest updating the documentation to include this information.