TabPFN
TabPFN copied to clipboard
Meaning of mix_activations
My understanding of mix_activations
is whether to mix activation functions within a batch when sampling from the MLP prior.
Now there's
if not (('mix_activations' in hyperparameters) and hyperparameters['mix_activations']):
s = hyperparameters['prior_mlp_activations']()
hyperparameters['prior_mlp_activations'] = lambda: s
which seems to support that. Now there's this comment:
config["mix_activations"] = False # False heisst eig True
which seems to imply True
and False
are switched.
Can you confirm that the comment is out of date, or am I missing something?