TabPFN icon indicating copy to clipboard operation
TabPFN copied to clipboard

Meaning of mix_activations

Open amueller opened this issue 1 year ago • 0 comments

My understanding of mix_activations is whether to mix activation functions within a batch when sampling from the MLP prior. Now there's

        if not (('mix_activations' in hyperparameters) and hyperparameters['mix_activations']):
            s = hyperparameters['prior_mlp_activations']()
            hyperparameters['prior_mlp_activations'] = lambda: s

which seems to support that. Now there's this comment:

config["mix_activations"] = False # False heisst eig True

which seems to imply True and False are switched. Can you confirm that the comment is out of date, or am I missing something?

amueller avatar Jan 04 '24 20:01 amueller