ebm_code_release icon indicating copy to clipboard operation
ebm_code_release copied to clipboard

Published 20 hours ago • verified publisher icon openai

FLAG.objective mode logsumexp and softplus

Open ghost opened this issue 4 years ago • 1 comments

Hi, I am curious what is the math background for the logsunexp and softplus case in FLAG.objective.

Thanks.

ghost avatar Jan 09 '20 21:01 ghost

Hi, You can find the interpretation of softplus here (appendix): https://arxiv.org/pdf/1811.02486.pdf If you assume that negative samples are drawn from the uniform distribution as opposed from the model distribution than you obtain the logsumexp interpretation.

yilundu avatar Jan 15 '20 21:01 yilundu