Implement NCE loss function for Ivy Experimental API

[MarkCodering]

Describe your changes

• Add an NCE loss function to the experimental API for Ivy

Issue number and link

• Number: #17012
• Issue Link: https://github.com/unifyai/ivy/issues/17012

[ivy-leaves]

If you are working on an open task, please edit the PR description to link to the issue you've created.

For more information, please check ToDo List Issues Guide.

Thank you :hugs:

[MarkCodering]

Understood, I have linked this issue with the Todo list on this link: https://github.com/unifyai/ivy/issues/3856

If there are any problems with the PR, feel free to let me know.

Best,

Mark

[MarkCodering]

@zhumakhan Are you the reviewer for this PR?

[zhumakhan]

Hi, thanks for your PR! Could you please provide sources you used to implement NCE loss? Simply it returns a tensor instead of scalar value to compute gradient from. Also looked at this source, it is quite different: https://github.com/parthaca/examples/blob/master/word_language_model/nce_loss.py

[MarkCodering]

Here is the reference: https://www.tensorflow.org/api_docs/python/tf/nn/nce_loss

[zhumakhan]

Hey, it seems ivy.log_softmax does not take dim argument. Also I could not verify this implementation with tf.nn.nce_loss. Could you please provide one example using ivy.nce_loss and one using tf.nn.nce_loss?

Thanks!

[ivy-seed]

This PR has been labelled as stale because it has been inactive for more than 7 days. If you would like to continue working on this PR, then please add another comment or this PR will be closed in 7 days.

[ivy-seed]

This PR has been closed because it has been marked as stale for more than 7 days with no activity.