learning-to-reweight-examples
learning-to-reweight-examples copied to clipboard
uber-research
minist reweight
why ex_wts_a is always zeros in minist.reweight.py? and it will cause the calculation of the gradient to be zero,why?
ex_wts_a = tf.zeros([bsize_a], dtype=tf.float32) ex_wts_b = tf.ones([bsize_b], dtype=tf.float32) / float(bsize_b) w_dict, loss_a, logits_a = get_model( inp_a, label_a, ex_wts=ex_wts_a, is_training=True, reuse=True) var_names = w_dict.keys() var_list = [w_dict[kk] for kk in var_names] grads = tf.gradients(loss_a, var_list, gate_gradients=gate_gradients)
That's right. The purpose is to build a computational graph to calculate the gradients to ex_wts_a
can ex_wts_a use other initial values?I cannot found the theory result depends on initial values of ex_wts_a。
Eq. 30 in the Appendix B relies on the fact that we are taking gradients when ex_wts_a is 0.
On Thu, Jul 23, 2020 at 9:27 AM zhegeliang2 [email protected] wrote:
can ex_wts_a use other initial values?I cannot found the theory result depends on initial values of ex_wts_a。
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/uber-research/learning-to-reweight-examples/issues/3#issuecomment-663006541, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHSPJ6ZHPIIH2DFS353IOCDR5A3FZANCNFSM4JZXZWEQ .
--
Mengye Ren
Sr. Research Scientist, Uber Advanced Technologies Group
[email protected] [email protected] | t.uber.com/atgresearch http://t.uber.com/atgresearch