Modifying Lucid to work with grayscale input networks

[vedantbhatia]

Hi,

I understand that Lucid needs a three channel input to the network in order to function.

Could someone explain: a. Why this is the case b. If there are any workarounds c. What I would have to do to make it work, in case (b) is negative

Thanks in advance!

[BadrYoubiIdrissi]

I am currently working on making lucid work with any number of channels. I only need to test my modifications now, and do a pull request

[vedantbhatia]

@BadrYoubiIdrissi that's great. I would still love an explanation to my questions though!

I saw your comment on another related issue, in which you explained a workaround - passing "param_f = param.color.to_valid_rgb(param.spatial.naive((1,28,28,1)))" to render_vis.

Unfortunately this throws the error: "_ValueError: Cannot execute operation using run(): No default session is registered. Use with sess.as_default(): or pass an explicit session to run(session=sess) "

Any ideas on what causes this?

[colah]

Hello!

Try doing this instead: param_f = lambda: param.color.to_valid_rgb(param.spatial.naive((1,28,28,1)))

[vedantbhatia]

Thanks @colah. I will try this and get back. Could you help me understand why Lucid requires a three channel network? Is it just something that was assumed during implementation or is there a reason behind it?

[colah]

Lucid needs an input parameterization that matches the model. The default one is 3 channels. If you have a model with a different number of channels, you need to change the input parameterization away from the default.

[vedantbhatia]

Hello!

Try doing this instead: param_f = lambda: param.color.to_valid_rgb(param.spatial.naive((1,28,28,1)))

Hi, this did not work either. Raised the same error: "ValueError: Cannot execute operation using run(): No default session is registered. Use with sess.as_default(): or pass an explicit session to run(session=sess)"

def render_vis(model, objective_f, param_f, steps=512):
    T = render.make_vis_T(model, objective_f, param_f, optimizer=None, transforms=[])
    loss, vis_op, t_image = T("loss"), T("vis_op"), T("input")
    sess = tf.Session()
    sess.run(tf.global_variables_initializer())
    for step in range(steps):
        sess.run(vis_op)
    return sess.run(t_image)

logit_images = []
print("Logit ", end="")
for logit_idx in range(10):
  print("{}...".format(logit_idx), end="")
  with tf.Graph().as_default():
    image = render_vis(model, "softmax:{}".format(logit_idx),
                       steps=128, param_f = lambda: param.color.to_valid_rgb(param.spatial.naive((1,240,320,1))))
    logit_images.append(image)
show_images(logit_images)

[colah]

Ah, you are using make_vis_T() instead of render_vis(). Yep, you need to create a session. :)

Try doing this:

logit_images = []
print("Logit ", end="")
for logit_idx in range(10):
  print("{}...".format(logit_idx), end="")
  with tf.Graph().as_default(), tf.Session():
    image = render_vis(model, "softmax:{}".format(logit_idx),
                       steps=128, param_f = param.color.to_valid_rgb(param.spatial.naive((1,240,320,1))))
    logit_images.append(image)
show_images(logit_images)

[vedantbhatia]

Ah, you are using make_vis_T() instead of render_vis(). Yep, you need to create a session. :)

Try doing this:

logit_images = []
print("Logit ", end="")
for logit_idx in range(10):
  print("{}...".format(logit_idx), end="")
  with tf.Graph().as_default(), tf.Session():
    image = render_vis(model, "softmax:{}".format(logit_idx),
                       steps=128, param_f = param.color.to_valid_rgb(param.spatial.naive((1,240,320,1))))
    logit_images.append(image)
show_images(logit_images)

Thanks @colah, that worked.

May I ask what I can infer if my output is just noise? The images in my dataset are quite complex, could it be that optimization based visualization won't give discernible outputs for my data (or will I just require 1000x more steps), or is a implementation mistake from my side more plausible?

Thanks a ton for your help.

[vedantbhatia]

Ah, you are using make_vis_T() instead of render_vis(). Yep, you need to create a session. :) Try doing this:

logit_images = []
print("Logit ", end="")
for logit_idx in range(10):
  print("{}...".format(logit_idx), end="")
  with tf.Graph().as_default(), tf.Session():
    image = render_vis(model, "softmax:{}".format(logit_idx),
                       steps=128, param_f = param.color.to_valid_rgb(param.spatial.naive((1,240,320,1))))
    logit_images.append(image)
show_images(logit_images)

Thanks @colah, that worked.

May I ask what I can infer if my output is just noise? The images in my dataset are quite complex, could it be that optimization based visualization won't give discernible outputs for my data (or will I just require 1000x more steps), or is a implementation mistake from my side more plausible?

Thanks a ton for your help.

Adding that this is despite setting relu_gradient_override=True