swift-models icon indicating copy to clipboard operation
swift-models copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

Deep Dream Example

Open zaidalyafeai opened this issue 6 years ago • 10 comments

I am preparing a deep dream example.

image

zaidalyafeai avatar May 15 '19 01:05 zaidalyafeai

One Issue that I am facing is finding the gradient of a specific activation. For instance suppose that output of a layer has the shape [1, 7, 7, 32] and I need to only include the loss at [1, 7, 7, 0] (S4TF throws an error that indexing is not differentiable) . One workaround is to multiply by tensors of zeros except at the activation index. Any better ideas ?

zaidalyafeai avatar May 15 '19 01:05 zaidalyafeai

I wonder which specific indexing APIs are you using? Advanced indexing/slicing APIs were recently added and should be made differentiable. 🙂

cc @rxwei @eaplatanios

dan-zheng avatar May 15 '19 01:05 dan-zheng

@dan-zheng I am using the naive tensor[0..<1,0..<7, 0..<7, 0..<1]. For the record i am doing all my experiments of colab so the build might be old.

zaidalyafeai avatar May 15 '19 01:05 zaidalyafeai

I think we need inception model - @Hvass-Labs - has prepared a well documented tutorial here https://github.com/Hvass-Labs/TensorFlow-Tutorials/blob/master/14_DeepDream.ipynb

johndpope avatar May 31 '19 19:05 johndpope

@johndpope I used a vgg model and it is working. The inception model might create better visuals, however.

zaidalyafeai avatar May 31 '19 20:05 zaidalyafeai

fyi - @t-ae cut stylegan some months back - I wonder if there's interest to upgrade it to stylegan2 https://github.com/t-ae/stylegan-s4tf - seeing some amazing things on twitter by roadrunner https://twitter.com/ak92501

https://twitter.com/ak92501/status/1212784293723934726?s=20 Screen Shot 2020-01-03 at 21 45 42

@t-ae - did you consider doing a PR for stylegan here ? / it would be well welcomed

johndpope avatar Jan 03 '20 10:01 johndpope

@johndpope I have some reasons to think my code is not suitable for here.

  1. As I wrote in readme, I used LSGAN loss instead of WGAN-GP because S4TF doesn't support higher order differentiation for now. I suspect it leads poor results.
  2. My code generates 256x256 images, which is much smaller than original 1024x1024. And I have no resource to train 1024x1024 images.
  3. I used tensorboardx wrapper for visualizing training process. I think it's not good to add not-pure-swift things.

IMO StyleGAN demo with pre-trained weights from original TensorFlow implementation is more suitable for here than StyleGAN training code.

I don't read StyleGAN2 paper yet. I'll check if there's something I can do with S4TF.

t-ae avatar Jan 03 '20 12:01 t-ae

Hey @brettkoonce - I think instead of / (as well as socks) - need to be handing out some coupons for training on TPUs. How do we get SfTF to support higher order differentiation ? is there a ticket to track that functionality? I understood S4TF would reach some parity with tensorflow eventually.

johndpope avatar Jan 05 '20 23:01 johndpope

@johndpope probably a better question for @saeta than me. In general now that #146 has been merged, maybe this can be closed as well.

brettkoonce avatar Jan 06 '20 18:01 brettkoonce

my bad - Brennan Saeta @saeta - are there any efforts to get S4TF working on TPUs?

johndpope avatar Jan 07 '20 07:01 johndpope