style-transfer icon indicating copy to clipboard operation
style-transfer copied to clipboard

Published 20 hours ago • verified publisher icon fzliu

Poor output quality when using GoogleNet and CaffeNet

Open dniku opened this issue 8 years ago • 8 comments

image

Command:

python2 style.py -c "$ROOT_DIR/johannesburg.jpg" -s "$ROOT_DIR/starry_night.jpg" -o "$ROOT_DIR/starry_johannesburg.jpg" --model googlenet

image

Command:

python2 style.py -c "$ROOT_DIR/johannesburg.jpg" -s "$ROOT_DIR/starry_night.jpg" -o "$ROOT_DIR/starry_johannesburg.jpg" --model caffenet

Is this normal? I'm running Gentoo with a GeForce 750 Ti 2Gb, driver version 361.28, CUDA 7.0.28, Caffe built from git today. I'm getting out of memory errors when I try to run with the default neural network.

dniku avatar Jun 21 '16 19:06 dniku

Caffenet unfortunately doesn't work very well for style transfer. The Inception model will look better if you remove the LRN layers and fine-tune the network on Imagenet. This is something that I've gotten around to in the past, but never got around to hosting it somewhere appropriate.

fzliu avatar Jun 23 '16 01:06 fzliu

I am more concerned about the "grid" pattern which shows on both images. This looks a lot like a graphical artifact, as if there was something wrong with my video card. However, I've tried other implementations (https://github.com/anishathalye/neural-style on GPU and https://github.com/jcjohnson/neural-style on CPU) and they don't have this problem.

dniku avatar Jun 25 '16 09:06 dniku

@Pastafarianist This "grid" pattern may caused by the aggressive big stride of the early conv layer in caffenet and googlenet, which lead to a lot of information loss. So vgg like network has no "grid" pattern.

austingg avatar Jun 25 '16 09:06 austingg

Could someone with better hardware than mine run the same commands on the same images (from the repository) and make sure that the output is similar? If it is, then this issue should be closed.

dniku avatar Jun 25 '16 12:06 dniku

I'm getting the same grid artifacts using caffenet, to a lesser extent with googlenet.

crocodoyle avatar Jun 28 '16 00:06 crocodoyle

This version of the Inception model should produce fewer artifacts: https://www.dropbox.com/s/tdaowz2au059iqi/googlenet_style.caffemodel?dl=0

fzliu avatar Jun 28 '16 20:06 fzliu

hi @fzliu ,
What's the difference between this version and the original inception model ?

austingg avatar Jun 29 '16 01:06 austingg

This model is fine-tuned from the original Inception model after removing local response normalization (LRN).

fzliu avatar Jun 30 '16 07:06 fzliu