Why the models I trained is much larger than downloaded ones?

[wlxyhy]

The model I trained is nearly 100MB, while the downloaded model is just about 10~20MB?

[filmo]

Wondering the same thing. I'm using instance_norm and my models are all about 92MB

[htoyryla]

Are you aware of that the checkpoint file contains, not only the model but also the loss history. See here https://github.com/jcjohnson/fast-neural-style/blob/master/train.lua#L288-L307

I guess one could load the checkpoint, remove the loss histories and save the checkpoint again, to see how much effect the loss histories do make.

[filmo]

I've already commented out the loss history during checkpointing and it made little difference. Maybe a few MB. My models are consistently in the 90 to 110MB range rather than 15 to 20MB. I'm not good enough with Torch yet to have figured out why this is. Seems like some sort of intermediate state information is being preserved.

I had assumed model:clearState() would purge that, but it is already in the code and the files are still large. I'm going to experiment with cloning the saved model and extracting only the weights and biases.

    local checkpoint = {
        opt=opt,
  --      train_loss_history=train_loss_history,
  --      val_loss_history=val_loss_history,
  --      val_loss_history_ts=val_loss_history_ts,
  --      style_loss_history=style_loss_history,
    }

[htoyryla]

This could also depend on how torch serialization works. There are options to save as binary or ascii. https://github.com/torch/torch7/blob/master/doc/serialization.md

In a project of my own I currently have a model (of comparable complexity) which makes 800MB when saved, so the more general issue what affects the saved file size interests me too.

One thing to note is that changing the model architecture through the -arch flag will affect the model size.

[htoyryla]

"Extracting only the weights and biases"

Unlike Caffe, where the model architecture and the weights are stored in different files (prototxt and caffemodel), torch.save saves the complete model object, and the corresponding torch.load reconstructs the model object. Saving only weights and biases would require that, at load time, the model can be reconstructed through other means so that the weights can be applied.

[htoyryla]

I had a look at how model architecture affects the size of the saved model. The default architecture is 'c9s1-32,d64,d128,R128,R128,R128,R128,R128,u64,u32,c9s1-3', i.e. all residual layers have 128 filters while the pretrained instance_norm/udnie.t7 (size 11.2M) only has 64 filters.

But eccv16/la_muse.t7 uses the default architecture and the size is still not more that 25.5M. So it looks like the architecture does not explain the differences either (assuming the use of the defaults and not experimenting with larger model architectures).

So back to experimenting with torch serialization. Could it be that the serialization works differently depending on the environment. It does not look like so: if I load udnie.t7 and save it again (with default settings) the file is still 11.2M. Saving in ascii results in 30.3M; it should be easy to check if your files are ascii, but I'd be surprised if they are.

[htoyryla]

BTW, I had a look at one of my own models, size around 130M. It turned out that I was using an architecture with up to 512 filters. So check your -arch parameter.

[filmo]

I'll check that. I was just using the default arch that shipped with the code. My guess is that the problem lies there.