optimize-net
optimize-net copied to clipboard
fmassa
OptNet - Reducing memory usage in torch neural nets
Hello, awesome work with this package! I have an issue trying to visualize nn.Contiguous. Do you know what might be causing it? If it's not an easy fix, is there...
Why the occupied memory is larger than the` mem = optnet.countUsedMemory(net)` mem.total_size = 4400 M while I using `nvidia-smi` checking the gpu memory occupied up to almost 11000 M
Adds more information about how the memory sharing is performed. cc @nicholas-leonard
Could you guys expand on the how does it work section in the docs? Specifying how the output, gradInput, weight, gradWeight and intermediate state buffers are handled would be a...
This will help improve the memory optimizations in `training` mode, as for the moment the backward graph in `optimizeMemory` is very simplistic and has several redundant/unwanted links.
Creating a graph did not work for me. I get errors. It turns out that apply_func in graphgen.lua runs the original updateOutput again (after the first forward() call) via base_func....
repro: ``` lua require 'cunn' require 'cudnn' local generateGraph = require 'optnet.graphgen' local iterm = require 'iterm' local model = nn.DataParallelTable(1) model:add(cudnn.SpatialConvolution(3,96,7,7,3,3),1) model:add(cudnn.SpatialConvolution(3,96,7,7,3,3),2) model:cuda() local input = torch.randn(32,3,224,224):cuda() iterm.dot(generateGraph(model, input))...
Should fix https://github.com/fmassa/optimize-net/issues/6 There is a lot of magic going on in here: - temporarily overwrite torch `__index` and `select` to keep track of the modules in which those functions...
The current way of generating the graph in `graphgen.lua` is to rely on the `input` and `self.output` of each module. The underlying assumption is that those tensors will be the...
Does this work relating to OptNet: Differentiable Optimization as a Layer in Neural Networks? https://github.com/locuslab/optnet Thank you very much.
