channel_prune
channel_prune copied to clipboard
eeric
你好，看到你修改了pytorch_prune代码到resnet上
我运行代码的时候
def compute_rank(self, grad):
activation_index = len(self.activations) - self.grad_index - 1
activation = self.activations[activation_index]
values = torch.sum((activation * grad), dim = 0). sum(dim=2).sum(dim=3)[0, :, 0, 0].data
报错dimension out of range (expected to be in range of [-2, 1], but got 3)，但是看你们都没有这个错误。钩子函数，好像跟不进去。我用的python3.5，pytorch0.2
@eeric 有一个地方不太懂，在选择候选裁剪通道的时候，应该只要进行一次前向和后向，为什么代码利用要train_epoch()进行一个周期的运算？
我的理解是，先选出需要裁剪的通道，然后一个通道一个通道的裁剪，裁剪完，再进行finetune恢复。
def get_candidates_to_prune(self, num_filters_to_prune):
self.prunner.reset()
self.train_epoch(rank_filters = True) #这个地方好像没有更新权值，只是在不停的调用compute_rank()计算排序通道重要性，是不只需要计算一次？
self.prunner.normalize_ranks_per_layer()
return self.prunner.get_prunning_plan(num_filters_to_prune)
麻烦你，有个地方没看太懂，请教下，15行为什么要给每个通道号减去一个变量i？
1 ~ def get_prunning_plan(self, num_filters_to_prune):
2 ~ ~ filters_to_prune = self.lowest_ranking_filters(num_filters_to_prune)
3
4 ~ ~ # After each of the k filters are prunned,
5 ~ ~ # the filter index of the next filters change since the model is smaller.
6 ~ ~ filters_to_prune_per_layer = {}
7 ~ ~ for (l, f, _) in filters_to_prune:
8 ~ ~ ~ if l not in filters_to_prune_per_layer:
9 ~ ~ ~ ~ filters_to_prune_per_layer[l] = []
10 ~ ~ ~ filters_to_prune_per_layer[l].append(f)
11
12 ~ ~ for l in filters_to_prune_per_layer:
13 ~ ~ ~ filters_to_prune_per_layer[l] = sorted(filters_to_prune_per_layer[l])
14 ~ ~ ~ for i in range(len(filters_to_prune_per_layer[l])):
15 ~ ~ ~ ~ filters_to_prune_per_layer[l][i] = filters_to_prune_per_layer[l][i] - i
16
17 ~ ~ filters_to_prune = []
18 ~ ~ for l in filters_to_prune_per_layer:
19 ~ ~ ~ for i in filters_to_prune_per_layer[l]:
20 ~ ~ ~ ~ filters_to_prune.append((l, i))
21
22 ~ ~ return filters_to_prune
现在把yolo的裁剪改进去了，一次裁512个通道，才裁剪一次，恢复了20epoch，还是下降1-4个点，怀疑是否裁剪错了通道，上面那个通道为什么要减变量i，没太懂，是否这里可能有问题。
@hewumars 您好，我最近也尝试了对YOLOv2进行剪枝，但是我每次只能减一个filter，不然准确率就都下降为0，可以问问您在源代码基础上做了哪些改进吗？以及学习率方面的设置？
@hewumars values = torch.sum((activation * grad), dim = 0). sum(dim=2).sum(dim=3)[0, :, 0, 0].data 这个问题怎么解决的啊。我知道要SUM出Channel可以这么写： torch.sum((activation * grad), dim = 0). sum(dim=1).sum(dim=2)。但是后面维度扩展该怎么办啊。
Are you programming the code using python2.7? The usage of ordered dictionaries is not correct. like: x = self.model.features._modules.items()[layer][1][kt].bn1(x)
@yangzhikai according to model design, like this x = self.features._modules.items()[layer][1][kt].bn1(x), please try on.
@eeric Ordered dictionary In python3 cannot execute this operation：new_conv = self.model.features._modules.items()[layer][1][kt].bn1。And I try a command like : new_conv = model._modules['4'][0].conv1 。That may be achieve the same effect. But I'm not sure the type of "new_conv " above is a conv_layer like : conv2d(64,128，stride=0) .....
@hewumars
你好，我在运行python finetune.py --train的时候遇到这个问题：
Traceback (most recent call last):
File "/home/wade/Downloads/finetune-res.py", line 351, in
@nizihan @hewumars 我的理解是这样的： 根源在于prune.py中的剪枝函数的剪枝操作。conv层删除相应的filter，next_conv也需要跟着作调整——1，next_conv输入的通道数变少；2,输入通道的序号，比如conv中6号filter被删除，8号filter被删除，那么next_conv原来的7号现在应该是6,原来的9号，现在应该是7。
我遇到了另一个问题，在运行python finetune.py --train的过程中
我输出的精度一直为100%
Epoch: 0
Accuracy : 100.0%
Epoch: 1
Accuracy : 100.0%
Epoch: 2
Accuracy : 100.0%
Epoch: 3
Accuracy : 100.0%
Epoch: 4
Accuracy : 100.0%
这是训练集的问题吗？我是在kaggle官网上下载的Dogs vs. Cats
程序里，由于pytorch版本问题，我添加了
values =
torch.sum((activation * grad), dim = 0,keepdim=True).
sum(dim=2,keepdim=True).sum(dim=3,keepdim=True)[0, :, 0, 0].data
请问问题出在哪里呢……
@tearhupo121031 https://github.com/eeric/channel_prune/issues/3#issuecomment-375931777
as I running the code, I found that accuracy useing layers respectively is lower than useing a sequencial model which layers in it. Do you encounter this problem? For instance ，In the model_forward of filterprunner . Using x=x.sequencal(x) yeild a higher accuracy than Using " for layer in emuerate model.features()".