PyTorch-MFNet icon indicating copy to clipboard operation
PyTorch-MFNet copied to clipboard

Published 20 hours ago • verified publisher icon cypw

Question about counting the parameter number and FLOPs

Open gyguo opened this issue 6 years ago • 7 comments

I counted the parameter number and FLOPs of MFNet, the parameter number is computed by the code

model = MFNET_3D(num_classes=101)
params = sum(p.numel() for p in model.parameters())

outputs a same result 7996368 as shown in the paper: 8.0 M.

But the FLOPs I got is different from the result in the paper, so whether you can show me the code you compute the FLOPS, especially for the nn.Conv3d layer, I think I made some mistake of computing the FLOPS of nn.Conv3d layer.

gyguo avatar Nov 22 '18 08:11 gyguo 

% Matlab Code
% FLOPs for nn.Conv3d, without bias
A_size = [num_out/group, prod(kernel)*num_input/group];       % weight
C_size = [prod(kernel)*num_input/group, out_h*out_w*out_t];   % im2col
flops = 0;
for i_group = 1:group    
    flops = flops + A_size(1)*A_size(2)*C_size(2);
end

cypw avatar Nov 22 '18 12:11 cypw 

% Matlab Code
% FLOPs for nn.Conv3d, without bias
A_size = [num_out/group, prod(kernel)*num_input/group];       % weight
C_size = [prod(kernel)*num_input/group, out_h*out_w*out_t];   % im2col
flops = 0;
for i_group = 1:group    
    flops = flops + A_size(1)*A_size(2)*C_size(2);
end

Thank you very much

gyguo avatar Nov 22 '18 14:11 gyguo

It is a nice work, the experiment results could be reproduced with the pretrained model provided by the author. I'd like to count the FLOPs of the 3D-MFnet next, where can I find the code, if anyone expert? Thanks for your kind help~

China-LiuXiaopeng avatar Dec 28 '18 04:12 China-LiuXiaopeng

I am counting the FLOPs of 3D MFNet, which should be 11.1G FLOPs (paper), however, I figure out 11.18G a little bigger. Do anyone have time to check my calculations, thanks!

Input shape: [1, 3, 16, 224, 224]
Output shape: [1, 400]
Conv3d(3, 16, kernel_size=(3, 5, 5), stride=(1, 2, 2), padding=(1, 2, 2), bias=False) FLOPs: 722534400
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 3211264
ReLU(inplace) FLOPs: 3211264
MaxPool3d(kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), dilation=1, ceil_mode=False) FLOPs: 3211264
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 802816
ReLU(inplace) FLOPs: 802816
Conv3d(16, 24, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 19267584
BatchNorm3d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(24, 16, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 19267584
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 802816
ReLU(inplace) FLOPs: 802816
Conv3d(16, 96, kernel_size=(3, 3, 3), stride=(2, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 65028096
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 802816
ReLU(inplace) FLOPs: 802816
Conv3d(16, 96, kernel_size=(1, 1, 1), stride=(2, 1, 1), bias=False) FLOPs: 38535168
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 24, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(24, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 24, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(24, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(48, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 192, kernel_size=(3, 3, 3), stride=(1, 2, 2), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 195084288
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 192, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False) FLOPs: 115605504
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(48, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(48, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(48, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(96, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 384, kernel_size=(3, 3, 3), stride=(1, 2, 2), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 195084288
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 384, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False) FLOPs: 115605504
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(192, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 768, kernel_size=(3, 3, 3), stride=(1, 2, 2), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 195084288
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 768, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False) FLOPs: 115605504
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 75264
ReLU(inplace) FLOPs: 75264
Conv3d(192, 768, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 75264
ReLU(inplace) FLOPs: 75264
Conv3d(192, 768, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
AvgPool3d(kernel_size=(8, 7, 7), stride=(1, 1, 1), padding=0) FLOPs: 301056
Linear(in_features=768, out_features=400, bias=True) FLOPs: 307200
Flops:  11.18G 
Params: 8.0M

China-LiuXiaopeng avatar Dec 28 '18 07:12 China-LiuXiaopeng

@China-LiuXiaopeng where do you find the code to count?

Zhuysheng avatar Jan 09 '19 07:01 Zhuysheng

I counted the parameter number and FLOPs of MFNet, the parameter number is computed by the code

model = MFNET_3D(num_classes=101) params = sum(p.numel() for p in model.parameters()) outputs a same result 7996368 as shown in the paper: 8.0 M.

But the FLOPs I got is different from the result in the paper, so whether you can show me the code you compute the FLOPS, especially for the nn.Conv3d layer, I think I made some mistake of computing the FLOPS of nn.Conv3d layer.

XZJie-AILab avatar Jan 26 '19 08:01 XZJie-AILab

I am counting the FLOPs of 3D MFNet, which should be 11.1G FLOPs (paper), however, I figure out 11.18G a little bigger. Do anyone have time to check my calculations, thanks!

Input shape: [1, 3, 16, 224, 224]
Output shape: [1, 400]
Conv3d(3, 16, kernel_size=(3, 5, 5), stride=(1, 2, 2), padding=(1, 2, 2), bias=False) FLOPs: 722534400
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 3211264
ReLU(inplace) FLOPs: 3211264
MaxPool3d(kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), dilation=1, ceil_mode=False) FLOPs: 3211264
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 802816
ReLU(inplace) FLOPs: 802816
Conv3d(16, 24, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 19267584
BatchNorm3d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(24, 16, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 19267584
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 802816
ReLU(inplace) FLOPs: 802816
Conv3d(16, 96, kernel_size=(3, 3, 3), stride=(2, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 65028096
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 802816
ReLU(inplace) FLOPs: 802816
Conv3d(16, 96, kernel_size=(1, 1, 1), stride=(2, 1, 1), bias=False) FLOPs: 38535168
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 24, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(24, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 24, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(24, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 96, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(48, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 192, kernel_size=(3, 3, 3), stride=(1, 2, 2), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 195084288
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 2408448
ReLU(inplace) FLOPs: 2408448
Conv3d(96, 192, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False) FLOPs: 115605504
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(48, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(48, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 48, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(48, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 192, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(96, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 384, kernel_size=(3, 3, 3), stride=(1, 2, 2), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 195084288
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 1204224
ReLU(inplace) FLOPs: 1204224
Conv3d(192, 384, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False) FLOPs: 115605504
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 96, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 150528
ReLU(inplace) FLOPs: 150528
Conv3d(96, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 384, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(192, 384, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 115605504
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 768, kernel_size=(3, 3, 3), stride=(1, 2, 2), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 195084288
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 231211008
BatchNorm3d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 602112
ReLU(inplace) FLOPs: 602112
Conv3d(384, 768, kernel_size=(1, 1, 1), stride=(1, 2, 2), bias=False) FLOPs: 115605504
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 75264
ReLU(inplace) FLOPs: 75264
Conv3d(192, 768, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 192, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 75264
ReLU(inplace) FLOPs: 75264
Conv3d(192, 768, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) FLOPs: 57802752
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=16, bias=False) FLOPs: 390168576
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
Conv3d(768, 768, kernel_size=(1, 3, 3), stride=(1, 1, 1), padding=(0, 1, 1), groups=16, bias=False) FLOPs: 130056192
BatchNorm3d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) FLOPs: 301056
ReLU(inplace) FLOPs: 301056
AvgPool3d(kernel_size=(8, 7, 7), stride=(1, 1, 1), padding=0) FLOPs: 301056
Linear(in_features=768, out_features=400, bias=True) FLOPs: 307200
Flops:  11.18G 
Params: 8.0M

Hello, my friend. I want to ask you what code is used to calculate the FLOPs.

ys-dpc avatar Apr 01 '19 10:04 ys-dpc