spikingjelly
spikingjelly copied to clipboard
fangwei123456
How to measure the engery consumption of SNN within SpikingJelly
Issue type
- [ ] Bug Report
- [ ] Feature Request
- [ ] Help wanted
- [x] Other
SpikingJelly version
0.0.0.0.13
Description As we all know, SNNs exhibit a higher energy efficiency as compared to other forms of ANNs.
I wonder how to measure the engery consumption of SNN within SpikingJelly.
Thanks in advance!
You can refer sec3.2 in this paper:
https://arxiv.org/pdf/2301.11929.pdf
And here is a library to count ops:
https://github.com/iCGY96/syops-counter
您好，感谢您推荐的文章和代码库。我已计算出该模型的AC和MAC数量，并估计了其理论能耗，这表明SNN确实更加energy-efficient。
然而，我有一个实际的问题想请教您：使用惊蛰训练SNN模型时，单轮耗时远高于结构和参数量完全相同的ANN模型，并且GPU功率也更高（通过nvidia-smi查看）。测试阶段也面临相同的问题。因此，SNN的耗时更长，功率更高，这似乎与其energy-efficient的特性相悖。我们认为这个问题是由仿真步长T所导致的，但即使我们将T设置为2，这个问题仍然存在。我想请问您：
（1）对于SNN在实际训练/测试中的能耗更高的现象，您有什么解释和分析？
（2）这是否会影响SNN所具有的energy-efficient特性？
感谢您的耐心阅读和解答，希望不会太耽误您的时间。
补充：我们的模型结构和代码均参考[使用卷积SNN识别Fashion-MNIST]
Hi all, we're actually measuring the performance and the emissions, together with the power consumption of SNNs vs ANNs. We're using CodeCarbon to measure the emissions. We measured the performances over the MNIST, CIFAR, FLOWERS102 (with small, medium and large networks - same architecture for the SNN and ANN ofc) and some pretrained models converted.
However it seems like that none of the SNN is more efficient of any ANN. Do you think that the standard image classification task is not a good test ground? we know that SNN should work better with neuromorphic datasets or videos, despite this, we're kinda concerned about these results.
Below ill attach some plots regarding CO2 emissions (the first one) and power consumptions (second one), related to the MNIST dataset.
NB: apologies for the text over the barplot, is still wip, but for each SNN we tested T=5,20,50, thus three bar for model. We also point out that all the SNN have as step mode 'm', as backend 'cupy', and use IF neurons
ill also show you what we tracked:
tracker, out_name = setup_tracker('MNIST')
tracker.start()
train_val_mnist = fit_and_eval(model_snn, epochs, optimizer, criterion, mnist_nclasses,
train_dataload_mnist, val_dataload_mnist, device,
model_snn.T, runs_path, ann=False, validation_step=True, convolution=False, onehot=False)
emissions_mnist_snn: float = tracker.stop()
where the 'fit_and_eval' functions is nothing but a standard training and evaluation loop.
Any toughts about this? All the tests have been performed in Colab.
@fangwei123456 @tianyunzhe
@NicolaCST SNNs show the energy efficiency only on neuromorphic chips. Running on GPUs can not prove this because GPUs do not use any optimization for spikes.
This case is similar to quantized neural network. For example, your 8-bits ANN will use less energy consumption on FPGA, but in GPU it still uses the float32 operation (just like spike 1 will be regarded as float 1.0 in GPU rather than a bool value) and has no difference with 32-bits ANN in energy consumption and running speed.
@fangwei123456 Ye we were also considering that, thank you for the advice and ur time. Did you know if there are any publicly available spaces that could allow us to use/simulate that type of architecture?
You can refer sec3.2 in this paper:
https://arxiv.org/pdf/2301.11929.pdf
And here is a library to count ops:
https://github.com/iCGY96/syops-counter
@NicolaCST You can refer to this. In SNNs, the power consumption is estimated by SOP * energy of each SOP, or AC * energy of each AC.
Thank you @fangwei123456
After importing the SyOPS library and fixing some issues (not working on latest versions of SpikingJelly due to the requirement of the module 'clock_driven' and some old neurons like 'MultiStepLIF' - I've opened an issue about it), i've managed to achieve this result, using a simple two-layer SNN with T=20 on MNIST
#SyOPS, AC, MAC ['5.32 K Ops', '5.32 K Ops', '1.0 Ops']
Computational complexity ACs: 5.32 K Ops Computational complexity MACs: 1.0 Ops Number of parameters: 203.26 k
As you mentioned in the previous answer, i should multiply ACs for the energy of each AC, but How can i know this value? should i estimate it? By asking to ChatGPT is seems like that he tends to make an estimation or assume that is a known value
''' Similarly, if we know that each action potential requires 0.1 microjoules of energy, and our SNN requires 1,000 action potentials per inference pass, we can estimate the power consumption as follows:
Power consumption = AC * energy per AC = 1,000 * 0.1e-6 = 0.0001 watts '''
Thank you in advance
@NicolaCST
i should multiply ACs for the energy of each AC, but How can i know this value?
You can refer to the sec.5 in this paper: https://arxiv.org/pdf/2301.11929.pdf
@fangwei123456 您好，我看了您推荐的论文和代码。 论文“Training Full Spike Neural Networks via Auxiliary Accumulation Pathway”中提到snn的理论计算功耗范围为 $\mathrm E(\mathcal F)\in[T\cdot E_{m a c}\cdot O_{m a c},T\cdot(E_{a c}\cdot O_{a c}+E_{m a c}\cdot O_{m a c})] \quad(15)$ , 可以根据模型的时间步长T和 $O_{ac}$ , $O_{mac}$ 计算得到。 但是我发现在相同结构的模型中SNN的MAC操作数量会明显大于ANN模型中的MAC操作数。您推荐的 syops-counter 在 benchmark中也提到了在相同的模型结构中例如sew_resnet18与resnet18中的功耗对比。但是从结果上来看似乎难以得出SNN更energy-efficient。
| Model | Input Resolution | Params(M) | ACs(G) | MACs(G) | Energy (mJ) | Acc@1 | Acc@5 |
|---|---|---|---|---|---|---|---|
| sew_resnet18 | 224x224 | 11.69 | 0.50 | 2.75 | 13.10 | 63.18 | 84.53 |
| resnet18 | 224x224 | 11.69 | 0.00 | 1.82 | 8.372 | 69.76 | 89.08 |
如果使用公式（15）计算功耗范围，论文Training Full Spike Neural Networks via Auxiliary Accumulation Pathway中计算了SEW ResNet-18 (ADD)的Estimated Consumption和Dynamic Consumption。
| Network | Methods | Acc@1 | T | EC(mJ) | Oac(G) | Omac(G) | DC(mJ) |
|---|---|---|---|---|---|---|---|
| SEW ResNet-18 (ADD) (Fang et al., 2021a) | MPSNN | 63.18 | 4 | [12.65, 16.40] | 0.51 | 2.75 | 13.11 |
SEW ResNet-18 (ADD)的动态功耗似乎也比resnet18要高。
下面是我的一些疑问： SNN相比于ANN具有更多的mac操作，如果使用论文“Training Full Spike Neural Networks via Auxiliary Accumulation Pathway”中提到SNN的理论计算功耗方法公式(15),以及syops-counter 中的benchmark中提供的信息，似乎SNN的计算功耗更大（在resnet18中），这有可能是什么原因造成的？
感谢您的耐心阅读和解答，希望不会太耽误您的时间。
sew resnet如果用ADD操作符的话是非二值的，功耗会高一些。 例如，如果ADD累加的spike数量为n，估算功耗时应该按n次AC或1次MAC。syops-counter是怎么做的，需要看它具体实现。
@fangwei123456 您好！我在使用syops库时发现，其标准的Resnet18，MACs操作数为1.82G，但ACs操作仅为0(计算能耗时也仅仅考虑了MAC操作的能耗，并不是一个MACs=MAC+AC)，正常来说对于一个Resnet18，MACs和ACs操作数理论上是近似的，所以我不太明白这一块，来请教您一下。 提前感谢您的阅读和耐心解答
