distiller

distiller copied to clipboard

ValueError occurs when I run quantization-aware-training using PACT. The error message is "ValueError: saturation_min must be smaller than saturation_max" and it seems like the error occurs when the clipping value of ReLU is below zero because the clipping value cannot be negative theoretically.

setting:

• architecture: preact_resnet20_cifar
• cifar10
• yaml: distiller/examples/quantization/quant_aware_train/preact_resnet20_cifar_pact.yaml (act_clip_decay: 0.01 added, file attached)

================= Error Log ============================ File "compress_classifier.py", line 212, in main() File "compress_classifier.py", line 78, in main app.run_training_loop() File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 192, in run_training_loop top1, top5, loss = self.train_validate_with_scheduling(epoch) File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 130, in train_validate_with_scheduling top1, top5, loss = self.train_one_epoch(epoch, verbose) File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 116, in train_one_epoch loggers=[self.tflogger, self.pylogger], args=self.args) File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 577, in train output = model(inputs) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(*inputs[0], **kwargs[0]) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/home/jelim/f_github/test_distiller/distiller/distiller/models/cifar10/preresnet_cifar.py", line 153, in forward x = self.layer1(x) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/home/jelim/f_github/test_distiller/distiller/distiller/models/cifar10/preresnet_cifar.py", line 76, in forward out = self.relu(out) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/home/jelim/f_github/test_distiller/distiller/distiller/quantization/clipped_linear.py", line 65, in forward scale, zero_point = asymmetric_linear_quantization_params(self.num_bits, 0, self.clip_val, signed=False) File "/home/jelim/f_github/test_distiller/distiller/distiller/quantization/q_utils.py", line 91, in asymmetric_linear_quantization_params raise ValueError('saturation_min must be smaller than saturation_max') ValueError: saturation_min must be smaller than saturation_max

====================================================

error_saturation_min_PACT.log pact_yaml.zip

Hi, did you find a solution to this issue?