nncf Mypy checks for NNCF common pruning code

Changes

This PR closes issue https://github.com/openvinotoolkit/nncf/issues/2494 by addressing various mypy checks

Resolved specific mypy errors related to type inconsistencies.
Utilized # type:ignore for cases requiring significant refactoring due to untyped packages, attributes not defined etc.
Added the directory nncf/common/pruning to .mypy.ini to include additional files for type checking.

Tests

Pytest and make pre-commit test were run to ensure that the changes did not change any logic and that the codebase remained functional.

Mar 30 '24 11:03 RitikaxShakya

@vshampor @p-wysocki Hello! Please take a look at my PR and let me know if i would need to do anything more. Thank you :)

Mar 30 '24 11:03 RitikaxShakya

Codecov Report

Attention: Patch coverage is 34.30657% with 90 lines in your changes are missing coverage. Please review.

Project coverage is 28.01%. Comparing base (fa8b702) to head (159e2dd). Report is 10 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #2613       +/-   ##
============================================
- Coverage    91.17%   28.01%   -63.17%     
============================================
  Files          494      494               
  Lines        45428    45379       -49     
============================================
- Hits         41420    12713    -28707     
- Misses        4008    32666    +28658

Files	Coverage Δ
nncf/common/pruning/clusterization.py	`30.88% <88.88%> (-66.18%)`	:arrow_down:
nncf/common/pruning/model_analysis.py	`24.46% <77.77%> (-74.47%)`	:arrow_down:
nncf/common/pruning/symbolic_mask.py	`44.44% <75.00%> (-55.56%)`	:arrow_down:
nncf/common/pruning/node_selector.py	`20.00% <0.00%> (-78.74%)`	:arrow_down:
nncf/common/pruning/utils.py	`35.35% <56.25%> (-59.60%)`	:arrow_down:
nncf/common/pruning/shape_pruning_processor.py	`29.41% <18.18%> (-68.21%)`	:arrow_down:
nncf/common/pruning/weights_flops_calculator.py	`23.07% <0.00%> (-75.00%)`	:arrow_down:
nncf/common/pruning/mask_propagation.py	`30.30% <21.42%> (-69.70%)`	:arrow_down:
nncf/common/pruning/operations.py	`46.91% <31.57%> (-50.22%)`	:arrow_down:
nncf/common/pruning/schedulers.py	`18.07% <9.09%> (-80.70%)`	:arrow_down:

... and 386 files with indirect coverage changes

Flag	Coverage Δ
COMMON	`?`
ONNX	`?`
OPENVINO	`?`
TENSORFLOW	`14.20% <34.30%> (-15.90%)`	:arrow_down:
TORCH	`24.01% <34.30%> (-41.91%)`	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
common	`33.66% <34.30%> (-60.14%)`	:arrow_down:
torch	`30.18% <ø> (-63.31%)`	:arrow_down:
tensorflow	`30.62% <ø> (-63.13%)`	:arrow_down:
onnx	`0.00% <ø> (-93.10%)`	:arrow_down:
openvino	`0.00% <ø> (-94.17%)`	:arrow_down:
ptq	`18.67% <ø> (-71.45%)`	:arrow_down:

Mar 30 '24 12:03 codecov[bot]

Greeting, @RitikaxShakya! Looks like some pre-commit checks haven't been passed, please try make test-torch on your local machine

Apr 03 '24 09:04 daniil-lyakhov

@daniil-lyakhov Hello! I passed make test-torch on my local machine and pushed again The errors were same on both branch develop and issue#2494

Apr 03 '24 11:04 RitikaxShakya

@daniil-lyakhov Hello! I passed make test-torch on my local machine and pushed again The errors were same on both branch develop and issue#2494

Looks like an environment problem, our internal CI fails with following error:

tests/torch/accuracy_aware_training/test_accuracy_aware_config.py:16: in <module>
    from nncf.torch.initialization import register_default_init_args
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

please try to reinstall nncf and launch torch tests

Apr 04 '24 12:04 daniil-lyakhov

@daniil-lyakhov Hello! I reinstalled and run all tests Pytest , make pre-commit and make test-torch on my local machine and pushed again. For make pre-commit , all tests were passed, but for make test-torch, the errors are same as before.

Apr 06 '24 09:04 RitikaxShakya

@daniil-lyakhov Hello! I reinstalled and run all tests Pytest , make pre-commit and make test-torch on my local machine and pushed again. For make pre-commit , all tests were passed, but for make test-torch, the errors are same as before.

Hello! Sorry for a delay, but as the error states your python interpreter has no site packages directory (please see following call https://github.com/openvinotoolkit/nncf/blob/develop/tests/shared/paths.py#L23). Could you please try to run following commands to check your python is working correctly?

python3.8 -m venv nncf_env
source nncf_env/bin/activate
cd nncf/
make install-torch-test

Apr 10 '24 09:04 daniil-lyakhov

@daniil-lyakhov Hello! Thank you! There were some packages missing, after running those commands and after make test-torch , I get these errors, but make pre-commit test were all passed :

ERROR tests/torch/accuracy_aware_training/test_accuracy_aware_config.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/accuracy_aware_training/test_runner.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/accuracy_aware_training/test_training_loop.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/automl/test_quantization_env.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/composite/test_sparsity_quantization.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/experimental/search_building_blocks/test_search_building_blocks.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/experimental/search_building_blocks/test_transformer_blocks.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/models_hub_test/test_hf_transformers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/models_hub_test/test_timm.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/models_hub_test/test_torchvision_models.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/modules/test_rnn.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_all_elasticity.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_elastic_depth.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_elastic_kernel.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_elastic_width.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_flops.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_ps_controller.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_sanity_sample.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_search.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_search_space.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_state.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/experimental/test_nodes_grouping.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_evolutionary_optimization.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_flops_pruning.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_layers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_legr.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_set_pruning_rate.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_common.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_distributed.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_model_pruning_analysis.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_schedulers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_utils.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_calculation_quantizer_params.py
ERROR tests/torch/ptq/test_fast_bias_correction.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_fq_params_calculation.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_graphs.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_min_max.py
ERROR tests/torch/ptq/test_ptq_params.py
ERROR tests/torch/ptq/test_quantizer_config.py
ERROR tests/torch/ptq/test_smooth_quant.py
ERROR tests/torch/ptq/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_weights_compression.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_wrap_model.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/qat/test_qat_classification.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/qat/test_qat_object_detection.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/qat/test_qat_segmentation.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_adjust_padding.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_algo_quantization.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_autoq_precision_init.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_functions.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_hawq_precision_init.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_hw_config.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_logarithm_scale.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_manual_precision_init.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_overflow_issue_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_quantization_metric.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_range_init.py
ERROR tests/torch/quantization/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_unified_scales.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/const/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_helpers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_modules.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_components.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_model_saving.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_structured_mask.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_training.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/rb/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/test_common.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_algo_common.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_api_behavior.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_backward_compat.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_compressed_graph.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_compression_lr_multiplier.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_compression_training.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_context_independence.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_custom_modules.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_distributed_data_parallel_mode.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_extractor.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_frozen_layers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_graph_building.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_init_data_loader.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_input_management.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_knowledge_distillation.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_layer_attributes.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_load_model_state.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_model_analyzer.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_model_graph_manager.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_model_transformer.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_nncf_network.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_no_compression_algorithm.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_no_nncf_trace_patching.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_pytorch_patch.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_resume_from_checkpoint.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_sanity_sample.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_sota_checkpoints.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_statistics_aggregator.py
ERROR tests/torch/test_telemetry.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_tracing_context.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_utils.py - TypeError: 'type' object is not subscriptable
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 112 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================== 28 deselected, 3 warnings, 112 errors in 7.91s ===========================================
make: *** [Makefile:146: test-torch] Error 2```

Apr 10 '24 15:04 RitikaxShakya

@RitikaxShakya , Let's take a look on one of the error:

tests/torch/accuracy_aware_training/test_runner.py:23: in <module>
    from nncf.torch.accuracy_aware_training.runner import PTAccuracyAwareTrainingRunner
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

here you are trying to use dict as a type, but it brakes python as dict is actual a type and it is not subscriptable. Please do this here and elsewhere

from typing import Dict
...
def __init__(self, controller: CompressionAlgorithmController, params: Dict[str, Any]):
...

Apr 12 '24 14:04 daniil-lyakhov

@RitikaxShakya , Let's take a look on one of the error:

tests/torch/accuracy_aware_training/test_runner.py:23: in <module>
    from nncf.torch.accuracy_aware_training.runner import PTAccuracyAwareTrainingRunner
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

here you are trying to use dict as a type, but it brakes python as dict is actual a type and it is not subscriptable. Please do this here and elsewhere

from typing import Dict
...
def __init__(self, controller: CompressionAlgorithmController, params: Dict[str, Any]):
...

Hello @daniil-lyakhov Sorry for so late reply. I made the changes as you suggested seems like its still not passing checks. Also the page for details of NNCF TF precommit (Ubuntu 20.04), NNCF Torch + common precommit are not opening.

Apr 24 '24 06:04 RitikaxShakya

@RitikaxShakya , Let's take a look on one of the error:
tests/torch/accuracy_aware_training/test_runner.py:23: in <module>
    from nncf.torch.accuracy_aware_training.runner import PTAccuracyAwareTrainingRunner
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable
here you are trying to use dict as a type, but it brakes python as dict is actual a type and it is not subscriptable. Please do this here and elsewhere
from typing import Dict
...
def __init__(self, controller: CompressionAlgorithmController, params: Dict[str, Any]):
...
Hello @daniil-lyakhov Sorry for so late reply. I made the changes as you suggested seems like its still not passing checks. Also the page for details of NNCF TF precommit (Ubuntu 20.04), NNCF Torch + common precommit are not opening.

Hi @RitikaxShakya, no problems! Please try to rebase your PR. Besides following error appeared in TF pre-commit

   def generate_output_mask(
        cls, node: NNCFNode, graph: NNCFGraph, tensor_processor: Type[NNCFPruningBaseTensorProcessor]
    ) -> Optional[NNCFTensor]:
        """
        Generate output mask from input masks with all None replaced by identity masks.
        If all input masks is None return None.
    
        :param node: Node to determine it's sources.
        :param graph: NNCF graph to work with.
        :param tensor_processor: Interface with tensor processing methods.
        :return: Filled input masks.
        """
        input_edges = graph.get_input_edges(node)
        previous_nodes = [edge.from_node for edge in input_edges]
        input_masks = [input_node.attributes["output_mask"] for input_node in previous_nodes]
        input_masks = [
            input_mask[node.node_name] if isinstance(input_mask, dict) else input_mask for input_mask in input_masks
        ]
    
        not_empty_masks: List[NNCFTensor] = [mask for mask in input_masks if mask is not None]
        if not not_empty_masks:
            return None
    
        first_non_empty_mask = not_empty_masks[0]
    
        device = first_non_empty_mask.device
        filled_input_masks = []
        for i, mask in enumerate(input_masks):
            if mask is None:
                concat_axis = node.layer_attributes.axis if node.layer_attributes is not None else None  # type:ignore
>           if concat_axis is not None:
E           UnboundLocalError: local variable 'concat_axis' referenced before assignment

Please try to run pytorch and tensorflow pre-commit locally, as pre-commit works fine on develop branch you should be able to find bugs in your pr

Apr 24 '24 08:04 daniil-lyakhov

@RitikaxShakya , do you plan to continue to work on this PR?

May 14 '24 12:05 MaximProshin

@MaximProshin Hello! sorry for late reply, I was busy with my semester exams, so slowly working on issues, its almost completed, there are few more errors i need to fix.

May 16 '24 18:05 RitikaxShakya

hey @RitikaxShakya, will you have a time to finish it?

Jul 25 '24 05:07 mlukasze

hey @RitikaxShakya, will you have a time to finish it?

Yes, i will try to finish it this week, sorry for being so late.

Jul 25 '24 07:07 RitikaxShakya

thanks! do not worry :)

Jul 25 '24 07:07 mlukasze

@RitikaxShakya please rebase your PR.

Aug 06 '24 13:08 alexsu52

@RitikaxShakya please rebase your PR.

Thank you for informing, and i am trying to complete this issue soon

Aug 06 '24 15:08 RitikaxShakya

@daniil-lyakhov @RitikaxShakya what is the status of this PR? Is it still active?

Sep 03 '24 06:09 MaximProshin

nncf nncf copied to clipboard

Mypy checks for NNCF common pruning code

Changes

Tests

Codecov Report

nncf
nncf copied to clipboard