Finalize Example-Level Shuffling (for non-GDT EBFs) Implementation and Update Test Cases 3/n

[yeyingxiao]

Summary: This diff completes the implementation of example-level shuffling for non-GDT format EBF and updates feature comparison and test utilities to support the new data structure. Key changes include:

Core Implementation

• Finalize group_events_by_example and flatten_events in sigrid.py
  • These methods now fully support grouping and flattening features by example, handling lists of tensors as required.

Test Utilities

• Update find_perturbed_features in sigrid.py
  • The function now supports comparison of both tensors and lists of tensors, ensuring correct detection of perturbed features regardless of feature structure.
• Add assertTupleOfListOfTensorsAlmostEqual in basic.py
  • Enhanced to assert equality for tuples of lists of tensors, supporting the new output format from example-level shuffling.
• Add extracted_features_equal in basic.py
  • Now recursively checks equality for tensors, lists of tensors, and tuples containing either, ensuring robust feature comparison.

Test Cases

• Update Test Cases in test_feature_transform.py

  • When test_ebf_example_shuffling is enabled, uses assertTupleOfListOfTensorsAlmostEqual for feature comparison.
  • Otherwise, falls back to assertTensorTuplesAlmostEqual.

• Update/Expand Test Cases in feature_transform_tests.py

  • Test cases for EBF example-level shuffling are updated to reflect the finalized grouping/flattening behavior.

Summary

This diff ensures that the EBF example-level shuffling logic is fully implemented and that all relevant comparison and test utilities are compatible with the new extracted feature structure. All related test cases are updated to validate the new behavior, providing a robust foundation for future development and maintenance.

Differential Revision: D87590987

[meta-codesync[bot]]

@yeyingxiao has exported this pull request. If you are a Meta employee, you can view the originating Diff in D87590987.

[meta-codesync[bot]]

This pull request has been merged in meta-pytorch/captum@0e759814749592147943bfd900f25b992208fd3b.