super-gradients Feature Extraction using YOLO-NAS

💡 Your Question

Is it possible to do feature extraction using YOLO-NAS. If yes what are the steps in doing that?

Versions

--2024-03-14 06:31:49-- https://raw.githubusercontent.com/pytorch/pytorch/main/torch/utils/collect_env.py Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 22068 (22K) [text/plain] Saving to: 'collect_env.py'

collect_env.py 100%[===================>] 21.55K --.-KB/s in 0.001s

2024-03-14 06:31:50 (14.6 MB/s) - 'collect_env.py' saved [22068/22068]

Collecting environment information... PyTorch version: 2.1.2 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64) GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 Clang version: Could not collect CMake version: version 3.16.3 Libc version: glibc-2.31

Python version: 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0] (64-bit runtime) Python platform: Linux-5.15.133+-x86_64-with-glibc2.31 Is CUDA available: True CUDA runtime version: 12.1.105 CUDA_MODULE_LOADING set to: LAZY GPU models and configuration: GPU 0: Tesla T4 GPU 1: Tesla T4

Nvidia driver version: 535.129.03 cuDNN version: Probably one of the following: /usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.0 HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True

CPU: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 46 bits physical, 48 bits virtual CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) CPU @ 2.00GHz Stepping: 3 CPU MHz: 2000.194 BogoMIPS: 4000.38 Hypervisor vendor: KVM Virtualization type: full L1d cache: 64 KiB L1i cache: 64 KiB L2 cache: 2 MiB L3 cache: 38.5 MiB NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Mitigation; PTE Inversion Vulnerability Mds: Mitigation; Clear CPU buffers; SMT Host state unknown Vulnerability Meltdown: Mitigation; PTI Vulnerability Mmio stale data: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown Vulnerability Retbleed: Mitigation; IBRS Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT Host state unknown Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat md_clear arch_capabilities

Versions of relevant libraries: [pip3] flake8==7.0.0 [pip3] msgpack-numpy==0.4.8 [pip3] mypy-extensions==1.0.0 [pip3] numpy==1.23.0 [pip3] onnx==1.13.0 [pip3] onnx-simplifier==0.4.36 [pip3] onnxruntime==1.13.1 [pip3] pytorch-ignite==0.4.13 [pip3] pytorch-lightning==2.2.0.post0 [pip3] torch==2.1.2 [pip3] torchaudio==2.1.2 [pip3] torchdata==0.7.1 [pip3] torchinfo==1.8.0 [pip3] torchmetrics==0.8.0 [pip3] torchtext==0.16.2 [pip3] torchvision==0.16.2 [conda] magma-cuda121 2.6.1 1 pytorch [conda] mkl 2023.1.0 h213fc3f_46344
[conda] msgpack-numpy 0.4.8 pypi_0 pypi [conda] numpy 1.23.0 pypi_0 pypi [conda] pytorch-ignite 0.4.13 pypi_0 pypi [conda] pytorch-lightning 2.2.0.post0 pypi_0 pypi [conda] torch 2.1.2 pypi_0 pypi [conda] torchaudio 2.1.2 pypi_0 pypi [conda] torchdata 0.7.1 pypi_0 pypi [conda] torchinfo 1.8.0 pypi_0 pypi [conda] torchmetrics 0.8.0 pypi_0 pypi [conda] torchtext 0.16.2 pypi_0 pypi [conda] torchvision 0.16.2 pypi_0 pypi

Mar 14 '24 06:03 LakshmySanthosh

Nope, it is not supported out of the box. You can however load the model and manually replace heads with identity layers to obtain fester maps from the neck output.

Mar 14 '24 06:03 BloodAxe

How to call or replace the layers manually?

Mar 14 '24 06:03 LakshmySanthosh

@LakshmySanthosh Here's an example:

from super_gradients.training import models
from super_gradients.common.object_names import Models
from torch.nn import Identity
import torch
model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")
model.heads = Identity()
model.eval()
x = torch.randn((5,3,640,640))
features = model(x)

You can additionally set the neck to Identity id you want to extract the features straight from the backbone.

Apr 15 '24 08:04 shaydeci