gudrb

transformations in MiniViT paper

9

Hello, I have a question about the transformations in the MiniViT paper. I could find the first transformation (implemented in the MiniAttention class) in the code: https://github.com/microsoft/Cream/blob/4a13c4091e78f9abd2160e7e01c02e48c1cf8fb9/MiniViT/Mini-DeiT/mini_vision_transformer.py#L104 However, i couldn't...

assert num_buckets == self.num_buckets error

6

I am trying to use the mini_deit_tiny_patch16_224 with finetuning another subtask having different sequence size of 18 (num of patches) with dimension 192. when operate under code for blk in...

CPU memory increase while training

6

During the training step, the code block: for blk in self.v.blocks: x = blk(x) causes an increase in CPU memory, eventually leading to full CPU memory. Do you know any...

Compatibility Issue with H100 GPU, CUDA 12.2, and PyTorch 2.1 - AttributeError: module 'rpe_index_cpp' has no attribute 'forward_gpu'

3

Hello, I recently upgraded my GPU to an H100 and updated CUDA to version 12.2. I'm using PyTorch version 2.1. After setting up the irpe.py file, the setup was successful....