SJShin-AI

I tried hard to construct the minimal example. Sorry for providing the still lengthy code, which is also difficult to understand the learning framework. My task is dataset condensation, which...

I think that this implementation should provide a lot of utilities on per-sample gradient computation. One of famous utilization is "https://arxiv.org/abs/2109.02934", which updates the model parameter based on gradient variance...

Hi, i checked your solution and it slightly reduces the memory leak. However, the memory increases in very small increments as the iteration goes on. It seems like some parts...

Hi, first, thanks for your generous reply on my question. I really appreciate your sincere reply with efforts. As you referred, the cause of the memory leakage problem seems to...

Hi. Thanks for your kind suggestion. Actually i already have tried that. The choice leads to the following error below: ``` File "condense_example.py", line 102, in loss.backward() File "/usr/local/lib/python3.8/dist-packages/torch/_tensor.py", line...

I think the backpack is the only way to realistically calculate the gradient variance of each model parameter through a quick computation of the per-sample gradient. However, the problem from...

Thanks. I appreciate your suggestion. Did you try any kind of time comparison between backpack and functorch? I have heard that functorch is still expensive to compute per-sample gradient, which...

Thanks for your response. Then, is the provided baselines are all from the author's code? or not? I think that the strength of the claim depends upon the reproducibility of...

During the check, i just found out that some loss functions are wrongly calculated as follows: For instance, in the Grad-Match method code, ```loss = self.criterion(torch.nn.functional.softmax(outputs, dim=1), targets.to(self.args.device)).sum()``` the self.criterion...