.with_buffers are really work?

[bwnjnOEI]

I use torch.nn.utils.prune to pruning model, and it generates a mask buffer. However, when I use make_functional_with_buffers, the error is RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment. Is it because of the use of deepcopy, or am I missing something?

My test code as follows:

import torch
import torch.nn as nn
from torch.nn.utils.prune import l1_unstructured
from functorch import make_functional_with_buffers

m = nn.Linear(100,100)
l1_unstructured(m, 'weight', amount=0.5)
print(m.weight_mask.is_leaf)
func, params, buffers = make_functional_with_buffers(m)

Pls, help!

[bwnjnOEI]

I remove-register_buffer like this

prune.remove(m, 'weight')
m.register_buffer('weight_mask', mask)

It worked, but I don't understand why?

[zou3519]

I'm not sure what is going on here, but we'll take a deeper look

[bwnjnOEI]

I have a basic question: When we use .make_functional_with_buffers or .make_functional to extract func and params, can we also associate and copy the execution of ._forward_pre_hooks in module?

[zou3519]

I'm not sure -- does _forward_pre_hooks get copied by deepcopy?

In general functorch doesn't work very well with module hooks at the moment