Benjamin Bossan

This sounds good, thanks. If you have some code already, don't hesitate to create an early draft PR, even without tests etc., so that I can give quick feedback.

Thanks for the update @yaswanth19. Indeed I think approach 1 will be difficult to make work. Regarding approach 2, could you give an example of adapter related keys that are...

Good point about VeRA. The point of VeRA is that the `vera_A` and `vera_B` layers are shared among all layers, so that they don't need to be stored for each...

@thusinh1969 are you also using LoRA/QLoRA or normal fine-tuning? @nivibilla Could you please show your train script, or at the very least how the base model and PEFT model are...

Thanks @nivibilla. I assume you're on the latest versions of the relevant libraries (PEFT, accelerate, transformers)? With your setting, I'm not sure if we'll get `fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP`, which I believe...

@nivibilla: Yes, I think it should be possible like that. @wizeng23 Thanks for your detailed report. Based on that, I ran my own experiments. What I found: When using transformers...

Note that in #1078, we changed skorch's net classes to inherit from `BaseEstimator` (there is no release yet, so installing from source is required). From my understanding, this should be...

> `net.set_fit_request(Z=True)` > `# fails, and is ultimately what I need to do: scale X, not Z, pass Z to neural net.` I could reproduce, `set_fit_request` does not exist. I...

Thanks for the pointer. So basically, we would just have to copy more or less the same method in skorch and we're good? Probably this is the relevant part: ```python...

Okay, so I tested this: ```python # net.py: add to NeuralNet def set_fit_request(self, **kwargs): from sklearn.utils.metadata_routing import MetadataRequest self._metadata_request = MetadataRequest(owner=self.__class__.__name__) for param, alias in kwargs.items(): self._metadata_request.fit.add_request(param=param, alias=alias) return self...