Custom layer filters that support regular expressions

[tehybel]

Old-style literal filters are still supported, since they are valid regular expressions, too.

[dxqb]

• printing to the console which layers were selected (and which not) would be helpful imho
• when this is done I might want to use it for a PR to filter layers in full finetuning. Finetuning some layers such as the embedders seems counterproductive

[O-J1]

Given Thomas has been MIA for awhile now I have gone and refactored it.

@dxqbYD Added logging of the selected/unselected layers/

[dxqb]

Here https://github.com/Nerogar/OneTrainer/blob/d472c65a66fa71e5e05babc3c02202882cfc079e/modules/modelSetup/ChromaFineTuneSetup.py#L49 is now code that applies a layer filter to full tinetuning - although technically requires_grad should also set to False for these parameters

[dxqb]

one possible solution for the some of the complications above: do we really need regex? why not a simpler substring matching, and an "ignore pattern" that allows to exclude certain layers

[Calamdor]

I think that would allow filtering of “single” for the flux-ish models (flux, Hunyuan, i assume Chroma) to only grab a double block(s).

[dxqb]

I think that would allow filtering of “single” for the flux-ish models (flux, Hunyuan, i assume Chroma) to only grab a double block(s).

yes. any other use cases that would really require regex?

[O-J1]

I think that would allow filtering of “single” for the flux-ish models (flux, Hunyuan, i assume Chroma) to only grab a double block(s).

yes. any other use cases that would really require regex?

I think that would allow filtering of “single” for the flux-ish models (flux, Hunyuan, i assume Chroma) to only grab a double block(s).

yes. any other use cases that would really require regex?

Anyone wanting to to select multiple non-contiguous blocks or complex layer selection. Comma delimited lists still work fine and that was the intention of this

[dxqb]

yes. any other use cases that would really require regex?

Anyone wanting to to select multiple non-contiguous blocks or complex layer selection. Comma delimited lists still work fine and that was the intention of this

okay, if we want to support regex let's do that for expert users, but following the same sentiment as your screenshot quote I'd like a way to reject a pattern for non-expert users. I'd consider myself among those non-expert users, considering how difficult it is to reject something using regex:

^(?:(?!guidance_layer|context_embedder|x_embed).)*$

What do you think about:

• keep the current lora_layers (but rename it to layers_accept either in this PR or in a later one, because we also need it for FFT)
• add a [lora_]layers_reject for simple substring pattern matching to reject layers
• both can be used as regex if you switch it on

This solves the complexity, but also this https://github.com/Nerogar/OneTrainer/pull/778#discussion_r2293132065 because we don't need regex presets anymore.

[O-J1]

This solves the complexity, ... don't need regex presets anymore.

Sumarising convo on Discord:

1. Regex presets are already supported
2. We agreed substring matching would be better as the default, to be contributed in a seperate PR (new UI field)

I believe all review comments have been addressed now or are relevant to a new PR