Fix safety checker for some models

[wbruna]

The first changeset fixes safety checker behavior for models which do not incorporate the safety model internally (safetensors and some diffusers models).

The other two are somewhat-related minor memory optimizations.

Unfortunately, I couldn't make the check work on OpenVINO at all, regardless of this change (but I can't test it very well, it's too memory-hungry for my PC). If the filter is enabled on OpenVINO, and not working, this change may cause a crash instead; but it shouldn't interfere on a correctly working checker.