setting xformers to false still tries doing stuff with xformers

[zethfoxster]

im on a 4090 and use sdp attention with cuda 12.1, xformers is limited to 11...I set xformers to false, as i assumed it would not not be used. however, due to not having it installed i recieve on error about xformers, which leads me through of string of fixes which involve downgrading to cuda 11, which i am not going to do. please, if you set xformers to false....don't assert on xformers.....its not to be used for a reason. thank you.