Consolidate `torch.backends.cuda.matmul.allow_tf32 = False` in tests

[williamberman]

The module with PipelineTesterMixin already sets this. We shouldn't repeatedly set this in all tests and instead consolidate where we set it