Cleanup ``performance_test`` and ``test_benchmarks``

[Transurgeon]

Description

Please include a short summary of the change. This PR deletes some unused tests that have coverage elsewhere in the codebase. Benchmarks have all been ported over to the benchmarks repo and are currently mostly being skipped. Issue link (if applicable):

Type of change

• [ ] New feature (backwards compatible)
• [ ] New feature (breaking API changes)
• [ ] Bug fix
• [x] Other (Documentation, CI, ...)

Contribution checklist

• [x] Add our license to new files.
• [x] Check that your code adheres to our coding style.
• [x] Write unittests.
• [x] Run the unittests and check that they’re passing.
• [x] Run the benchmarks to make sure your change doesn’t introduce a regression.