`cacheDir` not honored if `deps.optimizer.{mode}.enabled` is true

[Pistonight]

Describe the bug

cacheDir not honored if deps.optimizer.{mode}.enabled is true. vitest will save cache to node_modules/.vite as if cacheDir is not specified.

Reproduction

Run

npm create vite@latest
npm i

Edit the config

/// <reference types="vitest/config" />
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'

export default defineConfig({
  cacheDir: ".vite",
  plugins: [react()],
  test: {
    deps: {
      optimizer: {
        ssr: {
          enabled: true // toggling this to repro
        }
      }
    }
  }
})

Create a test src/App.test.tsx

import { describe, expect, test } from "vitest";
describe("App", () =>{
    test("works", () => {
        expect(1).not.toBe(2);
    });
});

Run

npx vitest

1. Observe cache is unexpectedly created in node_modules/.vite.
2. Toggle test.deps.optimizer.ssr.enabled to false, and run npx vitest again
3. Observe cache is created in .vite as expected

System Info

System:
    OS: Windows 11 10.0.26120
    CPU: (32) x64 AMD Ryzen 9 5950X 16-Core Processor
    Memory: 41.33 GB / 63.91 GB
  Binaries:
    Node: 18.20.4 - ~\dotbin\extra\portable\nvm\symlink\node.EXE
    npm: 10.7.0 - ~\dotbin\extra\portable\nvm\symlink\npm.CMD
    bun: 1.1.6 - ~\.bun\bin\bun.EXE
  Browsers:
    Edge: Chromium (127.0.2651.86)
    Internet Explorer: 11.0.26100.1
  npmPackages:
    @vitejs/plugin-react: ^4.3.2 => 4.3.2
    vite: ^5.4.8 => 5.4.9
    vitest: ^2.1.3 => 2.1.3

Used Package Manager

npm

Validations

• [X] Follow our Code of Conduct
• [X] Read the Contributing Guidelines.
• [X] Read the docs.
• [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• [X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• [X] The provided reproduction is a minimal reproducible example of the bug.