`--no-cache` still using cache

[huafu]

🐛 Bug Report

After trying to debug kulshekhar/ts-jest#603, @GeeWee (who was sure it was a cache issue) suggested me to clear jest cache directory instead of just using --no-cache.

To Reproduce

Unfortunately the issue is not really reproducible since it's related to wrong data which shouldn't be kept in cache. Tho, downgrading jest from latest (23.2.0 at the time of writing this) to < 23 did fix my issue (any 23.x.x would fail).

Expected behavior

I was expecting --no-cache option to make Jest act as if the cache directory has been removed.

Link to repl or repo (highly encouraged)

nope :-/

Run npx envinfo --preset jest

Paste the results here:

  System:
    OS: macOS High Sierra 10.13.5
    CPU: x64 Intel(R) Core(TM) i5-4288U CPU @ 2.60GHz
  Binaries:
    Node: 10.4.0 - ~/.nvm/versions/node/v10.4.0/bin/node
    Yarn: 1.7.0 - /usr/local/bin/yarn
    npm: 6.1.0 - ~/.nvm/versions/node/v10.4.0/bin/npm
  npmPackages:
    jest: ^23.2.0 => 23.2.0

[thymikee]

Interesting, I don't recall we change something in this part of codebase recently. However it's true that --no-cache will still write the cache but shouldn't use it.

[huafu]

The fact that regressing to <23 did fix my issue might only be valid in my own case, because for example some cached data related to my issue in v23 could have been in another subfolder of the cache in v22 or so.

@thymikee for more precisions, I am unsure this issue is related to/started with v23. What I'm pretty sure tho is that the cache (or part of it) seems to be read/used with --no-cache while it shouldn't.

[StephanBijzitter]

Also suffering from this:

jest: failed to cache transform results in: /tmp/jest_rt/jest-transform-cache-36552e3bd32f852336b6be9fcd10c72c-d0e7c439bf6eb5a4bf3a6371fdd259d1/78/eventtarget_78075c08a7fe48775451e946a8e8469b.map
Failure message: ENOMEM: not enough memory, write

That's a message from my CI which is running a massive testing suite. I'm using Jest 24.0.23 with --no-cache, but the writing of the cache (even if not reading) is still problematic.

Weird thing though, it doesn't always happen, so sometimes it appears to have enough space. But before every testrun, --clearCache is used.

[sibelius]

any workaround for this?

[JasonGore]

This still appears to be an issue with v27.0.6. I'm passing in --no-cache and the .jest-cache directory is still created and populated. Other args like --showConfig and --clearCache are working as expected, so --no-cache appears to have no effect.

[SimenB]

As pointed out in https://github.com/facebook/jest/issues/6610#issuecomment-402767524, --no-cache doesn't actually mean "don't ever use a cache", it just means "clear out the cache before running tests" - the cache will then be populated by tests during the run. It's to ensure there's no stale caches, not to disabling caching entirely.

(I realise the name of the flag is misleading, but it is what it is 😅)

[JasonGore]

We have certain tests that fail only when the cache is first populated, which seems to indicate that initially populating the cache is actually hurting test perf. We are also in an environment where we don't need the cache: the environment is completely clean on every run. Is there any way to disable creating the cache entirely?

[SimenB]

Is there any way to disable creating the cache entirely?

Not at the moment

[pauldraper]

(I realise the name of the flag is misleading, but it is what it is sweat_smile)

But ideally, the flag would actual disable caching?

No point in writing to a cache (and causing OOM/disk errors etc) if you're not even using the thing, right?

[dmichon-msft]

One reason I was hoping to leverage this flag is that I have a custom transformer that is implemented by literally reading a different file directly from the file system, and having the cache enabled incurs a write and reads that serve no useful purpose whatsoever in that scenario.

[dmichon-msft]

@SimenB , would you accept a PR to make --no-cache actually disable writing to the cache?

[SalvatorePreviti]

The ability to not have cache at all (writing and reading) does make sense especially if you are using @swc/jest like I am, compilation is so fast that actually the cache is slower!

[Havunen]

I have been trying to optimize our test run it takes more than a minute and is becoming annoying. Writing to cache is one of the slow downs.

[SalvatorePreviti]

I have been trying to optimize our test run it takes more than a minute and is becoming annoying. Writing to cache is one of the slow downs.

Especially in CI that has usually slower IO than local and where cache is lost at every new run...

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[sibelius]

can we disable all cache usage?

[pauldraper]

can we disable all cache usage?

---

Is there any way to disable creating the cache entirely?

Not at the moment

[Gu7z]

Does this issue has any workarounds? Or is it planned to be fixed? The same spec copied running with --no-cache seems to be caching Or at least there is something eating time at the first one

The spec

describe("testing", () => {
  it("slow spec", () => {
    expect(1).toBe(1);
  });
});

[rmathewsbeyond]

Any new updates or workarounds for this issue?

[frandroid]

I had this problem and it turns out that I had duplicated test files in my __tests__ folder, so the issue was not that the cache was not emptying, but that I still had a copy of my broken test.

[joewolschon]

Any new updates or workarounds for this issue?

[wratho]

We are experiencing this as well, curious if anyone found any solutions?

[Fa7]

I wrote 5 patches that address this issue so based on the config.cache set in the jest.config you can tell the transformer to not write to the cache dir.

I hope this works for anyone experiencing this issue.n

Jest 29.7.0

--- a/node_modules/@jest/core/build/cli/index.js
+++ b/node_modules/@jest/core/build/cli/index.js
@@ -247,7 +247,7 @@ const buildContextsAndHasteMaps = async (
   const hasteMapInstances = Array(configs.length);
   const contexts = await Promise.all(
     configs.map(async (config, index) => {
-      (0, _jestUtil().createDirectory)(config.cacheDirectory);
+      config.cache && (0, _jestUtil().createDirectory)(config.cacheDirectory);
       const hasteMapInstance = await _jestRuntime().default.createHasteMap(
         config,
         {

--- a/node_modules/@jest/test-sequencer/build/index.js
+++ b/node_modules/@jest/test-sequencer/build/index.js
@@ -271,8 +271,10 @@ class TestSequencer {
         ];
       }
     });
-    this._cache.forEach((cache, context) =>
-      fs().writeFileSync(this._getCachePath(context), JSON.stringify(cache))
+    
+    this._cache.forEach((cache, context) => {
+      context?.config?.cache && fs().writeFileSync(this._getCachePath(context), JSON.stringify(cache))
+    }
     );
   }
   hasFailed(test) {

--- a/node_modules/@jest/transform/build/ScriptTransformer.js
+++ b/node_modules/@jest/transform/build/ScriptTransformer.js
@@ -507,11 +507,11 @@ class ScriptTransformer {
         sourceMapPath,
         'We should always have default sourceMapPath'
       );
-      writeCacheFile(sourceMapPath, sourceMapContent);
+      this._config.cache && writeCacheFile(sourceMapPath, sourceMapContent);
     } else {
       sourceMapPath = null;
     }
-    writeCodeCacheFile(cacheFilePath, code);
+    this._config.cache && writeCodeCacheFile(cacheFilePath, code);
     return {
       code,
       originalCode: content,
@@ -550,7 +550,8 @@ class ScriptTransformer {
         transformerConfig
       });
     }
-    (0, _jestUtil().createDirectory)(path().dirname(cacheFilePath));
+
+    this._config.cache && (0, _jestUtil().createDirectory)(path().dirname(cacheFilePath));
     return this._buildTransformResult(
       filename,
       cacheFilePath,

--- a/node_modules/jest-haste-map/build/index.js
+++ b/node_modules/jest-haste-map/build/index.js
@@ -711,10 +711,12 @@ class HasteMap extends _events().EventEmitter {
    * 4. serialize the new `HasteMap` in a cache file.
    */
   _persist(hasteMap) {
-    (0, _gracefulFs().writeFileSync)(
-      this._cachePath,
-      (0, _v().serialize)(hasteMap)
-    );
+    if (this._options.cache) {
+      (0, _gracefulFs().writeFileSync)(
+        this._cachePath,
+        (0, _v().serialize)(hasteMap)
+      );
+    }
   }
 
   /**

jest-environment-enzyme 7.1.2

--- a/node_modules/jest-environment-enzyme/node_modules/@jest/transform/build/ScriptTransformer.js
+++ b/node_modules/jest-environment-enzyme/node_modules/@jest/transform/build/ScriptTransformer.js
@@ -305,7 +305,7 @@ class ScriptTransformer {
     const cachePath = (0, _slash().default)(
       _path().default.join(cacheDir, cacheFilenamePrefix + '_' + cacheKey)
     );
-    (0, _jestUtil().createDirectory)(cacheDir);
+    this._config.cache && (0, _jestUtil().createDirectory)(cacheDir);
     return cachePath;
   }
 
@@ -489,12 +489,12 @@ class ScriptTransformer {
         typeof transformed.map === 'string'
           ? transformed.map
           : JSON.stringify(transformed.map);
-      writeCacheFile(sourceMapPath, sourceMapContent);
+      this._config.cache && writeCacheFile(sourceMapPath, sourceMapContent);
     } else {
       sourceMapPath = null;
     }
 
-    writeCodeCacheFile(cacheFilePath, code);
+    this._config.cache && writeCodeCacheFile(cacheFilePath, code);
     return {
       code,
       mapCoverage,

[Fa7]

I wrote 5 patches that address this issue so based on the config.cache set in the jest.config you can tell the transformer to not write to the cache dir.

I hope this works for anyone experiencing this issue.n

Jest 29.7.0

--- a/node_modules/@jest/core/build/cli/index.js
+++ b/node_modules/@jest/core/build/cli/index.js
@@ -247,7 +247,7 @@ const buildContextsAndHasteMaps = async (
   const hasteMapInstances = Array(configs.length);
   const contexts = await Promise.all(
     configs.map(async (config, index) => {
-      (0, _jestUtil().createDirectory)(config.cacheDirectory);
+      config.cache && (0, _jestUtil().createDirectory)(config.cacheDirectory);
       const hasteMapInstance = await _jestRuntime().default.createHasteMap(
         config,
         {

--- a/node_modules/@jest/test-sequencer/build/index.js
+++ b/node_modules/@jest/test-sequencer/build/index.js
@@ -271,8 +271,10 @@ class TestSequencer {
         ];
       }
     });
-    this._cache.forEach((cache, context) =>
-      fs().writeFileSync(this._getCachePath(context), JSON.stringify(cache))
+    
+    this._cache.forEach((cache, context) => {
+      context?.config?.cache && fs().writeFileSync(this._getCachePath(context), JSON.stringify(cache))
+    }
     );
   }
   hasFailed(test) {

--- a/node_modules/@jest/transform/build/ScriptTransformer.js
+++ b/node_modules/@jest/transform/build/ScriptTransformer.js
@@ -507,11 +507,11 @@ class ScriptTransformer {
         sourceMapPath,
         'We should always have default sourceMapPath'
       );
-      writeCacheFile(sourceMapPath, sourceMapContent);
+      this._config.cache && writeCacheFile(sourceMapPath, sourceMapContent);
     } else {
       sourceMapPath = null;
     }
-    writeCodeCacheFile(cacheFilePath, code);
+    this._config.cache && writeCodeCacheFile(cacheFilePath, code);
     return {
       code,
       originalCode: content,
@@ -550,7 +550,8 @@ class ScriptTransformer {
         transformerConfig
       });
     }
-    (0, _jestUtil().createDirectory)(path().dirname(cacheFilePath));
+
+    this._config.cache && (0, _jestUtil().createDirectory)(path().dirname(cacheFilePath));
     return this._buildTransformResult(
       filename,
       cacheFilePath,

--- a/node_modules/jest-haste-map/build/index.js
+++ b/node_modules/jest-haste-map/build/index.js
@@ -711,10 +711,12 @@ class HasteMap extends _events().EventEmitter {
    * 4. serialize the new `HasteMap` in a cache file.
    */
   _persist(hasteMap) {
-    (0, _gracefulFs().writeFileSync)(
-      this._cachePath,
-      (0, _v().serialize)(hasteMap)
-    );
+    if (this._options.cache) {
+      (0, _gracefulFs().writeFileSync)(
+        this._cachePath,
+        (0, _v().serialize)(hasteMap)
+      );
+    }
   }
 
   /**

jest-environment-enzyme 7.1.2

--- a/node_modules/jest-environment-enzyme/node_modules/@jest/transform/build/ScriptTransformer.js
+++ b/node_modules/jest-environment-enzyme/node_modules/@jest/transform/build/ScriptTransformer.js
@@ -305,7 +305,7 @@ class ScriptTransformer {
     const cachePath = (0, _slash().default)(
       _path().default.join(cacheDir, cacheFilenamePrefix + '_' + cacheKey)
     );
-    (0, _jestUtil().createDirectory)(cacheDir);
+    this._config.cache && (0, _jestUtil().createDirectory)(cacheDir);
     return cachePath;
   }
 
@@ -489,12 +489,12 @@ class ScriptTransformer {
         typeof transformed.map === 'string'
           ? transformed.map
           : JSON.stringify(transformed.map);
-      writeCacheFile(sourceMapPath, sourceMapContent);
+      this._config.cache && writeCacheFile(sourceMapPath, sourceMapContent);
     } else {
       sourceMapPath = null;
     }
 
-    writeCodeCacheFile(cacheFilePath, code);
+    this._config.cache && writeCodeCacheFile(cacheFilePath, code);
     return {
       code,
       mapCoverage,

One thing to note is this works and stops IO operations but UT are extremely slow due to transform caching no longer happens so each test has to re build each file transformed. So it seems this flow of caching files is a hard requirement for massive suites of tests that have an excessive amount of external deps.