Concurrent shared CacheDir writes cause problems

[bdbaddog]

This issue was originally created at: 2010-08-30 02:41:27. This issue was reported by: tymek.

tymek said at 2010-08-30 02:41:27

There is problem with sharing cache:

if at the same time two users compile and try to save the same file, the second user will have interrupted compilation (permission denied for saving file)

Maybe it's good idea to:

• change permissions of stored files (-rw-r--r-- => -rw-rw-rw-)
• ignore problems caused by IOError/OSError while saving cache

gregnoel said at 2010-09-27 17:41:19

Bug party triage.

tymek said at 2010-10-01 04:21:50

I saw that any IOError while retrieving cache are ignored in scons 2.0.1

tymek said at 2010-10-01 04:25:30

What if file in cache becomes inaccessible for some reason?

klimkin said at 2010-11-30 10:24:21

I have attached a modified version of 'CacheDir', which has fixes for problems with sharing. It also adds an additional level of caching. Files pushed only into local cache. Files checked in local cache first, then in global cache. This helps to keep cache intact on regression disk.

To use, add in site_init.py:

import SCacheDir
SCons.CacheDir = SCacheDir

in SConstruct:

CacheDir((local_cache_dir, global_cache_dir))

[mwichmann]

no attachment?

[bdbaddog]

I think this falls into the need a cachedir server to really fix the issue category.. Likely a dup or a sibling of other issues.

[mwichmann]

having the storage handled by a server would certainly help aribtrate conflicts between users ... possibly at the cost of a bit more slowdown. Haven't heard much about this work recently - wasn't someone looking at using the bazel remote-cache "server" stuff?

[bdbaddog]

having the storage handled by a server would certainly help aribtrate conflicts between users ... possibly at the cost of a bit more slowdown. Haven't heard much about this work recently - wasn't someone looking at using the bazel remote-cache "server" stuff?

Yes. One of the community members. I think maybe someone who works at VMWare?

[mwichmann]

Updated the summary, issue is talking about problem during writes