lfs.lock_dir really a good idea?

[rrthomas]

Locking is a complicated subject, and lfs.lock_dir implements its own mechanism. It's about 100 lines of C, and it's far from clear that it's useful or robust. It is hard to tell whether the behaviour is the same on Window and non-Windows system (it doesn't look like it). There are no tests for it. Something similar could easily be implemented using lfs.lock in a few lines of Lua. I suggest it be removed, or, failing that, tests be added. Contrary to the documentation, there seems to be no second parameter to check for staleness either on Windows or non-Windows systems (if a second parameter is passed, it is ignored).

In summary, this has the air of application-specific code that is not ready to be put in a library.

[mascarenhas]

lfs.lock does not work with the usual Lua threading patterns: coroutine schedulers like Copas, or threading + multiple states packages like Lanes. Because the different Lua "threads" are in the same OS process lfs.lock always succeeds, so does not lock anything.

lfs.lock_dir uses atomic filesystem operations that guarantee that it will always work correctly even in those cases. They do have the same behavior on Windows and non-Windows systems, the differences in the code are by design, because the atomic operations are different in each system. Windows also automatically disposes of the lock file when closing the file handle, while other systems need to explicitly unlink the lockfile after we are done with it.

Ignoring the staleness parameter is a bug... I will amend the documentation for now, and will implement this correctly later.

You are right about the tests, of course, but there were no tests for lfs.lock either... this simple test would show that it useless in the single-process case:

f = io.open("foo", "w+") lock1, err = lfs.lock(f, "w") -- exclusive lock! assert(lock1) lock2, err = lfs.lock(f, "w") assert(not lock2) -- oops!

[rrthomas]

Your comments are reassuring. It would help to have the explanation about single-process usage in the manual. Also, it would help to explain (or at least assert) that there is no simple way to do this with any of the common system locking primitives.

On non-Windows systems, how about instead binding liblockfile, which solves precisely this problem, and is battle-tested (e.g. it is installed on all Debian/Ubuntu systems)? It has a straightforward API, and a very carefully thought-through algorithm. (I had a quick look but I don't see an existing Lua binding for this library.) It also includes an important parameter which lock_dir lacks, namely, the time to spend trying to obtain the lock before giving up. Its man page also discusses problems with deciding whether a lockfile is stale.

There is also the Python lockfile module:

http://code.google.com/p/pylockfile/

which provides cross-platform code.

The fact that these implementations are in their own libraries again suggests this is a good task for a specialist package, and not LFS. The interesting thing about the Python library is that it doesn't contain any C. It also does include tests, and is for per-thread locks. If the guess in the documentation is correct and mkdir locking is sufficient for UNIX, then the job is small: translate 162 lines of Python to Lua. To translate both mkdir locking and link locking (links being the currently-preferred way on UNIX) make it up to 220 lines in total. This is longer than the current implementation in LFS, but simply looking at its well-commented code with copious documentation gives one rather more confidence, plus it requires no C. What's not to like?