pyjuliapkg Possible concurrency issues with resolve on a cluster

I think I am seeing multiple processes trying to simultaneously resolve julia versions and dependencies, mirroring https://github.com/JuliaPy/CondaPkg.jl/issues/40. Maybe locking is needed here too?

Oct 30 '23 23:10 amilsted

Yeah makes sense.

Oct 31 '23 10:10 cjdoris

Oh, by the way, I wonder if "pidfile" locking makes sense on a cluster? I think it probably doesn't, since there is no global list of processes.

Oct 31 '23 16:10 amilsted

I just took a look at the source:

https://github.com/vtjnash/Pidfile.jl/blob/67c3f23b25d4e43f1f5785ce77c2cac6952fefcf/src/Pidfile.jl#L128

Pidfile.jl uses both the PID and hostname, which presumably means it works with shared filesystems too.

Nov 02 '23 09:11 cjdoris

Ah, that's smart! Much better than https://github.com/mosquito/python-pidfile/blob/master/pidfile/pidfile.py

https://github.com/keiranmraine/pidlock sees to use hostname too.

Nov 02 '23 16:11 amilsted

Oh yeah of course this is a python package so need a python implementation of pidfiles.

Nov 02 '23 16:11 cjdoris

pidlock looks good thanks

Nov 02 '23 16:11 cjdoris

Btw, is it really necessary to write out the meta file every time? Perhaps we could check for changes first?

Nov 02 '23 22:11 amilsted