Tiledb using NFS

[CMCDragonkai]

What are the implications of using TileDB on NFS? What would happen if 2 tiledb instances were using the same underlying directory?

[stavrospapadopoulos]

Here is what could happen:

• If the two tiledb processes are both reading, both writing, or one reading and one writing, then there is no issue (no locking is used). However, please note that TileDB follows the eventual consistency model.
• If one tiledb process is performing consolidation while the other is reading, then there may be issues. This is because tiledb uses filelocks to exclusively lock the array in one (brief) stage of consolidation, namely when the old fragments are deleted. NFS does not support filelocking. If the other process is reading the fragments being deleted, issues will probably occur.

[Hoeze]

@stavrospapadopoulos this is not correct:

• If one tiledb process is performing consolidation while the other is reading, then there may be issues. This is because tiledb uses filelocks to exclusively lock the array in one (brief) stage of consolidation, namely when the old fragments are deleted. NFS does not support filelocking. If the other process is reading the fragments being deleted, issues will probably occur.

NFS does support file locking, only flock is not supported. fcntl should work: http://nfs.sourceforge.net/ https://linux.die.net/man/2/flock

[stavrospapadopoulos]

@Hoeze thanks for the clarification. We are using fcntl so the file locking may work properly on NFS. In that case, there would not be any issue with running two tiledb processes simultaneously. We have not experimented with NFS yet, but if any issue comes up we will be happy to investigate.

[Hoeze]

fyi, we very successfully run TileDB on both NFS and CephFS :)

[stavrospapadopoulos]

Thanks @Hoeze. @CMCDragonkai is it ok to close this issue?