[leo_gateway] Caching enabled on leo_gateway may make GET ignore R consistency

[mocchira]

Right now caching makes gateway ignore R consistency setting as copy in cache is verified only against primary node. This behavior should be changed to verify also against other nodes if needed according to the R consistency setting.

[yosukehara]

We need to deliver a configuration of objects' consistency which are HEAD requests (from LeoGateway to LeoStorage) in case of enabling the cache function, which is to strictly check the consistency of an object or not because systems' performance are affect.

LeoFS adopts Primary Backup + Quorum Based Replication. An object is copied to the primary node then it is replicated to the secondary node. On the other hand regarding READ, LeoGateway also requests the primary node of an object. To sum up, we do not face this situation at regular cases.

In near future, after supporting the multi tenancy feature, we need to also delver its feature to each tenant which is to be able to separately set the configuration.