shard: Blobstor `file exists` errors move shard into degraded mode

[cthulhu-rider]

looks like a bug but needs to be investigated. When local storage is loaded by objects with existing files in blobstor, shard shards switches to degraded mode fast and stops storage service

warn        engine/engine.go:160        could not put object to shard        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error count": 1, "error": "could not put object to BLOB storage: file exists"}
warn        engine/engine.go:160        could not put object to shard        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error count": 2, "error": "could not put object to BLOB storage: file exists"}
warn        engine/engine.go:160        could not put object to shard        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error count": 3, "error": "could not put object to BLOB storage: file exists"}
warn        engine/engine.go:160        could not put object to shard        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error count": 4, "error": "could not put object to BLOB storage: file exists"}
...
info        shard/mode.go:28        setting shard mode        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "old_mode": "READ_WRITE", "new_mode": "DEGRADED_READ_ONLY"}
info        shard/mode.go:69        shard mode set successfully        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "mode": "DEGRADED_READ_ONLY"}
info        engine/engine.go:105        shard is moved in degraded mode due to error threshold        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error count": 30}
...
warn        engine/put.go:137        could not put object to shard        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error": "shard is in read-only mode"}
warn        engine/engine.go:142        could not select objects from shard        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error": "shard is in degraded mode"}
warn        engine/engine.go:142        can't perform `TreeGetOpLog`        {"shard_id": "ERHCRoaBixN42afWgnJrzA", "error": "pilorama is in a degraded mode"}

it seems pretty strange to me that attempts to rewrite files are considered as blunders and lead to the shard's degradation. Lets analyze can shard feel OK to object exists write exceptions and not degrade.

[cthulhu-rider]

afaik shard won't go back to read-write mode by itself