quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Warn user when using local filesystem as default index root URI

Open PSeitz opened this issue 3 years ago • 1 comments

We could print a warning with a link to the docs outlining the limitations when using the filesystem as datadir or meta-store. It may ease onboarding new users.

PSeitz avatar Sep 21 '22 10:09 PSeitz

datadir on local fs is fine and is actually the recommended way to deploy Quickwit (SSDs).

What is not fine (in cluster mode), or at least risky if don't know what you're doing:

  • file-backed metastore (S3 is ok in read-only mode)
  • file-backed indexes
  • default index root URI on local file system

I've recently added some warning but adding a warning for the default index root URI on local file system and linking to the docs would be nice.

guilload avatar Sep 21 '22 19:09 guilload

@guilload Maybe we can add the check and warning logic after the call of load_node_config_with_env() https://github.com/quickwit-oss/quickwit/blob/main/quickwit/quickwit-config/src/node_config/mod.rs#L281

JerryKwan avatar Sep 11 '23 13:09 JerryKwan

@guilload Or add the check and warning logic around https://github.com/quickwit-oss/quickwit/blob/main/quickwit/quickwit-config/src/node_config/serialize.rs#L257

JerryKwan avatar Sep 12 '23 13:09 JerryKwan

Or add the logic in the validate function https://github.com/quickwit-oss/quickwit/blob/main/quickwit/quickwit-config/src/node_config/serialize.rs#L300

JerryKwan avatar Sep 14 '23 09:09 JerryKwan

And should add warning when validating the index config at https://github.com/quickwit-oss/quickwit/blob/main/quickwit/quickwit-config/src/index_config/serialize.rs#L88

JerryKwan avatar Sep 14 '23 09:09 JerryKwan

@guilload @PSeitz Please help to review the pull request https://github.com/quickwit-oss/quickwit/pull/3839 which added some warning about file-backed meta-store and index

JerryKwan avatar Sep 15 '23 05:09 JerryKwan