service.log_level does not accept lowercased log levels

[ppf2]

On 8.7.0.

elasticsearch.log_level setting accepts both uppercase and lowercase log levels, e.g. DEBUG and debug.

However, when using lowercase debug log level for service.log_level, it fails:

Traceback (most recent call last):
  File "<path>/connectors-python/bin/elastic-ingest", line 33, in <module>
    sys.exit(load_entry_point('elasticsearch-connectors', 'console_scripts', 'elastic-ingest')())
  File "<path>/connectors-python/connectors/cli.py", line 185, in main
    return run(args)
  File "<path>/connectors-python/connectors/cli.py", line 144, in run
    set_logger(
  File "<path>/connectors-python/connectors/logger.py", line 54, in set_logger
    logger.setLevel(log_level)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/logging/__init__.py", line 1452, in setLevel
    self.level = _checkLevel(level)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/logging/__init__.py", line 198, in _checkLevel
    raise ValueError("Unknown level: %r" % level)
ValueError: Unknown level: 'debug'
make: *** [run] Error 1

DEBUG works. It will be nice to make these log_level settings consistent to also allow lowercased log levels.

[tarekziade]

What is your goal @ppf2 ? Python's log levels are upper case, it's the standard. if the goal is to have a unified lower case in the cloud dashboard, we could lower() the level in the filebeat output

[ppf2]

@tarekziade The goal is to have a more unified experience in the dashboard as well as the connector service configuration so lower() in the filebeat output can certainly help with the visualization piece.

On the configuration side, we default to use lowercasing for the ES log_level, and upper casing for the second one. Perhaps we could go with uppercasing for both of them as the default, and then mention the list of accepted values (["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]) for log_level in both places in the documentation?

  backoff_multiplier: 2
  log_level: info

service:
  idling: 30
  heartbeat: 300
  max_errors: 20
  max_errors_span: 600
  max_concurrent_syncs: 1
  job_cleanup_interval: 300
  log_level: INFO