luigi icon indicating copy to clipboard operation
luigi copied to clipboard

Silent UnconsumedParameterWarning for core.autoload_range

Open starhel opened this issue 2 years ago • 3 comments

Description

autoload_range set in luigi.cfg emits UnconsumedParameterWarning as value is used only in luigi.__init__. I've added internal field _ignore_unconsumed to ignore this parameter in core class.

Motivation and Context

My motivation in to silent all warnings emitted by luigi in my developed code as it's add some noise in tests.

Have you tested this? If so, how?

I've run simple graph with this change to ensure that's warning is no longer emitted.

starhel avatar Jun 10 '22 18:06 starhel

Potentially the same is required for the no_configure_logging option.

poofeg avatar Jun 21 '22 13:06 poofeg

Where do you feel would be a good place to document this functionality? Are there other core-related parameters that should also have warnings suppressed?

dlstadther avatar Jun 23 '22 01:06 dlstadther

There are probably more of them:

  • core.hdfs_tmp_dir (deprecated)
  • core.history_filename (probably shouldn't be in core)
  • core.max_shown_tasks (it's mentioned in docs in core, but it's in scheduler config on master)
  • core.max_graph_nodes (same)
  • core.rpc_connect_timeout (used in RemoteScheduler class, should be rewritten to use core class)
  • core.rpc_retry_attempts (same)
  • core.rpc_retry_wait (same)
  • core.no_configure_logging (should be rewritten to use core class)

This is only core analysis, so there are probably some other config classes with options defined only in the config file. Unfortunately, for now, I have no time to prepare a proper refactor, so I can only add more options to the ignored list.

starhel avatar Jun 23 '22 17:06 starhel