NewMatchPathValueComboDetector: Error in persistence when missing values occur

[landauermax]

Running the NewMatchPathValueComboDetector with allow_missing_values set to True and then having both normal values and missing values (None) in the data results in an error when persisting:

Traceback (most recent call last):
  File "/usr/bin/aminer", line 794, in <module>
    main()
  File "/usr/bin/aminer", line 479, in main
    run_analysis_child(aminer_config, program_name)
  File "/usr/bin/aminer", line 89, in run_analysis_child
    child_return_status = child.run_analysis(3)
  File "/usr/lib/logdata-anomaly-miner/aminer/AnalysisChild.py", line 459, in run_analysis
    PersistenceUtil.persist_all()
  File "/usr/lib/logdata-anomaly-miner/aminer/util/PersistenceUtil.py", line 86, in persist_all
    component.do_persist()
  File "/usr/lib/logdata-anomaly-miner/aminer/analysis/NewMatchPathValueComboDetector.py", line 159, in do_persist
    PersistenceUtil.store_json(self.persistence_file_name, sorted(list(self.known_values_set)))
TypeError: '<' not supported between instances of 'NoneType' and 'bytes'

I assume this is because both None and other values are in the known_values_set, which cannot be sorted. Sorting is not necessary since this is a set, not a list.

[ernstleierzopf]

Running the NewMatchPathValueComboDetector with allow_missing_values set to True and then having both normal values and missing values (None) in the data results in an error when persisting:

Traceback (most recent call last):
  File "/usr/bin/aminer", line 794, in <module>
    main()
  File "/usr/bin/aminer", line 479, in main
    run_analysis_child(aminer_config, program_name)
  File "/usr/bin/aminer", line 89, in run_analysis_child
    child_return_status = child.run_analysis(3)
  File "/usr/lib/logdata-anomaly-miner/aminer/AnalysisChild.py", line 459, in run_analysis
    PersistenceUtil.persist_all()
  File "/usr/lib/logdata-anomaly-miner/aminer/util/PersistenceUtil.py", line 86, in persist_all
    component.do_persist()
  File "/usr/lib/logdata-anomaly-miner/aminer/analysis/NewMatchPathValueComboDetector.py", line 159, in do_persist
    PersistenceUtil.store_json(self.persistence_file_name, sorted(list(self.known_values_set)))
TypeError: '<' not supported between instances of 'NoneType' and 'bytes'

I assume this is because both None and other values are in the known_values_set, which cannot be sorted. Sorting is not necessary since this is a set, not a list.

Sorting of sets is not necessary, but it also is no problem, because sets are expected to be unsorted. Sorting these sets before persisting gives more reproducable results, which helps with testing. I think the performance overhead is not too big of a deal.