duprule icon indicating copy to clipboard operation
duprule copied to clipboard

Allow to compare 2 files and remove duplicates

Open Shooter3k opened this issue 3 years ago • 1 comments

I was wondering if it would be possible to have duprule run between 2 files and remove the duplicates that are already contained within the target file.

So, something along the lines of: duprule new_rule.rule main_rules.rule > new_rules_not_in_main_rules.rule

new_rule.rule = source file main_rules.rule = file you want to compare new_rule.rule against new_rules_not_in_main_rules.rule = file that contains the records only found in new_rule.rule and not found in main_rules.rule

Although the syntax and 'how it does it' doesn't really matter that much as long as a person is able to remove rules that you already had in your 'main list'.

Shooter3k avatar Feb 04 '22 21:02 Shooter3k

Hmm would you be able to achieve this by:

1- cat main_rules.rule new_rule.rule > combined.rule the order of combining is relevant. 2- duprule < combined.rule > combined-uniq.rule: keep first precedence or rule. 3- remove rules that are in combined-uniq.rule & main_rules and thus you'd have rules that are in new rule but not in main_rules.

mhasbini avatar Feb 05 '22 08:02 mhasbini