pylint ``duplicate-code`` are always counted on the last module checked

Steps to reproduce

Have pylint check multiple modules in a single invocation. There should be duplicate lines among these modules.

pylint -ry *.py

Current behavior

The R0801 messages will be issued on the last module checked, regardless of which modules contained the duplicate lines:

xmlgen.py:1:0: R0801: Similar lines in 2 files
==databaselib:11
==projectlib:18
import logging
import os
import os.path
import re
import time (duplicate-code)

This distorts the statistics, in particular the % errors / warnings by module report. The xmlgen.py module in the code I'm checking gets blamed for 68% of refactor issues, even though it contains 0 issues if checked individually.

Expected behavior

One option would be to issue duplicate code on all modules that contain it. That would inflate its count a bit, but I don't think that would be a major problem.

Another option could be to create a pseudo-module on which all duplicates are reported. So it would be much like the current situation, but without an existing module's stats being distorted.

pylint --version output

pylint 2.0.1
astroid 2.0.1
Python 3.4.6 (default, Mar 22 2017, 12:26:13) [GCC]

Jul 31 '18 02:07 mthuurne

Thanks for creating an issue! We should definitely emit the message for each file where this occurs, not just for the last one.

Aug 09 '18 06:08 PCManticore

Maybe my description wasn't clear, but the message is emitted for other files, but under the wrong file name: always with the name of the last file checked. In the example I included, there is code duplicated between databaselib and projectlib, but it is reported on xmlgen.py which is an unrelated module that does not contain the duplicate code.

Aug 14 '18 13:08 mthuurne

@mthuurne I got that, your description was clear enough.

Aug 15 '18 16:08 PCManticore

This one is a bit annoying when using build automation tools and it reports a lint error on a file you didn't actually change (or duplicate).

Jul 24 '19 12:07 rtaft

This may also be a duplicate of #2145

Jul 24 '19 12:07 rtaft

Any updates on the above? I've found I am getting the same issue

pylint --version output

pylint 2.6.0
astroid 2.4.2
Python 3.8.5 (default, Nov 10 2020, 13:10:08)

Nov 20 '20 11:11 AntObr

@Antobr no there has been no volunteer to work on this issue. Feel free to propose a PR if you are interested. :wink:

Nov 26 '20 17:11 hippo91

This behavior also leads to incorrect emission of the 'useless-suppression' message if a user chooses to disable=duplicate-code for a file or a block. It's because Pylint thinks the pragma should be placed in the last module.