flit_core does not filter out Python bytecode when not operating on vcs

[nanonyme]

When flit_core is generating allowed files into sdist https://github.com/pypa/flit/blob/ec6e6b7c8994d5e79d0fd951d9e4c26a7c5d9977/flit_core/flit_core/sdist.py#L109-L118 it does not use https://github.com/pypa/flit/blob/ec6e6b7c8994d5e79d0fd951d9e4c26a7c5d9977/flit/sdist.py#L98-L101 for filtering on output files which results in Python bytecode ending up in package. Looks like there is filtering done through https://github.com/pypa/flit/blob/ec6e6b7c8994d5e79d0fd951d9e4c26a7c5d9977/flit_core/flit_core/common.py#L79-L83 but it appears not to cover all cases. This is almost certainly undesired behaviour as there is no portability with Python bytecode across other Python versions.

[takluyver]

Can you describe a specific scenario where this is going wrong? The filtering works in somewhat different contexts, and I often live with little bits of duplication rather than tying modules pieces more closely together.

[layday]

When building with flit-core, __pycache__ folders are only excluded from the "module" folder, so if you were to add e.g. your tests folders in the sdist, tests/__pycache__ won't be filtered out. IIUC, the flit builder excludes these from everywhere in the sdist.

[takluyver]

Aha, I see. So the _include function is fine, because Module.iter_files() is only going through the files in the importable module - other files you configure to include in the sdist are added by SdistBuilder.apply_includes_excludes().

Maybe the simplest way to do it is to always add **/__pycache__ and **.pyc to the exclude patterns? Excludes override inclues, so there would be no way to configure including them, but I can't think of a reason you'd need that (and Flit doesn't aim to cover every possible scenario anyway).

[nanonyme]

@takluyver do we really need also the directory? Does flit package empty directories?