pypa
clean command doesn't cleanup the build dir
Originally reported by: Ionel Maries Cristian (Bitbucket: ionelmc, GitHub: ionelmc)
Eg: after running python setup.py clean I still have all the the stuff that supposed to be deleted in build/lib.linux-x86_64-2.7.
- Bitbucket: https://bitbucket.org/pypa/wheel/issue/117
@ionelmc would you mind providing instructions to reproduce? Is this problem really specific to wheel?
Wheel has no clean command, nor does it hook into setuptools's clean in any way. What should be done about this?
I'm closing this as invalid since this has nothing to do with wheel and the original poster has not responded in 6 months.
So here's an example. I didn't expect you'd close the issue without even trying to reproduce the problem, that's why I didn't even respond in the fist place.
ionel:xenial ~/osp/python-hunter master#> rm -rf dist build 0.00s => 0 13:51
ionel:xenial ~/osp/python-hunter master#> python3.6 setup.py bdist_wheel 0.07s => 0 13:51
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/hunter
copying src/hunter/__init__.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/tracer.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/actions.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/predicates.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/remote.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/const.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/util.py -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/event.py -> build/lib.linux-x86_64-3.6/hunter
running egg_info
writing src/hunter.egg-info/PKG-INFO
writing dependency_links to src/hunter.egg-info/dependency_links.txt
writing entry points to src/hunter.egg-info/entry_points.txt
writing requirements to src/hunter.egg-info/requires.txt
writing top-level names to src/hunter.egg-info/top_level.txt
reading manifest file 'src/hunter.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '.isort.cfg'
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.dylib' found anywhere in distribution
writing manifest file 'src/hunter.egg-info/SOURCES.txt'
copying src/hunter/_event.c -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_event.pxd -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_event.pyx -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_predicates.c -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_predicates.pxd -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_predicates.pyx -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_tracer.c -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_tracer.pxd -> build/lib.linux-x86_64-3.6/hunter
copying src/hunter/_tracer.pyx -> build/lib.linux-x86_64-3.6/hunter
running build_ext
building 'hunter._event' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/src
creating build/temp.linux-x86_64-3.6/src/hunter
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Isrc/hunter -I/usr/include/python3.6m -c src/hunter/_event.c -o build/temp.linux-x86_64-3.6/src/hunter/_event.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/src/hunter/_event.o -o build/lib.linux-x86_64-3.6/hunter/_event.cpython-36m-x86_64-linux-gnu.so
building 'hunter._predicates' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Isrc/hunter -I/usr/include/python3.6m -c src/hunter/_predicates.c -o build/temp.linux-x86_64-3.6/src/hunter/_predicates.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/src/hunter/_predicates.o -o build/lib.linux-x86_64-3.6/hunter/_predicates.cpython-36m-x86_64-linux-gnu.so
building 'hunter._tracer' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Isrc/hunter -I/usr/include/python3.6m -c src/hunter/_tracer.c -o build/temp.linux-x86_64-3.6/src/hunter/_tracer.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/src/hunter/_tracer.o -o build/lib.linux-x86_64-3.6/hunter/_tracer.cpython-36m-x86_64-linux-gnu.so
copying src/hunter.pth -> build/lib.linux-x86_64-3.6
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
copying build/lib.linux-x86_64-3.6/hunter.pth -> build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/__init__.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_event.pyx -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_event.c -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_event.pxd -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/tracer.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/actions.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_predicates.pyx -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_tracer.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/predicates.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_predicates.c -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_predicates.pxd -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_predicates.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_event.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_tracer.pxd -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/remote.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_tracer.pyx -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/const.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/util.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/event.py -> build/bdist.linux-x86_64/wheel/hunter
copying build/lib.linux-x86_64-3.6/hunter/_tracer.c -> build/bdist.linux-x86_64/wheel/hunter
running install_egg_info
Copying src/hunter.egg-info to build/bdist.linux-x86_64/wheel/hunter-2.0.2-py3.6.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/hunter-2.0.2.dist-info/WHEEL
creating '/home/ionel/osp/python-hunter/dist/hunter-2.0.2-cp36-cp36m-linux_x86_64.whl' and adding '.' to it
adding 'hunter.pth'
adding 'hunter/__init__.py'
adding 'hunter/_event.c'
adding 'hunter/_event.cpython-36m-x86_64-linux-gnu.so'
adding 'hunter/_event.pxd'
adding 'hunter/_event.pyx'
adding 'hunter/_predicates.c'
adding 'hunter/_predicates.cpython-36m-x86_64-linux-gnu.so'
adding 'hunter/_predicates.pxd'
adding 'hunter/_predicates.pyx'
adding 'hunter/_tracer.c'
adding 'hunter/_tracer.cpython-36m-x86_64-linux-gnu.so'
adding 'hunter/_tracer.pxd'
adding 'hunter/_tracer.pyx'
adding 'hunter/actions.py'
adding 'hunter/const.py'
adding 'hunter/event.py'
adding 'hunter/predicates.py'
adding 'hunter/remote.py'
adding 'hunter/tracer.py'
adding 'hunter/util.py'
adding 'hunter-2.0.2.dist-info/DESCRIPTION.rst'
adding 'hunter-2.0.2.dist-info/entry_points.txt'
adding 'hunter-2.0.2.dist-info/metadata.json'
adding 'hunter-2.0.2.dist-info/top_level.txt'
adding 'hunter-2.0.2.dist-info/WHEEL'
adding 'hunter-2.0.2.dist-info/METADATA'
adding 'hunter-2.0.2.dist-info/RECORD'
ionel:xenial ~/osp/python-hunter master#> python3.6 setup.py clean 13.98s => 0 13:51
running clean
removing 'build/temp.linux-x86_64-3.6' (and everything under it)
ionel:xenial ~/osp/python-hunter master#> tree build 0.42s => 0 13:51
build
├── bdist.linux-x86_64
└── lib.linux-x86_64-3.6
├── hunter
│ ├── actions.py
│ ├── const.py
│ ├── _event.c
│ ├── _event.cpython-36m-x86_64-linux-gnu.so
│ ├── _event.pxd
│ ├── event.py
│ ├── _event.pyx
│ ├── __init__.py
│ ├── _predicates.c
│ ├── _predicates.cpython-36m-x86_64-linux-gnu.so
│ ├── _predicates.pxd
│ ├── predicates.py
│ ├── _predicates.pyx
│ ├── remote.py
│ ├── _tracer.c
│ ├── _tracer.cpython-36m-x86_64-linux-gnu.so
│ ├── _tracer.pxd
│ ├── tracer.py
│ ├── _tracer.pyx
│ └── util.py
└── hunter.pth
3 directories, 21 files
Reproducing it was never the issue here. Just tell me how I am to fix an issue with a command that does not even belong to this project.
I took another look at the distutils.clean code. I'm wondering if I could do something about this by changing the names of directories created by wheel.
The crux of the problem seems to be that bdist_wheel installs the project to the build directory. That definitely makes this a valid issue for this project.
@ionelmc So here's what I've found while investigating: running setup.py build creates a build/lib directory which is not cleaned up by setup.py clean (both being distutils commands). Should wheel try to fix this even though it is not directly responsible? Both yes and not are acceptable answers, but I need to know where to go from here.
A naive way to fix this would be to remove any existing build/lib directory before running the build command, but I'm not sure this is the correct approach. Comments, @jaraco?
I'm not sure what to do here. Obviously, running a clean before each build would be a cleaner approach, but doing so probably invalidates any performance gains that might be had by re-using files from a previous build. I'm not familiar with the process or how much others would rely on those files remaining around. I do believe those build files are particularly important for 2to3 builds under 'develop' installs (because the files in the build directory are used to run). Even in that case, if one were to clean before running build, I wouldn't think that would cause problems... unless some strange project were somehow pre-populating that lib directory. Seems like an unlikely behavior. So probably worth a try.
But setup.py clean does not remove the build/lib directory, so even if one were to run clean, it wouldn't do anything.
@agronholm You can just advise the script that performs the build process to delete the build dir afterwards, instead of doing it by command:
import setuptools
import shutil
setuptools.setup(
...
)
shutil.rmtree('build', ignore_errors=True)
NOTE: Be careful with deleting non-empty folders, this will delete the directory no matter what comes after setup.py in your CLI command (every time the script is called or imported).
@agronholm Oh I'm sorry, I thought you just want the build/ folder to be cleaned after wheel generation...
The original complaint was that setup.py bdist_wheel does not clean up the build directory. Trouble is, wheel does clean up everything it creates directly, but the build directory is only created as a side effect of other setuptools commands that bdist_wheel invokes.