Tar Builder ought to use the tarfile module

[bdbaddog]

This issue was originally created at: 2008-01-21 06:48:55. This issue was reported by: petergummer.

petergummer said at 2008-01-21 06:48:55

The Tar builder is a very crude wrapper for the tar command line utility. It adds no real value compared with simply running tar directly within a Command builder; in fact, it obscures the interface a bit and can complicate things.

Gary Oberbrunner wrote, "I've noticed the same thing. Ideally IMHO it should be rewritten to use the python tarfile stuff internally, and just look like regular tar options to the user. Also tar's options aren't the same on all machines (IRIX doesn't even support -z, nor bundling args), another reason to redo it."

Thanks to Gary's suggestion, I've had a look at Python's tarfile module and it does add value; e.g., it allows you to specify each file's name within the archive, which can save an intermediate step of creating the desired directory structure. One thing that it lacks in Python 2.5.1 compared with the command-line tar utility is the ability to exclude specific files, but I notice that this feature has been added to Python 3:

https://mail.python.org/pipermail/python-3000-checkins/2007-June/001000.html

So I think Gary is right. The Tar builder should use the tarfile module.

gregnoel said at 2008-01-21 13:08:04

I've wanted this as well. And, although I agree that Tar() is the heavy lifter and should be fixed first, I'd like to see this capability extended to Jar() and Zip() as well. (Maybe those two should be farmed out into separate issues.) I also think it's a capability that should be explored for Pacakage() as it is integrated with the Autotools feature set (I'm less sure of this as the API could be a serious issue).

This is really an enhancement request, so I changed the issue type.

russel said at 2008-02-26 23:46:49

As noted on my email to the user list, the presence of https://www.scons.org/wiki/AccumulateBuilder and https://www.scons.org/wiki/DistTarBuilder on the wiki indicate some of the deficiencies of the Tar builder.

I agree that Zip (and Jar I guess) also need some attention.

garyo said at 2008-03-17 20:24:47

*** Issue 1360 has been marked as a duplicate of this issue. ***

garyo said at 2008-03-17 20:30:53

Tar builder, reassigning to Greg.

gregnoel said at 2008-03-17 22:44:25

Needs backported tarfile.py, which needs new-style classes, so I made this 2.x.

gregnoel said at 2009-04-18 17:25:40

Issue #2384 is developing the technology for this issue and issue #1351 uses the Zip() builder, so set the dependencies appropriately.

garyo said at 2012-09-01 10:04:01

de-assigning all tickets assigned to Greg Noel (no longer working on SCons)

Votes for this issue: 12.

gregnoel said this issue depends on #2384 at 2009-04-18 17:25:40.

gregnoel said this issue blocks #1351 at 2009-04-18 17:25:40.

garyo said this issue is duplicated by #1360 at 2008-03-17 20:24:49.

[mwichmann]

The only note would be that using tarfile module leaves the work being done in-process, which is not necessarily the best choice if there's lots going on and many threads building.

[bdbaddog]

The only note would be that using tarfile module leaves the work being done in-process, which is not necessarily the best choice if there's lots going on and many threads building.

True. Could such be run as a forked process instead of a thread as normal? It doesn't modify the Environment() it's connected to..

[mwichmann]

Add comment from Discord channel: there may be cases of people who have used the existing implementation to reach into esoteric tar features (esp. GNU tar) via flags passed through TARFLAGS, don't want to necessarily break those people.

OTOH, this would help resolve some of the existing issues with env.Tar not being as useful if your tar is not gnutar.