satis icon indicating copy to clipboard operation
satis copied to clipboard

The directory of downloaded packages is more great than the packages published

Open carlososiel opened this issue 8 years ago • 10 comments

The directory of downloaded packages is more great than the packages published, this happen because the dev-master packages is downloaded again if exist a new commit in the repository, but it keeps previous versions of package though not are published. Exist any way to clean these obsolete packages? The system can be delete these packages automatically with the features that exist today? If not exist this feature can be included?

carlososiel avatar Jul 21 '15 14:07 carlososiel

what do you mean with directory of downloaded packages ?

stof avatar Jul 21 '15 14:07 stof

The directory is web/dist, where the packages are stored in .zip format

carlososiel avatar Jul 21 '15 14:07 carlososiel

Well, this folder is not about packages downloaded by Satis but generated by Satis to be downloadable. The reason why Satis does not delete old ones is to keep them available for projects having a lock file referencing one of them (i.e. they have a dev version in their lock file, but new commits have been done in the meantime and Satis is regenerated). There is no command to clean old dist files currently, but this could be implemented (with a warning to make people aware of the drawback)

stof avatar Jul 21 '15 14:07 stof

@carlososiel Bear in mind that, although those dist files are no longer included in the packages.json file of your Satis instance, they may still be referenced in the composer.lock files of various projects which have been built using Composer. If you delete those files, composer install might fail for some of your projects until you run composer update.

Edit: @stof beat me!

joshdifabio avatar Jul 21 '15 14:07 joshdifabio

There also is the skip-dev option so things like dev-master are skipped when dumping, so it would not create a ne zip for every new commit. This does of course not work when you depend on master being downloadable as a dist.

aaukt avatar Jul 21 '15 14:07 aaukt

@joshdifabio I understand the risks, and i proceed with much care, but is important for me include this feature in the system to keep only the packages used in my server. @aaukt Is not a option for me, i use packages that depend of dev-master versions

carlososiel avatar Jul 21 '15 14:07 carlososiel

@carlososiel dev-master is still available, it just links directly to the git repository (and composer would clone it)

aaukt avatar Jul 21 '15 14:07 aaukt

@carlososiel the 'feature' cannot be implemented, as by its very nature composer nor satis have any way of identifying which packages are 'obsolete', as they don't know what's out there in the world. Your feature request implies reverse knowledge which would honestly only apply in your specific use case. And for single-user features I would recommend just writing a script on your end instead that cleans them up, rather trivial given the versioned filenames.

curry684 avatar Oct 25 '15 21:10 curry684

Sorry to revive an old issue but it seemed more sensible then creating a new one.

@curry684, Your explanation, is this the reason why the zips aren't removed even if the package itself is removed from the Satis repository? We just got rid of a couple of large packages (in terms of tags / branches and therefore number of zip archives) from our repository and noticed our dist folder is still filled with all it's zip archives.

mikebarlow avatar Aug 15 '16 21:08 mikebarlow

@mikebarlow the underlying 'issue' is that Composer supports (or rather: depends on) the composer.lock file which locks the state of the application to a verified and known-to-be-working one. For tags this is pretty easy, an install of v6.8.4 can just lock itself to that tag reference. A branch install, like dev-master most obviously, is a moving reference though, as it may have seen 5 new commits the next hour after installing. Therefore Composer locks itself unto commit hashes there, to still be able to ensure that the rest of your team and the production server get the exact same code that you tested and developed with.

Satis and Packagist have to keep the distributions for the old commits around for branches to be able to serve an older version of a branch reference to version locked projects. They have no reliable way without human intervention to decide which ones are never going to be used (again).

curry684 avatar Aug 15 '16 22:08 curry684