haikuporter icon indicating copy to clipboard operation
haikuporter copied to clipboard

Speed up entering chroot

Open diversys opened this issue 9 years ago • 16 comments

Entering chroot is painfully slow with many installed packages (I have ~600 packages here). It is okay if you only need to run haikuporter once but when porting one often needs to run the build over and over again.

Judging from ProcessController most of the time is spent printing the meta information for every single installed package with package -i command. Maybe it's possible to cache this info somehow?

diversys avatar Oct 27 '15 07:10 diversys

Maybe just disable it unless a --verbose or so switch is used, if it's only printing?

pulkomandy avatar Oct 27 '15 07:10 pulkomandy

I thought that getting the meta info is done to resolve recipe requires?

diversys avatar Oct 27 '15 07:10 diversys

Well, that's why when porting things I use my dev-perso script and my own checkout outside of haikuporter, and only when I get things working do I write a recipe. One could at least check all the scriptlets to optimize them to spare some forks (like some sed or tr could be done with variable expansion).

mmuman avatar Jan 09 '16 00:01 mmuman

Depending on how we use them, we could have a --all to package list that would list infos for all installed packages, that would save N-1 forks with N the package count.

mmuman avatar Jan 09 '16 00:01 mmuman

Should be fixed after the merge?

waddlesplash avatar Jan 30 '16 00:01 waddlesplash

I think this has been fixed with the use of the recipeCache folder.

DarkmatterVale avatar Feb 09 '16 01:02 DarkmatterVale

Except the current version fails badly if repository/ needs to be repopulated.

mmuman avatar Feb 09 '16 01:02 mmuman

It still checks all the available hpkgs for me (maybe because feature branches, so it have to update plenty things). Is there any news on this front?

extrowerk avatar Nov 06 '17 09:11 extrowerk

The package_repo update commit doesn't change anything with regards to this, as that is only used for package repo creation. It's not involved in the dependency resolution process where all the package infos are parsed from all the package files.

The change in 7004cf830aaed92dc8ddb1b047a4ff46d9884276 however introduces persistence of the PackageInfo cache, which should solve this issue.

mmlr avatar Nov 24 '17 22:11 mmlr

It seems like some recent commit broke it.

diversys avatar Apr 23 '18 12:04 diversys

Can you provide further information like what port you were building, what command line arguments you used and what packages were installed in the system?

Bisecting the recent commits would obviously be most helpful. I don't really see what commit could have changed things in that regard off-hand.

mmlr avatar Apr 23 '18 14:04 mmlr

Now I'm puzzled, I reverted to ca76b0642fe65d7ced819e03f7eb0b512479c173 and still see in ProcessController how it does "package list -i" for every package in BUILD_REQUIRES(). BUILD_REQUIRES.

I'm using this alias: alias hp='haikuporter -S -j1 --get-dependencies --no-source-packages'

haikuporter does't user cache over here somehow...

diversys avatar Apr 23 '18 15:04 diversys

If your alias is defined like that, it will run an haikuporter that's in your PATH. Are you sure it runs the one you checked out and not one from a package? Maybe it's time for an update to the haikuporter package?

pulkomandy avatar Apr 23 '18 15:04 pulkomandy

The repository format has changed due to the introduction of TEST_REQUIRES, so the repository will be repopulated once you cross that change. This will also, unneccessarily, reset the other caches since they reside in the same directory. The repository update should be obvious. The HPKG cache should then be repopulated only once.

mmlr avatar Apr 23 '18 15:04 mmlr

If your alias is defined like that, it will run an haikuporter that's in your PATH. Are you sure it runs the one you checked out and not one from a package?

Yes, I have only one porter as it's a new Haiku VM. I've switched to master let it repopulate repository and tried to build kdiff3 and I still see package list which takes about 5 seconds.

diversys avatar Apr 24 '18 09:04 diversys

This is still an issue for me. Is it for anyone else?

diversys avatar Dec 23 '21 15:12 diversys