Gentoo Support

[JoseJX]

This is a bit a rough so far, but I'm interested in adding Gentoo support to Crouton. I don't want to suggest that this should be integrated as is, but I'd like some feedback and was wondering if others would be interested in this.

This currently builds most of the targets, but I haven't yet worked on the KDE, Gnome or Kodi/XBMC targets. I'm currently using a Pixel 2 for development.

Thanks!

[DennisLfromGA]

Looking forward to this, thanx.

I tried to install it but got stuck near the end I think.

This is the command string I used to install it:

sudo sh ./installer/main.sh -p /var/crouton/ -r gentoo -t xiwi,xorg,cli-extra,keyboard,lxde-desktop,chrome-beta

Here's where the output ended:

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by x11-drivers/xf86-video-intel-2.99.917_p20170216::gentoo
# required by xf86-video-intel (argument)
>=x11-libs/libdrm-2.4.75 video_cards_intel

Use --autounmask-write to write changes to config files (honoring
CONFIG_PROTECT). Carefully examine the list of proposed changes,
paying special attention to mask or keyword changes that may expose
experimental or unstable packages.
Failed to complete chroot setup.
Unmounting /var/crouton/chroots/gentoo...

I haven't tried to fix it yet to see what you think.

-DennisL

[JoseJX]

Thanks for testing! I was adding targets one by one and didn't notice the conflict between xiwi and xorg. I'm working on a fix for that now.

[JoseJX]

I've added a bit more intelligence to the expand variables, which should fix this issue. I've also made a few tweaks to the xfce-desktop and lxde-desktop targets to more closely match what you're expecting from debian/ubuntu, although these are completely untested so far since I haven't had time to compile them yet.

Thanks for taking a look!

[DennisLfromGA]

@JoseJX,

I've added a bit more intelligence to the expand variables, which should fix this issue.

I may have been a bit premature but I just updated the repo and tried an update (via '-u') but am still getting the video card issue -

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by x11-drivers/xf86-video-intel-2.99.917_p20170216::gentoo
# required by xf86-video-intel (argument)
>=x11-libs/libdrm-2.4.75 video_cards_intel

I'll try to be more patient and wait until you have a chance to work on things... ;-)

-DennisL

[JoseJX]

No worries! I kicked off a new build with the exact same targets as you're using above, hopefully that will help me figure out what's going on for you! One difference is that I'm building the crouton executable and installing with that, but I'm not sure how that would change the result.

What Chrome device are you using?

You might want to try removing the VIDEO_CARDS and INPUT_DEVICES lines from /etc/portage/make.conf, it might help get over this bump!

Can you attach (from the chroot) /etc/portage/make.conf if it fails again?

[DennisLfromGA]

@JoseJX,

One difference is that I'm building the crouton executable and installing with that, but I'm not sure how that would change the result.

Actually, I do the same thing, I 'make' the crouton executable and install or update using it. I don't think it should make any difference using 'main.sh' though.

What Chrome device are you using?

I've got the Dell Chromebook 13 (7310) 'lulu' model (i5 / 8GB RAM / 256GB SSD) with ARC++ enabled and using -

Version 57.0.2987.115 beta (64-bit)
Platform 9202.54.0 (Official Build) beta-channel lulu
ARC Version 3828404
Firmware Google_Lulu.6301.136.57

You might want to try removing the VIDEO_CARDS and INPUT_DEVICES lines from /etc/portage/make.conf, it might help get over this bump!

I'll do that now and get back to you.

Can you attach (from the chroot) /etc/portage/make.conf if it fails again?

Will do.

Thanx, -DennisL

[DennisLfromGA]

@JoseJX,

Just an update, I know you're working on it now but I thought I'd let you know what I ran into. Here's my make.conf

chronos@localhost /var/crouton/chroots/gentoo/etc/portage $ cat make.conf
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /usr/share/portage/config/make.conf.example for a more
# detailed example.
CFLAGS="-O2 -pipe -march=native -mtune=native"
CXXFLAGS="${CFLAGS}"
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST="x86_64-pc-linux-gnu"
# These are the USE and USE_EXPAND flags that were used for
# buidling in addition to what is provided by the profile.
USE="bindist pulseaudio alsa X dbus xcb xkb libkms uxa dri3 xvmc opengl"
CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"
PORTDIR="/usr/portage"
DISTDIR="${PORTDIR}/distfiles"
PKGDIR="${PORTDIR}/packages"
ABI_X86="32 64"
MAKEOPTS="-j5"
##VIDEO_CARDS="dummy"
##INPUT_DEVICES="keyboard libinput mouse evdev"

And here are the errors I'm running into when I enter the chroot and try to let it finish the setup.

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by x11-drivers/xf86-video-ati-7.8.0::gentoo[glamor]
# required by x11-base/xorg-drivers-1.19::gentoo[video_cards_radeonsi]
>=x11-base/xorg-server-1.19.2 glamor

Use --autounmask-write to write changes to config files (honoring
CONFIG_PROTECT). Carefully examine the list of proposed changes,
paying special attention to mask or keyword changes that may expose
experimental or unstable packages.

emerge: there are no ebuilds built with USE flags to satisfy "x11-base/xorg-server[glamor]".
!!! One of the following packages is required to complete your request:
- x11-base/xorg-server-1.18.4::gentoo (Change USE: +glamor)
(dependency required by "x11-drivers/xf86-video-ati-7.8.0::gentoo[glamor]" [ebuild])
(dependency required by "x11-base/xorg-drivers-1.19::gentoo[video_cards_radeonsi]" [ebuild])
(dependency required by "x11-base/xorg-server-1.19.2::gentoo[xorg]" [ebuild])
(dependency required by "xorg-server" [argument])
Failed to complete chroot setup.
The chroot setup script may be broken. Your chroot is not fully configured.
Removing the chroot setup script. You may want to update your chroot again.
UID 1000 not found in gentoo
Unmounting /var/crouton/chroots/gentoo...
Sending SIGTERM to processes under /var/crouton/chroots/gentoo...

Let me know if you'd like me to try anything else, otherwise I'll just hold off for a while.

Thanx, -DennisL

[JoseJX]

Thanks for posting those files, that helps! So, I hadn't thought of the variables being commented out like that! I've changed it so that the match should check for the variable at the start of the line which I think should fix that issue. I'm currently away from my chromebook, so I'm not sure yet if my build that I started earlier was successful, I'll follow up on that later tonight.

The errors following your make.conf are from there not being a valid VIDEO_CARDS line set, which just defaults to a normally sane set of cards, but those require a few more use flags to be set.

[dnschneid]

This is fantastic! I'll review it in multiple passes (especially since you're still working on it), but at a quick glance it looks like a really good start.

[JoseJX]

No worries on the number of comments, and thanks for reviewing! I'll look into addressing some of the issues tonight.

[JoseJX]

I think I've addressed most of the issues from the first review. I still haven't tested most of the changes yet, so please don't feel like you need to review this again immediately. I'm going to try to rerun DennisLfromGA's setup and make sure everything is working. I'll update this once it finishes compiling.

Thanks again for looking at the code!

[JoseJX]

Okay! This is in a bit better shape now. I was able to install xiwi and xorg simultaneously and both seem to be working. DennisLfromGA, I think that you should be able to complete your installation now, I was able to do a clean install with the targets you specified above.

[JoseJX]

I've tested the additional architecture code with the x86 target and it seemed to work fine. I've dug out my old Samsung Chromebook 3 for testing the ARM targets. It's crazy how slow this machine is now. :(

I think I've addressed most of your comments from the second review, but I'll continue going over the code some more to tighten things up. This all needs to be tested again as well, which I'll work on tonight.

[DennisLfromGA]

@JoseJX,

I was not able to repair my first chroot with targets:

xiwi,xorg,cli-extra,keyboard,lxde-desktop,chrome-beta

I kept getting hung up on the video driver issue -

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by x11-drivers/xf86-video-ati-7.8.0::gentoo[glamor]
# required by x11-base/xorg-drivers-1.19::gentoo[video_cards_radeonsi]
>=x11-base/xorg-server-1.19.2 glamor

But I was able to successfully install & run a brand new one.
I just let it set the default xorg xmethod and it works fine.

name: gentoo
encrypted: no
Entering /var/crouton/chroots/gentoo...
crouton: version 1-20170327171751
release: gentoo
architecture: amd64
xmethod: xorg
targets: cli-extra,keyboard,lxde-desktop,chrome-beta
host: version 9202.54.0 (Official Build) beta-channel lulu 
kernel: Linux localhost 3.14.0 #1 SMP PREEMPT Sat Mar 18 09:21:47 PDT 2017 x86_64 Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz GenuineIntel GNU/Linux
freon: yes

I may get brave and try to add 'xiwi' but I'll back it up first.

Meanwhile I'll be studying the 'Gentoo Cheat Sheet' ;)

Thanx for all your efforts, -DennisL

[JoseJX]

Hey, that's great! I'm glad you were able to get it to work. I haven't tested a whole lot of the changes from last night since I'm not in front on my Chromebook right now, so expect some breakage, but I think it's getting closer to being actually usable!

[DennisLfromGA]

Just an update -

It seems the switching key-combo ctrl + alt + shift + left-arrow/right-arrow gets broken. Once I switch out of gentoo I can't get back to any of my running chroots, to include gentoo. The 'crouton integration' extension lists them but when clicked nothing happens. Also croutoncycle in gentoo works once and then won't work again in any other chroot, it just hangs on -

+ force=
+ [ cros != cros ]
+ cmd=0
+ mkdir -m 775 -p /tmp/crouton-lock
+ exec
+ chmod -Rf g+rwX /tmp/crouton-lock
+ chgrp -Rf crouton /tmp/crouton-lock
+ flock 3

I've tried croutoncycle [list|cros|:1|:2] and it hangs at the same point. So I'm kind of locked out of any running desktop gui.

-DennisL

[JoseJX]

Hmm, I'll have to try that later, I've never actually tried switching on my machine. I'd suspect that it's missing an application or configuration option (perhaps a Debian default?). In any case, thanks for pointing that out!

[JoseJX]

I tried last night to get this running on my old Samsung Chromebook 3, but I'm running into an issue where the unpacked stage tarball is bigger than /tmp. Do you have any suggestions for working around this?

I also need to look into the desktop switching bug that DennisLfromGA reported above, I can reproduce it on my Pixel 2.

Thanks again for reviewing!

[dnschneid]

I tried last night to get this running on my old Samsung Chromebook 3, but I'm running into an issue where the unpacked stage tarball is bigger than /tmp. Do you have any suggestions for working around this?

Whoa, how big is the stage3 tarball? Maybe we'll need to add logic to main.sh to provide (and clean up) a temp folder on the same medium as the installation directory, if not doing -d.

[JoseJX]

For ARM, it's about 1.1GB unpacked, plus the portage tree which is about 750MB. amd64/x86 should be around the same size. The files before unpacking are about 300MB for the stage tarball and around 90MB for the portage tree. We could probably save a lot of space by making some kind of sparse filesystem for the portage tree, the 4k blocksize my Pixel is using isn't really optimal for the huge number of small files in the portage tree.

[JoseJX]

@DennisLfromGA - It looks like the lock isn't being removed on the switch between consoles, I'm still looking into it by working back from traces from a working Xenial install. Hopefully this will make clear what I'm doing wrong. Thanks for bringing it to my attention!

@dnschneid - I think most of the other issues are fixed now, but I wanted to bring up one thing. I have been thinking about the *-desktop targets and was thinking that instead of maintaining giant lists of packages to install, it might make more sense to just not allow those targets to install on Gentoo. So long as the base KDE/Gnome/lxde/xfce/whatever DE is installed, the user could pretty easily maintain the rest without really needing anything else from crouton.

So, since they're really sort of crutches for installing software that's maintained by the package manager and don't really come with anything extra (like startlxde or similar) would it be better to just not support those packages on Gentoo?

[dnschneid]

I think most of the other issues are fixed now, but I wanted to bring up one thing. I have been thinking about the *-desktop targets and was thinking that instead of maintaining giant lists of packages to install, it might make more sense to just not allow those targets to install on Gentoo. So long as the base KDE/Gnome/lxde/xfce/whatever DE is installed, the user could pretty easily maintain the rest without really needing anything else from crouton.

So, since they're really sort of crutches for installing software that's maintained by the package manager and don't really come with anything extra (like startlxde or similar) would it be better to just not support those packages on Gentoo?

Yeah, I'd be okay with that. Just make the -desktop targets depend on the normal targets and make gentoo not add anything in particular. No need to fail out.

[dnschneid]

w.r.t. 0269a1824270d337199f9eeddba1241a5039c2c8, I was actually hoping it would be more automatic. Something like

1. main.sh creates two temporary directories. One is in /tmp, the other is in $CHROOTSRC/.crouton-tmp or something (that's the real path, not the $CHROOT shadow in /var; don't want to go through ecryptfs). If main.sh sees .crouton-tmp, it deletes it.
2. Bootstrap can pick which /tmp to use. Ideally it would see how much space is available in tmpfs, but blindly picking one on a per-distro basis is fine too.
3. When main.sh deletes the /tmp directory, it deletes .crouton-tmp as well.

[JoseJX]

After some package keywording (I'll take care of it on the Gentoo side, but it's not quite ready yet) I was able to run LXDE on my Chromebook 3 on ARM. Interestingly, console switching seems to be working just fine on that machine, so the issue is probably related to the Intel driver. I suspect that there will be further issues with ARM and some of the other packages related to keywords, it's something that will have to be resolved with more testing.

So here's my current list of things left to do:

• [ ] Figure out why the console switching isn't working on Intel
• [ ] Figure out why gnome isn't starting properly
• [ ] Add .desktop file for croutonurlhandler
• [ ] Gentoo documentation in the wiki

Gnome and KDE can't be installed simultaneously right now because I'm using the profiles to configure the environment. If this is something that we need (or if I need to handle it better), please let me know. I'm going to test these targets now, but it'll be a while since they need some time to build.

I've added a Kodi target and made a -common file so we can support both Kodi and XBMC. If this approach isn't okay, please let me know!

[JoseJX]

As an update:

• Gnome builds okay, but I'm having some issues getting it to start up correctly. It seems that the gnome-session-wrapper isn't actually starting the gnome session, so it defaults to twm/xterms. Still working on that.
• KDE runs well!
• Kodi builds and runs too!
• After some more testing, I found that I can successfully switch back to the running X session by sending signal USR1 to the PID in /tmp/.X#-lock. Obviously, this bypasses all of the safeguards that you've put into croutoncycle, but at least this shows that the newer Intel driver included with Gentoo probably isn't causing the issue. If I trace through the shell script, I was finding that the call to xrefresh was locking things up. Using strace on this led me to see that the poll on the local socket in this program (/tmp/.X11-unix/X#) is what's getting stuck. After commenting this out, the call to xprop is then failing similarly. I'm not sure yet why the socket it getting stuck, but I'll keep working on it.

[nbe1233]

If you have time perhaps you could implement musl and uclibc gentoo versions https://wiki.gentoo.org/wiki/Project:Hardened_uClibc https://wiki.gentoo.org/wiki/Project:Hardened_musl

[JoseJX]

Yeah, hardened targets, x32 support and funtoo might be other interesting targets too, but for now I'm going to keep working on getting what I have in a merge-able state. Thanks for looking at it though!

I haven't had much time this week to dig into this, and really won't be able to for the following week either. If anyone has any ideas on the terminal switching issue, I'd really appreciate the input. If not, I'll get back on this in a week or so. Thanks!

[zenmetsu]

Not sure what the current status of this branch is, but I did find a few issues. The first was that the ARCH case statement in the installer was bombing with bad substitution on my Samsung Chromebook2. uname -m reports arm7l on this machine. I just hard-coded it to get the installer rolling.

The second issue is more of a Gentoo thing. During the emerge world to update the base, there was a perl conflict which broke the process. The solution was to change the emerge line in targets/core to emerge -NDauv --with-bdeps y --backtrack 100 @world

[dnschneid]

@JoseJX, sorry for the silence here. Could you please update the list of what isn't working?

[JoseJX]

Sorry, I got busy with other work and haven't had too much time to work on this. I've still got the following issues open:

1. ARM changes to get it working for zenmetsu. I'd probably want to test on more ARM machines before calling any changes I make "fixed"
2. Console switching on Intel still seems to be broken. I can successfully switch back to Crouton, but not back to Gentoo from Crouton.
3. I haven't tested Gnome at all since I last posted here, I need to figure out what's going on with that not starting properly.
4. Documentation!

I'm hoping to have more free time in August. If you have any suggestions about the console switching issue, that's really the only big bug left.