lf-edge
Added interactive installer package in pkg/installer
- This patch introduces a new interactive installation mode for EVE-OS. It allows users to choose the installation parameters in a text-based interface, which is more user-friendly than editing grub.cfg.
- We store the installation parameters in a JSON file (installer.json), which we can use for bulk installations with similar configurations.
Dear all, This is a huge PR; unfortunately, I could not do it in parts. Please commend me for any questions or concerns.
We cannot use GPL code in the installer, only permissive licenses compatible with Apache 2.0
@rene which package is GPL? FWIW we do use the large GPL things which everybody thinks is OK (Linux kernel and base Linux packages, and Xen) but we want to avoid any other GPL licences packages.
We cannot use GPL code in the installer, only permissive licenses compatible with Apache 2.0
@rene which package is GPL? FWIW we do use the large GPL things which everybody thinks is OK (Linux kernel and base Linux packages, and Xen) but we want to avoid any other GPL licences packages.
@eriknordmark , according to @jsfakian he left some comments from sample codes on the sources by mistake, so he cleaned up the code and we don't have this issue anymore.... @jsfakian , please, clarify if needed....
Note that GH is unable to handle a PR this size when it comes to adding comments on particular lines of code.
Dear all, This is a huge PR; unfortunately, I could not do it in parts. Please commend me for any questions or concerns.
Can't you do the vendor additions as a separate commit? That would at least make it possible to review the diffs for the second commit with your non-vendor additions and modifications.
Given the separate work on configuring the network parameters, do we really want a subset or duplication in here? The key thing we are missing in the installer is selecting disks (by showing which disks are available) and any related ZFS options.
Also, I think this should document the format for the config file so that we can (later) add that as a PXE payload to drive installation parameters across the board.
And if this needs to include network parameters, then I think there should be a common local format for those parameters which can be used by this interactive installer and by the interactive observability and configuration tool. But I actually don't think we need network configuration in the installer until we add the config file as a PXE payload - selecting the disk config and then later selecting (and testing) the network config using the observability tool should be fine when it is done interactively.
I removed the code.
Dear all, This is a huge PR; unfortunately, I could not do it in parts. Please commend me for any questions or concerns.
Can't you do the vendor additions as a separate commit? That would at least make it possible to review the diffs for the second commit with your non-vendor additions and modifications.
I did it, I hope it helps.
@eriknordmark, @deitch, @rene, @rucoder, I think I covered most (if not all) of the comments. Should we merge, or do we need more changes?
I added some comments, so I will sum up here.
- README: thank you for updating with how it works. It needs a bit more, primarily anything that isn't clearly obvious, like how you got the vendored dependencies in place.
- rust installation: can we avoid doing that explicitly here? This is the first, but not the last, to require rust; we don't want to create new technical debt where each package installs its own rust.
- interactive installer on each boot: from what I read, the interactive installer will run on each boot of installed EVE, not just installer EVE. I don't think we want that, I think we want it solely on the installer image. But correct me.
Thank you @deitch.
- I will update the README with more information.
- I tried to add rust in Alpine as a package, but it installs an old version (for Alpine 3.16), which has issues with compiling. If we upgrade to Alpine 3.20, it will be fine. So, I can either do it when we upgrade or try once more with Alpine 3.16 to see if I can make it work.
- For the interactive installer on each boot, we do not need it anymore, because we will not have maintenance mode. I am going to need some help here. I think you (@deitch) did a similar task for @rene, can you help me?
@jsfakian, is this PR still in progress? If so, should we mark it as WIP and convert it to a draft?
@jsfakian, is this PR still in progress? If so, should we mark it as WIP and convert it to a draft?
Hey @OhmSpectator, this PR is still in progress. @deitch is finishing up the separation of the installer and live image. We are debugging now. When it is finished, I will change this PR according to @deitch's PR. We don't need to change anything, IMHO.
We don't need to change anything, IHMO
Actually, I hope it will become much simpler. It will be entirely self-container in pkg/installer as a single container, making it easier. That is my goal, anyways, to help out.
#4248 is in. You should be able to rebase on that one. But... you should not have to change anything except what is in pkg/installer. Everything outside of there should not need changes.
If you need to change how make-raw works, it might, but since you just call that, I wouldn't see why.
#4248 is in. You should be able to rebase on that one. But... you should not have to change anything except what is in
pkg/installer. Everything outside of there should not need changes.If you need to change how
make-rawworks, it might, but since you just call that, I wouldn't see why.
Thanks, @deitch. I merged my code in pkg/installer. There seems to be an issue with arm build, but I do not understand what it is.
There seems to be an issue with arm build, but I do not understand what it is.
It's the runners' problem. We should resolve it today.
This passed the eden tests. Are there outstanding review comments or is this ready to go?
This passed the eden tests. Are there outstanding review comments or is this ready to go?
It is ready to go.
@deitch @rene can you check your "changes requested" and clear those? I prefer that over me having to delete/ignore those review comments.
Almost everything is ready to go. I have one unanswered question:
Final question is vendor.tar.gz. Is that to work around the fact that we have network: no in pkg/installer/build.yml? How big is this thing that we are committing to git?
Can you answer, please @jsfakian ?
I'm sorry, I forgot. It is 33MB. Yes, we do this because otherwise, we need networking to install the Rust app. A concern is that it can get huge if we extend the code and add more libraries. But let's rethink about it later.
That grows the size of the repository on disk by ~5%, and the amount to download by 10%:
$ g clone https://github.com/lf-edge/eve.git
Cloning into 'eve'...
remote: Enumerating objects: 124047, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 124047 (delta 4), reused 10 (delta 4), pack-reused 124037 (from 1)
Receiving objects: 100% (124047/124047), 379.12 MiB | 34.38 MiB/s, done.
Resolving deltas: 100% (83305/83305), done.
Updating files: 100% (17094/17094), done.
$ du -s -h eve/
650M eve/
Are we concerned about that? Should we be? @eriknordmark ?
That grows the size of the repository on disk by ~5%, and the amount to download by 10%:
$ g clone https://github.com/lf-edge/eve.git Cloning into 'eve'... remote: Enumerating objects: 124047, done. remote: Counting objects: 100% (10/10), done. remote: Compressing objects: 100% (6/6), done. remote: Total 124047 (delta 4), reused 10 (delta 4), pack-reused 124037 (from 1) Receiving objects: 100% (124047/124047), 379.12 MiB | 34.38 MiB/s, done. Resolving deltas: 100% (83305/83305), done. Updating files: 100% (17094/17094), done. $ du -s -h eve/ 650M eve/Are we concerned about that? Should we be? @eriknordmark ?
We could store it on a remote server and download the container with ADD instead.
remote server
I don't understand?
I guess this is all because of the network: no, so rust cannot just download things?
remote server
I don't understand?
I guess this is all because of the
network: no, so rust cannot just download things?
What I mean is, setup an HTTP server, (e.g., http://centaur.med.uoc.gr/files), and add the following to the dockerfile:
# Add the vendor tar file
ADD http://centaur.med.uoc.gr/files/vendor.tar.gz /usr/local/my-installer/vendor.tar.gz
RUN tar xvf /usr/local/my-installer/vendor.tar.gz \
&& rm vendor.tar.gz
FWIW, pkg/pillar/vendor/ is 106MB:
$ du -s -h pkg/pillar/vendor/
106M pkg/pillar/vendor/
I guess that this is not an issue with rust, it is an issue in general. We should solve it for this repo for rust, but it is no different than what we do for golang.
In that case, the 33MB should not stop us, if the vendor/ in all the others did not.
Question: can we avoid the ugly .tar.gz, and instead use vendoring, like we do with other languages? To keep it consistent and instinctively understood?
(oops, hit the "Comment" button accidentally; continuing)
There is a cargo vendor, could we use that?
FWIW,
pkg/pillar/vendor/is 106MB:$ du -s -h pkg/pillar/vendor/ 106M pkg/pillar/vendor/I guess that this is not an issue with rust, it is an issue in general. We should solve it for this repo for rust, but it is no different than what we do for golang.
In that case, the 33MB should not stop us, if the
vendor/in all the others did not.Question: can we avoid the ugly
.tar.gz, and instead use vendoring, like we do with other languages? To keep it consistent and instinctively understood?(oops, hit the "Comment" button accidentally; continuing)
There is a
cargo vendor, could we use that?
It is 328MB uncompressed. I can do that if the size does not matter.
It is 328MB uncompressed. I can do that if the size does not matter.
Git does its own compression on transmission. I think 33MB matters and 328MB matters. Consistent approach matters more. But why is it so big?
@deitch though of a way to remove vendor.tar.gz and add vendor dir instead. So, we chose that approach for consistency.
Drat, that same yetus permissions issue. I think @yash-zededa had figured it out once before?
Getting spdx errors on all of the vendored files. Can you add pkg/installer/vendor/ to .spdxignore?