lime-packages
lime-packages copied to clipboard
libremesh
Do a new release (2020.1)
I have been thinking that as LibreMesh being a framework for building firmwares of many flavours we can do source only releases. Binary releases are very opinionated (routing algorithm, which packages, etc) so I don't know if providing binaries with a fixed selection is very useful in reality. What if we provide source releases for the time being? We will be able to point to the "chef" options available for further flavouring/customization and also to the building from source docs as we have been doing.
I believe that doing so we can get a release out with a lot less work and that it still is useful for most people. We will be having most of the upsides of having releases, remarcably:
- let eveyone know about the development news and improvements
- upgrading docs (from a realease to a release)
- simpler triaging and priorization of bugs / issues
This proposal does not imply that we won't be doing binary releases in the future if we want and we are able to do it.
What do you think?
I agree that a new release is needed soon. It's important to keep short iterations to move on :bike: , and be more on sync with openwrt. Your proposal makes the release decision reasonable less heavy. In fact, is more a release for devs / technical people, that are connected with community networks - users. Ideally, would be nice to have binaries, but it's more important to move on.
As this is something new-strange for the new release I would note that: this release is a source-only release and here are the docs to compile it yourself
A source release is much better than nothing :) And if we have a Chef-like convenient tool to build the binaries, that would be 100% perfect!
Regarding the fact that many possible combinations of packages are possible in LibreMesh: that's true but if we look at the routing algorithms, only the Batman-adv+Babeld combination is being officially supported and tested (as far as I know, please let me know if someone is testing other combinations!). For this reason I believe that most of the users simply use the standard packages set.
Some time ago I started triaging issues for the next release, in case you wanted to use or expand this list: https://github.com/orgs/libremesh/projects/2
I think the idea of releases is great, particularly in the networks I follow, since I started to adopt the releases as a standard has changed a lot the quality of the network and I can have more control than this and this has impacted the lives of many people. Thanks for doing it
A source release is much better than nothing :) And if we have a Chef-like convenient tool to build the binaries, that would be 100% perfect!
Regarding the fact that many possible combinations of packages are possible in LibreMesh: that's true but if we look at the routing algorithms, only the Batman-adv+Babeld combination is being officially supported and tested (as far as I know, please let me know if someone is testing other combinations!). For this reason I believe that most of the users simply use the standard packages set.
Some time ago I started triaging issues for the next release, in case you wanted to use or expand this list: https://github.com/orgs/libremesh/projects/2
Great, from the list there are two issues marked as blockers:
I think that #327 is not a release blocker as it has many subitems that are not necesarly all blocker, many ara already done, some not yet, but it will take time. So if anyone wants what they have before they can still use luci.
About #280: now that we have #700 and #701 and that firstbootwizard is not recommended by defaults I would say that it is not a blocker.
What do you think?
What we may be good to have is a recommended way to cook the firmwares. I lost track of what it is currently working. @aparcar can you bring some light ? :)
Hi, the idea is fine for me, ideally we come up with a stable and reliable CI that builds all packages and thereby provide at least binary packages, which are needed any Chef like tool.
I just recently realized that ImageBuilders use HTTP for package downloads and do not perform any signature verification, meaning all images ever created with Chef are somewhat garbage due to the possibility of a drive-by-download-attack. I proposed fixes upstream, until then I recommended to not use Chef (or ImageBuilders).
We should certainly wait for 20.x to be released and give it maybe a month to settle. Please do not do any release before 20.x!
Regarding the blockers, both are fine for me to be removed. @dangowrt comments?
We should certainly wait for 20.x to be released and give it maybe a month to settle. Please do not do any release before 20.x!
Given that this will be a source only release not tied to a specific openwrt revision why do you say to wait for Openwrt 20.x? Maybe we can wait to the openwrt 20.x release for the documentation about chef/imagebuilders part?
I just recently realized that ImageBuilders use HTTP for package downloads and do not perform any signature verification
There is a mitigation that is to build the entire imagebuilder, that could work even offline, the option is: CONFIG_IB_STANDALONE=y. I use offline imagebuilders extensively. I also feel this is more network efficient (for your server bandwidth and also to free resources from the openwrt side). That also helps you to make releases of particular imagebuilders. As they look more "static/finished", then it is easier to inspect and find consistency problems. I don't know if there is some script that could do a more static verification.
I hope that frees the blocker of "waiting for the 20.x release"
@aparcar what do you think?
I am thinking in tagging a new release next week, on Tue 22 if no critcal bug arises. As the LiMe release don't need to be tied to any openwrt release and we always can do another release soon I think that waiting for the OpenWrt 20.x release and then waiting for things to settle is not giving the release more value. I prefer to do releases more often, have the changelog and news and updated docs wheel moving. When we have the chef/IM support we can provide the "binary helpers", we don't need to wait for them.
Sounds good. What if we tag a release candidate, make a call for testing also on 19.07 (until now the testing was very limited on this front) and set the release tagging to November the first?
Sounds good. What if we tag a release candidate, make a call for testing also on 19.07 (until now the testing was very limited on this front) and set the release tagging to November the first?
Great! So, I taged the last commit as rc1: https://github.com/libremesh/lime-packages/releases/tag/v20.09-rc.1 I've created also a 20.09 branch so we can continue development in the master branch and backport fixes only to the 20.09 release :)
Now we have 40 days to update docs, news, changelog of only ~1200 commits!
Related https://github.com/libremesh/lime-packages/pull/728#issuecomment-697971466
So I updated the CI PR and now whenever a release/tag is set a new folder appears on my storage server, we should however figure out a destination which is community owned. We could also use GitLab/GitHub pages for that.
https://images.aparcar.org/minio/libremesh/v20.09-rc.1/
Hi! I've cherry-picked bug fixes and some other commits from master, but not all, and tagged 20.09-rc.2. Some enhancements from master that are not very well tested are not included. Also the libremesh.mk refactoring is not included as there are open PRs and some open questions. Correct me if I am wrong but it should not change functionality nor fix any user noticiable bug so we can try to settle on the matter but if we can't in the following 2 weeks I would ship it as is.
I've cherry-picked bug fixes and some other commits from master, but not all, and tagged 20.09-rc.2. Some enhancements from master that are not very well tested are not included.
Thanks @spiccinini for the superwork!
I'd like to see the libremesh.mk PR merged before the final release.
@aparcar do you want to argument this? Do you think this needs to be included in the 20.09 release? I have a very neutral opinion, both options are good for me.
I checked a bit the commits list which are in master and not in v20.09-rc.2:
- Add missing include from merge (libremesh.mk related)
- babeld wired option is deprecated (I would include this in the release, it will remove alarming warnings)
- check-internet: use uclient-fetch instead of wget (tested enough to be included, I just tested again and it works)
- fbw: do not scann networks on boot
- fbw: minnor improvements
- firstbootwizard: add dismiss feature
- firstbootwizard: do not use intermediate file for dismiss
- firstbootwizard: force uci cache reload
- libremesh.mk: Add package configuration file (libremesh.mk related)
- libremesh.mk: allow change of SECTION/CATEGORY (libremesh.mk related)
- lime-proto-babeld: Use libremesh.mk (libremesh.mk related)
- lime-proto-batadv: Use libremesh.mk (libremesh.mk related)
- lime-proto-bmx6: Use libremesh.mk (libremesh.mk related)
- lime-proto-bmx7: Use libremesh.mk (libremesh.mk related)
- lime-proto-wan: Use libremesh.mk (libremesh.mk related)
- lime-smart-wifi: Use libremesh.mk (libremesh.mk related)
- Specify the location of cotonete code (ok, it's a bit useless to have)
We can drop libremesh.mk entirely. It was intended to simplify and unify things, however if it's preferred to keep things heterogeneous, sure.
We can drop libremesh.mk entirely. It was intended to simplify and unify things, however if it's preferred to keep things heterogeneous, sure.
Don't get me wrong: I think libremesh.mk is useful and is good that it has been merged in master. My neutral opinion is about whether to include it in the 20.09 release or not.
Thanks Ilario for spoting which commits are not present!
I checked a bit the commits list which are in master and not in v20.09-rc.2:
* Add missing include from merge (libremesh.mk related) * babeld wired option is deprecated (I would include this in the release, it will remove alarming warnings)
* check-internet: use uclient-fetch instead of wget (tested enough to be included, I just tested again and it works)
Ok +1 I will cherry-pick into 20.09 branch
* fbw: do not scann networks on boot * fbw: minnor improvements * firstbootwizard: add dismiss feature * firstbootwizard: do not use intermediate file for dismiss * firstbootwizard: force uci cache reload
I prefer not to include any of these changes in this release.
* libremesh.mk: Add package configuration file (libremesh.mk related) * libremesh.mk: allow change of SECTION/CATEGORY (libremesh.mk related) * lime-proto-babeld: Use libremesh.mk (libremesh.mk related) * lime-proto-batadv: Use libremesh.mk (libremesh.mk related) * lime-proto-bmx6: Use libremesh.mk (libremesh.mk related) * lime-proto-bmx7: Use libremesh.mk (libremesh.mk related) * lime-proto-wan: Use libremesh.mk (libremesh.mk related) * lime-smart-wifi: Use libremesh.mk (libremesh.mk related) * Specify the location of cotonete code (ok, it's a bit useless to have)
If the packages keep their "LiMe" category, the "LibreMesh" category will still be present and will contain just the network profiles: https://github.com/libremesh/network-profiles/blob/42e4539f08dcf7b0fb49b1753e5bed739429e28a/profile.mk#L35 as they are suggested to include in the feeds: https://github.com/libremesh/lime-web/blob/f1b489229244a15ae99133bb21d7711c542b68d8/development.txt#L83
In my opinion the "LibreMesh" category makes more sense than the "LiMe" category.
I've backported fixes that landed in master in the last days and tagged v20.09-rc.3 https://github.com/libremesh/lime-packages/releases/tag/v20.09-rc.3 If there are no new big issues I will be releasing 20.09 with only modification to the lime-system Makefile (LIME_RELEASE, LIME_CODENAME, etc)
The new release 2020.1 has been officially announced! Now we should "just" update the website, choose a SDK (I propose to fork this one: https://github.com/aparcar/openwrt-metabuilder) and publish some basic binaries based on 18.06.9 and on 19.07.5 on the https://downloads.libremesh.org/ to ease the life to the new users.
The website should be updated now. We don't have a SDK. The firmware for the ath79 target are published on https://downloads.libremesh.org/expansive_emancipation/2020.1/targets/ath79/generic/ I think this is enough for considering this issue closed.