openvpn3-linux icon indicating copy to clipboard operation
openvpn3-linux copied to clipboard

Pre-built binary for openvpn3 client for the new stable release of Debian (version 12 codename bookworm)

Open jim-barber-he opened this issue 2 years ago • 72 comments

The latest Debian stable release came out on the 10th of June 2023. Currently there is no pre-built openvpn3 Debian package built for the bookworm distribution. This is a request for adding a new package to support this since the other .deb packages have dependencies on packages that are too old.

jim-barber-he avatar Jun 18 '23 03:06 jim-barber-he

I contacted support about this and they told me it's planned for the v21 release. I would love to have a timeline for when this will be available if possible.

mschlachter avatar Jun 22 '23 12:06 mschlachter

This is not a promise. But I do hope I can manage to have a v21 release out by end of August/early September. This will definitely include Debian 12 builds. It is holiday season now, so not much will happen in July.

dsommers avatar Jul 03 '23 10:07 dsommers

Hey @dsommers do you have an update on the release timeline? This would be a lifesaver.

samiramer avatar Aug 31 '23 03:08 samiramer

Related to #201 and #171

samiramer avatar Aug 31 '23 03:08 samiramer

The v21 release has been delayed due to work related to solve #171 as well as some issues reported in the Core library by our internal QA. Hopefully a v21 for stable distribution releases (aka LTS distros, Debian, Ubuntu LTS, RHEL/CentOS) will be ready for internal QA within 2 weeks. Since #171 has now been taken out of the v21 release, I'm already wrapping up the last outstanding issues to get a proper build and packaging done. Once QA has approved the release, it will be shipped.

The v21 release will ship with an updated OpenVPN 3 Core library, which is currently going through QA now. Even though issues found by QA in this library has so far had a very small impact on OpenVPN 3 Linux (it is also used in OpenVPN Connect), there are a still a few potential issues which might affect this project as well.

dsommers avatar Aug 31 '23 11:08 dsommers

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=904044

bhack avatar Sep 08 '23 21:09 bhack

@dsommers any further work on v21? I can muddle through with old openvpn2 client, but the v3 would be great to be able to use.

Drizzt321 avatar Sep 19 '23 21:09 Drizzt321

@Drizzt321 The v21 is going through QA these days. If all goes well, end of the month is the main goal for the final release.

I'm pushing out fixes needed for this release in the releaseprep/v21 branch.

dsommers avatar Sep 20 '23 10:09 dsommers

@mschlachter @jim-barber-he @samiramer @Drizzt321 I've attached here the latest QA build for Debian 12. Feel free to test it if you want. Any feedback would be valuable.

openvpn3_0-20230919git6361bd5-1+bookworm_amd64.deb.gz

Since GitHub is picky about such attachments, you need to gunzip this file before running:

 apt install ./openvpn3_0-20230919git6361bd5-1+bookworm_amd64.deb

dsommers avatar Sep 20 '23 10:09 dsommers

@mleeman is also preparing a regular/official Debian package at: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=904044#27

bhack avatar Sep 20 '23 12:09 bhack

Dang, busted :-P

I've added some comments in the ticket, the highlights are:

  1. The debian builders do not like git submodules (no network access). In order to avoid having to recompress the releases; I decided to make 2 packages: openvpn3 (library) and openvpn3-linux (client). The package names match the repository names; this allows easy maintenance with git-buildpackage when a new release is tagged
  2. I have added pkg-config support in openvpn3 for better support in the client compilation.
  3. Minor patches are applied to avoid git and git submodule checkout/update
  4. I need to add a binary conflict to openvpn3 to avoid problems with ppl that have both repos enabled

These should work for amd64/bookworm or newer; I have a compilation issue with bullseye that I need to figure out (not much of a C++ expert, got stuck at C/glib myself).

/usr//include/openvpn/client/ovpncli.hpp:571:5: warning: inline variables are only available with ‘-std=c++17’ or ‘-std=gnu++17’

i386 compilation is also failing, didn't look into that one either.

Just checkout master and run fakeroot dpkg-buildpackage --no-sign; sudo debi

Feel free to comment, review https://salsa.debian.org/televic-team/openvpn3 https://salsa.debian.org/televic-team/openvpn3-linux

mleeman avatar Sep 20 '23 13:09 mleeman

@mleeman That's really cool you're looking into this!

In regards to i386 ... There are some issues on some of the C++ templates converting between C++ data types to D-Bus data types which is not really happy on 32-bit platforms. Since the wast majority of users are on 64-bit platforms, this has not been a high priority to resolve.

The documentation need to be updated also, the minimum C++ standard now will be C++17; this is due to the OpenVPN 3 Core library moved to that a while back. Further, the glib2 refactoring for #171 will also require C++17.

In regards to source code, we will always publish source code as tarballs as well; which includes everything needed to build with ./configure. The v20 is pushed out here: https://swupdate.openvpn.net/community/releases/openvpn3-linux-20.tar.xz https://swupdate.openvpn.net/community/releases/openvpn3-linux-20.tar.xz.asc

If you want, I can provide a similar development source tarball for the binary posted a bit earlier today.

Please reach out to me if you see any changes on our end to make the packaging better and/or smoother. Ideally we should not need to ship our own builds as a third-party repos. And this goes both for potential packaging efforts of the OpenVPN 3 Core library as well as the OpenVPN 3 Linux project.

In regards to packaging, lintian will complain about D-Bus policies being too broad. I discussed a missing feature with the upstream "dbus-daemon" developers some years ago, and got acceptance for the send_destination_prefix support. IIRC, that arrived in dbus-daemon-1.14, but I'm uncertain about dbus-broker support. This is basically the main detail to get rid of the lintian complaints.

dsommers avatar Sep 20 '23 16:09 dsommers

To me, it does not really matter; packaging follows (as much as possible) upstream.

I have mirrored the packaging structure on the git repo; so if you tag your release in github, I can fetch the tarball from there.

$ cat debian/watch 
version=4
opts=filenamemangle=s/.+\/v?(\d\S+)\.tar\.gz/openvpn3-$1\.tar\.gz/ \
  https://github.com/openvpn/openvpn3/tags .*/v?(\d\S+)@ARCHIVE_EXT@

If you prefer to have full/integrated tarballs that you release on your website, I can change the watch files to match.

I have indeed added a number of lintian-overrides (needs to be reviewed over time), but we have a working solution atm.

As for patches, I'll send the ones that are valuable to the mailing list once the dust settles.

mleeman avatar Sep 21 '23 13:09 mleeman

FYI, there is some low hanging fruit (spelling errors) that got picked up by the debian tooling

https://mentors.debian.net/package/openvpn3-linux/ https://mentors.debian.net/package/openvpn3/

    authentification authentication [usr/bin/ovpncli]
    authentification authentication [usr/lib/x86_64-linux-gnu/_ovpncli.so]
    laoding loading [usr/bin/ovpncli]
    laoding loading [usr/lib/x86_64-linux-gnu/_ovpncli.so]

and

    Adress Address [usr/libexec/openvpn3-linux/openvpn3-service-netcfg]
    Transfered Transferred [usr/libexec/openvpn3-linux/openvpn3-service-configmgr]
    Transfered Transferred [usr/libexec/openvpn3-linux/openvpn3-service-sessionmgr]
    Unkown Unknown [usr/sbin/openvpn3-admin]
    authentification authentication [usr/bin/openvpn3]
    authentification authentication [usr/libexec/openvpn3-linux/openvpn3-service-client]
    laoding loading [usr/bin/openvpn3]
    laoding loading [usr/libexec/openvpn3-linux/openvpn3-service-aws]
    laoding loading [usr/libexec/openvpn3-linux/openvpn3-service-client]
    occured occurred [usr/sbin/openvpn3-admin]
    subsciber subscriber [usr/bin/openvpn3]
    subsciber subscriber [usr/sbin/openvpn3-admin]
    unkown unknown [usr/libexec/openvpn3-linux/openvpn3-service-client]

mleeman avatar Sep 22 '23 09:09 mleeman

  • Source tarball While I see the simplicity of pulling the tarball from github, it is not a proper distribution tarball; it's a git snapshot not including any dependencies. The tarballs we publish will have all dependencies and those tarballs are what is being used across all our builds - including Fedora Copr builds (and later upstream Fedora packaging). These tarballs are the one used through our QA processes and should also be prepared to comply to reproducible builds requirements; that the built binary will be identical - as long as the distribution and compiler chain is the same. And the authenticity of these tarballs can be verified through our PGP signature.

  • Packaging of OpenVPN 3 Core library is dubious It is primarily a source header-only library. While it can produce a libopenvpn3.so in some cases; that is only used for clients not written in C++ where a glue layer is needed - and it depends on swig to work. OpenVPN 3 Linux does not depend on a shared library (and probably won't go that path). Packaging OpenVPN 3 Core library should probably just be a -dev package with only the header files from ./client, ./doc and ./openvpn.

  • I'll look at all the typos and such; that should be fine to get fixed.

dsommers avatar Sep 22 '23 10:09 dsommers

@mleeman Two quick patches fixing the openvpn3-linux typ0s ...

  • https://codeberg.org/OpenVPN/openvpn3-linux/commit/2755c95728cae3bfe347c4153d6d4f959d02a4ca
  • https://codeberg.org/OpenVPN/openvpn3-linux/commit/8881bf35c835c7deab439c87a7efc09444caef84

These are queued up for the coming v21 release.

dsommers avatar Sep 22 '23 10:09 dsommers

fine with me, I will adjust the client to use your published snapshots.

Is there a page where the index can be queried of the releases, any other URL but https://swupdate.openvpn.net/community/releases/openvpn3-linux-21.tar.xz does not seem to work.

the openvpn maintainers use this page:

https://openvpn.net/index.php/open-source/downloads.html 

But openvpn3 is not listed there.

$ cat Development/salsa/openvpn/debian/watch 
version=4
https://openvpn.net/index.php/open-source/downloads.html \
(?:|.*/)openvpn(?:[_\-]v?|)(\d[^\s/]*)\.(?:tar\.xz|txz|tar\.bz2|tbz2|tar\.gz|tgz)

mleeman avatar Sep 22 '23 10:09 mleeman

@mleeman Good point on the publicity aspects of new releases. We will look into how to do that better. Perhaps a quick solution for now would be to have use this file I just now pushed out: https://swupdate.openvpn.net/community/releases/openvpn3-linux.latest

If you want a different layout here, we can probably script that a bit better.

The version scheme is also fairly simple. I'm going to use three "release categories":

  • stable releases: v20 - openvpn3-linux-20.tar.xz{,.asc}
  • beta releases: v19_beta - openvpn3-linux-19_beta.tar.xz{,.asc}
  • development releases: v22_dev - openvpn3-linux-22_dev.tar.xz{,.asc}

Stable releases will be the only releases which might have a minor version, like v20.1. This is only happening where the next stable release is "too far away" and there is an important bug or security fix needed. In this case, the version strings will be:

  • stable hotfix: v20.1 - openvpn3-linux-20.1.tar.xz{,.asc}

This openvpn3-linux.latest could perhaps be extended to have a "release category" tag as well. Like stable:20 and development:22_dev. With a "stable hotfix", the stable: line would be stable:20.1.

In a Debian and Ubuntu LTS context, I think it would be best stay on the stable releases in the longer run. For Ubuntu non-LTS, the beta releases might be applicable. The development builds might be best being a separate repository we update more frequently.

Let me know what you think about this.

dsommers avatar Sep 22 '23 16:09 dsommers

  • Starting from your snapshot and by removing all the patches (that I had previously added) gives a clean compilation, so your release tarball is an excellent starting point. If you prefer this route, then this is the one we'll be taking. It's easy and it's clean.
  • I agree that only the stable releases should come in Debian, it's fine if someone uses the build infrastructure to experiment, but that is outside of the scope of Debian itself.
  • One thing I was thinking about over the weekend is the name of the package. At the moment it is openvpn3 or openvpn3-linux. When comparing this to the openvpn2 releases, a better name might be openvpn3-client, since (AFAIK) it only contains the client part. Nothing changes at your end, just on mine.

Let me know what you prefer and I'll reset my salsa branches to your to the release tarball instead of the one generated from github.

mleeman avatar Sep 25 '23 14:09 mleeman

Regarding 1), that sounds like the best approach to me too; I generally would like package maintainers to add as few additional patches as possible - get them upstream first. (I'm also a Fedora package maintainer, so I have some packaging experience as well)

We are aligned on the "stable" versions in Debian, so nothing more to discuss there.

Packaging names ... okay, so this is "complicated". But it is related to some project names:

  • "OpenVPN 3 Linux" (openvpn3-linux) is this project; the (primarily, but not necessarily restricted to) Linux implementation of the "OpenVPN 3 generation"
  • "OpenVPN 3 Core Library" (openvpn3) is the implementation of the OpenVPN protocol as the 3rd generation.

Currently the openvpn3-linux project has only implemented the client side - but I would like to have a server side implementation too in the future. In the Fedora packaging, there are more sub-packages:

  • openvpn3 Provides the generic components; configuration manager (openvpn3-service-configmgr), log service (openvpn3-service-logger), network configuration (openvpn3-service-netcfg), the Python module [1], the openvpn3and openvpn3-admin CLI tools, the openvpn3-autoload (deprecated, but will be shipped at least until next summer - RHEL 7 EOL), as well as some support files for all of this and some docs.
  • openvpn3-client Provides the client side components: The client services (openvpn3-service-backendstart, openvpn3-service-client), the openvpn2 and openvpn3-as CLI tools and the openvpn3-systemd helper and the [email protected] systemd unit, plus other related support files, man pages and such
  • openvpn3-addon-aws - Contains an optional add-on component (for AWS-VPC integration), provides the openvpn3-service-aws and related support files and man page
  • openvpn3-selinux SELinux policy to confine the some of the services and allow D-Bus daemon to pass access to the tun/dco interfaces between two D-Bus enabled processes.
  • openvpn3-devel Contains some generated files which is installed under /usr/include/openvpn3 ... and they are not related to the Core library itself.

The intension behind these package names is that

  • openvpn3 itself is what is needed to run either a client or a server functionality. It does not relate to the OpenVPN 3 Core library (header) files, as that is not an end-user functionality.
  • openvpn3-client need to depend on openvpn3, since that component builds on the openvpn3 basic functionality to provide the client functionality. This is the same rationale as for the openvpn3-addon-aws package.
  • When a server functionality will come, that would ideally be an openvpn3-server sub-package; which can be installed without the openvpn3-client package installed - and vice versa.

If we would package the OpenVPN 3 Core library, that should be named libopenvpn3-devel or something like that. It does not provide an end-user product, but a library to implement the OpenVPN protocol. This code does contain a reference test client, but that is not something you would put into production as is.

The reason the Debian packaging does not have such splits as the Fedora is that I heard from a Debian maintainer many years ago that Debian generally prefers to avoid sub-packages and only use them when strictly needed. In the Fedora land, sub-packages are more often used to split out functionality which does not need to be installed to have something working. The -selinux package is also more a common way how to clearly indicate that this project will touch the SELinux policy on the system.

But you are of course free to do package this project in Debian as you see fit best with the Debian packaging guidelines.

dsommers avatar Sep 25 '23 14:09 dsommers

Debian packaging (in general) does tend to use smaller sub packages. What I would propose is to:

  1. start by packaging openvpn3-linux into a package openvpn3-client. It describes clearly what it is: a client for openvpn3 functionality. For starters, I'd bundle all the tools that are shipped with openvpn3-linux.
  2. it's great that there is a server component coming. At that point a bit more work will be needed; though I think that the starting point will probably not be the tarball we discussed before; but a number of different sub releases. When this is the case, we'll probably have packages with names like: libopenvpn3-dev (including the version to avoid conflicting with /usr/include/openvpn) and libopenvpn3 that are dependencies for openvpn3-client and openvpn3-server. To some extent, this was they way I started working; but in retrospect (and until there is a server component,) it is probably overkill.

This way, the packaging names match to some extent the ones you defined for rpm packaging. As the need arises and if it is useful, I can split off components into different packages. At the moment, I would focus on getting openvpn3 into Debian/Ubuntu as is, while staying very close to your releases: it give you a much better level of control on what you want to release and in what form.

How does this sound?

mleeman avatar Sep 26 '23 11:09 mleeman

That sounds great! Currently the packages (and docs) in our provided releases uses the openvpn3 package name. There are also some docs for CloudConnexa which also needs to be updated.

Perhaps we should move our (OpenVPN Inc provided) packaging to also use the openvpn3-client with a "Provides" openvpn3 for a short time as we move forward? It would be great if the packages we provide will be compliant with the upstream Debian ones and not causing conflicts in any way.

dsommers avatar Sep 26 '23 14:09 dsommers

In openvpn2 we use the official Debian/Ubuntu packages as base and update from that as needed for the Community releases. Once there is official Debian packaging for openvpn3 we should probably go to a similar model.

flichtenheld avatar Sep 26 '23 15:09 flichtenheld

I have an initial version, most of the code could be re-used

https://salsa.debian.org/televic-team/openvpn3-client

One lintian error popped up that will probably require me to re-package the code:

E: openvpn3-client source: license-problem-convert-utf-code [openvpn3-core/openvpn/common/unicode-impl.hpp]
N: 
N:   The following file source files include material under a non-free license from Unicode Inc. Therefore, it is not possible to ship this in main or contrib.
N:   
N:   This license does not grant any permission to modify the files (thus failing DFSG#3). Moreover, the license grant seems to attempt to restrict use to "products supporting the Unicode
N:   Standard" (thus failing DFSG#6).
N:   
N:   In this case a solution is to use libicu and to remove this code by repacking.
N: 
N:   Please refer to Bug#823100 for details.
N: 
N:   Visibility: error
N:   Show-Always: no
N:   Check: cruft
N: 
N:

Another one I need to look into, seems to be some data that is used during a test

E: openvpn3-client source: source-is-missing [openvpn3-core/test/unittests/comp-testdata/sum]

mleeman avatar Sep 27 '23 10:09 mleeman

We need to come up with a solution for that license issue in the OpenVPN 3 Core library. It's essentially a public domain license, which shouldn't really be a problem - but legalese is always annoying.

that comp-testdata/sum is essentially just a random binary which is only used for compression testing in the Core library unit tests. That file should also be replaced by some real random binary data and not a binary executable (which in this case is even for the Sparc CPU arch).

Both of these issues are coming from the OpenVPN 3 Core library.

dsommers avatar Sep 27 '23 11:09 dsommers

I am searching a bit and have found e.g.

https://llvm.org/doxygen/ConvertUTF_8h_source.html

it seems to be the same source and llvm is in main. The warning states:

N:   This license does not grant any permission to modify the files (thus failing DFSG#3). Moreover, the license grant seems to attempt to restrict use to "products supporting the Unicode
N:   Standard" (thus failing DFSG#6).

while the header file says (copy):

17  * Unicode, Inc. hereby grants the right to freely use the information
 18  * supplied in this file in the creation of products supporting the
 19  * Unicode Standard, and to make copies of this file in any form
 20  * for internal or external distribution as long as this notice
 21  * remains attached.

The file in llvm has a more recent copyright:

/*===--- ConvertUTF.h - Universal Character Names conversions ---------------===
 *
 * Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 * See https://llvm.org/LICENSE.txt for license information.
 * SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 *
 *==------------------------------------------------------------------------==*/
/*
 * Copyright © 1991-2015 Unicode, Inc. All rights reserved.
 * Distributed under the Terms of Use in
 * http://www.unicode.org/copyright.html.
 *
 * Permission is hereby granted, free of charge, to any person obtaining
 * a copy of the Unicode data files and any associated documentation
 * (the "Data Files") or Unicode software and any associated documentation
 * (the "Software") to deal in the Data Files or Software
 * without restriction, including without limitation the rights to use,
 * copy, modify, merge, publish, distribute, and/or sell copies of
 * the Data Files or Software, and to permit persons to whom the Data Files
 * or Software are furnished to do so, provided that
 * (a) this copyright and permission notice appear with all copies
 * of the Data Files or Software,
 * (b) this copyright and permission notice appear in associated
 * documentation, and
 * (c) there is clear notice in each modified Data File or in the Software
 * as well as in the documentation associated with the Data File(s) or
 * Software that the data or software has been modified.
 *
 * THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
 * ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
 * WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 * NONINFRINGEMENT OF THIRD PARTY RIGHTS.
 * IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS
 * NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL
 * DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
 * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
 * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 * PERFORMANCE OF THE DATA FILES OR SOFTWARE.
 *
 * Except as contained in this notice, the name of a copyright holder
 * shall not be used in advertising or otherwise to promote the sale,
 * use or other dealings in these Data Files or Software without prior
 * written authorization of the copyright holder.
 */

Maybe there has been an update since the 2001-2004 version that is in the core lib?

Another reference: https://sources.debian.org/src/desmume/0.9.11-2/src/utils/ConvertUTF.h/

mleeman avatar Sep 27 '23 11:09 mleeman

Good find! Thanks a lot for your research! Will dig into this and find a better licensed version.

This is anyhow something needed to be tackled in the openvpn3 repository and will need to come in the next OpenVPN 3 Core release before pulling it into this (openvpn3-linux) project.

dsommers avatar Sep 27 '23 11:09 dsommers

A bit unrelated: do you think that this package is going to conflict with network manager openvpn in distros? See https://github.com/OpenVPN/openvpn3-linux/issues/46

bhack avatar Sep 28 '23 11:09 bhack

I personally hope that NetworkManager won't use openvpn3 because it doesn't support fragment, which is mandatory with certain providers that tunnel traffic thus breaking PMTUD: https://forum.fibra.click/d/44630-lista-nera-dei-provider-che-multiplexano-le-sessioni-ppp-via-tunnel-l2tp

darkbasic avatar Sep 28 '23 13:09 darkbasic

A bit unrelated: do you think that this package is going to conflict with network manager openvpn in distros? See #46

@bhack No, it will not conflict with NetworkManager. That's a different set of package names with different package dependencies. A NetworkManager-openvpn3 add-on is wanted and requested by many, and that will use a very different integration than the approach the current one does.

dsommers avatar Sep 28 '23 15:09 dsommers