[OSSPlanet] Jenkins Project Application for Mirror Plan

[dduportal]

• Applicant: Jenkins (https://www.jenkins.io/)

• Project Info: The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project. Backed by the CDF foundation.

• Contact: Jenkins Infrastructure team

  • Mail: [email protected]
  • IM room (Element Matrix): https://www.jenkins.io/chat/#jenkins-inframatrix-org
  • Website: https://www.jenkins.io/projects/infrastructure/
  • GitHub: https://github.com/jenkins-infra

• Who I am: Damien DUPORTAL, current Jenkins Infrastructure elected Officer, technical infrastructure leader.

• Verification:

  • Originating issue from jenkins-infra GitHub organization: https://github.com/jenkins-infra/helpdesk/issues/4159

• Resource you need: we are applying for a mirror for the download service hosted at get.jenkins.io which uses etix/mirrorbits as a mirror redirector system. As such, we would need:

  • An HTTPS public service which serves the content mirrored from rsync://ftp-nyc.osuosl.org/jenkins/ or rsync://ftp-chi.osuosl.org/jenkins/ (OSUOSL has been a huge sponsor of the Jenkins project for years) updated at regular intervals (existing mirrors are syncing between every 1 hour and every 1 day)
  • We recommend disk space of 750 Gb, but usually it is closer to 250 Gb (as OSUOSL does garbage collect artifacts older than 2 years)
  • A rsync OR FTP service, which can be restricted by IP from https://reports.jenkins.io/infrastructure/v2/index.json (jq -r '.["get.jenkins.io"].outbound_ips) to allow our etix/mirrorbits system to scan the mirror at regular interval

• Ability to promote OSSPlanet: we will add blog post and mentions on the www.jenkins.io website, and in the public GitHub tracker github.com/jenkins-infra.

[bluet]

Hello @dduportal , glad to help.

I'm testing the rsync speed, and it doesn't seem well for now (only few MB per second). Do you have any speed limits currently?

[sahilister]

Rsync speeds from OSL are slower for my IN mirror as well. I rsync from rsync://ftp.halifax.rwth-aachen.de/jenkins/ (DE) or you can sync from rsync://mirror.freedif.org/jenkins/ (SG).

[dduportal]

Hello @dduportal , glad to help.

I'm testing the rsync speed, and it doesn't seem well for now (only few MB per second). Do you have any speed limits currently?

Hello, many thanks for your answer!

We also do have issues when rsync to OSUOSL on each Jenkins Core release (weekly) and I should contact them.

As pointed by @sahilister you can use rsync://ftp.halifax.rwth-aachen.de/jenkins/ (DE) or rsync://mirror.freedif.org/jenkins/ (SG) which are, currently, official mirrors (Aachen University has been a long time sponsor).

The Jenkins infra team will work, on our own, to define a reference rsync mirror instead of OSUOSL in the upcoming months: we'll let you know.

[dduportal]

Hi @bluet ! Thanks for running OSSPlanet!

Any news about this issue?

[bluet]

Thanks @dduportal . It's up and running at http://mirror.ossplanet.net/jenkins/ . Sorry I forgot to update here.

[dduportal]

Thanks @dduportal . It's up and running at http://mirror.ossplanet.net/jenkins/ . Sorry I forgot to update here.

Hi @bluet many thanks!

The HTTPS server is working as expected, but I can't check the rsync service:

rsync -av rsync://mirror.ossplanet.net         

almalinux       AlmaLinux
anthonos        Anthon OS Linux
apache-dist     apache-dist
armbian         armbian packages and releases
CPAN            Comprehensive Perl Archive Network
crystal-linux   Crystal Linux
debian-multimedia       debian-multimedia Project
deepin          Linux Deepin Archive Repository
deepin-cd       Linux Deepin ISO
download.xpud.org       A Browser OS with an App Store.
dvwa            DVWA - Damn Vulnerable Web Application
fdroid          F-Droid - Free and Open Source Android App Repository
FreeBSD         The FreeBSD Project
gnome           GNOME Desktop Environment
gnu             The Free Software Foundation (The GNU Project)
gutenberg       Project Gutenberg: Free eBooks
kali            Kali Linux Archive
kali-images     Kali Linux ISO images
LDP             The Linux Documentation Project
linuxmint       Linux Mint
linuxmint-iso   Linux Mint ISO
mariadb         MariaDB
mirror          All mirror archives
msys2           MSYS2 - Software Distribution and Building Platform for Windows
nongnu          NonGNU
peppermint      Peppermint OS
qtproject       Qt Project
raspbian        Raspbian for RaspberryPi
raspbian-downloads
regolith        Regolith Linux
rockylinux      Rocky Linux
ubuntu          Ubuntu Archive Repository
ubuntu-cdimage  cdimage.ubuntu.com
ubuntu-ports    Ubuntu Ports
ubuntu-releases releases.ubuntu.com

Is there a sync time that I should wait for?

[bluet]

@dduportal just added it into rsyncd.conf :+1: please check again

[dduportal]

@dduportal just added it into rsyncd.conf 👍 please check again

Thanks! Looks like it is working well:

rsync -a rsync://mirror.ossplanet.net/jenkins/TIME

-rw-r--r--             11 2024/09/06 16:57:28 TIME