Can't update/upgrade - /var/cache/deb-get/deb-get.json empty

[Xdavius]

Can't update/upgrade - /var/cache/deb-get/deb-get.json empty

can't install with one line install : curl -sL https://raw.githubusercontent.com/wimpysworld/deb-get/main/deb-get | sudo -E bash -s install deb-get

I's not the first time. Sometimes work, sometimes not. Today, i'm blocked. Yesterday everything good. I'ts not the first time it happens to me. Seems like a down server (?)

Expected behavior

Download new content of deb-get.json

Actual behavior

Error - can't use deb-get.

Steps to reproduce

deb-get update deb-get upgrade

System information

Run the following a paste the output below:

/etc/os-release

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

deb-get version

0.3.8

deb-get update

  [+] Updating /etc/deb-get/01-main
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  108k    0  108k    0     0   175k      0 --:--:-- --:--:-- --:--:--  351k
  [+] Updating cache of supported apps in the background
  [+] Updating /var/cache/deb-get/deb-get.json
  [*] WARNING! Updating /var/cache/deb-get/deb-get.json failed.
  [*] WARNING! Cached file /var/cache/deb-get/deb-get.json is empty or missing.

Thank you for this must have tool. Sorry for reporting a trouble.

[github-actions[bot]]

Hello there 👋 Thanks for submitting your first issue to the deb-get project 🐛 We'll try and take a look at your issue soon ⏲

In the meantime you might want to join the Wimpys World Discord 🗣 where we have a large community of Linux 🐧 enthusiasts and passionate open source developers 🧑‍💻

You might also be interested in following Wimpys World Twitch 📡 channel where Wimpy streams let's code video, including this project, several times a week. A back catalog of past live stream and other Linux related content is available on Wimpys World YouTube 📺 channel.

[philclifford]

Thanks for the feedback.

It may be that you haven't set up an API key for github - this is effectively required now that so many repos are supported unless you set a much longer cache retention time (e.g. export DEBGET_CACHE_RTN=240 - minutes, defaults to 60 ) and then repeat the update when you have refreshed the rate-limit allowance. (See https://github.com/wimpysworld/deb-get#github-api-rate-limits ) Another possibility is some temporary outage at github (I've seen a few of those either with github timing out or occasionally with a particular app being mid-workflow of a release and being temporarily "broken" until the release .deb asset is built) . In those cases a little patience and retry later is all I can suggest.

[seefood]

I get it consistently too. initial setup is impossible without the DEBGET_TOKEN set either, it just sits there stuck. may be worth adding a warning that failing to set it will slow things down considerably or fail gracefully.

also, I have an extra error there:

❯ deb-get update
  [+] Updating /etc/deb-get/01-main
tail: cannot open '/01-main.repo' for reading: No such file or directory
No URLs found in /01-main.repo.tmp.
  [+] Updating /var/cache/deb-get/deb-get.json
  [+] Updating cache of supported apps in the background
  [*] WARNING! Updating /var/cache/deb-get/deb-get.json failed.
  [*] WARNING! Cached file /var/cache/deb-get/deb-get.json is empty or missing.

seems $ETC_DIR and $ELEVATE need to be exported before used in the subshell around lines 710-720. I'll make a PR.

[seefood]

Not everything is perfect, but it's a step. what's supposed to be in /etc/deb-get/01-main.repo ? Mine is empty.

[philclifford]

@seefood it is the manifest from 01-main/ ... You only need the first line as update will then fetch a fresh/new copy

https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main

[seefood]

ok, adding that line to the file, running it seems to get stuck for a long time on wget.

❯ bash -x /usr/bin/deb-get update --repos-only
  [....]
+ REPO_URL=https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main
+ sudo wget -q https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main/manifest -O /etc/deb-get/01-main.repo

This never returns, and if I hit ctrl-C, I'l left with 01-main.repo empty again (So this needs a more graceful way to quit)

Why is it hung, well, could be due to my ISP, they don't give good access to IPv6, some destinations are not routed to.

❯ sudo wget -vvv https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main/manifest -O /etc/deb-get/01-main.repo
--2023-07-01 16:24:54--  https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main/manifest
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 2606:50c0:8002::154, 2606:50c0:8001::154, 2606:50c0:8000::154, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8002::154|:443... 

So timeout does the trick for me.

❯ sudo wget -vvv https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main/manifest -O /etc/deb-get/01-main.repo --timeout 5
--2023-07-01 16:27:08--  https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main/manifest
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 2606:50c0:8002::154, 2606:50c0:8003::154, 2606:50c0:8000::154, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8002::154|:443... failed: Connection timed out.
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8003::154|:443... failed: Connection timed out.
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8000::154|:443... failed: Connection timed out.
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8001::154|:443... failed: Connection timed out.
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2948 (2.9K) [text/plain]
Saving to: ‘/etc/deb-get/01-main.repo’

/etc/deb-get/01-main.repo         100%[=============================================================>]   2.88K  --.-KB/s    in 0.001s  

2023-07-01 16:27:28 (3.61 MB/s) - ‘/etc/deb-get/01-main.repo’ saved [2948/2948]

Should I add it in a PR as well?

[philclifford]

Please do. Ideally a separate PR. Thanks!

[FallCheetah7373]

Can confirm happens on my system linux mint 21.1

https://raw.githubusercontent.com/wimpysworld/deb-get/main/01-main

yeah I reinstalled deb-get and still have this issue, I did setup fine grained token.

[yermulnik]

Started to hit the same or similar today?

> deb-get update --quiet
  [+] Updating /etc/deb-get/01-main
  [+] Updating cache of supported apps in the background
  [+] Updating /var/cache/deb-get/deb-get.json
  [*] WARNING! Updating /var/cache/deb-get/deb-get.json failed.
  [*] WARNING! Cached file /var/cache/deb-get/deb-get.json is empty or missing.
[apt refresh packages output]
Fetched 14,4 kB in 16s (898 B/s)
Reading package lists...
  [+] Updating /var/cache/deb-get/deb-get.json
  [*] WARNING! Updating /var/cache/deb-get/deb-get.json failed.
  [*] WARNING! Cached file /var/cache/deb-get/deb-get.json is empty or missing.

> ls -la /var/cache/deb-get/deb-get.json
ls: cannot access '/var/cache/deb-get/deb-get.json': No such file or directory

Any hints?

ps: it also looks like that deb-get leaves /var/cache/deb-get/updating_supported.lock file when it exits unexpectedly. I couldn't repro this, but the lock file was there after failed auto-run from the cron during the night. I tried to look into what's happening, found this dead lock, removed it, but started to encounter the above outlined issue with missing deb-get.json.

[philclifford]

Does your Cron job run as normal user with passwordless sudo and GitHub API environment setup ? If deb-get has been run with sudo or by root this sort of failure can result.

[yermulnik]

It runs as root (from /etc/crontab). Had no issues like this before. It's broadly speaking as simple as this: timeout 15m deb-get update --quiet | fgrep 'has an update pending.' I tried to re-run it just now and the issue has gone with and w/o GH token provided. I'll pass GH token to crontab entry then to see to ensure it doesn't hit API rate limits. Thanks for the hint.

Hmm, now I'm getting this:

  [!] ERROR! Missing required information of github package croc:
URL=
VERSION_PUBLISHED=

It's probably not related, but how do I fix that please?

[philclifford]

Hmm, now I'm getting this:

  [!] ERROR! Missing required information of github package croc:
URL=
VERSION_PUBLISHED=

It's probably not related, but how do I fix that please?

Either drop the fixed package definition from an unmerged PR into /etc/deb-get/99-local.d/ as a temporary override or add an external repo with fixed definitions. See my response https://github.com/wimpysworld/deb-get/issues/1008#issuecomment-1943817070 for details. You might also grab and use the latest version of deb-get from main, but that only helps avoid the problems caused by broken package definitions, not the missing info itself until the corrected definitions are merged or overridden.

[yermulnik]

Thanks. Do I get it right that I can just wait for the get-deb fix to get merged and it will self-update?

[yermulnik]

JFYI: I dropped 91-fixes as suggested in the referred comment, though the below error is still there:

  [!] ERROR! Missing required information of github package croc:
URL=
VERSION_PUBLISHED=

[philclifford]

JFYI: I dropped 91-fixes as suggested in the referred comment, though the below error is still there Did you remove /var/cache/deb-get/croc.json ?

[yermulnik]

Oops, I did indeed not. Re-tried the step (dropping /etc/deb-get/91-fixes.repo file in) altogether with removal of croc.json cache file.

• 1st run of deb-get update --quite gave me [*] WARNING! Updating /var/cache/deb-get/deb-get.json failed. and the same [!] ERROR! Missing required information of github package croc:.
• 2nd run gave me the [!] ERROR! Missing required information of github package croc: only.
• 3rd run went w/o errors.
• 4th run — [*] WARNING! Cannot update cache of supported apps: /var/cache/deb-get/updating_supported.lock found (job still running?) and deb-get is indeed still running in an interruptible sleep state 🤔

  > ps axuww | fgrep deb-get
  root     3188673  0.0  0.0  15268  3224 pts/29   S    22:16   0:00 bash /bin/deb-get update --quiet
  root     3188682  0.2  0.0  15400  3480 pts/29   S    22:16   0:00 bash /bin/deb-get update --quiet
  root     3188685  0.0  0.0  12508  2176 pts/29   S    22:16   0:00 tee /var/cache/deb-get/supported.list.tmp
  root     3197182  0.0  0.0  21536  7552 pts/29   S    22:17   0:00 wget --quiet --no-use-server-timestamps https://code-industry.net/get-master-pdf-editor-for-ubuntu/?download -O /var/cache/deb-get/master-pdf-editor-5.html
  

• Killed the above processes and ran deb-get for the fifth time — ran w/o errors.

[yermulnik]

Thanks. Do I get it right that I can just wait for the get-deb fix to get merged and it will self-update?

I tbh do not want to keep in mind the temp fix that I need to manually remove once new version of deb-get is released, and hence asked the above — any approx timelines on the release? Thanks.