gnome-shell-extension-gsconnect icon indicating copy to clipboard operation
gnome-shell-extension-gsconnect copied to clipboard

journal/syslog floods, exhausting disk space

Open khaeru opened this issue 5 years ago • 21 comments

Describe the bug

Having GSConnect installed and enabled periodically leads to a full disk. As a result, rsyslogd takes up 100% CPU, lots of fan noise and heat, etc.

Inspecting the journal ($ journalctl) yields more than 1 million repetitions of messages like:

Aug 14 21:48:24 khaeru-desktop org.gnome.Shell.Extensions.GSConnect[7184]: == Stack trace for context 0x561a74ee80b0 ==
Aug 14 21:48:24 khaeru-desktop org.gnome.Shell.Extensions.GSConnect[7184]: #0   561a751579a8 i   /home/khaeru/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:744 (7f68dc0851f0 @ 1451)
Aug 14 21:48:24 khaeru-desktop gjs[7642]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Aug 14 21:48:24 khaeru-desktop gjs[7642]: The offending callback was SourceFunc().

…and the same four messages in /var/log/syslog. /var/log/syslog ends up filling the disk (at >10 GB); however the journal (/var/log/journal/*) is also >1 GB, probably only smaller because it's stored as binary rather than plain text.

Steps To Reproduce:

  1. Install & enable GSConnect.
  2. Wait.

Expected behavior Logs are not filled with GSConnect-related error messages. Disk space is not exhausted.

Support Log

The GSConnect settings dialog cannot be opened while the logs are flooding, and so a support log cannot be generated. It is necessary to kill the gjs process (e.g. 7642 in the above messages), after which it seems to be automatically restarted, and the dialog can be opened.

System Details (please complete the following information):

  • GSConnect version: 24
    • Installed from: GNOME Extensions Website
  • GNOME/Shell version: 3.32.2-2ubuntu1~ubuntu19.04.1
  • Distro/Release: Ubuntu 19.04

GSConnect environment (if applicable):

  • Paired Device(s): None; bug occurs when no device is paired.

khaeru avatar Aug 14 '19 19:08 khaeru

1 million, huh? That is... excessive! Wow.

Those areas of the code have been reworked quite a bit since v24 was released. (Ages ago, in gnome-shell-extension years, since it'll be 3 months old on Friday.) With a v25 release beginning to take shape on the horizon, I was wondering if you'd be willing to try installing the v25-rc3 release candidate to see if it clears the problem up?

You can find "manual" upgrade instructions for installation from zipfile on the Wiki's "Installation" page.

ferdnyc avatar Aug 14 '19 20:08 ferdnyc

Thanks, I will try this at the next opportunity and reply with any results.

khaeru avatar Aug 23 '19 11:08 khaeru

Hi, just to report I had the same experience today. My logs grew to over 50GB then my machine pretty much just died (also on V24). The messages were mostly:

Sep  2 00:07:27 home org.gnome.Shell.Extensions.GSConnect[32155]: == Stack trace for context 0x55a6063e60b0 ==
Sep  2 00:07:27 home gjs[32590]: The offending callback was SourceFunc().
Sep  2 00:07:27 home org.gnome.Shell.Extensions.GSConnect[32155]: #0   55a6066560d8 i   /home/james/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:744 (7f8ce50851f0 @ 1451)

JamesSwift avatar Sep 02 '19 23:09 JamesSwift

@JamesSwift , @khaeru

Now that v25 v26 is released, can you confirm whether this issue is still occurring with the updated version?

ferdnyc avatar Sep 10 '19 15:09 ferdnyc

I've had no problems since trying the beta, and now the latest release.

JamesSwift avatar Sep 12 '19 23:09 JamesSwift

Thanks @JamesSwift , glad to hear it!

I'm going to close this, if this same problem occurs (for anyone) with v26, please open a new issue with relevant log messages.

ferdnyc avatar Sep 13 '19 15:09 ferdnyc

GSConnect V37. Ubuntu 20.04

Syslog about 50 gigabytes!

May  5 22:58:38 X550JX org.gnome.Shell.Extensions.GSConnect[3882]: == Stack trace for context 0x5595d63af1a0 ==
May  5 22:58:38 X550JX gjs[3882]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
May  5 22:58:38 X550JX org.gnome.Shell.Extensions.GSConnect[3882]: #0   5595d64607e0 i   /home/nikhil/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:1125 (49cecf8d178 @ 1324)
May  5 22:58:38 X550JX org.gnome.Shell.Extensions.GSConnect[3882]: == Stack trace for context 0x5595d63af1a0 ==
May  5 22:58:38 X550JX gjs[3882]: The offending callback was SourceFunc().

lamyergeier avatar May 05 '20 21:05 lamyergeier

Did you recently upgrade from Ubuntu 18.04? If so you may have to start with a fresh configuration for GSConnect. See this page in the wiki.

andyholmes avatar May 05 '20 23:05 andyholmes

Seeing this right now under Manjaro. I have community/gnome-shell-extension-gsconnect 37-1 installed.

dniku avatar May 18 '20 16:05 dniku

I had this problem today on Ubuntu 20.04 and it had filled up my syslog with 331gb. Full disks on linux isnt pretty and I thought my machine was dead since X wouldnt start. Had to ctrl-alt-F2 to TTY to find and delete the syslog.1 -file in order to get my (x) to boot properly.

I was about to buy a new computer, as I assumed my 4yo computer had died.

If im correct with posting this experience on this issue, which maybe not correct...?

These are the two lines repeating in my logs (331gb): ** The offending callback was SourceFunc(). Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget

**

finnjohnsen avatar Dec 10 '20 09:12 finnjohnsen

Hello, I just had the same issue on a family desktop.. The weird thing was that no one installed anything on it, and I never installed gsconnect on it..

After debugging, turned out that snapd was the cause root, apparently it installed multiple gnome related things by default, and probably did an auto update.. → Don't have an other explanation, the system was installed more than 6 months ago, and no one touched it since.

After disabling all snap related services and emptying the syslog, things are back to normal.

Distro/Release: Ubuntu 20.04

elacheche avatar Dec 29 '20 00:12 elacheche

@finnjohnsen could you post some lines of the actual log? I'm especially interested in what is the interval at which these repeat. Are you sure that GSConnect was causing the log spam, or could it have been something else?

To the original post, one reason the journal is smaller is that it has actual settings for max storage usage as well as limits of how much at a minimum to try and keep free.

daniellandau avatar Dec 29 '20 08:12 daniellandau

Just had the same problem - Ubuntu 20.04.1 LTS

I dual use the computer to 1) run my TV with gnome and 2) run a nextcloud server. I can see from Prometheus that my disk started filling at ~6.30am today and I caught it after it consumed ~135gig at 10.30am - a few hundred meg away from zero disk space.

I've already truncated syslog to keep the server alive, but I've captured the following from the console as I investigated:

Syslog: Jan 3 10:09:30 viper gjs[2613]: The offending callback was SourceFunc(). Jan 3 10:09:30 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked. Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 == Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:729 (56ff1b8d178 @ 963) Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 == Jan 3 10:09:30 viper gjs[2613]: The offending callback was SourceFunc(). Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:729 (56ff1b8d178 @ 963) Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 == Jan 3 10:09:30 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked. Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensionschrichrchris@vicchris@viper:~$ sudo tail /var/log/syslog Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 == Jan 3 10:12:48 viper gjs[2613]: The offending callback was SourceFunc(). Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:729 (56ff1b8d178 @ 963) Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 == Jan 3 10:12:48 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked. Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:729 (56ff1b8d178 @ 963) Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 == Jan 3 10:12:48 viper gjs[2613]: The offending callback was SourceFunc(). Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:729 (56ff1b8d178 @ 963) Jan 3 10:12:48 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs.

Top: chris@viper:~$ top

top - 10:09:59 up 7 days, 12:44, 2 users, load average: 3.45, 2.81, 2.56 Tasks: 282 total, 4 running, 278 sleeping, 0 stopped, 0 zombie %Cpu(s): 35.0 us, 48.6 sy, 0.0 ni, 16.1 id, 0.2 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 7629.2 total, 138.5 free, 1248.8 used, 6242.0 buff/cache MiB Swap: 2048.0 total, 686.9 free, 1361.1 used. 6033.7 avail Mem Unknown command - try 'h' for help PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
885 syslog 20 0 363616 88224 3152 R 154.5 1.1 133:43.08 rsyslogd 306 root 19 -1 543864 307864 306192 R 97.3 3.9 210:01.44 systemd+ 2613 jarvis 20 0 3663072 33164 17788 R 75.1 0.4 151:10.25 gjs
884 prometh+ 20 0 1548828 65104 14132 S 5.3 0.8 31:22.05 prometh+ 883 prometh+ 20 0 1445720 16052 6036 S 1.7 0.2 68:10.26 prometh+ 1128915 www-data 20 0 351944 44860 34980 S 1.0 0.6 5:29.12 php-fpm+ 1218559 root 20 0 1644216 38680 16704 S 1.0 0.5 1:12.83 grafana+ 850 message+ 20 0 9832 6032 3592 S 0.7 0.1 17:28.12 dbus-da+ 11029 www-data 20 0 787056 3704 3328 S 0.3 0.0 17:28.36 loolwsd
1269271 www-data 20 0 2242312 10132 6488 S 0.3 0.1 0:00.57 apache2
1 root 20 0 171552 11552 6876 S 0.0 0.1 11:48.21 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.21 kthreadd 3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_par+ 9 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 mm_perc+ 10 root 20 0 0 0 0 S 0.0 0.0 0:15.70 ksoftir+ 11 root 20 0 0 0 0 I 0.0 0.0 13:02.60 rcu_sch+

ps:

chris@viper:~$ ps ax | grep gsconnect 2613 ? Rl 154:36 gjs /home/jarvis/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js 1305553 pts/0 R+ 0:00 grep --color=auto gsconnect

Resolution: chris@viper:~$ sudo killall gjs chris@viper:/var/log$ sudo truncate syslog --size 0

After killing gjs and performing no other action I can't see any negative impact on the system and everything seems to be running as normal.

chrisharden42 avatar Jan 03 '21 10:01 chrisharden42

I'm sure the daemon probably shouldn't crash at all, but that doesn't look like enough volume to fill >100GB in a matter of a few hours. I'd very much like to help, but so far I have no good idea where to look.

daniellandau avatar Jan 03 '21 20:01 daniellandau

Possibly related: https://gitlab.gnome.org/GNOME/polari/-/issues/148 and https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/1868

sonnyp avatar Feb 20 '21 18:02 sonnyp

I'm also seeing this error, it's intermittent but when it happens my gnome-shell completely freezes. I am running gsconnect, I'll disable it for a little while and see if the crashing goes away.

gregharvey avatar Jul 21 '21 08:07 gregharvey

In my case I don't think it was gsconnect, after more debugging I disabled hardware acceleration in my browser (Brave) and I haven't had a crash since...

gregharvey avatar Jul 22 '21 12:07 gregharvey

Ubuntu 21.04 & GSConnect 45. I had the same issue as the OP (my syslog grew as much as 900 Gb). I needed to uninstall the extension, reboot my computer and install GSConnect again. After that all seems to work ok.

santiagofn avatar Sep 15 '21 02:09 santiagofn

Weirdly, after a few months of this not happening, I'm now experiencing regular desktop freezes with this same message again. Will try removing GSConnect and see what happens...

gregharvey avatar Sep 20 '21 11:09 gregharvey

I've just experienced the same problem. My entire root partition has been filled by /var/log/syslog.

OS: Trisquel Nabia
GNOME Shell: 3.36.9
GSConnect: 53

Logs fragment:

Mar 21 14:04:36 trisquel gjs[2092]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Mar 21 14:04:36 trisquel gjs[2092]: The offending callback was SourceFunc().

Also, this warning shows up randomly:

Mar 21 14:04:36 trisquel org.gnome.Shell.Extensions.GSConnect[2092]: == Stack trace for context 0x55698b23d1a0 ==
Mar 21 14:04:36 trisquel org.gnome.Shell.Extensions.GSConnect[2092]: #0 55698b2ecd40 i /home/***/.local/share/gnome-shell/extensions/[email protected]/service/daemon.js:729 (37899718d178 @ 963)

anedroid avatar Mar 21 '22 14:03 anedroid