Massive Log File (daemon.log)

[dombarnes]

Sanity checks

• [X] I have searched github.com/withfig/fig/issues and there are no duplicates of my issue
• [X] I have run fig doctor in the affected terminal session
• [X] I have run fig restart and tested again (tell us if that fixed it)

Issue Details

Description:

I was looking into a recent loss of disc space and in doing so found that inside ~/.fig/log/ was a file daemon.log which had grown to 14GB in size since March 2022. I tail'd the file and found it full of entries like this over and over every few ticks:

2022-08-05T14:53:34.689294Z ERROR fig_cli::daemon: 536: Error while connecting to websocket: refresh token expired
2022-08-05T14:53:34.690484Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-05T14:53:34.693750Z  WARN aws_smithy_client::builder: 245: Retries require a `sleep_impl`, but none was passed into the builder. No retries will occur with the current configuration. If this was intentional, you can suppress this message with `Client::set_sleep_impl(None). Otherwise, unless you have a good reason to use the low-level service client API, consider using the `aws-config` crate to load a shared config from the environment, and construct a fluent client from that. If you need to use the low-level service client API, then pass in a sleep implementation to make timeouts and retry work.

I have since deleted the file, quit and relaunched Fig (which also required me to log in - maybe related or not). And now when tail-ing this file, I don't see these messages. It seems the issue is somewhat resolved for now. But it seemed worthy of reporting

Environment

# Fig Diagnostics
## Fig details:
  - Fig version: Version 1.0.59 (B489) [British]
  - Bundle path: /Applications/Fig.app
  - Autocomplete: true
  - Settings.json: true
  - Accessibility: true
  - Number of specs: 0
  - Symlinked dotfiles: true
  - Only insert on tab: false
  - Keybindings path:
  - Installation Script: true
  - PseudoTerminal Path: /Users/domster83/.google-cloud-sdk/bin:/Users/domster83/laptop/bin:/Users/domster83/.rbenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/domster83/.cargo/bin:/Users/domster83/.fig/bin:/Users/domster83/.local/bin:/Users/domster83/.yarn/bin:/usr/local/opt/[email protected]/bin:/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin:/usr/local/opt/curl/bin:/usr/local/heroku/bin:/usr/local/opt/node@14/bin:/Users/domster83/.config/yarn/global/node_modules/.bin:/usr/local/lib/node_modules:/usr/local/sbin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin:/Users/domster83/.flutter/bin:/Users/domster83/.android/sdk/emulator:/Users/domster83/.android/sdk/platform-tools:/Users/domster83/.android/sdk/cmdline-tools/latest/bin:/Users/domster83/.android/sdk/cmdline-tools/tools/bin:/Users/domster83/.webos/webOS_Signage_SDK/CLI/bin:LG_WEBOS_SIGNAGE_SDK_HOME:/Users/domster83/.pub-cache/bin
  - SecureKeyboardInput: false
  - SecureKeyboardProcess: <none>
## Hardware Info:
  - Model Name: MacBook Pro
  - Model Identifier: MacBookPro15,2
  - Chip:
  - Cores: 4
  - Memory: 16 GB
## OS Info:
  - macOS 12.5.0 (21G72)
## Environment:
  - User Shell: /bin/zsh
  - Current Directory: /Users/domster83
  - CLI Installed: true
  - Executable Location: /usr/local/bin/fig
  - Current Window ID: 28262/% (com.googlecode.iterm2)
  - Active Process: zsh (7603) - /dev/ttys011
  - Terminal: iterm
  - Environment Variables:
    - TERM_SESSION_ID=w0t0p0:165D3654-CF34-4D15-A5F5-8B2AEE2BD2E1
    - PATH=/Users/domster83/.google-cloud-sdk/bin:/Users/domster83/laptop/bin:/Users/domster83/.rbenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/domster83/.cargo/bin:/Users/domster83/.fig/bin:/Users/domster83/.local/bin:/Users/domster83/.yarn/bin:/usr/local/opt/[email protected]/bin:/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin:/usr/local/opt/curl/bin:/usr/local/heroku/bin:/usr/local/opt/node@14/bin:/Users/domster83/.config/yarn/global/node_modules/.bin:/usr/local/lib/node_modules:/usr/local/sbin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin:/Users/domster83/.flutter/bin:/Users/domster83/.android/sdk/emulator:/Users/domster83/.android/sdk/platform-tools:/Users/domster83/.android/sdk/cmdline-tools/latest/bin:/Users/domster83/.android/sdk/cmdline-tools/tools/bin:/Users/domster83/.webos/webOS_Signage_SDK/CLI/bin:LG_WEBOS_SIGNAGE_SDK_HOME:/Users/domster83/.pub-cache/bin
    - TERM=xterm
    - FIG_INTEGRATION_VERSION=8
    - FIG_TERM=1
    - FIG_TERM_VERSION=4.3.0
    - FIG_PID=7603
## Integrations:
  - SSH: false
  - TMUX: false
  - iTerm: installed!
  - Hyper: application is not present.
  - Visual Studio Code: installed!
  - Docker: true

[Chizkiyahu]

My file is 335MB from 2022-07-07 and is have inside full HTML pages

Click to expand! to see HTML log

``` [2m2022-08-01T21:42:13.525164Z[0m [31mERROR[0m [2mfig_cli::daemon[0m[2m:[0m [2m539:[0m Error while connecting to websocket:

[Chizkiyahu]

inside the log data

[mschrage]

@dombarnes we were having issues with our backend and have since switched our hosting provider.

I believe we clear logs whenever Fig is restarted. Let me know if this issue persists.

[aljrico]

I am seeing the exact same issue. The daemon.log file is now at 72GB. I have restarted Fig and the laptop multiple times. I am on version 1.0.60.

I only noticed because the laptop ran out of storage. I will now manually remove the daemon.log unless told otherwise

[mschrage]

@grant0417 anything change with our logging?

@aljrico totally fine to delete the logs.

[grant0417]

Seems like this log file has been hanging around a bit too long! Will ensure it is is deleted more often to ensure this doesnt happen!

[grant0417]

Also there were some issues we were having with our old server host which is what caused the file to be filled with html pages.

[aljrico]

For completeness, I am not logged in. Not sure if that is related with this issue

[dombarnes]

Given how often some developers restart their computers (monthly for some?) it may be worth doing a max size cycle or daily?

[buzzcutts]

same issue here- noticed process 'fig darwin universal' cranking at 25% cpu. Daemon.log grew to ~80GB quit and deleted .log

[grant0417]

Is anyone able to send the last 100-1000 lines of the log files, not too sure why they are growing so quickly, the fix should be out soon however to clean them up.

[Chizkiyahu]

• restart the fig or computer don't delete the log file
• at the last 6 days I have file of 5MB
• no HTML in the log 👍
• 18,000 ping msg like [2m2022-08-15T19:15:13.568376Z[0m [34mDEBUG[0m [2mfig_cli::daemon::websocket[0m[2m:[0m [2m188:[0m Websocket ping
• 6000 pong msg like [2m2022-08-15T19:22:07.337865Z[0m [34mDEBUG[0m [2mfig_cli::daemon::websocket[0m[2m:[0m [2m192:[0m Websocket pong

[nnmrts]

Is or could there be sensitive information in my log file? Ready to share mine as well, although it is almost 5GBs big...

[grant0417]

Thanks @Chizkiyahu, looks like something has caused debug level logging in the log files, typically they only log INFO or higher, I believe the issues should be resolved (as i mentioned earlier) but that has been helpful!

[davrax]

Same issue here- I'd installed Fig (with Homebrew) 4-5 days ago, started it, but did no further setup. Observed that I was rapidly running out of storage space, and found that daemon.log has grown to 19 GB of this:

2022-08-18T17:09:56.806421Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.807037Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.808307Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.808525Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.812232Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.812539Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.813695Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.813939Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.815125Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.815292Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.816467Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.816635Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.819256Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.819440Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.820733Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.820899Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.822045Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.822171Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.826751Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket
2022-08-18T17:09:56.826892Z ERROR fig_cli::daemon: 538: Error while connecting to websocket: missing field `expiration_time` at line 3 column 1
2022-08-18T17:09:56.828037Z  INFO fig_cli::daemon::websocket: 68: Connecting to websocket

[grant0417]

@davrax, that is strange, can you try running fig update

[sthales]

Same issue here. fig update resolved.

[parssak]

this has been resolved in newer versions of Fig, if you experience this, make sure to update to the latest version

[maddler]

This is still happening in latest version

❯ fig version fig_cli 2.15.0

On top of that, after running 'fig uninstall' the 'fig daemon' would be left running and once killed will, in turn cause logs to be flooed with

Jan 23 22:30:02 dracula systemd[349]: fig-daemon.service: Service RestartSec=5s expired, scheduling restart.
Jan 23 22:30:02 dracula systemd[349]: fig-daemon.service: Scheduled restart job, restart counter is at 1.
Jan 23 22:30:02 dracula systemd[13131]: fig-daemon.service: Failed to execute command: No such file or directory
Jan 23 22:30:02 dracula systemd[13131]: fig-daemon.service: Failed at step EXEC spawning /usr/bin/fig: No such file or directory
Jan 23 22:30:02 dracula systemd[349]: fig-daemon.service: Main process exited, code=exited, status=203/EXEC
Jan 23 22:30:02 dracula systemd[349]: fig-daemon.service: Failed with result 'exit-code'.

Of course the service files would've been deleted during the uninstall. Pretty messy and upsetting to be honest.

[Amine27]

@maddler Did you managed to fix this problem? I am having the same logs. Thank you.

[maddler]

I had to re-install, fully disable the service and the uninstall the app itself. But after the lack of support I fully removed the app from all my servers and computers. Can't risk having a disk filled by that.

[boxed]

I tried out fig for a very short time. Got a 40gig log file.

[adhamsalama]

@maddler They don't allow installing Fig on Linux anymore, so I can't even try your solution. :/