docker-rtorrent-rutorrent icon indicating copy to clipboard operation
docker-rtorrent-rutorrent copied to clipboard

Published 20 hours ago • verified publisher icon crazy-max

rutorrent UI is not showing up but "loading"

Open Ariloum opened this issue 1 year ago • 50 comments

Support guidelines

I've found a bug and checked that ...

  • [X] ... the documentation does not mention anything about my problem
  • [X] ... there are no open or closed issues that are related to my problem

Description

After updating the rutorrent-crazymax for unraid docker rutorrent shows no more UI (endless "loading" text), but as I can see by the network traffic it still downloading/uploading some torrents

Expected behaviour

entering http://192.168.8.121:8095/ opens the web-ui

Actual behaviour

I'm getting endless "Loading" message instead of web-ui.

Steps to reproduce

I think I didn't updated the rutorrent-crazymax for long like 6-8 months, after update it today to the latest version (Last Update: Oct 25, 2023) ui stopped working

Docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 4
  Running: 1
  Paused: 0
  Stopped: 3
 Images: 4
 Server Version: 20.10.21
 Storage Driver: btrfs
  Build Version: Btrfs v5.10.1 
  Library Version: 102
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1c90a442489720eec95342e1789ee8a5e1b9536f
 runc version: v1.1.4-0-g5fd4c4d1
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.19.17-Unraid
 Operating System: Slackware 15.0 x86_64 (post 15.0 -current)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.04GiB
 Name: Tower
 ID: GW6T:RCDH:BJCE:LPCW:ZYVX:YX5A:FAPL:AXUY:OVMH:GIL7:PTPY:PEL4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Docker Compose config

No response

Logs

tail rutorrent.log 
<fault>
<value><struct>
<member><name>faultCode</name>
<value><i4>-506</i4></value></member>
<member><name>faultString</name>
<value><string>Method 'd.multicall' not defined</string></value></member>
</struct></value>
</fault>
</methodResponse>

Additional info

No response

Ariloum avatar Oct 25 '23 14:10 Ariloum

looks like it fixed itself after awhile somehow

Ariloum avatar Nov 21 '23 12:11 Ariloum

I have a similar issue. Since last update 4 days ago, rutorrent won't load anymore (or at least I believe it is since the last update). It leads to rtorrent become unhealthy and stops seeding. Closing rutorrent helps rtorrent to recover after a couple of minutes.

When rutorrent is not started or closed, rtorrent seeds properly.

I have tried older tags like 4.2.9-0.9.8-0.13.8 and 4.0-0.9.8-0.13.8 but it didn't help.

I am running this on Synology with 32GB of RAM. I tried to stop all other containers, down to 4 as shown in the docker info below, thinking it could have been a resources problem but it didn't help.

docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 38
  Running: 4
  Paused: 0
  Stopped: 34
 Images: 45
 Server Version: 20.10.23
 Storage Driver: btrfs
  Build Version: Btrfs v4.0
  Library Version: 101
 Logging Driver: db
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs db fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b23a389d8c181697302d163356e97dec04eb8d88
 runc version: 5af893d
 init version: ed96d00
 Security Options:
  apparmor
 Kernel Version: 4.4.302+
 Operating System: Synology NAS

 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 31.32GiB
 Name: NAS
 ID: O3K4:UNEN:B7S5:QDHO:BMYV:M6C2:PZKF:QSTK:QU4D:RMMS:NQNK:6Z3L
 Docker Root Dir: /volume1/@docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Compose

---
name: rtorrent-rutorrent

services:
  geoip-updater:
    image: crazymax/geoip-updater:latest
    container_name: geoip-updater
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    networks:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/geoip:/data"
    env_file:
      - "./geoip-updater.env"
    restart: unless-stopped

  rtorrent-rutorrent:
    image: crazymax/rtorrent-rutorrent:latest
    container_name: rtorrent-rutorrent
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    networks:
      - rtorrent-rutorrent
    expose:
      - "6881/udp"
      - "8000"
      - "8080"
      - "9000"
      - "50000"
    ports:
      - target: 6881
        published: 6881
        protocol: udp
      - target: 8000
        published: 8000
        protocol: tcp
      - target: 8080
        published: 8080
        protocol: tcp
      - target: 9000
        published: 9000
        protocol: tcp
      - target: 50000
        published: 50000
        protocol: tcp
    env_file:
      - "rtorrent-rutorrent.env"
      - ".env"
    volumes:
      - "/volume1/docker/rtorrent:/data"
      - "/volume1/docker/download:/downloads"
      - "/volume1/docker/rtorrent/passwd:/passwd"
      #-  "/volume1/docker/rtorrent/rtorrent/watch:/watch"
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
    restart: unless-stopped

  rtorrent-logs:
    image: bash
    container_name: rtorrent-rutorrent-logs
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    command: bash -c 'tail -f /log/*.log'
    network_mode: none
    depends_on:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/rtorrent/log:/log"
    restart: unless-stopped

networks:
  rtorrent-rutorrent:
    name: rtorrent-rutorrent

The latest entry in rutorrent.log is from 2 months ago.

docker logs as soon as rutorrent is kicked in

stderr
22/11/2023
11:58:54
2023/11/22 11:58:54 [info] 560#560: *576 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: REDACTED, server: , request: "GET /php/getplugins.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm/php-fpm.sock:", host: "rutorrent.REDACTED", referrer: "https://REDACTED"

Radarr and Sonarr, both querying port 8000 on a regular basis, report successful connection when rutorrent is not opened. However they time out when rutorrent is started:

stderr
22/11/2023
12:06:50
2023/11/22 12:06:50 [error] 562#562: *630 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "REDACTED:8000"

I am not sure where to go next. Any help would be greatly appreciated!

tofkentof avatar Nov 22 '23 12:11 tofkentof

I've added stop_grace_period: 300s to my compose as I am seeding a very large number of torrents and read that it could lead to some issues. Although I doubt it relates to this one.

I have also added a couple of new env variables but it didn't help:

RU_HTTP_TIME_OUT=300 # was 30 by default
RU_RPC_TIME_OUT=300 # was 5 by default

tofkentof avatar Nov 22 '23 17:11 tofkentof

Ive been seeing a similar issue, where rutorrent is unable to connect to rtorrent for quite a while, before it graduates to taking several minutes to load before failing, and eventually ending in a state where rutorrent loads, but all plugins fail to load and almost all torrents have tracker time out errors.

wkpatrick avatar Nov 28 '23 02:11 wkpatrick

I am still facing the issue. I let the container run for a week without interacting with rutorrent at all hoping time would help (as it did for OP) but sadly it did not.

Any pointers would be really appreciated.

tofkentof avatar Dec 06 '23 11:12 tofkentof

@tofkentof @wkpatrick Are you using torrents with UDP trackers by any chance?

stickz avatar Dec 14 '23 21:12 stickz

@stickz in my .rtorrent.rc, trackers.use_udp.set is commented so I guess I only communicate over tcp?

What is your reasoning? I'm curious :)

Jut to confirm that my problem has been happening for over a month. It impacts only rutorrent. rtorrent behind works fine and seeds properly everything.

tofkentof avatar Dec 16 '23 19:12 tofkentof

What is your reasoning? I'm curious :)

rTorrent has known issues with UDP trackers. This docker image does not implement UDNS. There is also anther issue with tracker scraping that could cause this issue to happen.

I'm trying to figure out the best solution for the problem. If your rTorrent software is timing out after 5 minutes and crashing this problem can be easily resolved with a patch to the docker container. How many torrents approximately are you running?

stickz avatar Dec 16 '23 23:12 stickz

Also please try docker edge. The author just merged my pull request with stability patches which could resolve this problem. https://github.com/crazy-max/docker-rtorrent-rutorrent/pull/286 docker pull crazymax/rtorrent-rutorrent:edge

stickz avatar Dec 16 '23 23:12 stickz

@stickz I planned to cut a release with your patches. I was thinking of this kind of semantic for versioning: 4.2.9-0.9.8+1-0.13.8+1-r0

+<num> marks changes for patches so we can track changes and user can pin to specific tag.

WDYT?

crazy-max avatar Dec 17 '23 09:12 crazy-max

@crazy-max sounds good to me. I will be submitting more patches in the future.

stickz avatar Dec 17 '23 15:12 stickz

https://github.com/crazy-max/docker-rtorrent-rutorrent/releases/tag/4.2.9-0.9.8_1-0.13.8_1-r0

crazy-max avatar Dec 17 '23 15:12 crazy-max

I'm trying to figure out the best solution for the problem. If your rTorrent software is timing out after 5 minutes and crashing this problem can be easily resolved with a patch to the docker container. How many torrents approximately are you running?

Something like 15k.

Also please try docker edge

I've pulled the new image from the tag edge but now rtorrent stays in an unhealthy state.

From rtorrent logs, I can see for each torrent:

Upload unchoked slots adjust; currently:0 adjust:0
Upload unchoked slots adjust; currently:0 adjust:0
Download unchoked slots adjust; currently:0 adjust:0
Download unchoked slots adjust; currently:0 adjust:0
7FD8EE0017EC9108422C8A25FFB379E13CAD681D->download_list: Closing download directly.
7FD8EE0017EC9108422C8A25FFB379E13CAD681D->download_list: Inserting download.
7FD8EE0017EC9108422C8A25FFB379E13CAD681D->download_list: Resuming download: flags:0.
84E7711519E8DD17DCC1A05E074A9A4BD545C726->tracker_list: added tracker (group:0 url:https://REDACTED)

It represents over 140k line of logs since rtorrent was restarted few hours ago.

From docker logs, i can see that the radarr and sonarr time out when trying to connect to rtorrent:

2023/12/17 13:43:29 [error] 763#763: *133 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:13:43:29 +0000] "POST /RPC2 HTTP/1.1" 504 160 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 13:44:34 [error] 761#761: *137 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:13:44:34 +0000] "POST /RPC2 HTTP/1.1" 504 160 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 14:44:14 [error] 763#763: *333 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:14:44:14 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 14:44:20 [error] 762#762: *335 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:14:44:20 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 15:37:38 [error] 763#763: *545 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:37:38 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "XML-RPC.NET"
2023/12/17 15:37:39 [error] 763#763: *545 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:37:39 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "XML-RPC.NET"
2023/12/17 15:44:25 [error] 763#763: *574 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:44:25 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 15:44:30 [error] 760#760: *578 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:44:30 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"

I did not have this problem when switching back and forth from different tags (4.0, 4.2.9, ...).

I am not sure what rtorrent is doing at the moment. I don't think it's loading the different torrents because i could see some from different trackers in the rtorrent logs but the trackers show that I'm not seeding anything.

I am tempted to leave like that until tomorrow morning and if it doesn't revert to an healthy state. If not I'll roll back to tag latest and see if that helps.

tofkentof avatar Dec 17 '23 16:12 tofkentof

I am tempted to leave like that until tomorrow morning and if it doesn't revert to an healthy state. If not I'll roll back to tag latest and see if that helps.

Yes, please try tag latest. If that doesn't work, I'll submit a pull request for the tracker scrape patch to the docker container.

stickz avatar Dec 17 '23 23:12 stickz

I reverted to latest but it did not help. However symptoms seem to change (my assumption). Now basically rtorrent cannot announce to any tracker:

1702890213 I 738185956C8B1E7FF1F283F31732AAFB14A6E346->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 219B167AA641B0B810902B06068EDA2300ED4650->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 9FD8FCC02C6090A2E429117CCD39994745BE550C->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I CF23CD39F05895A733881D64B0E742152E999075->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I B39E3CFE6E4B76C289E925A28F6DB4E855E3FC38->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 2033869D84463585FADB43E541CBCCE14E368B5E->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 3E9746BAB324E7E602468B8A4BAB60CF2D181A4D->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 360742B99B188AEE55ADA7891897C0410E074901->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 0F51AD5BE0E5320701B89548EF632FC7101AC6FA->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 9E9C92E4CD83EE28C1B647391567B3B4D1065CA9->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I F05C211747605B1AD52FF65F9777A95CFA785B46->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I CCC52A29DA125BCCB8D081A3BB7FE35801AE2AB8->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 25185043DC93631CE33F9AADD0F41DC79013BD4B->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)

I've got 16379 lines like this, all with a different hash, which seems to be the number of torrents I've got in rutorrent currently.

I suppose all torrents were parsed and tried to be announced to the trackers but timed out. Although I would expect to see those same lines again with the next reannouncement but rtorrent.log has not been populated for the last 90 minutes.

tofkentof avatar Dec 18 '23 10:12 tofkentof

I suppose all torrents were parsed and tried to be announced to the trackers but timed out.

Maybe rate-limit?

crazy-max avatar Dec 18 '23 11:12 crazy-max

Maybe rate-limit?

Sorry what do you mean by rate limit?

If you mean upload limit, I do not specify any.

Should I post my .rtorrent.rc file here?

tofkentof avatar Dec 18 '23 11:12 tofkentof

@crazy-max @tofkentof Nope, the problem probably related to #288. This should resolve the tracker issues.

stickz avatar Dec 18 '23 11:12 stickz

@tofkentof Perhaps try edge now? We just pushed #288, which should resolve the tracker problem. docker pull crazymax/rtorrent-rutorrent:edge

stickz avatar Dec 19 '23 12:12 stickz

just to be sure should I use edge or master? #288 has been merged to master already however no new build was released.

tofkentof avatar Dec 19 '23 14:12 tofkentof

just to be sure should I use edge or master? #288 has been merged to master already however no new build was released.

Try edge. It is the latest commit from master. It's not required to tag a release, to use the changes. https://hub.docker.com/r/crazymax/rtorrent-rutorrent/tags

stickz avatar Dec 19 '23 15:12 stickz

sI've been running on edge for pretty much the last 24 hours. I'm a bit worried because I cannot get back to a state where I was before my initial post: rtorrent was running fine and seeding consistently when rutorrent was not running. Since moving to edge, the container goes from healthy to unhealthy every few hours. Torrents go from seeded to unseeded. Moved initially back to latest but same problem is occuring. Now back again on edge and no improvement.

I'm clueless.

I was considering creating a complete new blank container from latest (or edge) using a new volume for /data (for example: "/volume1/docker/rtorrent2:/data") and restoring only .rtorrent.rc. I would also need to create 2 new volumes to point to the previous container for retrieve all my torrents if I am correct:

"/volume1/docker/rtorrent/rtorrent/.session:/data/rtorrent/.session"
"/volume1/docker/rtorrent/rutorrent/share/torrents:/data/rutorrent/share/torrents"

Although I'm not sure if I need to restore other files for this scenario to work.

My files in my rtorrent folder (with some exceptions that are not relevant here I suppose):

xxx@NAS:/volume1/docker/rtorrent$ find . -regex '.*' | grep -vE '\.session|\.csv$|/share/torrents|\.ico$'
.
./rtorrent
./rtorrent/log
./rtorrent/log/rtorrent.log
./rtorrent/log/tracker.log
./rtorrent/log/storage.log
./rtorrent/watch
./rtorrent/rtorrent-cross-seed.sh
./rtorrent/.rtorrent.rc~
./rtorrent/.rtorrent.rc
./rutorrent
./rutorrent/conf
./rutorrent/conf/users
./rutorrent/conf/users/users
./rutorrent/conf/access.ini
./rutorrent/conf/plugins.ini
./rutorrent/plugins
./rutorrent/plugins-conf
./rutorrent/share
./rutorrent/share/users
./rutorrent/share/users/rtorrent
./rutorrent/share/users/rtorrent/settings
./rutorrent/share/users/rtorrent/settings/erasedata
./rutorrent/share/users/rtorrent/settings/rss
./rutorrent/share/users/rtorrent/settings/rss/cache
./rutorrent/share/users/rtorrent/settings/rss/cache/info
./rutorrent/share/users/rtorrent/settings/trafic
./rutorrent/share/users/rtorrent/settings/trafic/trackers
./rutorrent/share/users/rtorrent/settings/trafic/torrents
./rutorrent/share/users/rtorrent/settings/ratio.dat
./rutorrent/share/users/rtorrent/settings/which.dat
./rutorrent/share/users/rtorrent/settings/loginmgr.dat
./rutorrent/share/users/rtorrent/settings/extsearch.dat
./rutorrent/share/users/rtorrent/settings/rtorrent.dat
./rutorrent/share/users/rtorrent/settings/history_data.dat
./rutorrent/share/users/rtorrent/settings/scheduler.dat
./rutorrent/share/users/rtorrent/torrents
./rutorrent/share/users/rtorrent/tmp
./rutorrent/share/settings
./rutorrent/share/settings/erasedata
./rutorrent/share/settings/rss
./rutorrent/share/settings/rss/cache
./rutorrent/share/settings/rss/cache/info
./rutorrent/share/settings/trafic
./rutorrent/share/settings/trafic/trackers
./rutorrent/share/settings/trafic/torrents
./rutorrent/share/settings/ratio.dat
./rutorrent/share/settings/which.dat
./rutorrent/share/settings/labels
./rutorrent/share/settings/cpu.dat
./rutorrent/share/settings/trackers
./rutorrent/share/settings/theme.dat
./rutorrent/share/settings/cookies.dat
./rutorrent/share/settings/history_data.dat
./rutorrent/share/settings/WebUISettings.dat
./rutorrent/share/settings/loginmgr.dat
./rutorrent/share/settings/scheduler.dat
./rutorrent/share/settings/extsearch.dat
./rutorrent/share/settings/rtorrent.dat
./rutorrent/themes
./rutorrent/rutorrent.log
./geoip
./geoip/GeoLite2-City.mmdb
./geoip/GeoLite2-Country.mmdb
./geoip/geoip-updater.env
./passwd
./passwd/rpc.htpasswd
./passwd/rutorrent.htpasswd
./passwd/webdav.htpasswd
./rtorrent-rutorrent.env

What do you think?

tofkentof avatar Dec 20 '23 12:12 tofkentof

What do you think?

Could rtorrent-cross-seed.sh be causing that problem to happen? Also please provide your .rtorrent.rc file. This will verify tracker scrape delay is working properly.

Since moving to edge, the container goes from healthy to unhealthy every few hours.

This is expected for the torrent session saving process.

  1. Does the container recover after 15 minutes?
  2. Do you have adequate disk space available to save your session files?
  3. Do you know if any of the session files are corrupted? This problem was just fixed recently.

stickz avatar Dec 20 '23 12:12 stickz

I have modified slightly my message above since you wrote your messae :)

I always had rtorrent-cross-seed.sh but I have moved it one level above just to confirm it is not the problem. Will keep you posted. I am also commenting out the line in my .rtorrent.rc that references this script.

.rtorrent.rc (as it was before commenting out rtorrent-cross-seed.sh reference)

# Maximum and minimum number of peers to connect to per torrent
throttle.min_peers.normal.set = 1
throttle.max_peers.normal.set = 100

# Same as above but for seeding completed torrents (-1 = same as downloading)
throttle.min_peers.seed.set = 1
throttle.max_peers.seed.set = 50

# Maximum number of simultanious uploads per torrent
throttle.max_uploads.set = 15

# Global upload and download rate in KiB. "0" for unlimited
throttle.global_down.max_rate.set_kb = 0
throttle.global_up.max_rate.set_kb = 0

# Enable DHT support for trackerless torrents or when all trackers are down
# May be set to "disable" (completely disable DHT), "off" (do not start DHT),
# "auto" (start and stop DHT as needed), or "on" (start DHT immediately)
dht.mode.set = auto

# Enable peer exchange (for torrents not marked private)
protocol.pex.set = yes

# Check hash for finished torrents. Might be usefull until the bug is
# fixed that causes lack of diskspace not to be properly reported
pieces.hash.on_completion.set = yes

# Set whether the client should try to connect to UDP trackers
#trackers.use_udp.set = yes

# Set the max amount of memory address space used to mapping file chunks. This refers to memory mapping, not
# physical memory allocation. Default: 1GB (max_memory_usage)
# This may also be set using ulimit -m where 3/4 will be allocated to file chunks
#pieces.memory.max.set = 1GB

# Alternative calls to bind and ip that should handle dynamic ip's
#schedule2 = ip_tick,0,1800,ip=rakshasa
#schedule2 = bind_tick,0,1800,bind=rakshasa

# Encryption options, set to none (default) or any combination of the following:
# allow_incoming, try_outgoing, require, require_RC4, enable_retry, prefer_plaintext
protocol.encryption.set = allow_incoming,try_outgoing,enable_retry

# Set the umask for this process, which is applied to all files created by the program
system.umask.set = 0022

# Add a preferred filename encoding to the list
encoding.add = UTF-8

# Watch a directory for new torrents, and stop those that have been deleted
directory.watch.added = (cat,(cfg.watch)), load.start
#schedule = watch_directory,5,5,"load.normal=/watch/*.torrent,d.delete_tied="
schedule2 = untied_directory, 5, 5, (cat,"stop_untied=",(cfg.watch),"*.torrent")

# Close torrents when diskspace is low
schedule2 = monitor_diskspace, 15, 60, ((close_low_diskspace,1000M))

# Move finished (no need Autotools/Automove plugin on ruTorrent)
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="
method.set_key=event.download.finished,cross_seed,"execute={/data/rtorrent/rtorrent-cross-seed.sh,$d.name=}"

# Logging
log.open_file = "rtorrent", /data/rtorrent/log/rtorrent.log
log.open_file = "tracker", /data/rtorrent/log/tracker.log
log.open_file = "storage", /data/rtorrent/log/storage.log

log.add_output = "info", "rtorrent"
log.add_output = "critical", "rtorrent"
log.add_output = "error", "rtorrent"
log.add_output = "warn", "rtorrent"
log.add_output = "notice", "rtorrent"
log.add_output = "debug", "rtorrent"

log.add_output = "dht_debug", "tracker"
log.add_output = "tracker_debug", "tracker"

log.add_output = "storage_debug", "storage"

This is expected for the torrent session saving process. Does the container recover after 15 minutes? Do you have adequate disk space available to save your session files? Do you know if any of the session files are corrupted? This problem was just fixed recently.

I'll have to check about the 15 minutes mark. I'm not sure but it seems to take longer.

I have 7TB free so it should be fine :) Before the initial problem, I did not have any session files corrupted. Is there a tool I could script to parse all session files and check for consistency?

tofkentof avatar Dec 20 '23 12:12 tofkentof

It looks like you're missing a bunch of values for your .rtorrent.rc file. You can replace @varriable@ with the correct values. https://github.com/crazy-max/docker-rtorrent-rutorrent/blob/master/rootfs/tpls/etc/rtorrent/.rtlocal.rc#L37-L47

trackers.delay_scrape in particular will fail to launch rTorrent, if you don't have the latest patch installed. You're also missing session saving improvements as well. We increased that from 20M to 3H so you can keep seeding.

stickz avatar Dec 20 '23 12:12 stickz

Thanks for that @stickz.

I've altered .rtorrent.rc file that now looks like that:

throttle.min_peers.normal.set = 1
throttle.max_peers.normal.set = 100
throttle.min_peers.seed.set = 1
throttle.max_peers.seed.set = 50
throttle.max_uploads.set = 15
throttle.global_down.max_rate.set_kb = 0
throttle.global_up.max_rate.set_kb = 0
dht.mode.set = auto
protocol.pex.set = yes
pieces.hash.on_completion.set = yes
protocol.encryption.set = allow_incoming,try_outgoing,enable_retry
system.umask.set = 0022
encoding.add = UTF-8
directory.watch.added = (cat,(cfg.watch)), load.start
schedule2 = untied_directory, 5, 5, (cat,"stop_untied=",(cfg.watch),"*.torrent")
schedule2 = monitor_diskspace, 15, 60, ((close_low_diskspace,1000M))
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="
network.xmlrpc.size_limit.set = 1M
schedule2 = session_save, 1200, 3600, ((session.save))
method.set_key = event.download.inserted, 2_save_session, ((d.save_full_session))
trackers.delay_scrape = true
log.open_file = "rtorrent", /data/rtorrent/log/rtorrent.log
log.open_file = "tracker", /data/rtorrent/log/tracker.log
log.open_file = "storage", /data/rtorrent/log/storage.log
log.add_output = "info", "rtorrent"
log.add_output = "critical", "rtorrent"
log.add_output = "error", "rtorrent"
log.add_output = "warn", "rtorrent"
log.add_output = "notice", "rtorrent"
log.add_output = "debug", "rtorrent"
log.add_output = "dht_debug", "tracker"
log.add_output = "tracker_debug", "tracker"
log.add_output = "storage_debug", "storage"

I have also added the new variable RT_TRACKER_DELAY_SCRAPE to my compose.

I'll monitor how it's going and will report.

tofkentof avatar Dec 20 '23 14:12 tofkentof

As an update, the situation is worse now :) rtorrent is not seeding anything, I believe consistently.

in rtorrent.log, I can see that it times out on the announcement for each and every tracker (i've got something like 30 trackers that are suppose to announce).

I have been running on Edge for the last 20 hours maybe.

My assumption is that it never got to announce to any tracker for the last 20 hours.

Anything (logs, cfg, ...) you would like me to provide?

tofkentof avatar Dec 21 '23 11:12 tofkentof

Anything (logs, cfg, ...) you would like me to provide?

Yes, I need to know if you're using any UDP trackers. This docker container doesn't support it properly. Also, try ensuring that IPV6 is disabled at the operating system layer. rTorrent does not function properly with IPV6.

Furthermore, consider limiting the amount of trackers that can announce at once.

network.http.max_open.set = 1024
network.max_open_files.set = 2048
network.max_open_sockets.set = 1024

Lastly, this variable needs to be 16M for the amount of torrents you have. We recently added support for 16M. network.xmlrpc.size_limit.set = 16M

stickz avatar Dec 21 '23 12:12 stickz

I've done everything you mentioned except disabling ipv6 on the hosts (synology distro). Instead I have disabled ipv6 in the container itself by adding:

    sysctls:
      net.ipv6.conf.all.disable_ipv6: 1

Should I disable ipv6 at the host level?

The uptime for the container is 3 hours. Nothing is currently seeding unfortunately.

In rtorrent.log, I can see that each torrent is being parsed (successfully?):

1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I A84D132950124EF464997E88A2C072E711DB4AF1->download_list: Closing download directly.
1703164956 I A84D132950124EF464997E88A2C072E711DB4AF1->download_list: Inserting download.
1703164956 I A84D132950124EF464997E88A2C072E711DB4AF1->download_list: Resuming download: flags:0.
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->tracker_list: added tracker (group:0 url:https://REDACTED)
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->download_list: Closing download directly.
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->download_list: Inserting download.
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->download_list: Resuming download: flags:0.
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->tracker_list: added tracker (group:0 url:https://REDACTED)
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->download_list: Closing download directly.
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->download_list: Inserting download.
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->download_list: Resuming download: flags:0.
1703164956 I 8D8443912CB6DA1D507B0A4E3A1D1BD91C03576D->tracker_list: added tracker (group:0 url:http://REDACTED)

My .rtorrent.rc now looks like this:

throttle.min_peers.normal.set = 1
throttle.max_peers.normal.set = 100
throttle.min_peers.seed.set = 1
throttle.max_peers.seed.set = 50
throttle.max_uploads.set = 15
throttle.global_down.max_rate.set_kb = 0
throttle.global_up.max_rate.set_kb = 0
dht.mode.set = auto
protocol.pex.set = yes
pieces.hash.on_completion.set = yes
trackers.use_udp.set = no
protocol.encryption.set = allow_incoming,try_outgoing,enable_retry
system.umask.set = 0022
encoding.add = UTF-8
directory.watch.added = (cat,(cfg.watch)), load.start
schedule2 = untied_directory, 5, 5, (cat,"stop_untied=",(cfg.watch),"*.torrent")
schedule2 = monitor_diskspace, 15, 60, ((close_low_diskspace,1000M))
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="
network.xmlrpc.size_limit.set = 16M
network.http.max_open.set = 1024
network.max_open_files.set = 2048
network.max_open_sockets.set = 1024
schedule2 = session_save, 1200, 3600, ((session.save))
method.set_key = event.download.inserted, 2_save_session, ((d.save_full_session))
trackers.delay_scrape = true
log.open_file = "rtorrent", /data/rtorrent/log/rtorrent.log
log.open_file = "tracker", /data/rtorrent/log/tracker.log
log.open_file = "storage", /data/rtorrent/log/storage.log
log.add_output = "info", "rtorrent"
log.add_output = "critical", "rtorrent"
log.add_output = "error", "rtorrent"
log.add_output = "warn", "rtorrent"
log.add_output = "notice", "rtorrent"
log.add_output = "debug", "rtorrent"
log.add_output = "dht_debug", "tracker"
log.add_output = "tracker_debug", "tracker"
log.add_output = "storage_debug", "storage"

My compose looks like this:

---
name: rtorrent-rutorrent

services:
  geoip-updater:
    image: crazymax/geoip-updater:latest
    container_name: geoip-updater
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    networks:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/geoip:/data"
    env_file:
      - "./geoip-updater.env"
    restart: unless-stopped

  rtorrent-rutorrent:
    image: crazymax/rtorrent-rutorrent:edge
    container_name: rtorrent-rutorrent
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - RU_HTTP_TIME_OUT=300
      - RU_RPC_TIME_OUT=300
      - RT_SESSION_SAVE_SECONDS=10800
      - RT_TRACKER_DELAY_SCRAPE=true
    networks:
      - rtorrent-rutorrent
    expose:
      - "6881/udp"
      - "8000"
      - "8080"
      - "9000"
      - "50000"
    ports:
      - target: 6881
        published: 6881
        protocol: udp
      - target: 8000
        published: 8000
        protocol: tcp
      - target: 8080
        published: 8080
        protocol: tcp
      - target: 9000
        published: 9000
        protocol: tcp
      - target: 50000
        published: 50000
        protocol: tcp
    env_file:
      - "rtorrent-rutorrent.env"
      - ".env"
    volumes:
      - "/volume1/docker/rtorrent:/data"
      - "/volume1/docker/download:/downloads"
      - "/volume1/docker/rtorrent/passwd:/passwd"
      #-  "/volume1/docker/rtorrent/rtorrent/watch:/watch"
    sysctls:
      net.ipv6.conf.all.disable_ipv6: 1
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
    stop_grace_period: 300s
    restart: unless-stopped

  rtorrent-logs:
    image: bash
    container_name: rtorrent-rutorrent-logs
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    command: bash -c 'tail -f /log/*.log'
    network_mode: none
    depends_on:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/rtorrent/log:/log"
    restart: unless-stopped

networks:
  rtorrent-rutorrent:
    name: rtorrent-rutorrent

tofkentof avatar Dec 21 '23 16:12 tofkentof

sit tight @tofkentof. I have #292 coming. I realized the rTorrent software stack was not being built with the intended flags. Also, c-ares + curl is outdated. This will need to be resolved with anther pull request.

This docker container is lacking maintenance. I don't know if bringing it up to date will resolve your 16K torrent problem though.

stickz avatar Dec 21 '23 22:12 stickz