invidious icon indicating copy to clipboard operation
invidious copied to clipboard

Published 20 hours ago • verified publisher icon iv-org

[Bug] Index out of bounds (IndexError): user preferences

Open fhemberger opened this issue 1 year ago • 9 comments

Describe the bug When saving user preferences, the following message is shown:

Title: `Index out of bounds (IndexError)`
Date: `2023-12-09T12:07:39Z`
Route: `/preferences?referer=%2Ffeed%2Fpopular`
Version: `2023.12.06-9e8baa3 @ master`

<details>
<summary>Backtrace</summary>
<p>
   
\```
Index out of bounds (IndexError)
  from src/invidious/routes/preferences.cr:180:8 in 'update'
  from lib/kemal/src/kemal/route.cr:12:9 in '->'
  from src/invidious/helpers/handlers.cr:30:37 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:94:12 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:151:29 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/ext/kemal_static_file_handler.cr:112:11 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client'
  from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
  from ???
\```
</p>
</details>

Steps to Reproduce Postgres version: 15.5 Invidious version: 2023.12.06-9e8baa3 Database SQL files used from master.

Invidious config:

database_url: postgres://invidious:##redacted##@##postgres hostname##/invidious
check_tables: true
domain: "##invidious hostname##"
external_port: 443
https_only: true
hsts: true
captcha_enabled: false
channel_threads: 4
feed_threads: 4
hmac_key: "##redacted##"
default_user_preferences:
  locale: en-US
  comments: [""]
  quality: dash
  vr_mode: false
  save_player_pos: true
  show_nick: false
  extend_desc: true

Steps to reproduce:

  1. Start Postgres and Invidious container
  2. Go to start page, click "login", create account
  3. Open preferences
  4. Save preferences (you don't have to change anything)

Logs Docker logs since startup:

Unhandled exception:  (DB::ConnectionRefused)
  from lib/sqlite3/src/sqlite3/connection.cr:10:11 in '->'
  from lib/db/src/db/pool.cr:214:27 in 'build_resource'
  from /usr/share/crystal/src/int.cr:545:11 in 'build_database'
  from src/invidious.cr:63:15 in '__crystal_main'
  from /usr/share/crystal/src/crystal/main.cr:115:5 in 'main'
  from src/env/__libc_start_main.c:95:2 in 'libc_start_main_stage2'
Caused by: Cannot establish connection (PQ::ConnectionError)
  from lib/pg/src/pq/connection.cr:34:9 in 'new'
  from lib/pg/src/pg/connection.cr:13:23 in '->'
  from lib/db/src/db/pool.cr:214:27 in 'build_resource'
  from /usr/share/crystal/src/int.cr:545:11 in 'build_database'
  from src/invidious.cr:63:15 in '__crystal_main'
  from /usr/share/crystal/src/crystal/main.cr:115:5 in 'main'
  from src/env/__libc_start_main.c:95:2 in 'libc_start_main_stage2'
Caused by: Error connecting to 'invidious-postgres15:5432': Connection refused (Socket::ConnectError)
  from /usr/share/crystal/src/crystal/system/unix/socket.cr:13:11 in 'new'
  from lib/pg/src/pg/connection.cr:13:23 in '->'
  from lib/db/src/db/pool.cr:214:27 in 'build_resource'
  from /usr/share/crystal/src/int.cr:545:11 in 'build_database'
  from src/invidious.cr:63:15 in '__crystal_main'
  from /usr/share/crystal/src/crystal/main.cr:115:5 in 'main'
  from src/env/__libc_start_main.c:95:2 in 'libc_start_main_stage2'
[production] Invidious is ready to lead at http://0.0.0.0:3000
2023-12-09 12:05:24 UTC [info] jobs: running ClearExpiredItems job
2023-12-09 12:05:24 UTC [info] jobs: ClearExpiredItems done.
2023-12-09 12:07:02 UTC [info] 302 GET / 11.53ms
2023-12-09 12:07:02 UTC [info] 200 GET /feed/popular 28.64ms
2023-12-09 12:07:07 UTC [info] 200 GET /login?referer=%2Ffeed%2Fpopular 4.77ms
2023-12-09 12:07:14 UTC [info] 302 POST /login?referer=%2Ffeed%2Fpopular&type=invidious 337.62ms
2023-12-09 12:07:14 UTC [info] 200 GET /feed/popular 5.72ms
2023-12-09 12:07:14 UTC [info] 200 GET /api/v1/auth/subscriptions?fields=authorId 17.09ms
2023-12-09 12:07:30 UTC [warn] i18n: Missing translation key "auto"
2023-12-09 12:07:30 UTC [info] 200 GET /preferences?referer=%2Ffeed%2Fpopular 14.58ms
2023-12-09 12:07:31 UTC [info] 200 GET /api/v1/auth/subscriptions?fields=authorId 14.35ms
Exception: Index out of bounds (IndexError)
  from src/invidious/routes/preferences.cr:180:8 in 'update'
  from lib/kemal/src/kemal/route.cr:12:9 in '->'
  from src/invidious/helpers/handlers.cr:30:37 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:94:12 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:151:29 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/ext/kemal_static_file_handler.cr:112:11 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client'
  from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
  from ???

fhemberger avatar Dec 09 '23 12:12 fhemberger

Could you try with postgresql 14 if that makes a difference?

unixfox avatar Dec 18 '23 22:12 unixfox

Same with the latest master and Postgres 14.10:

Title: `Index out of bounds (IndexError)`
Date: `2023-12-19T07:46:13Z`
Route: `/preferences?referer=%2Ffeed%2Fpopular`
Version: `2023.12.18-97c4165 @ master`

<details>
<summary>Backtrace</summary>
<p>
   
\```
Index out of bounds (IndexError)
  from src/invidious/routes/preferences.cr:180:8 in 'update'
  from lib/kemal/src/kemal/route.cr:12:9 in '->'
  from src/invidious/helpers/handlers.cr:30:37 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:94:12 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:151:29 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/ext/kemal_static_file_handler.cr:112:11 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client'
  from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
  from ???
\```
</p>
</details>

fhemberger avatar Dec 19 '23 07:12 fhemberger

Please post a screen recording of what you are doing. Thank you

unixfox avatar Dec 19 '23 07:12 unixfox

Prerequisites

SQL dumps/docker-entrypoint.sh taken from:

  • https://github.com/iv-org/invidious/tree/master/config/sql
  • https://github.com/iv-org/invidious/blob/master/docker/init-invidious-db.sh

Ansible tasks

- name: Start Postgres
  community.docker.docker_container:
    container_default_behavior: no_defaults
    detach: yes
    env:
      POSTGRES_DB: invidious
      POSTGRES_PASSWORD: "##REDACTED##"
      POSTGRES_USER: "invidious"
    image: "postgres:14-alpine"
    labels:
      traefik.enable: "false"
    name: "invidious-postgres14"
    network_mode: default
    networks_cli_compatible: yes
    networks: [name: "{{ docker_network_name }}"]
    ports:
      - "5433:5432"
    pull: yes
    purge_networks: yes
    restart_policy: unless-stopped
    state: started
    volumes:
      - "invidious-postgres14-data:/var/lib/postgresql/data"
      - "{{ docker_mountpoint }}/config/invidious/sql:/config/sql"
      - "{{ docker_mountpoint }}/config/invidious/init-invidious-db.sh:/docker-entrypoint-initdb.d/init-invidious-db.sh"

- name: Start Invidious
  community.docker.docker_container:
    cap_drop: [ALL]
    container_default_behavior: no_defaults
    detach: yes
    env:
      # Please read the following file for a comprehensive list of all available
      # configuration options and their associated syntax:
      # https://github.com/iv-org/invidious/blob/master/config/config.example.yml
      INVIDIOUS_CONFIG: |
        database_url: postgres://invidious:##REDACTED##@invidious-postgres14/invidious
        check_tables: true
        domain: invidious.example.com
        external_port: 443
        https_only: true
        hsts: true
        # popular_enabled: false
        captcha_enabled: false
        channel_threads: 4
        feed_threads: 4
        hmac_key: ##REDACTED##
        default_user_preferences:
          locale: en-US
          region: DE
          comments: [""]
          quality: dash
          vr_mode: false
          save_player_pos: true
          show_nick: false
          extend_desc: true
    image: quay.io/invidious/invidious:latest
    labels:
      prometheus.enable: "false"
      traefik.enable: "true"
      traefik.http.routers.invidious.entrypoints: "web-secure"
      traefik.http.routers.invidious.rule: "Host(`##REDACTED##`)"
      traefik.http.routers.invidious.tls: "true"
    name: invidious
    network_mode: default
    networks_cli_compatible: yes
    networks: [name: "{{ docker_network_name }}"]
    pull: yes
    purge_networks: yes
    restart_policy: unless-stopped
    state: started

Opening Invidious in the browser

https://github.com/iv-org/invidious/assets/153481/2008c514-77f8-458c-8367-97998f73b0af

fhemberger avatar Dec 21 '23 12:12 fhemberger

You are using postgresql 14 alpine not postgresql 14 (Debian).

Please try with postgresql 14 (Debian)

unixfox avatar Dec 21 '23 12:12 unixfox

Same effect with postgres:14 Docker image.

fhemberger avatar Dec 21 '23 13:12 fhemberger

I had the same error just now. I was testing something in my Invidious Debug instance and I got this error when trying to change the Proxy videos setting from off to on.

HOWEVER, this error only happens if you don't have an account, if you have an account, the Proxy videos option works well. I don't know if this is intended but I guess it shouldn't display Index out of bounds (IndexError)

https://github.com/iv-org/invidious/assets/61166695/2c2e8720-5771-425e-8cc8-963c3ea123df

Fijxu avatar Dec 28 '23 04:12 Fijxu

Finally figured this out. It's because of the way we tried to disable comments in the config. I hit this when I had the incorrect number of elements in the comments array.

Accepted values: a two-entries array.

  ##
  ## Type of comments to display below video.
  ##
  ## Accepted values: a two-entries array.
  ## Each entry can be one of: "youtube", "reddit", ""
  ##
  ## Default: ["youtube", ""]
  ##
  comments: ["", ""]

sixdrum avatar Mar 18 '24 03:03 sixdrum

Probably separate issue, but "Missing translation key "auto"" occurs for me on every GET for /preferences

dud1337 avatar Apr 05 '24 05:04 dud1337