krr icon indicating copy to clipboard operation
krr copied to clipboard

Using VictoriaMetrics leads `Connection reset by peer` after 2 minutes

Open Dentrax opened this issue 1 year ago • 2 comments

Describe the bug Using VictoriaMetrics leads Connection reset by peer after 2 minutes.

To Reproduce Steps to reproduce the behavior:

  1. krr simple --verbose --prometheus-url https://vmselect.test.com/select/0/prometheus
  2. See error:
During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/furkan.turkal/robusta_krr/core/runner.py:202 in run                                       │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/robusta_krr/core/runner.py'           │
│                                                                                                  │
│ /Users/furkan.turkal/robusta_krr/core/runner.py:177 in _collect_result                           │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/robusta_krr/core/runner.py'           │
│                                                                                                  │
│ /Users/furkan.turkal/robusta_krr/core/runner.py:138 in _gather_objects_recommendations           │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/robusta_krr/core/runner.py'           │
│                                                                                                  │
│ /Users/furkan.turkal/robusta_krr/core/runner.py:113 in _calculate_object_recommendations         │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/robusta_krr/core/runner.py'           │
│                                                                                                  │
│ /Users/furkan.turkal/robusta_krr/core/integrations/prometheus/loader.py:97 in gather_data        │
│                                                                                                  │
│ [Errno 2] No such file or directory:                                                             │
│ '/Users/furkan.turkal/robusta_krr/core/integrations/prometheus/loader.py'                        │
│                                                                                                  │
│                                     ... 6 frames hidden ...                                      │
│                                                                                                  │
│ /Users/furkan.turkal/requests/sessions.py:635 in post                                            │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/requests/sessions.py'                 │
│                                                                                                  │
│ /Users/furkan.turkal/requests/sessions.py:587 in request                                         │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/requests/sessions.py'                 │
│                                                                                                  │
│ /Users/furkan.turkal/requests/sessions.py:745 in send                                            │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/requests/sessions.py'                 │
│                                                                                                  │
│ /Users/furkan.turkal/requests/models.py:899 in content                                           │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/requests/models.py'                   │
│                                                                                                  │
│ /Users/furkan.turkal/requests/models.py:818 in generate                                          │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/furkan.turkal/requests/models.py'                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ChunkedEncodingError: ("Connection broken: ConnectionResetError(54, 'Connection reset by peer')", ConnectionResetError(54, 'Connection reset by peer'))

Expected behavior Maybe worth adding some resiliency with retries and timeouts.

Screenshots -

Desktop (please complete the following information):

  • OS: macOS
  • Browser -
  • Version v1.4.0

Dentrax avatar Jul 18 '23 15:07 Dentrax

on 0: [INFO] Prometheus connected successfully for kubernetes-admin@foo cluster
Calculating Recommendation |                                        | █▆▄ 0/111 [0%] in 1:38 (~0s, 0.0/s)

2mins passed but it seems progress bar get stuck. 🤔

Dentrax avatar Jul 20 '23 08:07 Dentrax

Could you please send me the full log with verbose mode?

LeaveMyYard avatar Jul 27 '23 09:07 LeaveMyYard