Derek Su
Derek Su
I saw lots of timeout error messages in the longhorn-manager pod ``` 2024-07-08T00:46:46.778602213+01:00 time="2024-07-07T23:46:46Z" level=error msg="Failed to get info from backup store" func="controller.(*BackupTargetController).reconcile" file="backup_target_controller.go:384" controller=longhorn-backup-target cred=contabo-s3-secret error="failed to list backup...
Best effort in v1.7.0
cc @mantissahz
@conall88 How many backups in your S3 repo? The issue is caused by list timeout.
@mantissahz IIRC, you were working on the adjustable timeout setting for listing backups. Has it been implemented?
What we need to check and improvement - paging (https://github.com/longhorn/longhorn/issues/1904) - configurable timeout (https://github.com/longhorn/longhorn/issues/8319, https://github.com/longhorn/longhorn/issues/8368)
@conall88 If we building a custom image with an increased timeout, can you help verify it in your environment? cc @mantissahz
Set the milestone to v1.10.0, but it will be backported to v1.9.1.
cc @roger-ryao Can you help verify this? Thanks.
~~@yangchiu Does the issue also happen in master-head and v1.6.2?~~ The issue can be easily reproduced in v1.6.2 and master branches.