incubator-devlake
incubator-devlake copied to clipboard
apache
[feat-2578]: Download pipeline logging archives API
…ve them
⚠️ Pre Checklist
Please complete ALL items in this checklist, and remove before submitting
- [ ] I have read through the Contributing Documentation & PR Template
- [ ] This PR is using a
label(bug, feature etc.) - [ ] My code is has necessary documentation (if appropriate)
- [ ] I have added any relevant tests
- [ ] This section (⚠️ Pre Checklist) will be removed when submitting PR
Summary
Does this close any open issues?
#2578
Screenshots
sample tar file for pipeline 58 using CURL
directory structure of logging directory in the container
Other Information
n/a
Hi, @keon94 , don't forget to prefix the issue number with "Closes" or "Fixes", so the issue could be closed automatically when the PR gets merged.
Both standalone and temporal workflows have been tested. Standalone behaves as expected and Temporal has the limitation that the log files from the workers are not downloaded, but the directory structure and the messages in them look correct. #2693 will tackle that limitation.
I find another bug here:
pipeline failed but Config-ui was not updated right.
I'm sure it's a backend bug and it does not exist in branch main.
I find another bug here:
I'm sure it's a backend bug and it does not exist in branch main.
Did you try refreshing the UI page? I tried to recreating a similar problem and refreshing the UI fixed it
I find another bug here:
Did you try refreshing the UI page? I tried to recreating a similar problem and refreshing the UI fixed it
Yes, refreshed. I'm sure that task failed but pipelines are still underway.
It's right on the main branch.
Some logs:
[GIN] 2022/08/19 - 00:28:52 | 200 | 3.771125ms | ::1 | GET "/pipelines/65/tasks"
[GIN] 2022/08/19 - 00:28:57 | 201 | 55.501917ms | ::1 | POST "/pipelines"
[GIN] 2022/08/19 - 00:28:57 | 200 | 8.266458ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:28:57 | 200 | 11.2455ms | ::1 | GET "/pipelines"
[GIN] 2022/08/19 - 00:28:57 | 200 | 2.573542ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:28:57 | 200 | 4.56175ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:28:57 | 200 | 3.9035ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:28:57 | 200 | 19.362084ms | ::1 | GET "/pipelines"
[GIN] 2022/08/19 - 00:28:57 | 200 | 12.667625ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:28:57 | 200 | 3.836417ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:28:57 | 200 | 7.728625ms | ::1 | GET "/pipelines/66/tasks"
time="2022-08-19 00:28:58" level=info msg=" [pipeline service] wait for new pipeline"
time="2022-08-19 00:28:58" level=info msg=" [pipeline service] run pipeline, 66"
time="2022-08-19 00:28:58" level=info msg=" [task service] run task in background %!(EXTRA uint64=113)"
time="2022-08-19 00:28:58" level=info msg=" [pipeline service] [pipeline #66] [task #113] start executing task: 113"
time="2022-08-19 00:28:58" level=info msg=" [pipeline service] [pipeline #66] [task #113] [jenkins] start plugin"
time="2022-08-19 00:29:00" level=error msg=" [task service] task failed%!(EXTRA *fmt.wrapError=Failed to connect: dial tcp 172.26.129.157:443: i/o timeout)"
time="2022-08-19 00:29:00" level=error msg=" [pipeline service] [pipeline #66] run tasks failed: %!w(*errors.errorString=&{Failed to connect: dial tcp 172.26.129.157:443: i/o timeout})"
time="2022-08-19 00:29:00" level=error msg=" [pipeline service] failed to run pipeline, 66: error running pipeline 66: Failed to connect: dial tcp 172.26.129.157:443: i/o timeout"
time="2022-08-19 00:29:00" level=info msg=" [pipeline service] get lock and wait pipeline"
[GIN] 2022/08/19 - 00:29:02 | 200 | 10.055542ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:02 | 200 | 16.596458ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:07 | 200 | 19.045166ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:07 | 200 | 5.402916ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:12 | 200 | 7.442834ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:12 | 200 | 11.496042ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:17 | 200 | 16.53125ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:17 | 200 | 9.987458ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:22 | 200 | 10.203583ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:22 | 200 | 18.818083ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:27 | 200 | 19.99125ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:27 | 200 | 11.28075ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:33 | 200 | 6.968541ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:33 | 200 | 12.173833ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:37 | 200 | 11.030333ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:37 | 200 | 6.963459ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:42 | 200 | 8.45175ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:42 | 200 | 19.7775ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:48 | 200 | 7.312958ms | ::1 | GET "/pipelines/66"
[GIN] 2022/08/19 - 00:29:48 | 200 | 9.492375ms | ::1 | GET "/pipelines/66/tasks"
[GIN] 2022/08/19 - 00:29:51 | 200 | 19.625µs | ::1 | GET "/version"
[GIN] 2022/08/19 - 00:29:51 | 200 | 29.433375ms | ::1 | GET "/blueprints/5"
@likyh yeah, it was a bug in my code - I was throwing the task error before updating the task status. I've fixed it now, please try again.