harden-runner icon indicating copy to clipboard operation
harden-runner copied to clipboard

Published 20 hours ago • verified publisher icon step-security

Insights not generated if too many jobs

Open ben-manes opened this issue 2 years ago • 11 comments

I parallelized Caffeine's 2.5hr build by running tests across 112 jobs (117 jobs total), resulting in a 27 minute build time. I'd like to start separating the large, shared block of allowed endpoints. Unfortunately the dashboard fails to render, whereas 1 job workflows show up fine.

ben-manes avatar May 22 '22 18:05 ben-manes

Thanks for reporting @ben-manes! I will take a look later today. It hasn’t been run with so many jobs before so might have hit an unexpected issue.

varunsh-coder avatar May 22 '22 18:05 varunsh-coder

I looked into this and am getting a too many open files error in the backend. Will investigate and fix soon.

I also noticed that one of the allowed domains is not set properly in the workflow file, and so harden-runner is not running properly. Can you please fix the domain name in the workflow file? I think you just need to remove the domain part. Thanks!

domain artifacts.codacy.com::433

https://github.com/ben-manes/caffeine/blob/d8e8a303853192497c84601920987789fa11d658/.github/workflows/build.yml#L21

varunsh-coder avatar May 23 '22 05:05 varunsh-coder

oh good catch. Pushed a new build.

ben-manes avatar May 23 '22 05:05 ben-manes

I'm not sure why, but once fixed access is not granted

curl: (7) Failed to connect to artifacts.codacy.com port 443: Connection refused

ben-manes avatar May 23 '22 07:05 ben-manes

Oh, I think you mentioned this was an known bug wrt a domain resolving to multiple IPs

Mon, 23 May 2022 07:24:20 GMT:domain resolved: artifacts.codacy.com., ip address: 99.84.208.63, TTL: 60 Mon, 23 May 2022 07:25:20 GMT:domain resolved: artifacts.codacy.com., ip address: 99.84.208.90, TTL: 60 Mon, 23 May 2022 07:26:20 GMT:domain resolved: artifacts.codacy.com., ip address: 99.84.208.12, TTL: 60 Error: StepSecurity Harden Runner: Traffic to IP Address 99.84.208.90 was blocked

ben-manes avatar May 23 '22 07:05 ben-manes

Hi @ben-manes

Made progress on the original issue. The backend is able to analyze all the jobs, but it is now failing while storing the analysis results due to size limit. That is going to take a day or two to resolve.

The other issue related to artifacts.codacy.com is a new issue, it is not the same as the known issue, which only affects audit mode. @h0x0er is investigating this.

Will keep you posted. Thanks!

varunsh-coder avatar May 23 '22 18:05 varunsh-coder

thanks a bunch! no rush, this is from a hobby project for me so I won't hack on it again until next weekend.

ben-manes avatar May 23 '22 18:05 ben-manes

Found the issue with artifacts.codacy.com. That is because the allowed port is set to 433 instead of 443 :). It should work fine once you update that.

https://github.com/ben-manes/caffeine/blob/fa552a5bcef523db4ed6508342390952ebe8db9c/.github/workflows/build.yml#L11

varunsh-coder avatar May 24 '22 16:05 varunsh-coder

haha, ugh. 🙁 Thanks!

ben-manes avatar May 24 '22 17:05 ben-manes

It works, but it is missing the compile jobs in the list. Probably the UI needs an improvement to handle these large fan outs.

ben-manes avatar May 26 '22 18:05 ben-manes

It works, but it is missing the compile jobs in the list. Probably the UI needs an improvement to handle these large fan outs.

Yes. Working on that. It is only evaluating and showing the first 30 jobs as of now.

varunsh-coder avatar May 26 '22 18:05 varunsh-coder