toolkit icon indicating copy to clipboard operation
toolkit copied to clipboard

Prevent "too many open files" in artifact upload

Open rmunn opened this issue 1 year ago • 5 comments

Fixes https://github.com/actions/upload-artifact/issues/485.

The .file method uses "a lazystream wrapper to prevent issues with open file limits", which is precisely the scenario we're running into with https://github.com/actions/upload-artifact/issues/485.

This was originally part of https://github.com/actions/toolkit/pull/1723, but I pulled the bugfix out into its own PR so it can be reviewed separately from the bugfix that https://github.com/actions/toolkit/pull/1723 was for (preserving Unix file permissions in uploaded .zip files). This one is more urgent than the Unix file permissions, which can be worked around by uploading a .tar file.

rmunn avatar Jul 23 '24 10:07 rmunn

Once #1774 is merged, I'll rebase on top of the new state of main so that the tests can pass.

rmunn avatar Jul 24 '24 04:07 rmunn

Just rebased on top of main; let's see what the unit tests do now.

rmunn avatar Jul 26 '24 01:07 rmunn

@robherley - Looks like a team member needs to re-approve running the workflows again.

rmunn avatar Jul 26 '24 03:07 rmunn

Looks like this can be removed:

import {createReadStream} from 'fs'

joshmgross avatar Jul 26 '24 14:07 joshmgross

Looks like this can be removed:

import {createReadStream} from 'fs'

Done in commit 7c61054.

rmunn avatar Jul 27 '24 10:07 rmunn

@robherley - Your approving review is now considered stale because I rebased and pushed one new commit. Want to give this another look? I should be done making changes.

rmunn avatar Jul 31 '24 03:07 rmunn