During the codesigning phase Im getting a popup to enter a keychain password for the generated keychain

[7sharp9]

I have my machine setup as a local runner and when it gets to the code signing phase this pops up:

I moved to a local machine runner because I burned through my 2000 minutes quite quick and I think this was happening on the Github host too as it was runing for 40 minutes just on the code signing job.

It does look like the importing of the keys work as I see the keychain file. I can also look through the job commands and find the password to enter but this should not happen.

Did I miss sometihn I have to do or did Apple change something?