vcsjones
Doesn't handle keyvault throttling
When a keyvault throttles the signing process, the sign tools hangs. It would be better if it could detect this situation, sleep an amount of time, and retry. Below is the output from an AzDO pipeline that was throttled.
Unhandled exception. System.AggregateException: One or more errors occurred. (Error calling Key Vault) (Error calling Key Vault) ---> System.Security.Cryptography.CryptographicException: Error calling Key Vault ---> Azure.RequestFailedException: Service request failed. Status: 429 ()
Content: {"error":{"code":"Throttled","message":"Request was not processed because too many requests were received. Reason: VaultRequestTypeLimitReached"}}
Headers: Cache-Control: no-cache Pragma: no-cache x-ms-keyvault-region: REDACTED x-ms-client-request-id: 991a7dda-ca08-4415-81d3-3f8f898ae410 x-ms-request-id: 0e57042c-b600-47ba-bfac-27238ec41a3a x-ms-keyvault-service-version: REDACTED x-ms-keyvault-network-info: REDACTED X-Content-Type-Options: REDACTED Strict-Transport-Security: REDACTED Date: Mon, 04 Dec 2023 15:18:07 GMT Content-Length: 146 Content-Type: application/json; charset=utf-8 Expires: -1
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TContent,TResult](RequestMethod method, TContent content, Func`1 resultFactory, CancellationToken cancellationToken, String[] path) at Azure.Security.KeyVault.Keys.Cryptography.RemoteCryptographyClient.Sign(SignatureAlgorithm algorithm, Byte[] digest, CancellationToken cancellationToken) at Azure.Security.KeyVault.Keys.Cryptography.CryptographyClient.Sign(SignatureAlgorithm algorithm, Byte[] digest, CancellationToken cancellationToken) at RSAKeyVaultProvider.RSAKeyVault.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in //RSAKeyVaultProvider/RSAKeyVault.cs:line 45 --- End of inner exception stack trace --- at RSAKeyVaultProvider.RSAKeyVault.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in //RSAKeyVaultProvider/RSAKeyVault.cs:line 45 at AzureSign.Core.AuthenticodeKeyVaultSigner.SignCallback(IntPtr pCertContext, IntPtr pvExtra, UInt32 algId, Byte[] pDigestToSign, UInt32 dwDigestToSign, CRYPTOAPI_BLOB& blob) in /_/src/AzureSign.Core/AuthenticodeKeyVaultSigner.cs:line 239
I'm seeing the same problem. I'm signing about 35 files in a single call to AzureSignTool. My build server logs are showing this: "Signing failed with error 801901AD".
This seems to be the same problem as #135
When Azure Key Vault's API responds with a 429 status code, there needs to be a retry based on the time info (RetryAfter header?) included in the response. Azure has recommendations for retry behavior here: https://learn.microsoft.com/en-us/azure/key-vault/general/overview-throttling
Additionally Service limits are available here: https://learn.microsoft.com/en-us/azure/key-vault/general/service-limits
Added PR to try and properly handle throttling in Azure Keyvault #237
