OSError on macOS 12.2 running on arm

[behrtam]

Setup: macOS 12.1, oscrypto 1.2.1, Python 3.9.5

Not sure where to even start to look for this problem. Running the newest mac version on arm and not Intel. The OSStatus 62385568 did not give me any clues.

/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_mac/asymmetric.py:1027: in rsa_pkcs1v15_encrypt
    key_length = certificate_or_public_key.byte_size
/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_asymmetric.py:225: in byte_size
    return self.public_key.byte_size
/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_mac/asymmetric.py:255: in public_key
    handle_sec_error(res)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

error = 62385568, exception_class = <class 'OSError'>
    
>       raise exception_class(output)
E       OSError: OSStatus 62385568

/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_mac/_security.py:57: OSError

[wbond]

Do you know what kind of key it was (RSA, DSA, etc) and what size it was?

[wbond]

Actually, from the trace, it must have been an RSA key. If you can determine the key size, that would be helpful for improving the error message.

[behrtam]

SHA-256 with RSA Encryption ( 1.2.840.113549.1.1.11 ), Key Size=2.048 bits

[wbond]

Could you try a 4096 bit key and see if it errors? 2048 is pretty small for 2022. I recently renewed my code signing cert and Sectigo wouldn’t let me use a 2048 bit key. Perhaps Apple decided to error out on shorter keys now?

[behrtam]

4096 bits does run into the same error.

We are using oscrypto via pyas2lib.as2 ... will try to find some time to work on a minimal setup to reproduce this.

[wbond]

Is the certificate an RSAPSS cert?

[magicrobotmonkey]

I can recreate with a super simple self signed cert:

openssl req -newkey rsa:2048 -keyout test.key -x509 -out test.cer \
        -subj "/C=US/ST=CA/L=Thousand Oaks/O=test/OU=RegOps/CN=test" \
    -sha256 -days 1095

from oscrypto import asymmetric


with open("config/test.cer", "rb") as readme:
    cert = readme.read()

encryption_cert = asymmetric.load_certificate(cert)
print(encryption_cert.byte_size)

Traceback (most recent call last):
  File "/Users/abassett/work/ez_fda_as2/tests/test_oscrypto.py", line 8, in <module>
    print(encryption_cert.byte_size)
  File "/Users/abassett/venvs/oscrypto/lib/python3.10/site-packages/oscrypto/_asymmetric.py", line 234, in byte_size
    return self.public_key.byte_size
  File "/Users/abassett/venvs/oscrypto/lib/python3.10/site-packages/oscrypto/_mac/asymmetric.py", line 265, in public_key
    handle_sec_error(res)
  File "/Users/abassett/venvs/oscrypto/lib/python3.10/site-packages/oscrypto/_mac/_security.py", line 57, in handle_sec_error
    raise exception_class(output)
OSError: OSStatus 53899264

[magicrobotmonkey]

Appears to fire these messages in console:

[magicrobotmonkey]

Those might be a red herring? But they're also interspersed with these MacOS error issues:

[magicrobotmonkey]

It looks like it might be due to this deprecation? https://developer.apple.com/documentation/security/1396096-seccertificatecopypublickey

I did a sloppy port to SecCertificateCopyKey and it seems like it might be working. I'll try to pull together a proper PR.