lemur
lemur copied to clipboard
Netflix
create authority fails with nonetype
I'm deploying lemur in my environment now to test it out that is works.
We're using oauth2 with a oauth2 identity provider written in go. So oauth2 was broken for me until I changed the lowercase b to a capital B here: https://github.com/Netflix/lemur/compare/master...JohnTheodore:master
Here if the RFC and the built in library/function for golang: RFC: https://tools.ietf.org/html/rfc7617#section-2 and golang link: https://golang.org/src/net/http/request.go?s=27803:27869#L857
I think it's supposed to be capital Basic.
So once I fixed that, oauth2 works now.
I'm having an additional problem now. So my lemur.conf.py variables that matter more are here: https://gist.github.com/JohnTheodore/bfbd98105c7aa66b44d895018b9a4e35
After I can login as the first time (with my oauth2 user having the 'admin' role attached), I believe the next step I need to take is to create an authority. When I try to create an authority I get this error in the lemur journal: https://gist.github.com/JohnTheodore/19e8803f9c50a65be1b861aa67ce5222
There are dozens of fields in the form for creating an authority. I'm not sure which part of is is causing this error: TypeError: 'NoneType' object cannot be interpreted as an integer
Note, once I have things working and can cut a certificate, I'll make a small PR with any fixes or docs that can make things more clear.
Here are my kwargs from lemur/certificates/service.py(237)
https://gist.github.com/JohnTheodore/baae0d3cfd24881d5f3b88ac7af96f89
I can see 'body' is empty. When I make the authority am I also supposed to make a new cert as well?
For authorities, these are typically configured via the plugin itself: http://lemur.readthedocs.io/en/latest/administration.html#plugin-specific-options
You can typically configure which root/intermediate you need from your CA. When you create an authority for that plugin it will use these values. Some authorities create their values dynamically. Symantec/Verisign does not.
I set all those options in my lemur.conf.py: https://gist.github.com/JohnTheodore/bfbd98105c7aa66b44d895018b9a4e35
Though there are no authorities listed when I sign in.
I believe VERISIGN_INTERMEDIATE and VERISIGN_ROOT both need to be set, and be pem formatted strings. You can easily do open('path-to-pem', 'r').readlines() if thats what if you prefer.
hmm, I set all the variables for VERISIGN_* now.
VERISIGN_URL = 'certmanager-webservices.websecurity.symantec.com/vswebservices'
VERISIGN_PEM_PATH = '/etc/lemur/jt-vice-mpki-api.p7b'
VERISIGN_FIRST_NAME = 'lemur'
VERISIGN_LAST_NAME = 'viceapi'
VERISIGN_EMAIL = '[email protected]'
VERISIGN_INTERMEDIATE = '/etc/lemur/digitcert.intermediate.pem'
VERISIGN_ROOT = '/etc/lemur/digitcert.root.pem'
and these are set too:
# set the default plugin
LEMUR_DEFAULT_ISSUER_PLUGIN = 'verisign-issuer'
LEMUR_DEFAULT_AUTHORITY = 'verisign'```
but.. there is still no authority listed.
- Setting those files only allows the plugin to work, you still need to create an authority and select that plugin.
- The values need to be PEM strings not file paths (you have paths listed atm).
ok, these settings worked for creating an authority:
# MPKI settings
VERISIGN_URL = 'https://certmanager-webservices.websecurity.symantec.com/vswebservices'
VERISIGN_PEM_PATH = '/etc/lemur/jt-vice-mpki-api.p7b'
VERISIGN_FIRST_NAME = 'lemur'
VERISIGN_LAST_NAME = 'viceapi'
VERISIGN_EMAIL = '[email protected]'
VERISIGN_INTERMEDIATE = open('/etc/lemur/digitcert.intermediate.pem', 'r').read()
VERISIGN_ROOT = open('/etc/lemur/digitcert.root.pem', 'r').read()
When trying to create my first test certificate, I get this error:
[2018-05-24 07:42:14,716] ERROR in schema: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'PEM lib')]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/lemur/common/schema.py", line 160, in decorated_function
resp = f(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/views.py", line 270, in post
cert = service.create(**data)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 249, in create
cert_body, private_key, cert_chain, external_id, csr = mint(**kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 196, in mint
cert_body, cert_chain, external_id = issuer.create_certificate(csr, kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/plugins/lemur_verisign/plugin.py", line 197, in create_certificate
response = self.session.post(url, data=data)
File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 522, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.5/dist-packages/raven/breadcrumbs.py", line 326, in send
resp = real_send(self, request, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/adapters.py", line 423, in send
timeout=timeout
File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py", line 595, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py", line 352, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py", line 831, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connection.py", line 289, in connect
ssl_version=resolved_ssl_version)
File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 321, in ssl_wrap_socket
ctx.use_certificate_file(certfile)
File "/usr/local/lib/python3.5/dist-packages/OpenSSL/SSL.py", line 817, in use_certificate_file
_raise_current_error()
File "/usr/local/lib/python3.5/dist-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
from the debugger:
ipdb> data
{'validityPeriod': '1Y', 'email': '[email protected]', 'certProductType': 'Server', 'specificEndDate': '05/25/2018', 'csr': '-----BEGIN CERTIFICATE REQUEST-----\morestuffhere\n-----END CERTIFICATE REQUEST-----\n', 'signatureAlgorithm': 'sha256WithRSAEncryption', 'subject_alt_names': '', 'challenge': '12345', 'serverType': 'Apache', 'firstName': 'lemur', 'lastName': 'viceapi'}
ipdb> url
'https://certmanager-webservices.websecurity.symantec.com/vswebservices/rest/services/enroll'
ipdb> response = self.session.post(url, data=data)
*** OpenSSL.SSL.Error: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'PEM lib')]
ipdb>
I think you want readlines() not read() as you actually want the whole file read.
You will likely have to re-create the authority as it probably wasn't set correctly.
readlines puts it into an array. and the cryptography library that parse_certificate runs wants it to be a single string blob I think?. There were errors creating the authority when parse_certificates had the input as an array of strings, versus just one string for the body. read() does read the whole file as a single string.
This is the function that I think doesn't want an array of strings: https://github.com/Netflix/lemur/blob/master/lemur/common/utils.py#L54 https://cryptography.io/en/latest/x509/reference/#loading-certificates
I changed it to readlines(), and made a new authority, here is the error from that:
[2018-05-24 18:49:00,591] ERROR in schema: initializer for ctype 'char' must be a bytes of length 1, not str
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/lemur/common/schema.py", line 160, in decorated_function
resp = f(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/views.py", line 223, in post
return service.create(**data)
File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/service.py", line 108, in create
cert = upload(**kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 235, in upload
cert = Certificate(**kwargs)
File "<string>", line 4, in __init__
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 417, in _initialize_instance
manager.dispatch.init_failure(self, args, kwargs)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
return manager.original_init(*mixed[1:], **kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/models.py", line 144, in __init__
cert = lemur.common.utils.parse_certificate(kwargs['body'])
File "/usr/local/lib/python3.5/dist-packages/lemur/common/utils.py", line 54, in parse_certificate
return x509.load_pem_x509_certificate(body, default_backend())
File "/usr/local/lib/python3.5/dist-packages/cryptography/x509/base.py", line 43, in load_pem_x509_certificate
return backend.load_pem_x509_certificate(data)
File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 1132, in load_pem_x509_certificate
mem_bio = self._bytes_to_bio(data)
File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 436, in _bytes_to_bio
data_char_p = self._ffi.new("char[]", data)
TypeError: initializer for ctype 'char' must be a bytes of length 1, not str
initializer for ctype 'char' must be a bytes of length 1, not str
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/lemur/common/schema.py", line 160, in decorated_function
resp = f(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/views.py", line 223, in post
return service.create(**data)
File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/service.py", line 108, in create
cert = upload(**kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 235, in upload
cert = Certificate(**kwargs)
File "<string>", line 4, in __init__
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 417, in _initialize_instance
manager.dispatch.init_failure(self, args, kwargs)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
return manager.original_init(*mixed[1:], **kwargs)
File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/models.py", line 144, in __init__
cert = lemur.common.utils.parse_certificate(kwargs['body'])
File "/usr/local/lib/python3.5/dist-packages/lemur/common/utils.py", line 54, in parse_certificate
return x509.load_pem_x509_certificate(body, default_backend())
File "/usr/local/lib/python3.5/dist-packages/cryptography/x509/base.py", line 43, in load_pem_x509_certificate
return backend.load_pem_x509_certificate(data)
File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 1132, in load_pem_x509_certificate
mem_bio = self._bytes_to_bio(data)
File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 436, in _bytes_to_bio
data_char_p = self._ffi.new("char[]", data)
TypeError: initializer for ctype 'char' must be a bytes of length 1, not str
So the api credential (certificate) I got from Symantec (digicert) is a p7b file. The issue was the p7b needs to be in pem format. I can make a certificate now!
Is there a way I can force all certificates to never expires more than say 2 months after today's date?
also are there little scripts for renewing a cert for say apache, tomcat, and other services as cronjobs?
also what is the command to sync all the certs from mpki to lemur? meaning the older certs
hmm, if wanted to make API keys for say somesub.domain.com, and then have a script run that renews that cert every week say. Do I have to make a role, user, and API key each time I want to do that? the goal is to make all certificate renewals automated, run by cronjobs or something similar.