djangosaml2idp icon indicating copy to clipboard operation
djangosaml2idp copied to clipboard

Bug: TypeError: unhashable type: 'RequestedAuthnContext'

Open dmitry-viskov opened this issue 5 years ago • 11 comments

After such request:

[19/Mar/2019 21:10:58] "GET /idp/sso/redirect?SAMLRequest=fVNNj9owEL3vr0C5J44XwrIWRKLQDyQKEaQ99FKZeOhacuzUnuzSf1%2FHYSmtuvjgSON5b2bem0wdr1XD5i0%2B6R38bMHh3cCfU620Y%2BFxFrVWM8OddEzzGhzDiu3nn9fsPklZYw2ayqjoH9htFHcOLEqje9hqOYu2m%2Ffr7cfV5juMKM1Gk3FGsxRSfqDjDOBhKIajQzqhgo75IX1IM95Dv4J1nmcWedo%2BUljzLAXYja86i0CcElW7BP1kybmacy2stEOu0eNS%2Bhinw5g%2BlveU0ZRlk2993tJDpOYY6J8QG0aIcyYxDWgQbaAkUjRdkFgQ0kKFlx6CKu%2BkFlL%2FuC3GoU9y7FNZFnGx3Zc9yfxVpIXRrq3B7sE%2Bywq%2B7NaXfq7HI9y7SCpTNwoQCDY87pwgPVse7mkXYUEBm7%2FJEWA1IBcceeJNnZJr3B%2BmhnUqr5aFUbL6FeLd%2BWBszfHtqWlCQ0SK%2BBhSWatdA5U8ShDRhWaulHlZWODojUTbQjQgfxU%2FbyyIsL9eJoQTDhZeAW6l62yDE%2Fee5BfKfv7r9IXyy7iDY35zXytWdXk%2BXPjPi7Gis9j7DaK03DdvLJ5F%2Bi953zW50XZ%2B9%2Fp8%2FTPmvwE%3D&RelayState=ssotest HTTP/1.0" 302 0

I've got an error:

Internal Server Error: /idp/login/process/
Traceback (most recent call last):
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/utils/decorators.py", line 62, in _wrapper
    return bound_func(*args, **kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/utils/decorators.py", line 58, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/contrib/auth/mixins.py", line 52, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/djangosaml2idp/views.py", line 83, in dispatch
    return super(IdPHandlerViewMixin, self).dispatch(request, *args, **kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/django/views/generic/base.py", line 89, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/djangosaml2idp/views.py", line 153, in get
    AUTHN_BROKER.add(authn_context_class_ref(req_authn_context), "")
  File "/Users/dmitry/.pyenv/versions/sso-edx-venv/lib/python3.6/site-packages/saml2/authn_context/__init__.py", line 96, in add
    self.db["key"][key].append(_ref)
TypeError: unhashable type: 'RequestedAuthnContext'
[19/Mar/2019 21:10:58] "GET /idp/login/process/ HTTP/1.0" 500 136835

Could someone help me please? is it a bug or i do something wrong?

dmitry-viskov avatar Mar 19 '19 21:03 dmitry-viskov

Try to install the late code refactor from here https://github.com/peppelinux/djangosaml2idp

If the problem persists I'll take a look more in the deep, probably would it be Better to include SP and idp metadata or settings.py pysaml related configuration for Better acknowledgment

peppelinux avatar Mar 19 '19 21:03 peppelinux

As peppelinux said, please send your SP and IDP config sections from settings.py, as well as the metadata file for the relevant SP.

I decoded the SAMLRequest in the url you gave, and it came out as absolute gibberish. It's possible that the malformed SAMLRequest is causing this issue. Try removing SP Request encryption (if enabled) and making sure that the SP is sending a proper request, then try again. Let me know if there's improvement.

askvortsov1 avatar Mar 19 '19 22:03 askvortsov1

@askvortsov1 mind that samlrequest in http-redirect Is not a simple b64 but a zlib compressed before b64., Pass It to https://github.com/peppelinux/djangosaml2idp/blob/master/djangosaml2idp/utils.py#L8

I wrote It in a flash, possibile enhancement to It are also possibile, let me know

peppelinux avatar Mar 19 '19 22:03 peppelinux

@dmitry-viskov put logging level to debug, you'll see saml reqeust and response in stdout. In settings.py:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse'
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'djangosaml2idp': {
            'handlers': ['console'],
            'level': 'DEBUG',
        },
    }
}

peppelinux avatar Mar 20 '19 09:03 peppelinux

@peppelinux @askvortsov1 thanks for the help! i've used your branch https://github.com/peppelinux/djangosaml2idp - and it is working now! is it stable? when do you plan to release it?

Regarding my environment - i'm using djangosaml2idp as idp. Service provider is the part of OpedEdx project and not connected with djangosaml2idp. Settings:

INSTALLED_APPS += ('djangosaml2idp',)
IDP_BASE_URL = '%s/idp' % SSO_BASE_URL
SAML_IDP_NAME = "SSO IdP"
SAML_IDP_DESCRIPTION = "SSO IdP"
SAML_SETTINGS_VALID_FOR = 7 * 24
SAML_IDP_SPMETADATA = {
    'remote': [{'url': 'http://edx.lms.test/auth/saml/metadata.xml'}],
}
SAML_IDP_SPCONFIG = {
    'http://edx.lms.test/auth/saml/metadata.xml': {
        'processor': 'djangosaml2idp.processors.BaseProcessor',
        'attribute_mapping': {
            'email': 'email',
            'first_name': 'first_name',
            'last_name': 'last_name',
            'full_name': 'full_name',
            'is_staff': 'is_staff',
            'is_superuser': 'is_superuser',
            'id': 'uid'
        }
    }
}
SAML_IDP_CONFIG = {
        'debug': DEBUG,
        'xmlsec_binary': get_xmlsec_binary(['/opt/local/bin', '/usr/bin/xmlsec1']),
        'entityid': '%s/metadata' % IDP_BASE_URL,
        'description': SAML_IDP_DESCRIPTION,
        'service': {
            'idp': {
                'name': SAML_IDP_NAME,
                'endpoints': {
                    'single_sign_on_service': [
                        ('%s/sso/post' % IDP_BASE_URL, saml2.BINDING_HTTP_POST),
                        ('%s/sso/redirect' % IDP_BASE_URL, saml2.BINDING_HTTP_REDIRECT),
                    ],
                },
                'name_id_format': [NAMEID_FORMAT_EMAILADDRESS, NAMEID_FORMAT_UNSPECIFIED],
                'sign_response': True,
                'sign_assertion': True,
            },
        },
        'metadata': SAML_IDP_SPMETADATA,
        # Signing
        'key_file': BASE_DIR + '/saml2/certificates/private.key',
        'cert_file': BASE_DIR + '/saml2/certificates/public.cert',
        # Encryption
        'encryption_keypairs': [{
            'key_file': BASE_DIR + '/saml2/certificates/private.key',
            'cert_file': BASE_DIR + '/saml2/certificates/public.cert',
        }],
        'valid_for': SAML_SETTINGS_VALID_FOR,  # hours
}

http://edx.lms.test/auth/saml/metadata.xml:

<md:EntityDescriptor
    xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" cacheDuration="PT604800S" entityID="http://edx.lms.test/auth/saml/metadata.xml">
    <md:SPSSODescriptor AuthnRequestsSigned="false" WantAssertionsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
        <md:KeyDescriptor
            xmlns:ds="http://www.w3.org/2000/09/xmldsig#" use="signing">
            <ds:KeyInfo>
                <ds:X509Data>
                    <ds:X509Certificate>
MIICqjCCAZICCQCzt8BXga429zANBgkqhkiG9w0BAQsFADAXMRUwEwYDVQQDDAxlZHgubG1zLnRlc3QwHhcNMTkwMzE4MTkyNTUzWhcNMjkwMzE1MTkyNTUzWjAXMRUwEwYDVQQDDAxlZHgubG1zLnRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3bTrys8NoMZFwOOTkzGZqRQG2HrLwLUzd6KbEXascfYKaVpxW1IQqtoAWdiOlVjOffTvjwyKByN0uKAzKMNGv8IuX85AmxXoN2OXZ6pql25PrMxCG8ZbH49L7i+RQ+kpCMdB+dZ3aB1JPEqmbkzmwaC9HxcicKNWJPA5/O19PebCagbplpzBb5N4Hs8s/rOI9TH7iyK4hteT95zRiICiOZ++88KTnlbNpuMR0L65Y4EUxdVVBTxLKacQYRnE5hF5dVZOnHYQ01GVM/cg0pbKPIDSbaVWga46CPJS/NYS0knIrecp8iplky/FY/tR0Wb8TgHRHtzKJ6Oh9BoMr/dDFAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAEwt8GUH82AqlUK8MNKAO1HpYVUbd/4SfxqLn2q/WcFknNrlf+hHnAwGkWLuP6Txo6VD/Xg8Q0SNWt+99uf53S2Y9hYSOVtO2LpqJxBSJ+YEC7QLaSHaVLf63/39PYpLPkeVW0l1K6kl2+rWIEgT/AU2H8xT99ePXf8tPtGtBNAdcT6U8Dsxftm5Juj972dqsISkRxtK0PGk7r274Bu5RzEW3PcgIcazP3MMF3qB3IfhmdoZETpd4fKV8gej1LyWL3VJrwRmUe+2Zg7yp8ntF2Y1QjtK/g+hVK9TvyQpdS6VPZERGMttiREEj5f+dpkEz0zSZ0bGQI2b9YP8tDNS4Fo=
</ds:X509Certificate>
                </ds:X509Data>
            </ds:KeyInfo>
        </md:KeyDescriptor>
        <md:NameIDFormat>
urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified
</md:NameIDFormat>
        <md:AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="http://edx.lms.test/auth/complete/tpa-saml/" index="1"/>
    </md:SPSSODescriptor>
    <md:Organization>
        <md:OrganizationName xml:lang="en-US">edx.lms.test</md:OrganizationName>
        <md:OrganizationDisplayName xml:lang="en-US">edx.lms.test.</md:OrganizationDisplayName>
        <md:OrganizationURL xml:lang="en-US">http://edx.lms.test</md:OrganizationURL>
    </md:Organization>
    <md:ContactPerson contactType="technical">
        <md:GivenName>edx.lms.test Support</md:GivenName>
        <md:EmailAddress>[email protected]</md:EmailAddress>
    </md:ContactPerson>
    <md:ContactPerson contactType="support">
        <md:GivenName>edx.lms.test Support</md:GivenName>
        <md:EmailAddress>[email protected]</md:EmailAddress>
    </md:ContactPerson>
</md:EntityDescriptor>

dmitry-viskov avatar Mar 20 '19 10:03 dmitry-viskov

SP OpenEdx settings:

Screenshot 2019-03-20 в 13 21 31

Private key:

MIIEwAIBADANBgkqhkiG9w0BAQEFAASCBKowggSmAgEAAoIBAQC3bTrys8NoMZFw
OOTkzGZqRQG2HrLwLUzd6KbEXascfYKaVpxW1IQqtoAWdiOlVjOffTvjwyKByN0u
KAzKMNGv8IuX85AmxXoN2OXZ6pql25PrMxCG8ZbH49L7i+RQ+kpCMdB+dZ3aB1JP
EqmbkzmwaC9HxcicKNWJPA5/O19PebCagbplpzBb5N4Hs8s/rOI9TH7iyK4hteT9
5zRiICiOZ++88KTnlbNpuMR0L65Y4EUxdVVBTxLKacQYRnE5hF5dVZOnHYQ01GVM
/cg0pbKPIDSbaVWga46CPJS/NYS0knIrecp8iplky/FY/tR0Wb8TgHRHtzKJ6Oh9
BoMr/dDFAgMBAAECggEBAKhgnLwuSDJdTiS2FWfyOhFTRz7m71uKOgV6i5u2YFzG
2hzTZLBQZQLQhMQZrp8XgkvlqhRqDEMXH/I+0X7JUbMwHPRfoPP2UxRjBpwDv/KA
nkhJl+JH4yGbARHoWK39FC2vjLSJQ13hWq07dHZbr5lACibYaE7xLRPNDYG9/r/0
ZL//10gPP3HCAzrIW8hBErCAtr/uCu6J9ImtU9EEj2q4/LYBm8kOYIeaFMVg2xjm
lzKifiyukOIEm7vwMTmHLaRzK7voigQ+yyKAwzRyvmGjx6eouPuWxGUCzAp1EKg+
7xWAMK+dqT0wSqiCbdwYMXHyZRQAiJ0VN+Ak+5TcQAECgYEA8sXxbuv4wJmhfKxg
gXAEeJR66D/+NbDB8SLidU9hXMgy4UtqPvc5mzGqZnTPdGjtPSW+FkJs/BNggEkv
bBkFEMWZXw2gazzZ9EqkNhY5/s6lFradtJYeMNam2u38qdtXU8QY9ovEPDBpl3lm
6zjYA8ltTSdRzccHLcYlp55e4NUCgYEAwWuOj8yUzfv+QLCEwh1sp9B/cmlxXnJ/
g5FXr0jWWgHQZR69m4tqFFALheXR7SnzXxZdKoFqiqlT5U3zJrg5JRnGNpP0oTh8
SEnsvxnVbSEEM1wlooUPi3EARkndO61RWfWccbHNnIuyYsFO1CHhia4kwcnnc1dz
4xDmdXOlqDECgYEA33VpNCMJdSYz6RC1R4723pI0yz/T2lGXQkBM15O4V3GCu7Lx
2YEvyfSeFu5f4ZOPAqCR5eVAcYnYYlx6xPIGophvAnoFdY+uQljN62br/OjyNgTj
BkzC/0skjBFQZnwZVmYCYDTn7ErXdnBtVgeddWbbL5AIkPMoXW6sTCvOwv0CgYEA
mRfqCjXS3kuDzm4pVn/1vcpQ5h1Bp5EXDuterRYI28styKwRRemOeC93P29NvtuC
j5PjAD7ZNpGaHxiYyd/tE0mYUyC+81EDRjCFHCl1feAxM/nOrBmH2UYDAp96Zb7k
ckDuqCsk5LDJ7FK3cz6yBsj4vQ5zjR7+Ug7RasuzpjECgYEAr5HQnB3lQ7GE2evA
2+MYMI0sHp9k9Qtq44zFAH/ry8Z3Lc2JT0673KNBm7uqq/q9YaaKDqq0weHZ0j4Y
n2aKy/ffoVcbIkZ1cDR6diHEBwk5SJfrbhyRWyzOp1EX1ZyR7PXXZviYfE998ctC
h92+YRZucr+bl7eJ64zZI7O0NwM=

Public key:

MIICqjCCAZICCQCzt8BXga429zANBgkqhkiG9w0BAQsFADAXMRUwEwYDVQQDDAxl
ZHgubG1zLnRlc3QwHhcNMTkwMzE4MTkyNTUzWhcNMjkwMzE1MTkyNTUzWjAXMRUw
EwYDVQQDDAxlZHgubG1zLnRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC3bTrys8NoMZFwOOTkzGZqRQG2HrLwLUzd6KbEXascfYKaVpxW1IQqtoAW
diOlVjOffTvjwyKByN0uKAzKMNGv8IuX85AmxXoN2OXZ6pql25PrMxCG8ZbH49L7
i+RQ+kpCMdB+dZ3aB1JPEqmbkzmwaC9HxcicKNWJPA5/O19PebCagbplpzBb5N4H
s8s/rOI9TH7iyK4hteT95zRiICiOZ++88KTnlbNpuMR0L65Y4EUxdVVBTxLKacQY
RnE5hF5dVZOnHYQ01GVM/cg0pbKPIDSbaVWga46CPJS/NYS0knIrecp8iplky/FY
/tR0Wb8TgHRHtzKJ6Oh9BoMr/dDFAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAEwt
8GUH82AqlUK8MNKAO1HpYVUbd/4SfxqLn2q/WcFknNrlf+hHnAwGkWLuP6Txo6VD
/Xg8Q0SNWt+99uf53S2Y9hYSOVtO2LpqJxBSJ+YEC7QLaSHaVLf63/39PYpLPkeV
W0l1K6kl2+rWIEgT/AU2H8xT99ePXf8tPtGtBNAdcT6U8Dsxftm5Juj972dqsISk
RxtK0PGk7r274Bu5RzEW3PcgIcazP3MMF3qB3IfhmdoZETpd4fKV8gej1LyWL3VJ
rwRmUe+2Zg7yp8ntF2Y1QjtK/g+hVK9TvyQpdS6VPZERGMttiREEj5f+dpkEz0zS
Z0bGQI2b9YP8tDNS4Fo=

dmitry-viskov avatar Mar 20 '19 10:03 dmitry-viskov

Thank you @dmitry-viskov, The good news is that everything is working now, the code that you actually using seems to be quite stable and we didn't get any exception at this moment. Today I've merged the SLO branch into master, this means that Single LogOut service is now available. I'm pushing PR to @askvortsov1 repository to faster feedback, feel free to contribute.

I also suggest you the following things:

  • Change your private and public key, this is a public thread and I'm sure that you wan't go in production with a shared private key
  • Get it working with a stronger data integrity verification following this note

peppelinux avatar Mar 20 '19 10:03 peppelinux

thanks @peppelinux

Change your private and public key, this is a public thread and I'm sure that you wan't go in production with a shared private key

these keys are only for the local usage. I'm not going to use it somewhere else

dmitry-viskov avatar Mar 20 '19 11:03 dmitry-viskov

@mhindery we should merge this into master and continue to develop in a development branch if you agree. I think that this issue will remain opened until we done this if you agree

peppelinux avatar Mar 21 '19 17:03 peppelinux

This issue is solved by the repr_saml addition, right? It's unrelated to logout unless I'm missing something? I can merge that (being only that part from the referenced MR) into master and release a new version for this (as it's a bugfix vs new features and other backwards-incompatible changes, releasing just a bugfix-version might be useful)

mhindery avatar Mar 28 '19 21:03 mhindery

This issue is solved by the repr_saml addition, right? It's unrelated to logout unless I'm missing something? I can merge that (being only that part from the referenced MR) into master and release a new version for this (as it's a bugfix vs new features and other backwards-incompatible changes, releasing just a bugfix-version might be useful)

I don't believe so: repr_saml is only used to display the saml request in a pretty format for debug statements. After doing a tiny bit of digging, I think I might have found the actual fix: At some point during refactoring, req_info stopped being passed to the get_authn method, meaning that the context has been defaulting to PASSWORD regardless of what the request actually requested. This seems to have fixed OP's problem, but it's probably something that should be revisited later.

askvortsov1 avatar Mar 28 '19 22:03 askvortsov1