api-umbrella
api-umbrella copied to clipboard
NREL
External IdP validation feature for API Backends
I am Andrés Muñoz, from Technical University of Madrid, working with @aalonsog, I have been working on the idea of implementing an external validation service in API-Umbrella in order to verify if a user token is valid or not. For achieving that, I followed the flow showed in the diagram #349 by @aalonsog and the guidelines stated by you in the previous comments.
This new feature allows to the API-umbrella users the possibility of make request to a registered API backend, using an API key or an OAuth2 token. If the user uses a token in the request, this token is verified using an external IdP, once the token is validated the user information is retrieved and redirect to the API backend following the usual workflow. The list of IdP's included in this development are:
FIWARE
Google
Facebook
GitHub
Hence, after the inclusion of this feature, the gatekeeper architecture is modified as is showed in the next figure:
This work was made in collaboration with @aalonsog @agordillo and @jsalvachua.
If you have any coment or need more information about it, please let me know.
@anmunoz: Hi there! Sorry for the delay in getting back to you. We've been busy with other things, but I'll try to take a look at this in the coming weeks. Thanks for the contribution and getting this started!
Hi, I'm Urtza Iturraspe, works in Tecnalia and I am using Ckan, biz ecosystem, Fiware at general in an European Project. I am trying to make offer using BIZ from Ckan I can do it but from BIZ I have some problems, I have modified all of these files in api-umbrella folder. Now I am trying to create a product Specificacion using BIZ Ecosystem and CKAN Dataset API as asset type and it returns an error because app_id returned is "None". I have two API Backend defined in API-Umbrella admin window. In Postman I make a GET call
https://localhost:9222/api-umbrella/v1/apis.json&start=0&length=100
{'headers': {u'X-Api-Key': u'U7B9ysawwNN0hBjEvBDeoJEyVEbEODDbjx2b85dX', u'X-Admin-Auth-Token': u'nx4HYGgPM8SbCTqRsmlXxPS7EPTTPbmkalgQQqyB'}, 'verify': False}
And It returns:
{
"draw": 0,
"recordsTotal": 2,
"recordsFiltered": 2,
"data": [
{
"backend_host": "www.ckanreplicate.com",
"backend_protocol": "https",
"balance_algorithm": "least_conn",
"created_at": "2018-10-31T07:22:58Z",
"created_by": "be817f88-e2d2-4b91-85ad-23a695921a04",
"deleted_at": null,
"frontend_host": "www.ckanreplicate.com",
"name": "ckan",
"servers": [
{
"host": "www.ckanreplicate.com",
"port": 443,
"id": "545d9ad3-9406-4523-92d6-c91642e146a1"
}
],
"settings": {
"allowed_ips": null,
"allowed_referers": null,
"anonymous_rate_limit_behavior": null,
"api_key_verification_level": null,
"api_key_verification_transition_start_at": null,
"append_query_string": "",
"authenticated_rate_limit_behavior": null,
"disable_api_key": null,
"error_data": {},
"error_templates": {},
"http_basic_auth": null,
"pass_api_key_header": false,
"pass_api_key_query_param": false,
"rate_limit_mode": null,
"require_https": null,
"require_https_transition_start_at": null,
"require_idp": null,
"required_roles": [
"api-umbrella-contact-form",
"api-umbrella-key-creator",
"provider",
"issue_customer"
],
"required_roles_override": null,
"id": "3455d01d-0892-499f-a453-76e9b8a59f4e",
"default_response_headers": null,
"headers": null,
"override_response_headers": null,
"rate_limits": null
},
"sort_order": 0,
"updated_at": "2018-11-27T11:44:51Z",
"updated_by": "be817f88-e2d2-4b91-85ad-23a695921a04",
"url_matches": [
{
"backend_prefix": "/",
"frontend_prefix": "/",
"id": "8bfabbf6-16cc-465b-bc7d-9ed25fcfbfcf"
}
],
"version": 2,
"id": "d2a306b0-2203-4d69-8613-0dd17d416375",
"rewrites": null,
"sub_settings": null,
"frontend_prefixes": "/"
},
{
"backend_host": "www.bizreplicate.com",
"backend_protocol": "http",
"balance_algorithm": "least_conn",
"created_at": "2018-11-27T11:47:16Z",
"created_by": "be817f88-e2d2-4b91-85ad-23a695921a04",
"deleted_at": null,
"frontend_host": "www.bizreplicate.com",
"name": "biz",
"servers": [
{
"host": "www.bizreplicate.com",
"port": 8443,
"id": "b908f0f3-bf86-463b-99af-58e8165a9f2d"
}
],
"settings": {
"allowed_ips": null,
"allowed_referers": null,
"anonymous_rate_limit_behavior": null,
"api_key_verification_level": null,
"api_key_verification_transition_start_at": null,
"append_query_string": null,
"authenticated_rate_limit_behavior": null,
"disable_api_key": null,
"error_data": {},
"error_templates": {},
"http_basic_auth": null,
"pass_api_key_header": null,
"pass_api_key_query_param": null,
"rate_limit_mode": null,
"require_https": null,
"require_https_transition_start_at": null,
"require_idp": null,
"required_roles": [
"api-umbrella-contact-form",
"provider",
"issue_customer",
"api-umbrella-key-creator"
],
"required_roles_override": null,
"id": "8b8fbf95-90c9-4385-b2cb-124dc5176b69",
"default_response_headers": null,
"headers": null,
"override_response_headers": null,
"rate_limits": null
},
"sort_order": 10000,
"updated_at": "2018-11-27T11:47:32Z",
"updated_by": "be817f88-e2d2-4b91-85ad-23a695921a04",
"url_matches": [
{
"backend_prefix": "/",
"frontend_prefix": "/",
"id": "8b899aeb-2ddd-4e95-bc48-115a70a2e8b1"
}
],
"version": 2,
"id": "d77196d6-c2e5-4159-90e0-bc067697d63b",
"rewrites": null,
"sub_settings": null,
"frontend_prefixes": "/"
}
]
}
``
If I take a look umbrella_client.py file (ckan_api_dataset plugin) validate_service method:
def validate_service(self, path):
err_msg = 'The provided asset is not supported. '
'Only services protected by API Umbrella are supported'
print ("Umbrella_client.py##validte_service INI")
print ("Umbrella_client.py##validte_service##path:"+str(path))
# Split the path of the service
paths = [p for p in path.split('/') if p != '']
if not len(paths):
# API umbrella resources include a path for matching the service
raise PluginError(err_msg)
# Make paginated requests to API umbrella looking for the provided paths
url = '/api-umbrella/v1/apis.json'
def page_processor(api):
front_path = [p for p in api['frontend_prefixes'].split('/') if p != '']
return len(front_path) == 0 or (len(front_path) <= len(paths) and front_path == paths[:len(front_path)])
matching_elem = self._paginate_data(url, err_msg, page_processor)
data_string = json.dumps(matching_elem)
# If the API is configured to accept access tokens from an external IDP save its external id
**app_id = None
if 'idp_app_id' in matching_elem['settings'] and len(matching_elem['settings']['idp_app_id']):
app_id = matching_elem['settings']['idp_app_id']**
return app_id
I have not matching_elem['settings']['idp_app_id'] ???
Please someone can help me with this?
Thanks a lot
Urtza
Hi @urtza2 have you tried with the github repo of apinf (https://github.com/apinf/api-umbrella) forked from api-umbrella?, that repo have a stable release that includes external idp validation. On the other hand this repo have more features for authorization that I think that is not your case of use
No I have install using these instructions ... sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 379CE192D401AB61 $ echo "deb https://dl.bintray.com/nrel/api-umbrella-ubuntu xenial main" | sudo tee /etc/apt/sources.list.d/api-umbrella.list $ sudo apt-get update $ sudo apt-get install api-umbrella
And after I make the changes that appear in this issue. I try using it. Thanks
How can I install API-Umbrella of github repo of apinf (https://github.com/apinf/api-umbrella)?. Docker file contain information of NREL repository. I have download as zip from this repo an try doing this:
shell> ./configure and I get some errors like:
fatal: Not a git repository (or any of the parent directories): .git
date: invalid date ‘-128-NOTFOUND’
-- Checking for one of the modules 'uuid'
CMake Error at build/work/cmake/share/cmake-3.8/Modules/FindPkgConfig.cmake:641 (message):
None of the required 'uuid' found
Call Stack (most recent call first):
CMakeLists.txt:83 (pkg_search_module)
CMake Error at CMakeLists.txt:65 (MESSAGE):
Could not find autoconf
Call Stack (most recent call first):
build/cmake/rsyslog.cmake:4 (require_program)
CMakeLists.txt:96 (include)
-- Configuring incomplete, errors occurred!
See also "/home/user1/Downloads/api-umbrella-master_APINF/CMakeFiles/CMakeOutput.log".
I'll try doing the same with NREL repo: 1.- ./configure -> get Makefile 2.- make -> get a lot of error.
What is the correct way to install API-Umbrella of this repo (https://github.com/apinf/api-umbrella) ?
Thanks in advance Urtza