nipyapi
nipyapi copied to clipboard
Chaffelson
automated deployment with Nested PG and external controller services giving issues
- Nipyapi version: Latest
- NiFi version: 1.11.4
- NiFi-Registry version:
- Python version: 3.7
Description
I am trying to do deployment from one registry to another of my flow. My flow contains nested PG and external controller services and parameter contexts and RPG. when i run the deploy_version method of nipyapi, it gives below error
Traceback (most recent call last): File "C:\Program Files\Python\lib\site-packages\nipyapi\utils.py", line 520, in rest_exceptions yield File "C:\Program Files\Python\lib\site-packages\nipyapi\versioning.py", line 766, in deploy_flow_version version=target_flow.version File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\apis\process_groups_api.py", line 858, in create_process_group (data) = self.create_process_group_with_http_info(id, body, **kwargs) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\apis\process_groups_api.py", line 945, in create_process_group_with_http_info collection_formats=collection_formats) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\api_client.py", line 326, in call_api _return_http_data_only, collection_formats, _preload_content, _request_timeout) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\api_client.py", line 153, in __call_api _request_timeout=_request_timeout) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\api_client.py", line 371, in request body=body) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\rest.py", line 268, in POST body=body) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\rest.py", line 224, in request raise ApiException(http_resp=r) nipyapi.nifi.rest.ApiException: (500) Reason: Internal Server Error HTTP response headers: HTTPHeaderDict({'Date': 'Sun, 09 Aug 2020 09:16:10 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31540000', 'Content-Type': 'text/plain', 'Vary': 'Accept-Encoding', 'Content-Length': '79', 'Server': 'Jetty(9.4.19.v20190610)'}) HTTP response body: An unexpected error has occurred. Please check the logs for additional details.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/charu_chaudhary/Desktop/DataIng/Data_Pipeline/features/tests/automated_deployment -new - args.py", line 169, in
What I Did
same as description. I exported flow from registry 1 imported in bucket of registry2
nipyapi.versioning.deploy_flow_version(
parent_id=nipyapi.canvas.get_root_pg_id(),
location=(0, 0),
bucket_id=prod_bucket.identifier,
flow_id=prod_flow.identifier,
reg_client_id=reg_client.id,
version=None
)
Urgency
It is blocking me to do deployment in production environment deployment, please help asap. deployment is supposed to be next week.
Please help!!!!!
Nested PGs generally aren't supported anyway, and don't work well in registry. The error you received says that you need to check the NiFi log as the server was unable to process your submission, but what you are trying to do probably won't work anyway.
On Sun, 9 Aug 2020, 10:18 charu123409, [email protected] wrote:
- Nipyapi version: Latest
- NiFi version: 1.11.4
- NiFi-Registry version:
- Python version: 3.7
Description
I am trying to do deployment from one registry to another of my flow. My flow contains nested PG and external controller services and parameter contexts and RPG. when i run the deploy_version method of nipyapi, it gives below error
Traceback (most recent call last): File "C:\Program Files\Python\lib\site-packages\nipyapi\utils.py", line 520, in rest_exceptions yield File "C:\Program Files\Python\lib\site-packages\nipyapi\versioning.py", line 766, in deploy_flow_version version=target_flow.version File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\apis\process_groups_api.py", line 858, in create_process_group (data) = self.create_process_group_with_http_info(id, body, **kwargs) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\apis\process_groups_api.py", line 945, in create_process_group_with_http_info collection_formats=collection_formats) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\api_client.py", line 326, in call_api _return_http_data_only, collection_formats, _preload_content, _request_timeout) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\api_client.py", line 153, in __call_api _request_timeout=_request_timeout) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\api_client.py", line 371, in request body=body) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\rest.py", line 268, in POST body=body) File "C:\Program Files\Python\lib\site-packages\nipyapi\nifi\rest.py", line 224, in request raise ApiException(http_resp=r) nipyapi.nifi.rest.ApiException: (500) Reason: Internal Server Error HTTP response headers: HTTPHeaderDict({'Date': 'Sun, 09 Aug 2020 09:16:10 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31540000', 'Content-Type': 'text/plain', 'Vary': 'Accept-Encoding', 'Content-Length': '79', 'Server': 'Jetty(9.4.19.v20190610)'}) HTTP response body: An unexpected error has occurred. Please check the logs for additional details.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "C:/Users/charu_chaudhary/Desktop/DataIng/Data_Pipeline/features/tests/automated_deployment -new - args.py", line 169, in version=None File "C:\Program Files\Python\lib\site-packages\nipyapi\versioning.py", line 766, in deploy_flow_version version=target_flow.version File "C:\Program Files\Python\lib\contextlib.py", line 130, in exit self.gen.throw(type, value, traceback) File "C:\Program Files\Python\lib\site-packages\nipyapi\utils.py", line 523, in rest_exceptions raise ValueError(e.body) ValueError: An unexpected error has occurred. Please check the logs for additional details. What I Did
same as description. I exported flow from registry 1 imported in bucket of registry2
nipyapi.versioning.deploy_flow_version( parent_id=nipyapi.canvas.get_root_pg_id(), location=(0, 0), bucket_id=prod_bucket.identifier, flow_id=prod_flow.identifier, reg_client_id=reg_client.id, version=None )
Urgency
It is blocking me to do deployment in production environment deployment, please help asap. deployment is supposed to be next week.
Please help!!!!!
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Chaffelson/nipyapi/issues/214, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZAZOEN5SAT3A5YT3CWWCDR7ZSVJANCNFSM4PZDBUXA .
Thanks for your response, could you suggest how to handle this in that case? My parent PG has 4 PGs inside and processors. Also one question does this automated deployment cater external controller services an parameter context??
Controllers are imported if attached to the versioned PG. Parameter contexts are not importable in that way, you must recreate them via the GUI or API. The standard design is a flat set of PGs that you connect together with input/output ports
On Sun, 9 Aug 2020, 20:11 charu123409, [email protected] wrote:
Thanks for your response, could you suggest how to handle this in that case? My parent PG has 4 PGs inside and processors. Also one question does this automated deployment cater external controller services an parameter context??
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Chaffelson/nipyapi/issues/214#issuecomment-671089881, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZAZOF2XVOOYINRJF2SBVLR73YHDANCNFSM4PZDBUXA .
One more doubt, it won't even work from one bucket to another within same registry? then how do we do deployment in that case for nested PGs? please respond
Don't use nested PGs, or only version the top most one.
On Tue, 11 Aug 2020, 09:22 charu123409, [email protected] wrote:
One more doubt, it won't even work from one bucket to another within same registry? then how do we do deployment in that case for nested PGs? please respond
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Chaffelson/nipyapi/issues/214#issuecomment-671804261, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZAZOBREJCYGROTANTYVZLSAD5TJANCNFSM4PZDBUXA .
I tried the second part wherein only top most is versioned but dint work.
I removed versioning for all the sub PGs and versioned the top most one but still faced the above issue. I am not sure whats happening.
It appears like this although it has 4 PGs inside it and many processors