pxf
pxf copied to clipboard
Fix pxf home file structure in script
Now if we fly a pipeline for the multi-cluster branch, the pipeline will go red because the file structure under PXF_HOME has changed. See more details in PR 1089.
- This PR updates the related file paths in the scripts:
- Updated examples in README
- Renamed
TEMPLATES_DIR=$(PXF_HOME)/templatestoSERVER_TEMPLATES_DIR="$(PXF_HOME)/templates/servers" - Created
CONF_TEMPLATES_DIR="${PXF_HOME}/templates/conf"for copying conf templates fromPXF_HOMEtoPXF_BASE
- This PR also provides a workaround for the pipeline failure when using
PXF_HOMEas thePXF_BASE.
In the current configure_pxf_server function, if user uses the default value for PXF_BASE which is PXF_HOME, the script will skip running pxf prepare, and continues to copy pxf-site.xml from ${PXF_HOME}/templates/servers/pxf-site.xml to ${PXF_HOME}/servers/pxf-site.xml. The prepare script will populate conf and server directories under PXF_BASE. Since now the servers folder is nested inside templates in the PXF_HOME and the pxf prepare is skipped, the pipeline will fail.
The workaround is manually setting a PXF_BASE for the tasks which were using PXF_HOME as the default value. We need to revert them after we fix the issue:
- delete line 77 in
test-cli-tpl.yml - delete line 84 and line 98 in
test-file-tpl.yml - delete line 195 and line 216 in
test-multinode-tpl.yml - revert the change in line 82 and line 84 in
test-tpl.yml - delete line 41 in
test-upgrade-extension-tpl.yml
- Now the pipeline is almost green except one task
test-pxf-gp6-ext-hdp2-upgrade-extension-centos7. Since it usespxf 6.6.0 packagewhich has the old file structure, the pipeline fails forNo such file or directory.