fhir-data-pipes icon indicating copy to clipboard operation
fhir-data-pipes copied to clipboard

Published 20 hours ago • verified publisher icon google

External backing storage demo

Open bennsimon opened this issue 2 years ago • 3 comments
trafficstars

Description of what I changed

  • Adds a demonstration of deploying a spark thrift server with external backing storage (Postgresql).
    • Uses an intermediary image that provides all the JDBC drivers that will be used by hive.

E2E test

TESTED:

Tested on my local machine by running below:

docker-compose -f compose-controller-spark-sql-external-storage.yaml up

Checklist: I completed these to help reviewers :)

  • [x] I have read and will follow the review process.

  • [x] I am familiar with Google Style Guides for the language I have coded in.

    No? Please take some time and review Java and Python style guides.

  • [x] My IDE is configured to follow the Google code styles.

    No? Unsure? -> configure your IDE.

  • [ ] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

  • [ ] I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

  • [ ] All new and existing tests passed.

  • [x] My pull request is based on the latest changes of the master branch.

    No? Unsure? -> execute command git pull --rebase upstream master

bennsimon avatar Apr 28 '23 12:04 bennsimon

/gcbrun

bashir2 avatar Apr 28 '23 15:04 bashir2

Thanks again @bennsimon; I think we discussed this in other forums, can you please remind me if this is still needed and if yes, update it to be merged? It would be great to add a README.md to the docker/drivers-build directory describing that this is for demo purposes and not integrated into our continuous tests.

bashir2 avatar Jul 20 '23 16:07 bashir2

Thanks again @bennsimon; I think we discussed this in other forums, can you please remind me if this is still needed and if yes, update it to be merged? It would be great to add a README.md to the docker/drivers-build directory describing that this is for demo purposes and not integrated into our continuous tests.

Hey @bashir2, yeah i think it will be needed to demonstrate that one can setup spark thriftserver with database other than derby database.

I will add the README.md.

bennsimon avatar Jul 21 '23 06:07 bennsimon