Arrow-flight connector

[sabbasani]

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

• [ ] Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
• [ ] PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• [ ] Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• [ ] If release notes are required, they follow the release notes guidelines.
• [ ] Adequate tests were added if applicable.
• [ ] CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add Arrow Flight connector :pr:`23032`
* Add documentation for the :doc:`/connector/base-arrow-flight`  :pr:`23032`

If release note is NOT required, use:

== NO RELEASE NOTE ==

 

[steveburnett]

Consider adding documentation for the new connector.

Suggest revising the release note entry to follow the Release Note Guidelines:

== RELEASE NOTES ==

General Changes
* Add Arrow Flight connector :pr:`23032`

[steveburnett]

Suggest change to the release note entry as follows:

== RELEASE NOTES ==

General Changes
* Add Arrow Flight connector :pr:`23032`

The documentation for Arrow Flight Connector appears to be being added in #23212 , so it doesn't need to be mentioned in this release note.

[linux-foundation-easycla[bot]]

The committers listed above are authorized under a signed CLA.

• :white_check_mark: login: sabbasani (2bcce6b5daa507aa318afc8303e1dfe82ffbed65)

[sabbasani]

@tdcmeehan @BryanCutler @steveburnett we have addressed all comments above.Can you please review

[steveburnett]

Suggest update of the release note to include the PR number in both entries, and to link to the new doc from the release note.

== RELEASE NOTES ==

General Changes
* Add Arrow Flight connector :pr:`23032`
* Add documentation for the :doc:`/connector/base-arrow-flight`  :pr:`23032`

[tdcmeehan]

@sabbasani is this PR still being worked on?

[sabbasani]

@sabbasani is this PR still being worked on?

@sabbasani is this PR still being worked on?

Yes @tdcmeehan I am working on testcases as discussed earlier

[sabbasani]

@tdcmeehan I have added testcases for arrow flight connector. Can you please review.

[tdcmeehan]

1. Please add a new test that extends from AbstractTestIntegrationSmokeTest. Curious how this goes.
2. Please add a new CI job specifically for Arrow Flight, since this adds a significant number of tests.
3. Please add some unit level tests--this module is lacking any. Specifically, test the configuration class, test round trip for any class that is serialized (serialize to and from JSON and make sure they are equivalent), and unit level tests for page to Arrow (and vice versa). This is not comprehensive--please add unit tests in general for complex functionality, serialization and object equivalence. See presto-base-jdbc for examples of unit tests in addition to integration tests.

[elbinpallimalilibm]

1. Please add a new test that extends from AbstractTestIntegrationSmokeTest. Curious how this goes.
2. Please add a new CI job specifically for Arrow Flight, since this adds a significant number of tests.
3. Please add some unit level tests--this module is lacking any. Specifically, test the configuration class, test round trip for any class that is serialized (serialize to and from JSON and make sure they are equivalent), and unit level tests for page to Arrow (and vice versa). This is not comprehensive--please add unit tests in general for complex functionality, serialization and object equivalence. See presto-base-jdbc for examples of unit tests in addition to integration tests.

1. Created TestArrowFlightIntegrationSmokeTest
2. Created .github/workflows/arrow-flight-tests.yml
3. Added ArrowPageUtilsTest , TestArrowColumnHandle, TestArrowHandleResolver, TestArrowSplit, TestArrowTableHandle, TestArrowTableLayoutHandle

[prestodb-ci]

Saved that user @sabbasani is from IBM

[ethanyzhang]

Hi @sabbasani @elbinpallimalilibm based on the discussion we had with Shweta and Sandhya, we are gonna take over this work, @BryanCutler will help move this PR over the finishing line. We keep everyone who worked on this PR as co-authors. Thank you. cc @tdcmeehan

[BryanCutler]

Continuing this PR at https://github.com/prestodb/presto/pull/24427

[BryanCutler]

https://github.com/prestodb/presto/pull/24427 has been merged, will close this one