[Bug] PXF 6.10 doesn't compile with Cloudberry

[ostinru]

Apache Cloudberry version

1.6.0 and current main

What happened

Hi, cloudberry team! I am trying to compile PXF 6.10 (not cloudberry-pxf, as it quite lags behind original PXF v6.4 vs v6.10) for Cloudberry and facing compilation issue:

1.430 src/pxfdelimited_formatter.c: In function 'new_pxfdelimited_state':
1.430 src/pxfdelimited_formatter.c:237:76: error: 'FormatterData' has no member named 'fmt_conversion_proc'
1.430   237 |         fmt_state->enc_conversion_proc = ((FormatterData*) fcinfo->context)->fmt_conversion_proc;
1.430       |                                                                            ^~
1.437 make[1]: Leaving directory '/usr/local/pxf_src/external-table'
1.437 make[1]: *** [<builtin>: src/pxfdelimited_formatter.o] Error 1

As I can see this feature was in Greenplum for 10 years https://github.com/greenplum-db/gpdb-archive/blame/main/src/include/access/formatter.h#L66

And it was removed during "Initial Cloudberry code dump." 10538adb70ac5e81abbe0cb16a9e1cf66ad808ec (with no detailed reasoning available).

Also functions pg_custom_to_server and pg_server_to_custom marked as /* Obsolete? */

I have checked - there is no such file in vanilla postgres (v9.4 and v14.0)

So, question is: what is the reason for removing this fields and functions (hope, you have full history of cloudberry).

What you think should happen instead

No response

How to reproduce

cherry-pick all commits from original PXF to cloudberry-pxf, and trigger build.

following commit will break build: https://github.com/greenplum-db/pxf-archive/commit/b21768715c7dc73b8598951036d44bb286055ff0#diff-8f1c5b6536bab3ebb1a81b1f2c978599cc995fe59d3fd5c07ed1fb27b5be6f45R237

Operating System

ubuntu 22.04

Anything else

No response

Are you willing to submit PR?

• [ ] Yes, I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct.

[github-actions[bot]]

Hi, @ostinru welcome!🎊 Thanks for taking the time to point this out.🙌

[my-ship-it]

Hi @avamingli, please have a look when have time?

[ostinru]

@avamingli, have you had a chance to look at this?

[ostinru]

@my-ship-it , @avamingli - one more gentle reminder.

[yjhjstz]

@ostinru I will taka a look.

[ostinru]

I can see that pxf_fdw now in cloudberry/gpcontrib/ (not cloudberry-pxf!) and it seems that external-table framework were being deprecated[1] by pivotal for a long time:

PXF uses the External Table Framework in Greenplum 5 and 6 to access external data. Greenplum 6 introduces the Foreign Data Wrapper Framework to access external data, and extensions are starting to move to the foreign data wrapper (FDW) framework because the External Table Framework will be deprecated in later versions of Greenplum.

@yjhjstz, @tuhaihe and @gfphoenix78 let's decide on which framework(s) are supported in Cloudberry. If we think that external-tables are deprecated - than we can close this issue, because PXF 6.10 with pxf_fdw builds fine.

[1] https://github.com/greenplum-db/pxf-archive/blob/main/fdw/PXF_FDW.md

[tuhaihe]

Cool @ostinru We can start a discussion on the Dev mailing list or GitHub Discussions to gather more feedback.