astronomer-providers icon indicating copy to clipboard operation
astronomer-providers copied to clipboard

Published 20 hours ago • verified publisher icon astronomer

Add custom XCOM backends - GCS

Open kaxil opened this issue 2 years ago • 6 comments

Refer to https://astronomer.slack.com/archives/CGQSYG25V/p1658957159758039 for the relevant discussion, and add GCS XCOM Backend into the Google Provider

kaxil avatar Aug 17 '22 20:08 kaxil

Unit test for Pandas Dataframe in xcom is pending.

rajaths010494 avatar Aug 29 '22 06:08 rajaths010494

Googler test cases were failing because of protobuf <3.20 . Fixed it and other tests are running fine now.

rajaths010494 avatar Aug 30 '22 09:08 rajaths010494

fixed review comments. upload non listed datatypes as bytes to download review is pending.

rajaths010494 avatar Sep 15 '22 06:09 rajaths010494

Test it with astro-sdk by setting custom XCM backend

phanikumv avatar Sep 26 '22 11:09 phanikumv

trying astro-sdk to test and pyarrow to explore more datatypes.

rajaths010494 avatar Sep 26 '22 12:09 rajaths010494

Address requirements discussed in https://astronomer.slack.com/archives/C0387PMM2E7/p1665064469759629

phanikumv avatar Oct 10 '22 11:10 phanikumv