singer-python icon indicating copy to clipboard operation
singer-python copied to clipboard

--catalog vs -p (properties) parameters

Open chrisb-c01 opened this issue 6 years ago • 4 comments

It might be me, but the use of the --catalog and property -p parameters is somewhat unclear to me.

They are used intermixed throughout different sections of the docs (e.g. Allowing Users to Select Streams to Sync vs. Sync mode).

Source code mentions -p to be deprecated over --catalog, however replacing -p by --catalog leads to different behaviour: the "selected": true on a schema definition in the catalog file is not honoured by --catalog, but is by -p parameter.

It would be helpful if this could be clarified a bit more in the docs.

chrisb-c01 avatar Apr 11 '18 06:04 chrisb-c01

Thanks for the feedback, @ChrisCalculus. As you noted, the -p- is deprecated in favor of using the --catalog. We're shifting towards usage of metadata in the catalog for noting which fields should be selected. We agree that the documentation needs some work and plan to address that in the future. We'll update here when that happens.

briansloane avatar Apr 11 '18 14:04 briansloane

Thanks for replying @briansloane. Looking forward to the update.

chrisb-c01 avatar Apr 11 '18 15:04 chrisb-c01

Any plans to change the transform function to respect the selected setting?

pedromachados avatar Apr 12 '18 03:04 pedromachados

@pedromachados No plans to do that as of now. I'd recommend creating another issue to outline this request

briansloane avatar Apr 12 '18 14:04 briansloane