Configure timestamp downcast programmatically

[devinrsmith]

Feature Request / Improvement

During testing of 0.7.0rc1, I noticed timestamp downcast errors, introduced as part of https://github.com/apache/iceberg-python/pull/848.

I'd like to have a way to programmatically configure timestamp downcast in the python API without relying on environment variables or a yaml configuration file.

I would have assumed that something like

from pyiceberg.table import DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE

catalog = ...

table = catalog.create_table(
    ...
    properties={DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE: True},
)

or

from pyiceberg.table import DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE

catalog = load_catalog(
    ...,
    **{
        DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE: "True",
    }
)

would have been the proper way to configure it, but it doesn't seem to take in either case.

[sungwy]

Hi @devinrsmith thank you again for raising this suggestion. I'm working on a similar property driven feature in https://github.com/apache/iceberg-python/pull/986 where I'm proposing that we load the pyarrow file IO config property into the table property as you suggest in the bottom proposal

For example, it will look like:

from pyiceberg.io import PYARROW_DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE

catalog = load_catalog(
    ...,
    **{
        PYARROW_DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE: "True",
    }
)

When the proposed approach is accepted and merged, I will open a PR to suggest a similar configuration.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'