turbodbc
turbodbc copied to clipboard
flag `truncate_timestamps` added to fetcharrow functions
#207
Arrow only allows timestamps with years between 1400 to 9999.
Adding the new boolean parameter truncate_timestamps
to functions fetchallarrow
and fetcharrowbatches
to truncate the value to the valid range instead of throwing.
It looks like OSX if failing but I don't think it is related to this PR (I might be wrong though)
Hi @JorgeGarciaIrazabal - Could you please try rebasing to master and reopen the PR? OSX is fixed there.
Codecov Report
Merging #209 into master will increase coverage by
0.05%
. The diff coverage is100%
.
@@ Coverage Diff @@
## master #209 +/- ##
==========================================
+ Coverage 98.59% 98.65% +0.05%
==========================================
Files 136 145 +9
Lines 2999 3349 +350
==========================================
+ Hits 2957 3304 +347
- Misses 42 45 +3
Impacted Files | Coverage Δ | |
---|---|---|
cpp/turbodbc/Library/src/time_helpers.cpp | 100% <100%> (ø) |
:arrow_up: |
cpp/turbodbc_arrow/Library/src/python_bindings.cpp | 100% <100%> (ø) |
:arrow_up: |
python/turbodbc/cursor.py | 98.97% <100%> (ø) |
|
...pp/turbodbc_arrow/Library/src/arrow_result_set.cpp | 96.24% <100%> (-0.44%) |
:arrow_down: |
...urbodbc_arrow/Test/tests/arrow_result_set_test.cpp | 100% <100%> (ø) |
:arrow_up: |
python/turbodbc/constructors.py | 100% <0%> (ø) |
|
python/turbodbc/exceptions.py | 100% <0%> (ø) |
|
python/turbodbc/__init__.py | 100% <0%> (ø) |
|
python/turbodbc/options.py | 100% <0%> (ø) |
|
... and 5 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 8592759...c488cc5. Read the comment docs.
Hi @JorgeGarciaIrazabal - Could you please try rebasing to master and reopen the PR? OSX is fixed there.
@MathMagique I have a different error after rebasing, I am not 100% sure why it failed this time though.
@MathMagique running the job again, all the checks passed! let me know if you want me to change/add anything to the PR. Thanks in advance
This is not an issue with Arrow
but with boost::gregorian::date
in the date_to_days
function. I would rather fix that than have this truncation workaround.
This is not an issue with
Arrow
but withboost::gregorian::date
in thedate_to_days
function. I would rather fix that than have this truncation workaround.
Yeah, that makes more sense. Unfortunately, with my limited c++ experience, I can not think of a better alternative for the library. Are you guys planing on working on this any time soon?
@wesm, @xhochy what would be the approrpriate changes to make in Arrow, I took at look at the datetime parts of arrow-cpp but it wasn't clear to me what we can use instead of boost::gregorian ?