vertica
Custom string decoder
Some days ago we faced an utf-8 problem. As your guide says we modified out connection string by appending unicode_error=replace. But i consider this fix as a temporary solution.
We have a lot of tables in our database and we can not absolutely guarantee that all the data we collect will be utf-8 encoded.
In particular, we found that some values were latin-1 encoded (b'M\xe9xico' instead of b'M\xc3\xa9xico').
And our users have a butthurt when they see "m�xico". So my proposal is to make
def vertica_type_cast(type_code, unicode_error) customizable and let developers to handle utf-8 exceptions.
(I don't want to look in the direction of mankipatching yet)
@akariasmorum I merged a new feature in v1.1.0. I want to ask for your opinion about whether this feature is useful for you. Can customized converter per sql type serve your needs better?
Thanks a lot for a new feature! It is extremely helpful for us! We can now make our custom data-type converter. Without it we would have to make all our departments save only utf-8 encoded data and also we would have to check all our ETL processes whether they convert data types. So you saved tons of time for our developers.
2022-05-19 12:27 GMT+03:00, Siting Ren @.***>:
@akariasmorum I merged a new feature in master branch. I want to ask for your opinion about whether this feature is useful for you. Can customized converter per sql type serve your needs better?
-- Reply to this email directly or view it on GitHub: https://github.com/vertica/vertica-python/issues/448#issuecomment-1131460395 You are receiving this because you were mentioned.
Message ID: @.***>
You can check out this enhanced feature published in v1.3.5, which let you define customized converter per sql type.