PySUS icon indicating copy to clipboard operation
PySUS copied to clipboard

Comment dataframe decoding because of UnicodeDecodeError

Open luabida opened this issue 1 year ago • 2 comments

I've encountered a bug while trying to download data for 'Dengue' in 2001 to 2008. @esloch pointed that line 86 of file PySUS.pysus.online_data.__init__.py was causing the error.

luabida avatar Oct 27 '22 19:10 luabida

@luabida, did you run the tests after making this change?

fccoelho avatar Oct 27 '22 20:10 fccoelho

@fccoelho I intend to permanently fix the issue #99 in next commits, although I also need this change on the epigraphhub_py PR. Please hold this PR until I fix the issue.

luabida avatar Oct 27 '22 21:10 luabida

https://github.com/AlertaDengue/PySUS/blob/fe9def7b5959ec8e0c45472d1f51b64102582781/pysus/online_data/init.py#L86

NOTE: The line above is actually doing nothing, it decodes the DataFrame but is not further used bc has no variable instantiated.

luabida avatar Nov 07 '22 14:11 luabida

https://github.com/AlertaDengue/PySUS/blob/fe9def7b5959ec8e0c45472d1f51b64102582781/pysus/online_data/init.py#L86

NOTE: The line above is actually doing nothing, it decodes the DataFrame but is not further used bc has no variable instantiated.

Maybe something like this:

        table = pa.Table.from_pandas(
                           df.applymap(
                               lambda x: x.decode() if isinstance(x, bytes) else x
                               )
        )

esloch avatar Nov 07 '22 14:11 esloch

Waiting for review

luabida avatar Nov 07 '22 19:11 luabida

@esloch

luabida avatar Nov 17 '22 13:11 luabida

LGTM!

esloch avatar Nov 21 '22 12:11 esloch