UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 464: character maps to <undefined>

[HadiJahanshahi]

The example which you provided gives this error for me: UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 464: character maps to <undefined>

[CuriousJames]

I have also just come accross this issue when running it as python3, I don't have pyhon2 installed on this machine so I'm unable to test it on that.

I'm running this on Windows 10, Python3.8, and the reverse_geocode version is 1.4.1

I noticed that you have setup a demo too, so I ran the init.py as installed by pip3 and that too returns the error message mentioned on this issue.

For anyone having this issue I suspect switching to Python2 is the quick solution, but unfortunately my projects are somewhat tied to Python3 now.

[CuriousJames]

I've now resolved this issue (#3) - but have been unable to test on Python 2, but as that is not depreciated that shouldn't be a major concern, maybe?

Thanks, James.

[richardpenman]

I found python2 doesn't have the encoding flag so have removed this for now. Then I tested an example in both python 2 & 3 without this flag and didn't get an error - what was the problem you encountered in 3?

[CuriousJames]

Yeah, I had a feeling it might not work for Python 2.

Below is the error message I get on Python 3 for the latest master repo (with my pull request merged in)

I found the encoding option from a quick google search. On that page it also suggests using the io module, so that might be the way to go to support both 2 and 3, I'll try it on 3 first now.

[BoZenKhaa]

I think there was a pull request on bitbucket that was handling Python 2 and 3 in if/else block. Wouldn't that work?

[Pcb21]

Thank you @richardpenman for this wonderful utility.

I got the same error as @CuriousJames on Python 3.7 on Windows 10 using a typical command window. My default encoding is cp1252 (i.e "typical" English/Latin encoding on Windows), so there are many city names that can't be decoded. And, yes, adding encoding="utf-8" in the call to open() resolved it.

Adding ability to read utf-8-encoded csvs in Python 2 is a little bit more fiddly. To support it, you would need it wrap the csv.reader call with a unicode_csv_reader that is described at https://docs.python.org/2.7/library/csv.html . There is another worked example at https://stackoverflow.com/questions/5180555/python-2-and-3-csv-reader (search for "Original answer")

EDITED TO LATER TO BE MORE SPECIFIC: With the current version (1.4.1) the line to change is line 77 which becomes

rows = csv.reader(open(local_filename, encoding="UTF-8"))

[DevashishPrasad]

"adding encoding="utf-8" in the call to open() resolved it."

This worked for me as well. Thank you @Pcb21

[richardpenman]

Have dropped support for python2 and added this encoding parameter.