api-cookbook
api-cookbook copied to clipboard
[Bug]: Python on Windows defaults to cp1252 encoding for CSV files
Do I have the most recent api-cookbook code?
- [x] I am using the latest code available on the
mainbranch of the api-cookbook repo.
Is there an existing issue for this?
- [x] I have searched the existing issues.
Current Behavior
Running as-is results in an encoding exception:
pipenv run python pivotal_import.py
[Note] This importer adheres to the Shortcut API rate limit of 200 requests per minute.
It may pause for up to 70 seconds during processing to avoid request throttling.
Traceback (most recent call last):
File "C:\Users\user\Documents\DEV\shortcut-api-cookbook\pivotal-import\pivotal_import.py", line 664, in <module>
sys.exit(main(sys.argv))
~~~~^^^^^^^^^^
File "C:\Users\user\Documents\DEV\shortcut-api-cookbook\pivotal-import\pivotal_import.py", line 655, in main
process_pt_csv_export(ctx, cfg["pt_csv_file"], entity_collector)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Documents\DEV\shortcut-api-cookbook\pivotal-import\pivotal_import.py", line 596, in process_pt_csv_export
header = [col.lower() for col in next(reader)]
~~~~^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python313\Lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5110: character maps to <undefined>
Fix:
Change line 594 of pivotal_import.py to specify utf-8:
with open(pt_csv_file, mode="r", encoding="utf-8") as csvfile:
Expected Behavior
Import running as expected
Minimal Reproducible Example
pipenv run python pivotal_import.py in Windows Powershell with Py 3.13
Environment
- Operating system: Win11
- Python/JavaScript version 3.13
Further Information
No response