Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Python on Windows defaults to cp1252 encoding for CSV files #93

Open
2 tasks done
alexc-rw opened this issue Jan 23, 2025 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@alexc-rw
Copy link

Do I have the most recent api-cookbook code?

  • I am using the latest code available on the main branch of the api-cookbook repo.

Is there an existing issue for this?

  • I have searched the existing issues.

Current Behavior

Running as-is results in an encoding exception:

 pipenv run python pivotal_import.py
[Note] This importer adheres to the Shortcut API rate limit of 200 requests per minute.
       It may pause for up to 70 seconds during processing to avoid request throttling.
Traceback (most recent call last):
  File "C:\Users\user\Documents\DEV\shortcut-api-cookbook\pivotal-import\pivotal_import.py", line 664, in <module>
    sys.exit(main(sys.argv))
             ~~~~^^^^^^^^^^
  File "C:\Users\user\Documents\DEV\shortcut-api-cookbook\pivotal-import\pivotal_import.py", line 655, in main
    process_pt_csv_export(ctx, cfg["pt_csv_file"], entity_collector)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\Documents\DEV\shortcut-api-cookbook\pivotal-import\pivotal_import.py", line 596, in process_pt_csv_export
    header = [col.lower() for col in next(reader)]
                                     ~~~~^^^^^^^^
  File "C:\Users\user\AppData\Local\Programs\Python\Python313\Lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5110: character maps to <undefined>

Fix:

Change line 594 of pivotal_import.py to specify utf-8:

with open(pt_csv_file, mode="r", encoding="utf-8") as csvfile:

Expected Behavior

Import running as expected

Minimal Reproducible Example

pipenv run python pivotal_import.py in Windows Powershell with Py 3.13

Environment

  • Operating system: Win11
  • Python/JavaScript version 3.13

Further Information

No response

@alexc-rw alexc-rw added the bug Something isn't working label Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

1 participant