Google Drive: Unable to upload file containing UTF-8 character

[abstractOwl]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/google-api-python-client/issues
• Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

Environment details

• OS type and version: OS X Ventura 13.2.1
• Python version: 3.8.5
• pip version: 20.1.1
• google-api-python-client version: 2.82.0

Steps to reproduce

1. Run code example below. See that it successfully uploads the file to your Google Drive folder
2. Change "hello" to "龍" and run the example again. The example should fail with the stack trace below.

Code example

import csv
import io
import json
import os

from dotenv import load_dotenv
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.http import MediaIoBaseUpload


def upload_csv():
    account_info = os.environ.get("ACCOUNT_SECRET")
    containing_folder = os.environ.get("CONTAINING_FOLDER")
    target_filename = "test-file"

    output = io.StringIO()
    writer = csv.writer(output)

    writer.writerow(["hello"])

    output.flush()
    output.seek(0)

    credentials = service_account.Credentials.from_service_account_info(json.loads(account_info))
    client = build("drive", "v3", credentials=credentials).files()
    media = MediaIoBaseUpload(output, mimetype="text/csv")

    metadata = {
        "name": target_filename,
        "mimeType": "application/vnd.google-apps.spreadsheet",
        "parents": [containing_folder],
    }

    response = client.create(
        body=metadata,
        media_body=media,
    ).execute()


if __name__ == '__main__':
    load_dotenv()
    upload_csv()

requirements.txt

google-api-python-client==2.82.0
google-auth-oauthlib==1.0.0
python-dotenv

.env

ACCOUNT_SECRET=???
CONTAINING_FOLDER=???

Stack trace

Traceback (most recent call last):          
  File "main.py", line 46, in <module>
    upload_csv()                                                                                                        
  File "main.py", line 38, in upload_csv                                                                                
    response = client.create(
  File "/[removed]/env/lib/python3.8/site-packages/googleapiclient/discovery.py", line 1196, in method                                                                                                                   
    g.flatten(msgRoot, unixfrom=False)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 116, in flatten                                                                                                                                
    self._write(msg)                                                                                                                                                                                                                             
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 181, in _write                                                                                                                                 
    self._dispatch(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 214, in _dispatch
    meth(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 272, in _handle_multipart
    g.flatten(part, unixfrom=False, linesep=self._NL)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 116, in flatten
    self._write(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 181, in _write
    self._dispatch(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 214, in _dispatch
    meth(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 432, in _handle_text
    super(BytesGenerator,self)._handle_text(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 249, in _handle_text
    self._write_lines(payload)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/generator.py", line 406, in write
    self._fp.write(s.encode('ascii', 'surrogateescape'))
UnicodeEncodeError: 'ascii' codec can't encode character '\u9f8d' in position 0: ordinal not in range(128)

Note that u9f8d is the unicode code point for the "龍" character https://www.wikidata.org/wiki/Q56598286

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[agoncharov-reef]

One should use io.BytesIO to be able to upload non-ascii-encoded files