[Bitbucket Cloud] Improper payload encoding when POSTing with 'x-www-form-urlencoded' content-type

[sqwxl]

What I'm trying to do

The bitbucket cloud api offers an endpoint to commit a text file using the Content-Type: x-www-form-urlencoded header. Using curl the call looks something like:

curl -u username:password https://api.bitbucket.org/2.0/repositories/myorg/myrepo/src \
--data-urlencode '/file.txt=File content.' \
--data-urlencode 'author=Me McMyself <[email protected]>' \
--data-urlencode 'message=A commit message.' \
--data-urlencode 'branch=somebranch'

What I've tried

from atlassian.bitbucket import Cloud


session = Cloud(username=username, password=password, cloud=True)

repo = session.workspaces.get(myorg).repositories.get(myrepo)

data = {
    "/file.txt": "File content."
    "author": "Me McMyself <[email protected]>",
    "message": "A commit message.",
    "branch": "somebranch",
}

repo.post(
    "src",
    headers={"Content-Type": "application/x-www-form-urlencoded"},
    data=data
)

Expected behavior

The outgoing request should have a payload that looks like this:

/file.txt=File+content.&author=Me+McMyself+%3Cmy%40email.com%3E&message=A+commit+message.&branch=somebranch

Actual behavior

Instead, it looks like this:

'"{\"/file.txt\": \"File content.\", \"message\": \"A commit message.\", \"author\": \"Me McMyself <[email protected]>\", \"branch\": \"somebranch\"}"'

...which isn't correctly interpreted by bitbucket because they are expecting a urlencoded payload as above.

Explanation

AtlassianRestAPI's request method has the following lines

if files is None:
    data = None if not data else dumps(data)
    json_dump = None if not json else dumps(json)

which run before the request is passed along to the requests module...

Workaround

After trying all kinds of variations using this library, I ended up making a call with requests directly:

requests.request(
    "POST",
    repo.url + "/src",
    data=data,
    auth=(session.username, session.password),
)

[gonchik]

Thanks, let's think about possibility wrap up non json payload

[Spacetown]

Can you try to set params=data instead of data=data? This should add the parameters instead of a JSON payload.

[Shaked]

@Spacetown that won't work because the information has to be passed as data, not as a URL params. The reason @nilueps use "/" at the beginning of his example is because atlassian state this in its API:

There could be a field name clash if a client were to upload a file named "message", as this filename clashes with the meta data property for the commit message. To avoid this and to upload files whose names clash with the meta data properties, use a leading slash for the files, e.g. curl --data-urlencode "/message=file contents".

I did find a slightly better workaround for this issue, instead of using requests:

data = {
    "/file.txt": "File content."
    "author": "Me McMyself <[email protected]>",
    "message": "A commit message.",
    "branch": "somebranch",
}
repo.post(
    "src",
    headers={"Content-Type": "application/x-www-form-urlencoded"},
    data=data,
    files=[] 
)

As @nilueps stated, the problematic lines are https://github.com/atlassian-api/atlassian-python-api/blob/master/atlassian/rest_client.py#L222-L224, so with my workaround files won't be set to None.

@gonchik I guess there's no reason to transform the data to JSON when the Content-Type states otherwise. I am not sure why transforming it at all if there's a json parameter available. Maybe it should automatically define the Content-Type depending on what the call contains e,g json will automatically set the Content-Type to application/json and data will be set to application/x-www/form-urlencoded. Just sharing some ideas, what do you think?