Office365-REST-Python-Client 400 Client Error: Bad Request for url

I got this error when I use ctx.execute_batch() to update or add SharePoint list item.

Can someone help me?

office365.runtime.client_request_exception.ClientRequestException: (None, None, '400 Client Error: Bad Request for url: https://uk.sharepoint.com/_api/$batch')

Jun 17 '21 07:06 samlee00123

Hi there, to determine the root cause, it might be beneficial to share the error details , is anything else could be shared in this regard? Or maybe additional details such as how many items are getting updated, whether the same error occurs when item is updated via non batched request

Jun 17 '21 09:06 vgrem

Hi there, to determine the root cause, it might be beneficial to share the error details , is anything else could be shared in this regard? Or maybe additional details such as how many items are getting updated, whether the same error occurs when item is updated via non batched request

Here is the code snippet.

    for index, row in df.iterrows():
        list_source.add_item(
            {
                "Title": create_unique_name(),
                "EID": row['EID'],
                "Name": row['Name']
            }
        )
    ctx.execute_batch()

200 items are created. If I use ctx.execute_batch(), it comes to error. However, if I query one by one, there are no error.

    for index, row in df.iterrows():
        list_source.add_item(
            {
                "Title": create_unique_name(),
                "EID": row['EID'],
                "Name": row['Name']
            }
        ).execute_query()

Jun 18 '21 02:06 samlee00123

@vgrem

Actually, the 200 items was created successfully in the SharePoint list. But it just comes to this error at the end. Besides, I found that updating 200 items was OK, but creating 200 items would have this error.

Jun 18 '21 02:06 samlee00123

Hey @samlee00123,

it appears the error occurs when batch request is large enough and exceeds a certain limit in terms of amount of operations per batch request which is determined by design and implementation of SharePoint REST API.

2.3.5 version addresses this issue and ClientContext.execute_batch method also support items_per_bulk parameter which:

allows to control the max amount of operations per batch request (100 by default) In case batch request exceeds this value, it is getting splitted into a smaller chunks before submitting to the server.

Maybe you could give it another shot in the newer version (2.3.5 or above)?

Sep 14 '21 15:09 vgrem

Closing this one since considered as resolved

Sep 30 '21 20:09 vgrem

@vgrem

This is still an issue as originally reported. I tested it in 2.3.9 version

Dec 06 '21 01:12 srikanthpagadala

for other people the parameter is now called items_per_batch

but nevetheless I also still face that problem

But nevertheless thank you for that fantastic work

Mar 14 '23 15:03 reyemb

@vgrem This is still an issue in 2.4.4. I'm currently using 2.3.13 as I know it works on that version.

To add some more detail, I'm trying to create about 500 items. Setting items_per_batch fails with the following error after creating items_per_batch / 2.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/office365/runtime/client_request.py", line 59, in execute_query
    self.process_response(response, query)
  File "/usr/local/lib/python3.10/dist-packages/office365/runtime/odata/v3/batch_request.py", line 40, in process_response
    sub_resp.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: None for url: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...
    ctx.execute_batch(items_per_batch=100) # upload all items
  File "/usr/local/lib/python3.10/dist-packages/office365/sharepoint/client_context.py", line 158, in execute_batch
    batch_request.execute_query(qry)
  File "/usr/local/lib/python3.10/dist-packages/office365/runtime/client_request.py", line 62, in execute_query
    raise ClientRequestException(*e.args, response=e.response)
office365.runtime.client_request_exception.ClientRequestException: ('-1, Microsoft.SharePoint.Client.InvalidClientQueryException', 'Invalid request.', '400 Client Error: None for url: None')

The following may or may not be related, but I thought it was interesting nonetheless. This also fails when setting items_per_batch to a large number (I tried setting this to 5000 and it uploaded 444 items before giving the following error which makes sense...) so I'd have expected it to work when I tried to create fewer. This failed after creating 150 items (items_per_batch/2 again) with the same error as when it failed after creating 444 items while using the same data, so I'm not sure why it ran into this error so much earlier.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/office365/runtime/client_request.py", line 58, in execute_query
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://{...}.sharepoint.com/sites/{...}/_api/$batch

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...
    ctx.execute_batch(items_per_batch=300) # upload all items
  File "/usr/local/lib/python3.10/dist-packages/office365/sharepoint/client_context.py", line 158, in execute_batch
    batch_request.execute_query(qry)
  File "/usr/local/lib/python3.10/dist-packages/office365/runtime/client_request.py", line 62, in execute_query
    raise ClientRequestException(*e.args, response=e.response)
office365.runtime.client_request_exception.ClientRequestException: ('-1, Microsoft.Data.OData.ODataException', 'The maximum number of bytes allowed to be read from the stream has been exceeded. After the last read operation, a total of 1055977 bytes has been read from the stream; however a maximum of 1048576 bytes is allowed.', '400 Client Error: Bad Request for url: https://{...}.sharepoint.com/sites/{...}/_api/$batch')

Sep 20 '23 20:09 carter7k

Still getting this issue today. This is unresolved. If i increase the batch I get another error stating I exceeded the number of bytes, if I limit the batch I get Microsoft.SharePoint.Client.InvalidClientQueryException', 'Invalid request.', '400 Client Error: None for url: None'

I think the bug is in the add_item() method.

Sep 27 '23 21:09 jjhornx

I am facing this issue as well. My code always failed after 50 items are created. The interesting observation is that when I run the same code from my laptop it's working fine, but when I run the code from a Linux server this error occurs.

Sep 28 '23 05:09 nkgxgongxi

Closing this one since considered as resolved

@vgrem could you look into this issue again? There are several folks using this library also facing the issue after you marked it as resolved. Many thanks!!

Update: It seems like when I use version 2.4.1 it works, but 2.4.4 doesn't.

Sep 28 '23 05:09 nkgxgongxi

Experiencing this issue with version 2.5.2 as well.