Office365-REST-Python-Client icon indicating copy to clipboard operation
Office365-REST-Python-Client copied to clipboard

IndexError: list index out of range

Open guillaume-colas opened this issue 4 years ago • 23 comments

DEar Sir, Could you please help me to find out why I'm not connecting to sharepoint anymore?

With version 2.2.2 everything was working fine. Then issue #272 occured (An error occurred while retrieving token from XML response: AADSTS500069 ) so I upgraded to 2.3.0.1. Now the following code doesn't work either: def copy_sharepoint(self,filename=""): ctx = ClientContext(self.SITE_SHAREPOINT).with_credentials(UserCredential(self.WIN_USER,self.WIN_PASSWORD))
localpath=os.path.join(self.CHEMIN_XLSX ,self.NOM_TIR + '/',self.FICHIER_BI) self.print_debug("",localpath) self.print_debug("",self.WIN_PASSWORD) with open(localpath, 'rb') as content_file: file_content = content_file.read()
file = ctx.web.get_folder_by_server_relative_url(self.CHEMIN_SHAREPOINT).upload_file(self.FICHIER_BI, file_content).execute_query()

Traceback (most recent call last): File "/appli/sopra/tools/tir_bench.py", line 1327, in tir_bench.copy_sharepoint() File "/appli/sopra/tools/tir_bench.py", line 1278, in copy_sharepoint file = ctx.web.get_folder_by_server_relative_url(self.CHEMIN_SHAREPOINT).upload_file(self.FICHIER_BI, file_content).execute_query() File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/client_object.py", line 30, in execute_query self.context.execute_query() File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/client_runtime_context.py", line 138, in execute_query self.pending_request().execute_query() File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/client_request.py", line 73, in execute_query self.beforeExecute.notify(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/types/EventHandler.py", line 18, in notify listener(*args, **kwargs) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/sharepoint/client_context.py", line 162, in _build_modification_query self.ensure_form_digest(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/sharepoint/client_context.py", line 123, in ensure_form_digest self.request_form_digest() File "/home/benchsp/.local/lib/python3.9/site-packages/office365/sharepoint/client_context.py", line 130, in request_form_digest response = self.execute_request_direct(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/client_runtime_context.py", line 134, in execute_request_direct return self.pending_request().execute_request_direct(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/odata/odata_request.py", line 34, in execute_request_direct return super(ODataRequest, self).execute_request_direct(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/client_request.py", line 86, in execute_request_direct self.context.authenticate_request(request_options) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/sharepoint/client_context.py", line 151, in authenticate_request self._auth_context.authenticate_request(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/auth/authentication_context.py", line 84, in authenticate_request self._provider.authenticate_request(request) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/auth/providers/saml_token_provider.py", line 73, in authenticate_request self.ensure_authentication_cookie()

File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/auth/providers/saml_token_provider.py", line 80, in ensure_authentication_cookie self._cached_auth_cookies = self.get_authentication_cookie() File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/auth/providers/saml_token_provider.py", line 93, in get_authentication_cookie token = self._acquire_service_token_from_adfs(user_realm.STSAuthUrl) File "/home/benchsp/.local/lib/python3.9/site-packages/office365/runtime/auth/providers/saml_token_provider.py", line 136, in _acquire_service_token_from_adfs assertion_node = dom.getElementsByTagNameNS("urn:oasis:names:tc:SAML:1.0:assertion", 'Assertion')[0].toxml() IndexError: list index out of range

Could you please help me with that issue?

Thanks in advance, Guillaume

guillaume-colas avatar Dec 08 '20 16:12 guillaume-colas

Hello, In addition, here's the value of response.content.decode())

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://www.w3.org/2005/08/addressing/soap/fault</a:Action><a:RelatesTo>cfb81633-ff49-4e50-9544-7e39214d7122</a:RelatesTo></s:Header><s:Body><s:Fault><s:Code><s:Value>s:Sender</s:Value><s:Subcode><s:Value xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">a:InvalidSecurity</s:Value></s:Subcode></s:Code><s:Reason><s:Text xml:lang="fr-FR">An error occurred when verifying security for the message.</s:Text></s:Reason></s:Fault></s:Body></s:Envelope>

guillaume-colas avatar Dec 08 '20 16:12 guillaume-colas

Greetings!

thank you for the detailed log, at the moment im lacking any possibility to verify it against SharePoint/ADFS env.

@wreiner, maybe you could kindly confirm whether federated authentication is working as expected on your side for the latest release?

vgrem avatar Dec 08 '20 16:12 vgrem

Hi everyone

I am having similar issues, the following line gives me a 400 or 500 response.

https://github.com/vgrem/Office365-REST-Python-Client/blob/020c8c8176ce2eea3accbe95e2e4372af57ef6c8/office365/runtime/auth/providers/saml_token_provider.py#L133

In case I get the 400 response the body is empty resulting in a xml.parsers.expat.ExpatError: no element found: line 1, column 0 exception. the 500 response gives same exception as @guillaume-colas.

I have not yet gotten the package to work, so it could well be the sharepoint server, but I will try downgrading the package just to be sure. How can I ensure that sharepoint authentication is setup correctly?

Thx in advance!

mark-boer avatar Dec 09 '20 10:12 mark-boer

Hi,

I'm not at my computer to fully analyse but as I see it, when I install the latest version with pip the function acquire_service_token_from_adfs is missing from runtime/auth/providers/saml_token_provider.py.

In my pull request there is no removal of this function so I'm not sure why it is missing. Also my fix for the datetime format is not in the file. Maybe there was an error with the merge?

wreiner avatar Dec 09 '20 10:12 wreiner

@wreiner thank you for getting back, regarding

acquire_service_token_from_adfs is missing

its been refactored from public to private method.

my fix for the datetime format is not in the file

the format should be in place as were committed originally, but its been refactored abit in terms of moving the declaration to sts_profile.py

self.created = now.astimezone(timezone.utc).isoformat('T')[:-9] + 'Z' self.expires = (now + timedelta(minutes=10)).astimezone(timezone.utc).isoformat('T')[:-9] + 'Z'

But if something is missing after the merge, i should be blamed.

vgrem avatar Dec 09 '20 11:12 vgrem

Hi everyone

I am having similar issues, the following line gives me a 400 or 500 response.

https://github.com/vgrem/Office365-REST-Python-Client/blob/020c8c8176ce2eea3accbe95e2e4372af57ef6c8/office365/runtime/auth/providers/saml_token_provider.py#L133

In case I get the 400 response the body is empty resulting in a xml.parsers.expat.ExpatError: no element found: line 1, column 0 exception. the 500 response gives same exception as @guillaume-colas.

I have not yet gotten the package to work, so it could well be the sharepoint server, but I will try downgrading the package just to be sure. How can I ensure that sharepoint authentication is setup correctly?

Thx in advance!

Hi Mark,

thank you for confirming it, it indeed appears in the latest version federated authentication (SharePoint/ADFS) somewhat broken Could you please give it a try for the previous version (2.2.2)?

vgrem avatar Dec 09 '20 15:12 vgrem

I'm not through complete analysis but for the my changes I can say they return the security token and cookie correctly but the cookie jar does not contain a 'FedAuth' field which is looked for in is_valid_auth_cookies which is now being used in _get_authentication_cookie.

https://github.com/vgrem/Office365-REST-Python-Client/blob/93325ea33675b86aafc2e6e0fa381c184ce479e7/office365/runtime/auth/providers/saml_token_provider.py#L32

For federated auth I only see SPOIDCRL in the jar.

Currently I'm still unable to use the obtained auth data correctly, I'm still investigating but can't tell how long I'll need.

All in all I can tell that with the old version and my patches everything still works fine as expected but unfortunately not with the new version.

wreiner avatar Dec 09 '20 17:12 wreiner

Thanks for the quick responses :-)

@vgrem it get a similar error on version 2.2.2, so it seems to be the sharepoint server.

It might be nice to wrap the exception into a more readable one.

mark-boer avatar Dec 10 '20 12:12 mark-boer

I am one step further. Turns out I needed to reset my password. I'm now hitting the same error as @wreiner. The is_valid_auth_cookies returns that the cookie is not valid, because the dictionary only contains the SPOIDCRL key and no FedAuth key. If i just comment out this check on whether the cookie is valid or not. I am able to download files from sharepoint. So the authentication seems to work fine.

Maybe I'll try version 2.2.2 again.

mark-boer avatar Dec 11 '20 13:12 mark-boer

I have the same issue. is_valid_auth_cookies return False because of no FedAuth key in cookies dict. If I comment return code and simplify return True, authentication will work fine.

def is_valid_auth_cookies(values):
    return True
    # return any(values) and values.get('FedAuth', None) is not None

hungnguyen95 avatar Dec 14 '20 09:12 hungnguyen95

Hey @hungnguyen95 , thank you for confirming it!

Turns out is_valid_auth_cookies needs to be adjusted.

vgrem avatar Dec 14 '20 10:12 vgrem

Hello, Will my issue be fixed by modifying is_valid_auth_cookies ?

I'm quite lost here sorry. I thought that my issue was in _acquire_service_token_from_adfs. Do you have any idea why I'm getting response 'Invalid security' and hopefully a workaround to get authentifcated?

Thanks in advance for your help. Guillaume

guillaume-colas avatar Dec 14 '20 22:12 guillaume-colas

@guillaume-colas it seems that your issue is probably related to the server not allowing you access. Are you sure the password is correct and have you tried resetting your password? Resetting my password resolved this issue. After which I ran into the is_valid_auth_cookies issue.

mark-boer avatar Dec 14 '20 23:12 mark-boer

@mark-boer Yes I've verified my password and it's correct. Is it possible that's the authentification with 2FA (sms code) is blocking from authentificating?

Is it possible to use ClientCredentials to upload files to sharepoint?

Thanks in advance for your help

guillaume-colas avatar Jan 05 '21 14:01 guillaume-colas

Currently getting this error, password is correct and working for sharepy connections. Anything I should try?

burrisca avatar Jul 27 '21 13:07 burrisca

Having same issue with latest version (2.3.9), username and password which I'm using works well if I use old version (2.1.4)

filippzorin avatar Dec 08 '21 07:12 filippzorin

Hi, I think I have the same problem. I try to connect to sharepoint with python 3.8 (using Spyder). And also get 'IndexError: list index out of range'.

I used the example given by https://pypi.org/project/office365-rest-client/. My username and password are correct. I am not very experienced so I thought the error meant I did something wrong. But do I understand correctly that this is a general issue?

And is there any other way of fixing this, without using an older version?

AnoukVerdonschot avatar Dec 27 '21 15:12 AnoukVerdonschot

I am still receiving this error, any updates?

Here is what I'm getting: 2021-12-28 11:45:04,502 [ERROR] - no element found: line 1, column 0 Traceback (most recent call last): ... File "office365\runtime\client_object.py", line 41, in execute_query File "office365\runtime\client_runtime_context.py", line 132, in execute_query File "office365\runtime\client_request.py", line 92, in execute_query File "office365\runtime\types\EventHandler.py", line 18, in notify File "office365\sharepoint\client_context.py", line 192, in _build_modification_query File "office365\sharepoint\client_context.py", line 151, in ensure_form_digest File "office365\sharepoint\client_context.py", line 158, in get_context_web_information File "office365\runtime\client_runtime_context.py", line 129, in execute_request_direct File "office365\runtime\odata\odata_request.py", line 36, in execute_request_direct File "office365\runtime\client_request.py", line 105, in execute_request_direct File "office365\sharepoint\client_context.py", line 181, in authenticate_request File "office365\runtime\auth\authentication_context.py", line 86, in authenticate_request File "office365\runtime\auth\providers\saml_token_provider.py", line 77, in authenticate_request File "office365\runtime\auth\providers\saml_token_provider.py", line 84, in ensure_authentication_cookie File "office365\runtime\auth\providers\saml_token_provider.py", line 97, in get_authentication_cookie File "office365\runtime\auth\providers\saml_token_provider.py", line 139, in _acquire_service_token_from_adfs File "xml\dom\minidom.py", line 1998, in parseString File "xml\dom\expatbuilder.py", line 925, in parseString File "xml\dom\expatbuilder.py", line 223, in parseString xml.parsers.expat.ExpatError: no element found: line 1, column 0

amberzaragoza avatar Dec 28 '21 19:12 amberzaragoza

Has anyone solved this issue? I have been struggling with it for a while. Even changed to def is_valid_auth_cookies(values): return True I still face the same error message: IndexError: list index out of range

sound118 avatar Apr 20 '22 03:04 sound118

It sounds mad. Everything works find until recently I changed my password, I started seeing this error and have problem connecting to SharePoint. After struggling with it for days, it turns out in my case it cannot handle some special characters in my password. I changed my password again, and it works fine now. I am using Office365-REST-Python-Client-2.3.11.

fyang421 avatar May 06 '22 14:05 fyang421

It sounds mad. Everything works find until recently I changed my password, I started seeing this error and have problem connecting to SharePoint. After struggling with it for days, it turns out in my case it cannot handle some special characters in my password. I changed my password again, and it works fine now. I am using Office365-REST-Python-Client-2.3.11.

This worked for me, thank you very much. The old password had a special character that was not recognized

Michel2868 avatar Sep 13 '22 14:09 Michel2868

Hi did anyone reolved this issue. I am still seeing the same error

shrigit143 avatar Oct 05 '22 02:10 shrigit143

Hi, Is the special character issue still there in the current version of the package. I am getting a similar error along the lines of

cIndexError: list index out of range

ashish10alex avatar Jan 11 '24 17:01 ashish10alex