aries-cloudagent-python

aries-cloudagent-python copied to clipboard

I'm currently investigating what is causing this, but I'm seeing the following error occur randomly when creating a credential definition. The code path suggests that we are creating the tails registry, however, we fail to upload the tails file to the tails folder because we aren't able to get the tails file off the local filesystem. The problem may possibly be in https://github.com/hyperledger/indy-sdk, but I have yet to narrow it down that far yet. All I know for certain is that this function is returning None for some reason. ACA-Py is setup with multi-tenant wallets in author mode when the error is being observed. The endorser should be approving all transactions automatically. https://github.com/hyperledger/aries-cloudagent-python/blob/e7fd7e57faea226b4931840fc2c0048722160a6e/aries_cloudagent/indy/util.py#L42-L53

Tracestack that I'm running into:

Traceback (most recent call last):
File "/home/indy/aries_cloudagent/core/event_bus.py", line 120, in notify
    await processor()
File "/home/indy/aries_cloudagent/revocation/routes.py", line 1260, in on_revocation_tails_file_event
    max_attempts=5, # heuristic: respect HTTP timeout
File "/home/indy/aries_cloudagent/tails/indy_tails_server.py", line 51, in upload_tails_file
    max_attempts=max_attempts,
File "/home/indy/aries_cloudagent/utils/http.py", line 161, in put_file
    with open(file_path, "rb") as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType

The tails_path method should likely raise an exception when it can't determine the file path, rather than just returning None. I suspect it's finding more than one tails file in the directory although it's possible the file is missing too. I can look at this shortly along with another registry creation issue.

Some explicit throws would definitely be nicer to know what's going on. I could do at least that much. As far as further investigation goes, I'm trying to get it moving along but I do not have direct access to the environment where the exception is occuring.

Is this still considered an issue and more importantly, a must have for 0.7.4. Please provide an update when you can.

I do believe that it is still an issue but I do not have access to an environment to reproduce & debug. Those who do are currently busy with other tasks

I've tweaked the title and removed the 0.7.4 tag. We have several issues in the area of not reliably being able to create CredDefs when the Tails Server is not available, and while this is not exactly that -- it is in the same code section. I think we should go ahead with 0.7.4 and note this in the Changelog and look to get this in the next release.

Does that sound all right? In doing that, the big question is whether this is happening because of new code added in 0.7.4, or has this been there for a while. Obviously, if this is a regression, then we need to be more concerned.

Thanks

Hi,

This seems to be related, the Faber agent in the demo also fails with revocation enabled (with 1.0.0-rc1).

Schema ID: Pb4gEUKLwig3iv3PzAmNbV:2:degree schema:15.16.38 Faber | Revocation registry: (undetermined) state: init Faber | Revocation registry: Pb4gEUKLwig3iv3PzAmNbV:4:Pb4gEUKLwig3iv3PzAmNbV:3:CL:149046:faber.agent.degree_schema:CL_ACCUM:48f4d8fa-674a-47f1-ad41-d48401222440 state: generated Faber | Revocation registry: Pb4gEUKLwig3iv3PzAmNbV:4:Pb4gEUKLwig3iv3PzAmNbV:3:CL:149046:faber.agent.degree_schema:CL_ACCUM:48f4d8fa-674a-47f1-ad41-d48401222440 state: posted Faber | 2022-09-14 14:31:14,440 aries_cloudagent.core.event_bus ERROR Error occurred while processing event Faber | Traceback (most recent call last): Faber | File "/home/indy/aries_cloudagent/core/event_bus.py", line 121, in notify Faber | await processor() Faber | File "/home/indy/aries_cloudagent/revocation/routes.py", line 1342, in on_revocation_registry_init_event Faber | await shield(generate(registry_record)) Faber | File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/futures.py", line 327, in iter Faber | yield self # This tells Task to wait for completion. Faber | File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup Faber | future.result() Faber | File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/futures.py", line 243, in result Faber | raise self._exception Faber | File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/tasks.py", line 180, in _step Faber | result = coro.send(None) Faber | File "/home/indy/aries_cloudagent/revocation/routes.py", line 1296, in generate Faber | await rr_record.upload_tails_file(profile) Faber | File "/home/indy/aries_cloudagent/revocation/models/issuer_rev_reg_record.py", line 440, in upload_tails_file Faber | f"Tails file for rev reg {self.revoc_reg_id} failed to upload: {result}" Faber | aries_cloudagent.revocation.error.RevocationError: Tails file for rev reg Pb4gEUKLwig3iv3PzAmNbV:4:Pb4gEUKLwig3iv3PzAmNbV:3:CL:149046:faber.agent.degree_schema:CL_ACCUM:48f4d8fa-674a-47f1-ad41-d48401222440 failed to upload: Exceeded maximum upload attempts

On my own environment (stock aca-py, indy-tails) with a similar test I can see that indy-tails is reporting the following error

2022-09-14 10:46:54,176 aiohttp.server ERROR Error handling request Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 314, in data_received messages, upgraded, tail = self._request_parser.feed_data(data) File "aiohttp/_http_parser.pyx", line 546, in aiohttp._http_parser.HttpParser.feed_data aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'invalid HTTP method'"

Hope that is helpful,

John

Hi everyone, are there any updates on this issue? I am also trying to run the revocation demo with ./run_demo faber --revocation and run in to the same error:

Faber      | 2022-11-18 10:23:46,167 aries_cloudagent.core.event_bus ERROR Error occurred while processing event
Faber      | Traceback (most recent call last):
Faber      |   File "/home/indy/aries_cloudagent/core/event_bus.py", line 121, in notify
Faber      |     await processor()
Faber      |   File "/home/indy/aries_cloudagent/revocation/routes.py", line 1352, in on_revocation_registry_init_event
Faber      |     await shield(generate(registry_record))
Faber      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/futures.py", line 327, in __iter__
Faber      |     yield self  # This tells Task to wait for completion.
Faber      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup
Faber      |     future.result()
Faber      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/futures.py", line 243, in result
Faber      |     raise self._exception
Faber      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/asyncio/tasks.py", line 180, in _step
Faber      |     result = coro.send(None)
Faber      |   File "/home/indy/aries_cloudagent/revocation/routes.py", line 1306, in generate
Faber      |     await rr_record.upload_tails_file(profile)
Faber      |   File "/home/indy/aries_cloudagent/revocation/models/issuer_rev_reg_record.py", line 442, in upload_tails_file
Faber      |     f"Tails file for rev reg {self.revoc_reg_id} failed to upload: {result}"
Faber      | aries_cloudagent.revocation.error.RevocationError: Tails file for rev reg MMxT26Uze1Qa2tUWrjVHcp:4:MMxT26Uze1Qa2tUWrjVHcp:3:CL:8:faber.agent.degree_schema:CL_ACCUM:2cdee1a5-c549-4c25-ba53-49e293e7cda3 failed to upload: Exceeded maximum upload attempts

I am running Ubuntu 20.04.5 LTS and am on kernel version 5.4.0-1038-aws, Docker version 20.10.12, docker-compose version 1.25.0, and commit 75c057e2625fab3bf8b2742e2f890adba3bb8dbf of the aries-cloudagent-python.

I would appreciate any help.

Greetings Benedikt

Hmmm just tested it and it's working for me (faber demo with a local ledger and tails server)

We should clean up/improve the error handling and reporting around tails files (in aca-py as well as the tails server). I know we've had an issue in the past with the tails server not being able to create the file (permissions issue) ...

@b3n3d17 are you seeing any messages in the tails server log?

Might be related to #1743