aries-cloudagent-python
aries-cloudagent-python copied to clipboard
Published
20 hours ago •
hyperledger
hyperledger
Error during creation of credential definition with revocation enabled
Problem Description
I get the error an error during the creation of a credential definition with revocation enabled using anoncreds. The credential definition is actually created however, the error occurs during the revocation operation. I still haven't been able to determine what the cause of the error is. Here is a partial log:
File "/home/aries/.venv/lib/python3.12/site-packages/acapy_agent/wallet/askar.py", line 398, in get_local_did
raise WalletNotFoundError("Unknown DID: {}".format(did))
acapy_agent.wallet.error.WalletNotFoundError: Unknown DID: REDACTED
I can create the credential definition in a multitenant - endorsement setup. However, after the successful creation and endorsement of the Revocation Registry Definition (RevRegDef), the automatic process to create the corresponding initial Revocation Registry Entry (the list state) fails with the WalletNotFoundError shown above.
To clarify the sequence based on my logs:
A tenant agent (using askar-anoncreds wallet type, configured with an endorser) successfully creates a CredDef via endorsement. The same tenant agent successfully creates the associated RevRegDef via endorsement. The anoncreds::revocation-registry-definition::finished event is emitted. The event handler revocation_setup.py::on_rev_reg_def triggers automatically upon receiving this event. This handler successfully uploads the tails file and then calls revocation.py::create_and_register_revocation_list. This leads down a call stack eventually calling indy_vdr.py::send_revoc_reg_entry, which needs to look up the tenant's issuer DID using askar.py::get_local_did to prepare the transaction for endorsement. Failure: get_local_did raises WalletNotFoundError for the tenant's own DID at this point in the execution flow. It seems that while the initial request context (handled via API and endorsement flow) correctly resolves the tenant's wallet and DID, the execution context available within the on_rev_reg_def event handler (running asynchronously after the RevRegDef write is acknowledged) is somehow unable to access the same DID within the tenant's wallet.
Possibly related to #3586
@esune — heads up on this one that might be related to what you are doing.
Status Update: After days of debugging, I’m still stuck. The error persists when revocation is enabled in multitenant setups and askar-anoncreds wallet types. Here’s the simplest breakdown:
What Happens
- Success:
- ✅ Tenant creates Credential Definition (via endorser).
- ✅ Tenant creates Revocation Registry Definition (via endorser).
- Failure:
- ❌ Automatic next step (creating the revocation list) crashes with:
WalletNotFoundError: Unknown DID: [Tenant's DID] - The tenant’s DID exists in their wallet but disappears during this step.
- ❌ Automatic next step (creating the revocation list) crashes with:
What I’ve Tried
- Checked that the DID exists in the tenant wallet (it does).
- Verified the revocation registry/tails file setup works in single-tenant mode.
- Added explicit checks to confirm the DID is present before ledger calls.
Where It Breaks
The error happens here:
File "acapy_agent/wallet/askar.py", line 398, in get_local_did
Why? The code suddenly tries to find the tenant’s DID in the admin wallet instead of the tenant’s wallet.
The Mystery
- Why does the context switch to the admin wallet after the RevRegDef is created?
Request for Help:
If anyone has seen similar issues or knows how to preserve the tenant context during automatic revocation steps, please share insights. This blocks revocation in multitenant deployments.
Simplest Reproduction:
- Configure ACA-Py with multitenancy + endorser.
- Create a tenant wallet.
- Issue a credential definition with revocation enabled.
- Observe failure at revocation list creation.
Thank you for the troubleshooting @MonolithicMonk, really appreciated. I will try to take a look asap and see if I can help, haven't yet had a chance to dig into this specific issue. Will report back if/when I do, if anyone else has input please let's continue this conversation - we'll get to a resolution.
Full context of the error logs before failure:
DEBUG /acapy_agent/anoncreds/revocation.py:396 ENTER: create_and_register_revocation_list() - RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1, Options: {'revocation_registry_size': 100, 'support_revocation': True}
DEBUG /acapy_agent/anoncreds/revocation.py:402 Create and register revocation list profile: <Settings('trimmed', transport.max_outbound_retry=4, transport.ws.heartbeat_interval=3, transport.ws.timeout_interval=15, default_label=Testest Test, wallet.key=9c232979-23eb-4b4a-88ca-9d8f8961eede, wallet.name=Testest Test, wallet.storage_type=postgres_storage, wallet.type=askar-anoncreds)>
DEBUG /acapy_agent/anoncreds/revocation.py:405 Normalized options: {'revocation_registry_size': 100, 'support_revocation': True}
DEBUG /acapy_agent/anoncreds/revocation.py:408 Attempting to fetch revocation registry data from storage - RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1
DEBUG /acapy_agent/anoncreds/revocation.py:420 Storage fetch results - RevRegDef Entry: Found, Private Entry: Found
DEBUG /acapy_agent/anoncreds/revocation.py:455 Fetching associated credential definition: RANDOM2025DIDRANDOM101:3:CL:2738835:random1
INFO /acapy_agent/anoncreds/revocation_setup.py:134 Successfully uploaded tails file for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
DEBUG /acapy_agent/anoncreds/revocation_setup.py:158 Proceeding to create/register revocation list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
DEBUG /acapy_agent/anoncreds/revocation.py:396 ENTER: create_and_register_revocation_list() - RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0, Options: {'revocation_registry_size': 100, 'support_revocation': True}
DEBUG /acapy_agent/anoncreds/revocation.py:402 Create and register revocation list profile: <Settings('trimmed', transport.max_outbound_retry=4, transport.ws.heartbeat_interval=3, transport.ws.timeout_interval=15, default_label=Testest Test, wallet.key=9c232979-23eb-4b4a-88ca-9d8f8961eede, wallet.name=Testest Test, wallet.storage_type=postgres_storage, wallet.type=askar-anoncreds)>
DEBUG /acapy_agent/anoncreds/revocation.py:405 Normalized options: {'revocation_registry_size': 100, 'support_revocation': True}
DEBUG /acapy_agent/anoncreds/revocation.py:408 Attempting to fetch revocation registry data from storage - RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
DEBUG /acapy_agent/anoncreds/revocation.py:471 Successfully retrieved credential definition
DEBUG /acapy_agent/anoncreds/revocation.py:485 Deserializing revocation registry definition
DEBUG /acapy_agent/anoncreds/revocation.py:488 Successfully deserialized RevRegDef
DEBUG /acapy_agent/anoncreds/revocation.py:497 Deserializing credential definition
DEBUG /acapy_agent/anoncreds/revocation.py:500 Successfully deserialized CredDef
DEBUG /acapy_agent/anoncreds/revocation.py:509 Loading revocation registry private definition
DEBUG /acapy_agent/anoncreds/revocation.py:514 Successfully loaded private definition
DEBUG /acapy_agent/anoncreds/revocation.py:524 Updating tails location to local path for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1
DEBUG /acapy_agent/anoncreds/revocation.py:530 Tails location updated - Original: https://tails-test.vonx.io/hash/6aKH42FKW4bAEMcMBCvdEREh6teYzBZNCHa34XQF5igx, New: /home/.indy_client/tails/6aKH42FKW4bAEMcMBCvdEREh6teYzBZNCHa34XQF5igx
DEBUG /acapy_agent/anoncreds/revocation.py:537 Creating revocation status list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1
INFO /acapy_agent/anoncreds/revocation.py:549 Successfully created revocation status list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1
DEBUG /acapy_agent/anoncreds/revocation.py:562 Retrieving AnonCreds registry implementation
DEBUG /acapy_agent/anoncreds/revocation.py:564 Registry implementation: AnonCredsRegistry
DEBUG /acapy_agent/anoncreds/revocation.py:569 Registering revocation list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1
DEBUG /acapy_agent/anoncreds/revocation.py:420 Storage fetch results - RevRegDef Entry: Found, Private Entry: Found
DEBUG /acapy_agent/anoncreds/revocation.py:455 Fetching associated credential definition: RANDOM2025DIDRANDOM101:3:CL:2738835:random1
DEBUG /acapy_agent/anoncreds/revocation.py:471 Successfully retrieved credential definition
DEBUG /acapy_agent/anoncreds/revocation.py:485 Deserializing revocation registry definition
DEBUG /acapy_agent/anoncreds/revocation.py:488 Successfully deserialized RevRegDef
DEBUG /acapy_agent/anoncreds/revocation.py:497 Deserializing credential definition
DEBUG /acapy_agent/anoncreds/revocation.py:500 Successfully deserialized CredDef
DEBUG /acapy_agent/anoncreds/revocation.py:509 Loading revocation registry private definition
DEBUG /acapy_agent/anoncreds/revocation.py:514 Successfully loaded private definition
DEBUG /acapy_agent/anoncreds/revocation.py:524 Updating tails location to local path for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
DEBUG /acapy_agent/anoncreds/revocation.py:530 Tails location updated - Original: https://tails-test.vonx.io/hash/BJZD1uTuBsWeehBGJvzp1XZYjz8mkcdn9PLQAXzgkQRq, New: /home/.indy_client/tails/BJZD1uTuBsWeehBGJvzp1XZYjz8mkcdn9PLQAXzgkQRq
DEBUG /acapy_agent/anoncreds/revocation.py:537 Creating revocation status list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
INFO /acapy_agent/anoncreds/revocation.py:549 Successfully created revocation status list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
DEBUG /acapy_agent/anoncreds/revocation.py:562 Retrieving AnonCreds registry implementation
DEBUG /acapy_agent/anoncreds/revocation.py:564 Registry implementation: AnonCredsRegistry
DEBUG /acapy_agent/anoncreds/revocation.py:569 Registering revocation list for RevRegDef ID: RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:0
DEBUG /acapy_agent/ledger/indy_vdr.py:222 Opening the pool ledger
DEBUG /acapy_agent/ledger/indy_vdr.py:222 Opening the pool ledger
DEBUG /acapy_agent/ledger/indy_vdr.py:222 Opening the pool ledger
WARNING /acapy_agent/ledger/multiple_ledger/indy_vdr_manager.py:135 Did RANDOM2025DIDRANDOM101 not posted to ledger indicio-demo
DEBUG /acapy_agent/ledger/indy_vdr.py:1245 send_revoc_reg_entry.Wallet instance: <AskarWallet>
DEBUG /acapy_agent/ledger/indy_vdr.py:1246 send_revoc_reg_entry.Profile context: <Settings('trimmed', transport.max_outbound_retry=4, transport.ws.heartbeat_interval=3, transport.ws.timeout_interval=15, default_label=multitenant-admin, wallet.key=multitenant-admin-wallet-key, wallet.name=multitenant-admin-wallet, wallet.storage_type=postgres_storage, wallet.type=askar-anoncreds)>
DEBUG /acapy_agent/ledger/indy_vdr.py:1249 send_revoc_reg_entry.Public DID retrieved from wallet: None
ERROR /acapy_agent/anoncreds/revocation.py:584 Failed to register revocation list for ID RANDOM2025DIDRANDOM101:4:RANDOM2025DIDRANDOM101:3:CL:2738835:random1:CL_ACCUM:1: Unknown DID: RANDOM2025DIDRANDOM101
Traceback (most recent call last):
File "/acapy_agent/anoncreds/revocation.py", line 575, in create_and_register_revocation_list
result = await anoncreds_registry.register_revocation_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/registry.py", line 171, in register_revocation_list
return await registrar.register_revocation_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/default/legacy_indy/registry.py", line 890, in register_revocation_list
result = await self._revoc_reg_entry_with_fix(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/default/legacy_indy/registry.py", line 824, in _revoc_reg_entry_with_fix
rev_entry_res = await ledger.send_revoc_reg_entry(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/ledger/indy_vdr.py", line 1252, in send_revoc_reg_entry
did_info = await wallet.get_local_did(issuer_did)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/wallet/askar.py", line 398, in get_local_did
raise WalletNotFoundError("Unknown DID: {}".format(did))
acapy_agent.wallet.error.WalletNotFoundError: Unknown DID: RANDOM2025DIDRANDOM101
Edits to generate logs:
# acapy_agent/anoncreds/revocation_setup.py
async def on_rev_reg_def(self, profile: Profile, event: RevRegDefFinishedEvent):
"""Handle rev reg def finished."""
payload = event.payload
# --- Detailed Logging START ---
LOGGER.debug("=" * 80)
LOGGER.debug(f"Entered 'on_rev_reg_def' handler.")
LOGGER.debug(
"ENTER: on_rev_reg_def() - Received RevRegDefFinishedEvent. "
"Event ID: %s, RevRegDef ID: %s, Tag: %s",
event.payload,
payload.rev_reg_def_id,
payload.rev_reg_def.tag
)
LOGGER.debug("Full event payload: %s", payload)
LOGGER.debug("Profile settings at entry: %s", profile.context.settings)
auto_create_revocation = True
if is_author_role(profile):
LOGGER.debug("Agent is in author role, checking auto-create settings")
auto_create_revocation = profile.settings.get(
"endorser.auto_create_rev_reg", False
)
LOGGER.info(
"Author role auto-create revocation registry setting: %s",
auto_create_revocation
)
else:
LOGGER.debug("Agent is NOT in author role, proceeding with default auto-create")
if auto_create_revocation:
LOGGER.info(
"Auto-creation of revocation registry enabled. "
"Proceeding with revocation setup for RevRegDef ID: %s",
payload.rev_reg_def_id
)
revoc = AnonCredsRevocation(profile)
failed_to_upload_tails = False
# Tails file upload sequence
LOGGER.debug(
"Attempting tails file upload for RevRegDef ID: %s",
payload.rev_reg_def_id
)
try:
LOGGER.debug("Calling upload_tails_file() with RevRegDef: %s", payload.rev_reg_def)
await revoc.upload_tails_file(payload.rev_reg_def)
LOGGER.info(
"Successfully uploaded tails file for RevRegDef ID: %s",
payload.rev_reg_def_id
)
except AnonCredsRevocationError as err:
LOGGER.warning(
"FAILED to upload tails file for RevRegDef ID: %s. Error: %s",
payload.rev_reg_def_id,
str(err),
exc_info=True
)
failed_to_upload_tails = True
LOGGER.debug("Set failed_to_upload_tails flag to True")
if failed_to_upload_tails:
LOGGER.warning(
"Tails file upload failed. Adding 'failed_to_upload' flag to options. "
"Original options: %s",
payload.options
)
payload.options["failed_to_upload"] = True
LOGGER.debug("Updated options: %s", payload.options)
# Revocation list creation
LOGGER.debug(
"Proceeding to create/register revocation list for RevRegDef ID: %s",
payload.rev_reg_def_id
)
try:
await revoc.create_and_register_revocation_list(
payload.rev_reg_def_id,
payload.options
)
LOGGER.info(
"Successfully created and registered revocation list for RevRegDef ID: %s",
payload.rev_reg_def_id
)
except Exception as e:
LOGGER.error(
"Failed to create/register revocation list for RevRegDef ID: %s. Error: %s",
payload.rev_reg_def_id,
str(e),
exc_info=True
)
raise
# Active registry management
if payload.rev_reg_def.tag == str(0):
LOGGER.debug(
"First registry detected (tag=0). Attempting to set active registry. "
"RevRegDef ID: %s",
payload.rev_reg_def_id
)
try:
await revoc.set_active_registry(payload.rev_reg_def_id)
LOGGER.info(
"Successfully set active registry to: %s",
payload.rev_reg_def_id
)
LOGGER.debug("Current profile settings after activation: %s", profile.context.settings)
except Exception as e:
LOGGER.error(
"Failed to set active registry for ID: %s. Error: %s",
payload.rev_reg_def_id,
str(e),
exc_info=True
)
raise
else:
LOGGER.debug(
"Not setting active registry - tag %s is not the initial registry (0)",
payload.rev_reg_def.tag
)
else:
LOGGER.info(
"Auto-creation of revocation registry DISABLED. "
"Skipping further processing for RevRegDef ID: %s",
payload.rev_reg_def_id
)
LOGGER.debug(
"EXIT: on_rev_reg_def() - Completed processing for RevRegDef ID: %s",
payload.rev_reg_def_id
)
# acapy_agent/anoncreds/revocation.py
async def create_and_register_revocation_list(
self, rev_reg_def_id: str, options: Optional[dict] = None
):
"""Create and register a revocation list."""
LOGGER.debug(
"ENTER: create_and_register_revocation_list() - RevRegDef ID: %s, Options: %s",
rev_reg_def_id,
options
)
LOGGER.debug("Create and register revocation list profile: %s", self.profile.context.settings)
options = options or {}
LOGGER.debug("Normalized options: %s", options)
try:
LOGGER.debug(
"Attempting to fetch revocation registry data from storage - "
"RevRegDef ID: %s", rev_reg_def_id
)
async with self.profile.session() as session:
rev_reg_def_entry = await session.handle.fetch(
CATEGORY_REV_REG_DEF, rev_reg_def_id
)
rev_reg_def_private_entry = await session.handle.fetch(
CATEGORY_REV_REG_DEF_PRIVATE, rev_reg_def_id
)
LOGGER.debug(
"Storage fetch results - RevRegDef Entry: %s, Private Entry: %s",
"Found" if rev_reg_def_entry else "Missing",
"Found" if rev_reg_def_private_entry else "Missing"
)
except AskarError as err:
LOGGER.error(
"Storage error fetching revocation registry data for ID %s: %s",
rev_reg_def_id,
str(err),
exc_info=True
)
raise AnonCredsRevocationError(
"Error retrieving required revocation registry definition data"
) from err
# Validate required entries
missing = []
if not rev_reg_def_entry:
missing.append("revocation registry definition")
if not rev_reg_def_private_entry:
missing.append("revocation registry private definition")
if missing:
LOGGER.error(
"Missing required revocation registry components for ID %s: %s",
rev_reg_def_id,
", ".join(missing)
)
raise AnonCredsRevocationError(
f"Missing required revocation registry data: {', '.join(missing)}"
)
try:
cred_def_id = rev_reg_def_entry.value_json["credDefId"]
LOGGER.debug(
"Fetching associated credential definition: %s", cred_def_id
)
async with self.profile.session() as session:
cred_def_entry = await session.handle.fetch(
CATEGORY_CRED_DEF, cred_def_id
)
if not cred_def_entry:
LOGGER.error(
"Credential definition not found: %s", cred_def_id
)
raise AnonCredsRevocationError(
f"Credential definition {cred_def_id} not found"
)
LOGGER.debug("Successfully retrieved credential definition")
except AskarError as err:
LOGGER.error(
"Storage error fetching credential definition %s: %s",
cred_def_id,
str(err),
exc_info=True
)
raise AnonCredsRevocationError(
f"Error retrieving cred def {cred_def_id}"
) from err
# Deserialization phase
LOGGER.debug("Deserializing revocation registry definition")
try:
rev_reg_def = RevRegDef.deserialize(rev_reg_def_entry.value_json)
LOGGER.debug("Successfully deserialized RevRegDef")
except Exception as e:
LOGGER.error(
"Failed to deserialize revocation registry definition: %s",
str(e),
exc_info=True
)
raise
LOGGER.debug("Deserializing credential definition")
try:
cred_def = CredDef.deserialize(cred_def_entry.value_json)
LOGGER.debug("Successfully deserialized CredDef")
except Exception as e:
LOGGER.error(
"Failed to deserialize credential definition: %s",
str(e),
exc_info=True
)
raise
LOGGER.debug("Loading revocation registry private definition")
try:
rev_reg_def_private = RevocationRegistryDefinitionPrivate.load(
rev_reg_def_private_entry.value_json
)
LOGGER.debug("Successfully loaded private definition")
except Exception as e:
LOGGER.error(
"Failed to load revocation registry private definition: %s",
str(e),
exc_info=True
)
raise
# Tails location handling
LOGGER.debug(
"Updating tails location to local path for RevRegDef ID: %s",
rev_reg_def_id
)
original_tails = rev_reg_def.value.tails_location
rev_reg_def.value.tails_location = self.get_local_tails_path(rev_reg_def)
LOGGER.debug(
"Tails location updated - Original: %s, New: %s",
original_tails,
rev_reg_def.value.tails_location
)
# Revocation list creation
LOGGER.debug(
"Creating revocation status list for RevRegDef ID: %s",
rev_reg_def_id
)
try:
rev_list = RevocationStatusList.create(
cred_def.to_native(),
rev_reg_def_id,
rev_reg_def.to_native(),
rev_reg_def_private,
rev_reg_def.issuer_id,
)
LOGGER.info(
"Successfully created revocation status list for RevRegDef ID: %s",
rev_reg_def_id
)
except Exception as e:
LOGGER.error(
"Failed to create revocation status list: %s",
str(e),
exc_info=True
)
raise
# Registry interaction
LOGGER.debug("Retrieving AnonCreds registry implementation")
anoncreds_registry = self.profile.inject(AnonCredsRegistry)
LOGGER.debug(
"Registry implementation: %s",
type(anoncreds_registry).__name__
)
LOGGER.debug(
"Registering revocation list for RevRegDef ID: %s",
rev_reg_def_id
)
try:
native_rev_list = RevList.from_native(rev_list)
result = await anoncreds_registry.register_revocation_list(
self.profile, rev_reg_def, native_rev_list, options
)
LOGGER.info(
"Revocation list registration result for ID %s: %s",
rev_reg_def_id,
result.revocation_list_state.state
)
except Exception as e:
LOGGER.error(
"Failed to register revocation list for ID %s: %s",
rev_reg_def_id,
str(e),
exc_info=True
)
raise
# Handle failed upload state
if options.get("failed_to_upload", False):
LOGGER.warning(
"Tails file upload failed detected, setting revocation list state to FAILED"
)
original_state = result.revocation_list_state.state
result.revocation_list_state.state = RevListState.STATE_FAILED
LOGGER.debug(
"State transition - From: %s, To: %s",
original_state,
result.revocation_list_state.state
)
# Storage operation
LOGGER.debug("Storing revocation list result")
try:
await self.store_revocation_registry_list(result)
LOGGER.info(
"Successfully stored revocation list for RevRegDef ID: %s",
rev_reg_def_id
)
except Exception as e:
LOGGER.error(
"Failed to store revocation list result: %s",
str(e),
exc_info=True
)
raise
LOGGER.debug(
"EXIT: create_and_register_revocation_list() - RevRegDef ID: %s",
rev_reg_def_id
)
return result
@MonolithicMonk are the edits you made to generate logs something that would be useful to include in ACA-Py in general, or just for this troubleshooting? If you push changes to a branch might be easier to take them into account for further testing.
@esune sorry for late reply, I was busy outdoors today. The edits I made are strictly to debug the issue. They may be useful going forward but as is they are far too verbose
I've been able to track the issue down to send_revoc_reg_entry. It wasn't receiving the profile that have been passed down from the initial credential definition creation. I have added the following edits but I'm still resolving an additional error that I will resolve before pull request:
# acapy_agent/ledger/base.py
@abstractmethod
async def send_revoc_reg_entry(
self,
revoc_reg_id: str,
revoc_def_type: str,
revoc_reg_entry: dict,
issuer_did: Optional[str] = None,
write_ledger: bool = True,
endorser_did: Optional[str] = None,
profile: Optional[Profile] = None, # <----- adding optional parameter
) -> dict:
"""Publish a revocation registry entry to the ledger."""
# acapy_agent/ledger/indy_vdr.py
async def send_revoc_reg_entry(
self,
revoc_reg_id: str,
revoc_def_type: str,
revoc_reg_entry: dict,
issuer_did: Optional[str] = None,
write_ledger: bool = True,
endorser_did: Optional[str] = None,
profile: Optional[Profile] = None, # <---- adding optional parameter
) -> dict:
"""Publish a revocation registry entry to the ledger."""
current_profile = profile or self.profile
async with current_profile.session() as session:
# rest of code ...
if current_profile.context.settings.get("wallet.type") == "askar-anoncreds":
# rest of code ...
I've been able to track the issue down to
send_revoc_reg_entry. It wasn't receiving the profile that have been passed down from the initial credential definition creation. I have added the following edits but I'm still resolving an additional error that I will resolve before pull request:
Thank you!
@esune The following is the logs that arises after resolving the profile context switching issue. It is very similar fix however, it is leading me down a rabbit hole of injecting a profile parameter into several functions to get the proper context. I'm not comfortable going down that path because it spans several files that I'm not familiar with. Not to mention the unintended consequences. The code base obviously knew how to determine tenant context before without the need to explicitly pass in a profile to every function.
Here is the error logs for the new issue that arose:
DEBUG /acapy_agent/core/event_bus.py:101 Notifying subscribers: <Event topic=acapy::record::endorse_transaction::transaction_acked, payload={"trimmed"}>
DEBUG /acapy_agent/core/event_bus.py:101 Notifying subscribers: <Event topic=acapy::record::endorse_transaction::transaction_acked, payload={"trimmed"}>
DEBUG /acapy_agent/core/event_bus.py:101 Notifying subscribers: <Event topic=anoncreds::revocation-registry-definition::finished, ("trimmed")>
DEBUG /acapy_agent/core/event_bus.py:101 Notifying subscribers: <Event topic=anoncreds::revocation-registry-definition::finished, ("trimmed")>
DEBUG /acapy_agent/ledger/indy_vdr.py:222 Opening the pool ledger
DEBUG /acapy_agent/ledger/indy_vdr.py:222 Opening the pool ledger
DEBUG /acapy_agent/ledger/indy_vdr.py:222 Opening the pool ledger
ERROR /acapy_agent/core/event_bus.py:123 Error occurred while processing event
Traceback (most recent call last):
File "/acapy_agent/core/event_bus.py", line 121, in notify
await processor()
File "/acapy_agent/anoncreds/revocation_setup.py", line 105, in on_rev_reg_def
await revoc.create_and_register_revocation_list(
File "/acapy_agent/anoncreds/revocation.py", line 452, in create_and_register_revocation_list
result = await anoncreds_registry.register_revocation_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/registry.py", line 171, in register_revocation_list
return await registrar.register_revocation_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/default/legacy_indy/registry.py", line 891, in register_revocation_list
result = await self._revoc_reg_entry_with_fix(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/default/legacy_indy/registry.py", line 824, in _revoc_reg_entry_with_fix
rev_entry_res = await ledger.send_revoc_reg_entry(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/ledger/indy_vdr.py", line 1271, in send_revoc_reg_entry
resp = await legacy_indy_registry.txn_submit(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/anoncreds/default/legacy_indy/registry.py", line 1224, in txn_submit
return await shield(
^^^^^^^^^^^^^
File "/home/dgateman/.pyenv/versions/3.12.9/lib/python3.12/asyncio/futures.py", line 289, in __await__
yield self # This tells Task to wait for completion.
^^^^^^^^^^
File "/home/dgateman/.pyenv/versions/3.12.9/lib/python3.12/asyncio/tasks.py", line 385, in __wakeup
future.result()
File "/home/dgateman/.pyenv/versions/3.12.9/lib/python3.12/asyncio/futures.py", line 202, in result
raise self._exception.with_traceback(self._exception_tb)
File "/home/dgateman/.pyenv/versions/3.12.9/lib/python3.12/asyncio/tasks.py", line 314, in __step_run_and_handle_result
result = coro.send(None)
^^^^^^^^^^^^^^^
File "/acapy_agent/ledger/indy_vdr.py", line 1350, in txn_submit
resp = await self._submit(
^^^^^^^^^^^^^^^^^^^
File "/acapy_agent/ledger/indy_vdr.py", line 360, in _submit
await wallet.sign_message(request.signature_input, sign_did.verkey)
File "/acapy_agent/wallet/askar.py", line 729, in sign_message
raise WalletNotFoundError("Missing key for sign operation")
acapy_agent.wallet.error.WalletNotFoundError: Missing key for sign operation
@MonolithicMonk I attempted to create a minimal example of the issue you're seeing here: https://github.com/Indicio-tech/acapy-minimal-example/blob/test/rev-reg-entry-failure/examples/multitenancy/example.py
This example succeeds (it can fail occasionally but only from timeouts when a ledger is being particularly slow). You can run that example with docker-compose run --rm example; do you notice any differences between this setup and yours where you're encountering this issue?
@dbluhm See your logs, you will likely find WalletNotFoundError not found error logs
Actually, no, there aren't any WalletNotFoundErrors in the logs from this example. Full logs (log level debug):
@MonolithicMonk I attempted to create a minimal example of the issue you're seeing here: https://github.com/Indicio-tech/acapy-minimal-example/blob/test/rev-reg-entry-failure/examples/multitenancy/example.py
This example succeeds (it can fail occasionally but only from timeouts when a ledger is being particularly slow). You can run that example with
dc run --rm example; do you notice any differences between this setup and yours where you're encountering this issue
As a side note, I actually love your repro repo . It shows a lot about how acapy works internally
Re:
As a side note, I actually love your repro repo . It shows a lot about how acapy works internally
@dbluhm — perhaps we should add that repo to OWF as “acapy-debug” or something like that. @MonolithicMonk ’s comment is about the 10th I’ve seen about how useful that repo is. Thoughts? We could add some docs for it on https://aca-py.org and encourage it’s use.
Re:
As a side note, I actually love your repro repo . It shows a lot about how acapy works internally
@dbluhm — perhaps we should add that repo to OWF as “acapy-debug” or something like that. @MonolithicMonk ’s comment is about the 10th I’ve seen about how useful that repo is. Thoughts? We could add some docs for it on https://aca-py.org and encourage it’s use.
I think we'd be interested in that but double checking with the powers that be!
I have observed this error in version 1.3*.
I just tested with the py3.12-nightly-2025-04-16 image and still don't see WalletNotFoundErrors 🤔
I updated my example, in case that's useful for reference.
I just tested with the py3.12-nightly-2025-04-16 image and still don't see
WalletNotFoundErrors 🤔
After further testing, I have now discovered that the trigger for the error is setting the configuration value --genesis-transactions-list instead of genesis-url.
The test was conducted using my testing environment however I tested both your configuration and mine and discovered the difference are those values - genesis-url also works for me, genesis-transactions-list fails, even if only 1 ledger is configured.
ps: I used my testing environment because I think your test repo hard codes a check for genesis_url and I didn't want to dig through it.
By the way, could this have been related to this bug? #3646 Where auto-provisioning of a public did was broken
The fix would have been included in the py3.12-nightly-2025-04-16 image that @dbluhm tests with. Error might pop up before the fix, with py3.12-nightly-2025-04-13?
@MonolithicMonk Is it possible to replicate the issue from the latest main branch? Does it still persist?
I just tested with latest nightly and the error persists for genesis-transactions-list configuration. A quick glance at #3646 and related issue suggest that they have to do with incomplete DID data. This one is more closely related to the expected vs actual wallet carrying the DID.
Hmm, I see. That's a very strange issue.
AFAICT, it should be something to do with how settings["ledger.ledger_config_list"] is being configured in acapy_agent/config/argparse.py, and how it's handled downstream...
Because that seems to be primary difference with parsing genesis-transactions-list instead of genesis_url
I recently made some logging improvements to what's going on there. (#3332) Merged last week. So debug logs will be a bit more informative.
Could you possible share the startup debug logs? Mainly the ledger config stage, to see if anything stands out.
I was looking at the argparse and ledger config code just now, and figured it's worth it to make some of that more readable... kind of like a baby step to try debug this issue: #3664
The following is my startup logs:
acapy_agent.core.conductor DEBUG Starting setup of the Conductor
acapy_agent.config.default_context DEBUG Building new injection context
acapy_agent.config.default_context DEBUG Begin binding providers to context
acapy_agent.config.default_context DEBUG Initializing plugin registry
acapy_agent.core.plugin_registry DEBUG Registering package: acapy_agent.protocols
acapy_agent.utils.classloader DEBUG Scanning subpackages under package acapy_agent.protocols
acapy_agent.utils.classloader DEBUG Extracted main package: acapy_agent, sub-package: protocols
acapy_agent.utils.classloader DEBUG 17 sub-packages found under acapy_agent: ['acapy_agent.protocols.out_of_band', 'acapy_agent.protocols.actionmenu', 'acapy_agent.protocols.present_proof', 'acapy_agent.protocols.endorse_transaction', 'acapy_agent.protocols.tests', 'acapy_agent.protocols.introduction', 'acapy_agent.protocols.trustping', 'acapy_agent.protocols.problem_report', 'acapy_agent.protocols.routing', 'acapy_agent.protocols.didexchange', 'acapy_agent.protocols.notification', 'acapy_agent.protocols.discovery', 'acapy_agent.protocols.did_rotate', 'acapy_agent.protocols.basicmessage', 'acapy_agent.protocols.issue_credential', 'acapy_agent.protocols.coordinate_mediation', 'acapy_agent.protocols.revocation_notification']
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.out_of_band
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.actionmenu
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.present_proof
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.endorse_transaction
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.introduction
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.trustping
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.problem_report
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.routing
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.didexchange
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.notification
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.discovery
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.did_rotate
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.basicmessage
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.issue_credential
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.coordinate_mediation
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.protocols.revocation_notification
acapy_agent.config.default_context INFO Registering default plugins
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.holder
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.ledger
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.connections
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.messaging.jsonld
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.resolver
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.settings
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.vc
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.vc.data_integrity
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.wallet
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.wallet.keys
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.did.indy
acapy_agent.config.default_context INFO Registering askar plugins
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.messaging.credential_definitions
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.messaging.schemas
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.revocation
acapy_agent.config.default_context INFO Registering anoncreds plugins
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.anoncreds
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.anoncreds.default.did_indy
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.anoncreds.default.did_web
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.anoncreds.default.legacy_indy
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.revocation_anoncreds
acapy_agent.config.default_context INFO Registering multitenant admin API plugin
acapy_agent.core.plugin_registry DEBUG Registered plugin: acapy_agent.multitenant.admin
acapy_agent.config.default_context DEBUG Registering external plugin: multitenant_provider.v1_0
acapy_agent.core.plugin_registry ERROR Module doesn't exist: multitenant_provider.v1_0
acapy_agent.config.default_context DEBUG Registering external plugin: traction_innkeeper.v1_0
acapy_agent.core.plugin_registry ERROR Module doesn't exist: traction_innkeeper.v1_0
acapy_agent.config.default_context DEBUG Registering external plugin: basicmessage_storage.v1_0
acapy_agent.core.plugin_registry ERROR Module doesn't exist: basicmessage_storage.v1_0
acapy_agent.config.default_context DEBUG Registering external plugin: connection_update.v1_0
acapy_agent.core.plugin_registry ERROR Module doesn't exist: connection_update.v1_0
acapy_agent.config.default_context DEBUG Registering external plugin: rpc.v1_0
acapy_agent.core.plugin_registry ERROR Module doesn't exist: rpc.v1_0
acapy_agent.config.default_context DEBUG Registering external plugin: connections
acapy_agent.core.plugin_registry ERROR Module doesn't exist: connections
acapy_agent.core.plugin_registry DEBUG Initializing plugin context for 36 plugins
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.out_of_band.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.out_of_band.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.actionmenu.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.actionmenu.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.present_proof.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.present_proof.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.present_proof.v2_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.present_proof.v2_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.endorse_transaction.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.endorse_transaction.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.issue_credential.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.issue_credential.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.issue_credential.v2_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.issue_credential.v2_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.coordinate_mediation.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class Controller from module acapy_agent.protocols.coordinate_mediation.v1_0.controller
acapy_agent.utils.classloader DEBUG Successfully loaded class LegacyPeerDIDResolver from module acapy_agent.resolver.default.legacy_peer
acapy_agent.utils.classloader DEBUG Successfully loaded class KeyDIDResolver from module acapy_agent.resolver.default.key
acapy_agent.utils.classloader DEBUG Successfully loaded class JwkDIDResolver from module acapy_agent.resolver.default.jwk
acapy_agent.utils.classloader DEBUG Successfully loaded class IndyDIDResolver from module acapy_agent.resolver.default.indy
acapy_agent.utils.classloader DEBUG Successfully loaded class WebDIDResolver from module acapy_agent.resolver.default.web
acapy_agent.utils.classloader DEBUG Successfully loaded class WebvhDIDResolver from module acapy_agent.resolver.default.webvh
acapy_agent.utils.classloader DEBUG Successfully loaded class PeerDID1Resolver from module acapy_agent.resolver.default.peer1
acapy_agent.utils.classloader DEBUG Successfully loaded class PeerDID2Resolver from module acapy_agent.resolver.default.peer2
acapy_agent.utils.classloader DEBUG Successfully loaded class PeerDID3Resolver from module acapy_agent.resolver.default.peer3
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::record::connections::deleted'), processor <bound method PeerDID3Resolver.remove_record_for_deleted_conn of <acapy_agent.resolver.default.peer3.PeerDID3Resolver object at 0x78059726fce0>>
acapy_agent.utils.classloader DEBUG Successfully loaded class PeerDID4Resolver from module acapy_agent.resolver.default.peer4
acapy_agent.utils.classloader DEBUG Successfully loaded class DIDIndyRegistry from module acapy_agent.anoncreds.default.did_indy.registry
acapy_agent.anoncreds.default.did_indy.registry INFO Successfully registered DIDIndyRegistry
acapy_agent.utils.classloader DEBUG Successfully loaded class DIDWebRegistry from module acapy_agent.anoncreds.default.did_web.registry
acapy_agent.anoncreds.default.did_web.registry INFO Successfully registered DIDWebRegistry
acapy_agent.utils.classloader DEBUG Successfully loaded class LegacyIndyRegistry from module acapy_agent.anoncreds.default.legacy_indy.registry
acapy_agent.anoncreds.default.legacy_indy.registry INFO Successfully registered LegacyIndyRegistry
acapy_agent.core.plugin_registry DEBUG Registering protocol events for 36 plugins
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::core::startup?$'), processor <function on_startup_event at 0x7805970c0040>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::core::shutdown?$'), processor <function on_shutdown_event at 0x7805970e47c0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::published.*'), processor <function on_revocation_published at 0x780596ef4a40>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::clear-pending.*'), processor <function on_pending_cleared at 0x780596ef4ae0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::published.*'), processor <function on_revocation_published at 0x780596ef4f40>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::clear-pending.*'), processor <function on_pending_cleared at 0x780596ef6340>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::ENDORSE_DID::(.*)?$'), processor <function on_register_nym_event at 0x780597b7e8e0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::CRED_DEF::(.*)?$'), processor <function on_cred_def_event at 0x780597926fc0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::SCHEMA::(.*)?$'), processor <function on_schema_event at 0x7805977c9800>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::REGISTRY_INIT.*'), processor <function on_revocation_registry_init_event at 0x7805976158a0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::REGISTRY_ENDORSED.*'), processor <function on_revocation_registry_endorsed_event at 0x780597648180>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::SEND_ENTRY.*'), processor <function on_revocation_entry_event at 0x780597617d80>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::REVOCATION::REV_REG_ENTRY_TXN_FAILED.*'), processor <function on_rev_reg_entry_txn_failed at 0x780597648360>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('anoncreds::credential-definition::finished'), processor <bound method DefaultRevocationSetup.on_cred_def of <acapy_agent.anoncreds.revocation_setup.DefaultRevocationSetup object at 0x780597560230>>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('anoncreds::revocation-registry-definition::finished'), processor <bound method DefaultRevocationSetup.on_rev_reg_def of <acapy_agent.anoncreds.revocation_setup.DefaultRevocationSetup object at 0x780597560230>>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('anoncreds::revocation-list::finished'), processor <bound method DefaultRevocationSetup.on_rev_list of <acapy_agent.anoncreds.revocation_setup.DefaultRevocationSetup object at 0x780597560230>>
acapy_agent.core.conductor DEBUG Context built successfully
acapy_agent.core.conductor DEBUG Ledger config list found. Loading multiple genesis transactions
acapy_agent.config.ledger DEBUG Processing 1 ledger configs
acapy_agent.config.ledger INFO Fetching genesis transactions from: http://test.bcovrin.vonx.io/genesis
acapy_agent.config.ledger DEBUG Processed 1 ledger configs successfully
acapy_agent.core.conductor DEBUG Configuring the root profile and setting up public DID
acapy_agent.core.profile INFO Create profile manager: askar-anoncreds
acapy_agent.utils.classloader DEBUG Successfully loaded class AskarAnonProfileManager from module acapy_agent.askar.profile_anon
acapy_agent.config.wallet INFO Created new profile - Profile name: traction-wallet, backend: askar-anoncreds
acapy_agent.utils.classloader DEBUG Successfully loaded class AskarWallet from module acapy_agent.wallet.askar
acapy_agent.config.wallet INFO No public DID created
acapy_agent.core.conductor DEBUG Root profile configured successfully
acapy_agent.core.conductor DEBUG Setting up multiledger manager
acapy_agent.core.conductor DEBUG Ledger backend: indy-vdr, Profile backend: askar-anoncreds
acapy_agent.core.conductor DEBUG Binding IndyCredxVerifier for 'askar-anoncreds' backend.
acapy_agent.config.ledger DEBUG Configuring ledger for profile traction-wallet and public_did None
acapy_agent.ledger.indy_vdr DEBUG Opening the pool ledger
acapy_agent.ledger.indy_vdr DEBUG Wrote pool ledger config 'bcovrin-test'
acapy_agent.config.ledger DEBUG Checking transaction author agreement
acapy_agent.config.ledger INFO Ledger configuration complete
acapy_agent.core.conductor DEBUG Ledger configured successfully.
acapy_agent.core.conductor DEBUG Transport not disabled. Setting up inbound transports.
acapy_agent.core.conductor DEBUG Inbound transports registered successfully.
acapy_agent.core.conductor DEBUG Setting up outbound transports.
acapy_agent.core.conductor DEBUG Outbound transports registered successfully.
acapy_agent.core.conductor DEBUG Initializing dispatcher.
acapy_agent.core.conductor DEBUG Dispatcher initialized successfully.
acapy_agent.utils.classloader DEBUG Successfully loaded class PackWireFormat from module acapy_agent.transport.pack_format
acapy_agent.core.conductor DEBUG Multitenant is enabled. Binding MultitenantManagerProvider.
acapy_agent.core.conductor DEBUG Binding RouteManagerProvider.
acapy_agent.core.conductor DEBUG Binding OobMessageProcessor.
acapy_agent.core.conductor DEBUG Binding default DocumentLoader.
acapy_agent.core.conductor DEBUG Admin API is enabled. Attempting to register admin server.
acapy_agent.multitenant.manager_provider INFO Create multitenant manager: single-wallet-askar
acapy_agent.utils.classloader DEBUG Successfully loaded class SingleWalletAskarMultitenantManager from module acapy_agent.multitenant.single_wallet_askar_manager
acapy_agent.core.conductor DEBUG Admin server registered on 0.0.0.0:8031
acapy_agent.core.conductor DEBUG Starting the Conductor agent.
acapy_agent.utils.classloader DEBUG Successfully loaded class AskarStorage from module acapy_agent.storage.askar
acapy_agent.core.conductor WARNING Wallet type record not found.
acapy_agent.core.conductor INFO New agent. Setting wallet type to askar-anoncreds.
acapy_agent.core.conductor DEBUG Wallet type validated.
acapy_agent.core.conductor DEBUG Transport not disabled. Starting inbound transports.
acapy_agent.core.conductor DEBUG Inbound transports started successfully.
acapy_agent.core.conductor DEBUG Starting outbound transports.
acapy_agent.core.conductor DEBUG Outbound transports started successfully.
acapy_agent.core.conductor DEBUG Admin server present. Starting admin server.
acapy_agent.core.plugin_registry DEBUG Registering admin routes for 36 plugins
acapy_agent.core.plugin_registry DEBUG Post-processing routes for 36 plugins
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::webhook::(.*)$'), processor <bound method AdminServer._on_webhook_event of <acapy_agent.admin.server.AdminServer object at 0x780596d8d880>>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('^acapy::record::([^:]*)(?:::.*)?$'), processor <bound method AdminServer._on_record_event of <acapy_agent.admin.server.AdminServer object at 0x780596d8d880>>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::basicmessage::received'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f564680>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::problem_report'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f564720>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::ping::received'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f5645e0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::ping::response_received'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f564540>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::actionmenu::received'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f5644a0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::actionmenu::get\\-active\\-menu'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f564ae0>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::actionmenu::perform\\-menu\\-action'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f564a40>
acapy_agent.core.event_bus DEBUG Subscribed: topic re.compile('acapy::keylist::updated'), processor <function AdminServer.start.<locals>.<lambda> at 0x78057f5649a0>
acapy_agent.core.conductor DEBUG Admin server started successfully.
acapy_agent.core.conductor DEBUG Admin responder bound to injector.
acapy_agent.core.conductor DEBUG Agent label: Traction Agent
acapy_agent.config.banner INFO
::::::::::::::::::::::::::::::::::::::::::::::
:: Traction Agent ::
:: ::
:: ::
:: Inbound Transports: ::
:: ::
:: - http://0.0.0.0:8030 ::
:: ::
:: Outbound Transports: ::
:: ::
:: - http ::
:: - https ::
:: ::
:: Administration API: ::
:: ::
:: - http://0.0.0.0:8031 ::
:: ::
:: ver: 1.3.0rc1 ::
::::::::::::::::::::::::::::::::::::::::::::::
acapy_agent.config.banner INFO
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: ⚠ DEPRECATION NOTICE: ⚠ ::
:: -------------------------------------------------------------------------------- ::
:: Receiving a core DIDComm protocol with the `did:sov:BzCbsNYhMrjHiqZDTUASHg;spec` ::
:: prefix is deprecated. All parties sending this prefix should be notified that ::
:: support for receiving such messages will be removed in a future release. Use ::
:: https://didcomm.org/ instead. ::
:: -------------------------------------------------------------------------------- ::
:: Aries RFC 0160: Connection Protocol is deprecated and support will be removed in ::
:: a future release; use RFC 0023: DID Exchange instead. ::
:: -------------------------------------------------------------------------------- ::
:: Aries RFC 0036: Issue Credential 1.0 is deprecated and support will be removed ::
:: in a future release; use RFC 0453: Issue Credential 2.0 instead. ::
:: -------------------------------------------------------------------------------- ::
:: Aries RFC 0037: Present Proof 1.0 is deprecated and support will be removed in a ::
:: future release; use RFC 0454: Present Proof 2.0 instead. ::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
acapy_agent.core.conductor DEBUG Recording ACA-Py version in wallet if needed.
acapy_agent.core.conductor WARNING Wallet version storage record not found.
acapy_agent.core.conductor DEBUG Force upgrade flag: False, From version config: None
acapy_agent.core.conductor DEBUG Determined from_version: None
acapy_agent.core.conductor WARNING ('No upgrade from version was found from wallet or via --from-version startup argument. Defaulting to %s.', 'v0.7.5')
acapy_agent.core.conductor DEBUG Set upgrade.from_version to default: v0.7.5
acapy_agent.core.conductor DEBUG Available upgrade versions: ['v0.8.1']
acapy_agent.core.conductor INFO Upgrade configurations available. Initiating upgrade.
acapy_agent.utils.classloader DEBUG Successfully loaded class AskarStorageSearch from module acapy_agent.storage.askar
acapy_agent.commands.upgrade INFO No ACA-Py version found in wallet storage.
acapy_agent.commands.upgrade INFO Selecting v0.7.5 as --from-version from the config.
acapy_agent.commands.upgrade INFO Running upgrade process for v0.8.1
acapy_agent.utils.classloader DEBUG Successfully loaded class ConnRecord from module acapy_agent.connections.models.conn_record
acapy_agent.commands.upgrade INFO No records of <class 'acapy_agent.connections.models.conn_record.ConnRecord'> found
acapy_agent.commands.upgrade INFO acapy_version storage record set to v1.3.0rc1
acapy_agent.core.conductor DEBUG Mediation invite provided: None
acapy_agent.core.conductor DEBUG Mediation invite record retrieved successfully.
acapy_agent.core.conductor DEBUG Checking for wallet upgrades in progress.
acapy_agent.core.conductor DEBUG Wallet upgrades check completed.
acapy_agent.core.conductor DEBUG Notifying protocols of startup status.
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::core::startup, payload={}>
acapy_agent.utils.classloader DEBUG Successfully loaded class InvitationMessageSchema from module acapy_agent.protocols.out_of_band.v1_0.messages.invitation
acapy_agent.protocols.out_of_band.v1_0.manager DEBUG Trying to find existing connection for oob invitation with did ENDORSER2025DIDRANDOM1
acapy_agent.utils.classloader DEBUG Successfully loaded class InvitationMessageSchema from module acapy_agent.protocols.out_of_band.v1_0.messages.invitation
acapy_agent.resolver.default.legacy_peer DEBUG Checking if resolver supports DID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.default.legacy_peer DEBUG DID is valid IndyDID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.default.legacy_peer DEBUG Failed to fetch doc for did ENDORSER2025DIDRANDOM1
acapy_agent.resolver.did_resolver DEBUG Valid resolvers for DID did:sov:ENDORSER2025DIDRANDOM1: [<acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>]
acapy_agent.resolver.did_resolver DEBUG Resolving DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>
acapy_agent.ledger.multiple_ledger.manager_provider INFO Create multiple Indy ledger manager: single-wallet-askar
acapy_agent.utils.classloader DEBUG Successfully loaded class IndyVdrLedgerPool from module acapy_agent.ledger.indy_vdr
acapy_agent.utils.classloader DEBUG Successfully loaded class IndyVdrLedger from module acapy_agent.ledger.indy_vdr
rlp.codec DEBUG Consider installing rusty-rlp to improve pyrlp performance with a rust basedbackend. Not currently functional for Python 3.11
acapy_agent.utils.classloader DEBUG Successfully loaded class MultiIndyVDRLedgerManager from module acapy_agent.ledger.multiple_ledger.indy_vdr_manager
acapy_agent.ledger.indy_vdr DEBUG Opening the pool ledger
acapy_agent.ledger.indy_vdr DEBUG Pool ledger config 'bcovrin-test' is consistent, skipping write
acapy_agent.resolver.did_resolver DEBUG Resolved DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>: {'@context': ['https://www.w3.org/ns/did/v1'], 'id': 'did:sov:ENDORSER2025DIDRANDOM1', 'verificationMethod': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#key-1', 'type': 'Ed25519VerificationKey2018', 'controller': 'did:sov:ENDORSER2025DIDRANDOM1', 'publicKeyBase58': 'Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY'}], 'authentication': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'assertionMethod': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'service': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#did-communication', 'type': 'did-communication', 'serviceEndpoint': 'https://fake-endpoint.example.com', 'priority': 1, 'routing_keys': [], 'recipient_keys': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19']}]}
acapy_agent.utils.classloader DEBUG Successfully loaded class ServiceSchema from module acapy_agent.protocols.out_of_band.v1_0.messages.service
acapy_agent.protocols.out_of_band.v1_0.manager DEBUG Creating connection with public did ENDORSER2025DIDRANDOM1
acapy_agent.utils.classloader DEBUG Successfully loaded class MaybeStoredConnRecordSchema from module acapy_agent.connections.models.conn_record
acapy_agent.utils.classloader DEBUG Successfully loaded class MaybeStoredConnRecordSchema from module acapy_agent.connections.models.conn_record
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::record::connections::invitation, payload={'state': 'invitation', 'created_at': '2025-04-17T17:37:48.643563Z', 'updated_at': '2025-04-17T17:37:48.643563Z', 'connection_id': 'b36ff223-3348-4967-a0ac-7d5c8f7b75ac', 'their_label': 'bcovrin-test-endorser', 'their_role': 'inviter', 'connection_protocol': 'didexchange/1.1', 'rfc23_state': 'invitation-received', 'invitation_msg_id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'accept': 'auto', 'invitation_mode': 'once', 'alias': 'endorser-agent', 'their_public_did': 'ENDORSER2025DIDRANDOM1'}>
acapy_agent.utils.classloader DEBUG Successfully loaded class InvitationMessageSchema from module acapy_agent.protocols.out_of_band.v1_0.messages.invitation
acapy_agent.connections.base_manager DEBUG Resolving connection targets for DID ENDORSER2025DIDRANDOM1
acapy_agent.resolver.default.legacy_peer DEBUG Checking if resolver supports DID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.default.legacy_peer DEBUG DID is valid IndyDID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.did_resolver DEBUG Valid resolvers for DID did:sov:ENDORSER2025DIDRANDOM1: [<acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>]
acapy_agent.resolver.did_resolver DEBUG Resolving DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>
acapy_agent.resolver.did_resolver DEBUG Resolved DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>: {'@context': ['https://www.w3.org/ns/did/v1'], 'id': 'did:sov:ENDORSER2025DIDRANDOM1', 'verificationMethod': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#key-1', 'type': 'Ed25519VerificationKey2018', 'controller': 'did:sov:ENDORSER2025DIDRANDOM1', 'publicKeyBase58': 'Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY'}], 'authentication': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'assertionMethod': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'service': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#did-communication', 'type': 'did-communication', 'serviceEndpoint': 'https://fake-endpoint.example.com', 'priority': 1, 'routing_keys': [], 'recipient_keys': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19']}]}
acapy_agent.connections.base_manager DEBUG Resolved DID document: context=['https://www.w3.org/ns/did/v1'] id='did:sov:ENDORSER2025DIDRANDOM1' also_known_as=None controller=None verification_method=[Ed25519VerificationKey2018(id='did:sov:ENDORSER2025DIDRANDOM1#key-1', type='Ed25519VerificationKey2018', controller='did:sov:ENDORSER2025DIDRANDOM1', public_key_hex=None, public_key_base58='Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY', public_key_pem=None, public_key_multibase=None, blockchain_account_id=None, ethereum_address=None, public_key_jwk=None)] authentication=['did:sov:ENDORSER2025DIDRANDOM1#key-1'] assertion_method=['did:sov:ENDORSER2025DIDRANDOM1#key-1'] key_agreement=None capability_invocation=None capability_delegation=None service=[DIDCommV1Service(id='did:sov:ENDORSER2025DIDRANDOM1#did-communication', type='did-communication', service_endpoint='https://fake-endpoint.example.com', recipient_keys=['did:sov:ENDORSER2025DIDRANDOM1#key-1'], routing_keys=[], accept=['didcomm/aip1', 'didcomm/aip2;env=rfc19'], priority=1)]
acapy_agent.connections.base_manager DEBUG Resolved DIDComm services: [DIDCommV1Service(id='did:sov:ENDORSER2025DIDRANDOM1#did-communication', type='did-communication', service_endpoint='https://fake-endpoint.example.com', recipient_keys=['did:sov:ENDORSER2025DIDRANDOM1#key-1'], routing_keys=[], accept=['didcomm/aip1', 'didcomm/aip2;env=rfc19'], priority=1)]
acapy_agent.resolver.default.legacy_peer DEBUG Checking if resolver supports DID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.default.legacy_peer DEBUG DID is valid IndyDID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.did_resolver DEBUG Valid resolvers for DID did:sov:ENDORSER2025DIDRANDOM1: [<acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>]
acapy_agent.resolver.did_resolver DEBUG Resolving DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>
acapy_agent.resolver.did_resolver DEBUG Resolved DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>: {'@context': ['https://www.w3.org/ns/did/v1'], 'id': 'did:sov:ENDORSER2025DIDRANDOM1', 'verificationMethod': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#key-1', 'type': 'Ed25519VerificationKey2018', 'controller': 'did:sov:ENDORSER2025DIDRANDOM1', 'publicKeyBase58': 'Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY'}], 'authentication': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'assertionMethod': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'service': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#did-communication', 'type': 'did-communication', 'serviceEndpoint': 'https://fake-endpoint.example.com', 'priority': 1, 'routing_keys': [], 'recipient_keys': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19']}]}
acapy_agent.utils.classloader DEBUG Successfully loaded class MaybeStoredConnRecordSchema from module acapy_agent.connections.models.conn_record
acapy_agent.utils.classloader DEBUG Successfully loaded class MaybeStoredConnRecordSchema from module acapy_agent.connections.models.conn_record
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::record::connections::request, payload={'state': 'request', 'created_at': '2025-04-17T17:37:48.643563Z', 'updated_at': '2025-04-17T17:37:48.724109Z', 'connection_id': 'b36ff223-3348-4967-a0ac-7d5c8f7b75ac', 'my_did': 'did:peer:4zQmcrNYtZisVvz8w8kz5y6KkUarVGfCwRhSLXHhTuzDZjeN:zqnSoyMuJhPmvnGSRJyrk4V9ENoEdPsCM5bEUsbjb4eT1GHC57x9bT3AqddWsUaFSb1SB5UErayQeTtjZWpuvzHP723pQcF4i4DVsWs7v8tkLMdUeEJr1KfNBY3beYQCnbiCbcQ1JkdZrT4YJgdRj5KTShtxLQKRqDECr2tqyUiCGiwMgtLUrRDgtnt3RV2EhnA6FHhwRadAyH3axBPgmGrWkbbK9JgXooQH4RbPozt9jfhYXijP85N8ocFn8z8qFtUfimmNAqjRa5pebwz4NjgRx74aLKhFCT47qE42mMU4xNDtBVNFvowPLWxtQdte1VhkpsUDgrucQc1mdgcoxjC7hb2SS6ernF5hMSE3covJ8YGJYLaBDu8Jf6bZsZ5mMpicSxWM7EpdEFEb8mamcdDs5RvMtSXv9v7duABgXsbR5ecXjvL2yoCqoAagnSoPqDgoZGkbjLTZe75NFTpCm76FWLhRK8TaPNhpJMNQ8NF6tTDuG2JbRRreKAoGEqRFv7bofHFovy9MmLaZDcW29BwbXAa1M8W4z6Zm9rqDVg87AtLJfAkBUsgW5JaPe2FhsTGNjoDng5EVisXpwjPfkckLMzjavpqowWztsrmyPuRHB6ZNgkV9paN3XQv6kVCg', 'their_label': 'bcovrin-test-endorser', 'their_role': 'inviter', 'connection_protocol': 'didexchange/1.1', 'rfc23_state': 'request-sent', 'invitation_key': 'Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY', 'invitation_msg_id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'request_id': '629baa96-b99a-4b63-a560-129e1d11b0ee', 'accept': 'auto', 'invitation_mode': 'once', 'alias': 'endorser-agent', 'their_public_did': 'ENDORSER2025DIDRANDOM1'}>
acapy_agent.protocols.coordinate_mediation.v1_0.route_manager DEBUG Routing connection as invitee
acapy_agent.utils.classloader DEBUG Successfully loaded class DIDXRequestSchema from module acapy_agent.protocols.didexchange.v1_0.messages.request
acapy_agent.utils.classloader DEBUG Successfully loaded class ThreadDecoratorSchema from module acapy_agent.messaging.decorators.thread_decorator
acapy_agent.utils.classloader DEBUG Successfully loaded class InvitationMessageSchema from module acapy_agent.protocols.out_of_band.v1_0.messages.invitation
acapy_agent.resolver.default.legacy_peer DEBUG Checking if resolver supports DID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.default.legacy_peer DEBUG DID is valid IndyDID did:sov:ENDORSER2025DIDRANDOM1
acapy_agent.resolver.did_resolver DEBUG Valid resolvers for DID did:sov:ENDORSER2025DIDRANDOM1: [<acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>]
acapy_agent.resolver.did_resolver DEBUG Resolving DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>
acapy_agent.resolver.did_resolver DEBUG Resolved DID did:sov:ENDORSER2025DIDRANDOM1 with <acapy_agent.resolver.default.indy.IndyDIDResolver object at 0x7805976e3ad0>: {'@context': ['https://www.w3.org/ns/did/v1'], 'id': 'did:sov:ENDORSER2025DIDRANDOM1', 'verificationMethod': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#key-1', 'type': 'Ed25519VerificationKey2018', 'controller': 'did:sov:ENDORSER2025DIDRANDOM1', 'publicKeyBase58': 'Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY'}], 'authentication': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'assertionMethod': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'service': [{'id': 'did:sov:ENDORSER2025DIDRANDOM1#did-communication', 'type': 'did-communication', 'serviceEndpoint': 'https://fake-endpoint.example.com', 'priority': 1, 'routing_keys': [], 'recipient_keys': ['did:sov:ENDORSER2025DIDRANDOM1#key-1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19']}]}
acapy_agent.connections.base_manager DEBUG Not caching connection targets for connection in state (request)
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::outbound-message::queued_for_delivery, payload=<OutboundMessage(connection_id='b36ff223-3348-4967-a0ac-7d5c8f7b75ac', enc_payload=None, _endpoint=None, payload='{"@type": "https://didcomm.org/didexchange/1.1/request", "@id": "629baa96-b99a-4b63-a560-129e1d11b0ee", "~thread": {"thid": "629baa96-b99a-4b63-a560-129e1d11b0ee", "pthid": "15f78885-d982-4dee-8601-9cb0f8ebb8f2"}, "label": "Traction Agent", "did": "did:peer:4zQmcrNYtZisVvz8w8kz5y6KkUarVGfCwRhSLXHhTuzDZjeN:zqnSoyMuJhPmvnGSRJyrk4V9ENoEdPsCM5bEUsbjb4eT1GHC57x9bT3AqddWsUaFSb1SB5UErayQeTtjZWpuvzHP723pQcF4i4DVsWs7v8tkLMdUeEJr1KfNBY3beYQCnbiCbcQ1JkdZrT4YJgdRj5KTShtxLQKRqDECr2tqyUiCGiwMgtLUrRDgtnt3RV2EhnA6FHhwRadAyH3axBPgmGrWkbbK9JgXooQH4RbPozt9jfhYXijP85N8ocFn8z8qFtUfimmNAqjRa5pebwz4NjgRx74aLKhFCT47qE42mMU4xNDtBVNFvowPLWxtQdte1VhkpsUDgrucQc1mdgcoxjC7hb2SS6ernF5hMSE3covJ8YGJYLaBDu8Jf6bZsZ5mMpicSxWM7EpdEFEb8mamcdDs5RvMtSXv9v7duABgXsbR5ecXjvL2yoCqoAagnSoPqDgoZGkbjLTZe75NFTpCm76FWLhRK8TaPNhpJMNQ8NF6tTDuG2JbRRreKAoGEqRFv7bofHFovy9MmLaZDcW29BwbXAa1M8W4z6Zm9rqDVg87AtLJfAkBUsgW5JaPe2FhsTGNjoDng5EVisXpwjPfkckLMzjavpqowWztsrmyPuRHB6ZNgkV9paN3XQv6kVCg"}', reply_session_id=None, reply_thread_id='629baa96-b99a-4b63-a560-129e1d11b0ee', reply_to_verkey=None, reply_from_verkey=None, target=None, target_list=[<ConnectionTarget(did=None, endpoint='https://fake-endpoint.example.com', label='bcovrin-test-endorser', recipient_keys=['Ahcx2kTf7hvm2cnV6xeR3MTEnGpVxGuXRqJLujzXcGGY'], routing_keys=[], sender_key='3X4J5iZHC3QSFVkGxNnGu6hCj7b6oy5TWMyr5m9j9nM8')>], to_session_only=False)>>
acapy_agent.utils.classloader DEBUG Successfully loaded class MaybeStoredConnRecordSchema from module acapy_agent.connections.models.conn_record
acapy_agent.transport.outbound.http DEBUG Posting to https://fake-endpoint.example.com; Data: b'{"protected": "eyJlbmMiOiAieGNoYWNoYTIwcG9seTEzMDVfaWV0ZiIsICJ0eXAiOiAiSldNLzEuMCIsICJhbGciOiAiQXV0aGNyeXB0IiwgInJlY2lwaWVudHMiOiBbeyJlbmNyeXB0ZWRfa2V5IjogInRSQlJTLUJhSldiZ0lQX3RFWmExNkx6SXNZQWFRaG5KbUVSaXV2YjFqcV9xOFFtVjdXYnVlNUVTQm5CeG1ZMVQiLCAiaGVhZGVyIjogeyJraWQiOiAiQWhjeDJrVGY3aHZtMmNuVjZ4ZVIzTVRFbkdwVnhHdVhScUpMdWp6WGNHR1kiLCAic2VuZGVyIjogIk1uODJ3LTVMUWJvV21tM3dpZ292MWJkM2JJXy0yRVlwTUYyVTZKcThSWDBKSGtycnRIdU9BT1pyd01TWmlLUlFrX0twQk1DRDFaZ1NGQ1dWWjdpZTlWRnUzZmJSUHhLa0Z4OFFvNm96VHg0SHZBc25mZFRDXzg1Wm5UYyIsICJpdiI6ICJaSGdwSmRqTFJEWGFjTXBpcGw1V2l0N1J0eDI5MFZSZyJ9fV19", "iv": "VhOB1iBqC5taEcGd", "ciphertext": "YsAiR2GyS-KlG9ORwX8Bcr0iXMeRC9HWQTEYQ5ElSCteHOvDO7EsO1_kwNN355lJh0Hh-JcOTp1lii91DpmmHkTki2CNIMknwoVOdQ0ZbG4qYF8l-_Q_ORJ7nsb9HeRGNxqWRSfLIw8lxpRXoYN5fCN802t65i5rXx1d0GxTfdCKF_KN8kHSzgq_EfdLAyZG4Rd68pvNCT1Q5damBf91V8rhbCn7rQRAvAnn3yrn0kmUS8Z5Kwmf9WdX8AAkBokJaIsgV-G_4iGsXOqdlL7CXfmgmIiNRkOflGyplvhs19h6HYhPq5zqlECrWbUxl9x_h2vHlIonFnnkFQPUi0E2eam5prMHyX8Yq6VZvXkGC4QQYUdpc8sPxtpv5jNxQakoRe7-GIs1i1C0eSOJDfTC3MoUC1b4ZB9ihvhvBbN2brUeG6l7boIiK9Z9K6Jjh23l9PmLBdVacvss_UwYKsoSB2tIQn5XUEffUSXCfc0Xj7Q9Nb1UW0aY_Jr1dsOKovNORky5eeILr6K3q_DJVOiDu2yKNZsCnOKY6EuX8IJh_vH9QnY_c935qBMOT5JqBPBD84JG4Vxj_8PYwf7tzAaePowrIdTL0JBIZQ8tLE_H1QBGco-5B55SAIGvaSzjcIjqQYMH8HZ2mKBcAZeBQn7DuQ9E3YbLyEuRXfR_7O4Z9qBg8hY8jutuDpTg1MLV7hz8rAchGqIVGAS_f3cKo5nAeChPVpZRlyMHnT0ZIHPZOq6JqhbOnANPqjxjl6J-uztYwPESp2IhwS5E_ccL7QrHgRzzwSauNJ_bvmJqguYYUZHPUfvIwbh_usuWfyhrrpB579ixNSlxb60bhuUIoyVFfvs_9yPsr3L2kbOkZNgcScV5fmc30Dn9SZCpebJ68e_gUP5sRtvrxxcyqiyouMWQ7wcvOP103Fjjao0LPMHb4E4SM7d8ZPTsWYCLH00pXd8VxMp9EefOYwcNoor9PZVc3QiTY9I6kxEBlGpDHkykKJiDQRm7UBbd0SgEuMlnq4Mesxio-d2ljrZ6TzvI6fWRtHlnWYpgvOeFmEBj_9J4RrIQIal-BWgMBBzvoJsNcjng2PUs_QwIcYNgCWo9v65oWLsq3xS0sUsnHjDARCTesBoFJRfvTAL1gg-CpkC0hA_xmxWbzIiPE7E_x8eYI-CKQQtWy3S-SaXG3wL67dg87_aJAwsjV3YKFqgcFD02bHfU2t3gmc0vKkpYP0pK5IH5KoWc2npKwgcyd0gOZ5rSQUPf9tJKnA", "tag": "GlhmGc3LfpivY28FydAB5g"}'; Headers: {'Content-Type': 'application/didcomm-envelope-enc'}
acapy_agent.utils.classloader DEBUG Successfully loaded class OobRecordSchema from module acapy_agent.protocols.out_of_band.v1_0.models.oob_record
acapy_agent.utils.classloader DEBUG Successfully loaded class OobRecordSchema from module acapy_agent.protocols.out_of_band.v1_0.models.oob_record
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::record::out_of_band::initial, payload={'state': 'initial', 'created_at': '2025-04-17T17:37:48.797410Z', 'updated_at': '2025-04-17T17:37:48.797410Z', 'trace': False, 'oob_id': '0c8f5e4f-a063-4082-ae75-fddeca87bdb2', 'invi_msg_id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'invitation': {'@type': 'https://didcomm.org/out-of-band/1.1/invitation', '@id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'label': 'bcovrin-test-endorser', 'imageUrl': 'https://cdn.midjourney.com/a8c7b401-dc90-4ea6-86a0-cc43304092bb/0_0.png', 'handshake_protocols': ['https://didcomm.org/didexchange/1.1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19'], 'services': ['did:sov:ENDORSER2025DIDRANDOM1']}, 'connection_id': 'b36ff223-3348-4967-a0ac-7d5c8f7b75ac', 'role': 'receiver', 'multi_use': False}>
acapy_agent.protocols.out_of_band.v1_0.manager DEBUG Performed handshake with connection b36ff223-3348-4967-a0ac-7d5c8f7b75ac
acapy_agent.utils.classloader DEBUG Successfully loaded class OobRecordSchema from module acapy_agent.protocols.out_of_band.v1_0.models.oob_record
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::record::out_of_band::done, payload={'state': 'done', 'created_at': '2025-04-17T17:37:48.797410Z', 'updated_at': '2025-04-17T17:37:48.797410Z', 'trace': False, 'oob_id': '0c8f5e4f-a063-4082-ae75-fddeca87bdb2', 'invi_msg_id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'invitation': {'@type': 'https://didcomm.org/out-of-band/1.1/invitation', '@id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'label': 'bcovrin-test-endorser', 'imageUrl': 'https://cdn.midjourney.com/a8c7b401-dc90-4ea6-86a0-cc43304092bb/0_0.png', 'handshake_protocols': ['https://didcomm.org/didexchange/1.1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19'], 'services': ['did:sov:ENDORSER2025DIDRANDOM1']}, 'connection_id': 'b36ff223-3348-4967-a0ac-7d5c8f7b75ac', 'role': 'receiver', 'multi_use': False}>
acapy_agent.utils.classloader DEBUG Successfully loaded class OobRecordSchema from module acapy_agent.protocols.out_of_band.v1_0.models.oob_record
acapy_agent.core.event_bus DEBUG Notifying subscribers: <Event topic=acapy::record::out_of_band::deleted, payload={'state': 'deleted', 'created_at': '2025-04-17T17:37:48.797410Z', 'updated_at': '2025-04-17T17:37:48.797410Z', 'trace': False, 'oob_id': '0c8f5e4f-a063-4082-ae75-fddeca87bdb2', 'invi_msg_id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'invitation': {'@type': 'https://didcomm.org/out-of-band/1.1/invitation', '@id': '15f78885-d982-4dee-8601-9cb0f8ebb8f2', 'label': 'bcovrin-test-endorser', 'imageUrl': 'https://cdn.midjourney.com/a8c7b401-dc90-4ea6-86a0-cc43304092bb/0_0.png', 'handshake_protocols': ['https://didcomm.org/didexchange/1.1'], 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19'], 'services': ['did:sov:ENDORSER2025DIDRANDOM1']}, 'connection_id': 'b36ff223-3348-4967-a0ac-7d5c8f7b75ac', 'role': 'receiver', 'multi_use': False}>
acapy_agent.utils.endorsement_setup INFO Successfully connected to endorser from invitation, and setup connection metadata.
acapy_agent.core.conductor DEBUG Startup notification sent.
acapy_agent.core.conductor INFO Listening...
acapy_agent.ledger.indy_vdr DEBUG Closing pool ledger after timeout
acapy_agent.ledger.indy_vdr DEBUG Closing pool ledger after timeout
Thank you - I'm not 100% clued up on exactly how things should be, but that does help
Can you share how you are creating and configuring the connection between the issuer-tenant and the endorser? Is the endorser creating the oob invite, and the issuer tenant is accepting it? Or other way around? What does the body of the invitation-create request look like?
And do you configure any metadata on the connection after it's complete - like setting endorser/author roles or endorser info?
Edit: I may be on to a red herring, because the issue seems related to multi-ledger, but I just want to be sure
There does seem to be something amiss with the multi-ledger config. And it's being elucidated with this refactoring: #3664 Can't quite put my finger on it yet, but it'll become clearer once the code is more understandable
Hoping that we can get that refactored PR merged, with improved logging for multi-ledger case, and re-test what's going on here
I connect tenants to endorser using --endorser-invitation startup flag.
While the focus of #3664 doesn't directly address this issue, my suspicion is that the solution lies somewhere in one of the files edited - acapy_agent/askar/profile.py and / or acapy_agent/askar/profile_anon.py. I am not sure if something changed after 1.2.4 that affects how ledger instances are scoped or provided relative to the specific profile instance during dependency injection.
Yeah, the PR is just to try make it a bit more clear what's going on.
I was also looking at those 2 files, and thought it's to do with how ledger.ledger_config_list is used, and how ledger.genesis_transactions are used in many other instances - which isn't set up for multi-ledger config. Things just seemed odd to me, but I don't think it's the problem now.
There are unfortunately quite a lot of changes between 1.2.4 and the 1.3.0rc's, because 1.2.x has just been patched with cherry-picks. So the release date of Mar 13 for 1.2.4 is not reflective of the main branch at Mar 13.
As for acapy_agent/askar/profile_anon.py - this was the last change that touched that file: https://github.com/openwallet-foundation/acapy/pull/3470 @dbluhm
And it does modify the profile used in the request ... so, might be onto something there.
One way to possibly test if that's the breaking change, is to reset on the commit right before that (f30c77a38ba532752d71db9b15cc3707bda18f3b) and then reset to the one merging that PR (ef1ab196f9d708d777c5820a10170fa0b4093c46). See if the problem isn't there before, and pops up after. Only if you're up for testing on Good Friday :-)