iroha Data integrity in `World`

Last updated: May 26, 2025

Emulate RDBMS data integrity.

Motivation

We have relied on ad-hoc solutions to remove orphaned entries when parents are deleted. These are error-prone and have introduced bugs such as #5244.
Certain deletions should be restricted. For example, domain owners must not be deleted while they still own domains. Today they can self-delete, leaving "null pointers" referring to them.

Therefore, we need a systematic approach akin to foreign-key constraints in RDBMS.

Restrictions

No entry may be implicitly deleted (for example, via a custom Drop trait).
Every deletion must be recorded as part of a transaction.

Solution

Model each field of World as either a lookup table Map<ID_a, VALUE_a> or a junction table Map<(Ref<ID_a>, Ref<ID_b>), _>.
Implement an RDBMS-style RESTRICT rule using reference counters in lookup tables.
Use a dedicated data trigger to remove orphans in CASCADE fashion.

Previous description

Currently, domains, accounts, and assets are cascaded on unregister due to nested structure, but for shallow objects #3921 we might choose either of the following unregister policies:

autoremove orphans (ON DELETE CASCADE)

entity should have a set of references (IDs) of other entities

do not unregister as long as referenced (ON DELETE RESTRICT)

entity should have a reference counter that indicates how many other entities are referencing it

May 30 '24 14:05 s8sato

domains, accounts, and assets would keep cascading if that is implied in CanUnregisterDomain and CanUnregisterAccount
should an account be dominant for the triggers it has registered?
- related to #4670

Jun 03 '24 06:06 s8sato

To consider entity relationships, I drew what if the storages under the world were tables of relational database:

erDiagram
    %% parent tables
    DOMAINS {
        _ id PK
        _ logo
        _ metadata
        _ admin FK
    }
    ACCOUNTS {
        _ id PK
        _ metadata
    }
    ASSETS {
        _ id PK
        _ total_quantity
        _ is_mintable
        _ logo
        _ metadata
        _ admin FK
    }
    %% child tables
    DOMAIN_ACCOUNTS {
        _ domain_id PK, FK
        _ account_id PK, FK
    }
    DOMAIN_ASSETS {
        _ domain_id PK, FK
        _ asset_id PK, FK
    }
    ASSET_ACCOUNTS_NUMERIC {
        _ asset_id PK, FK
        _ account_id PK, FK
        Numeric balance
    }
    ASSET_ACCOUNTS_STORE {
        _ asset_id PK, FK
        _ account_id PK, FK
        Metadata datastore
    }
    %% relationships
    DOMAINS ||--o{ DOMAIN_ACCOUNTS : has
    DOMAINS ||--o{ DOMAIN_ASSETS : has
    ACCOUNTS ||--o{ DOMAINS : administrates
    ACCOUNTS ||--o{ ASSETS : administrates
    ACCOUNTS ||--o{ DOMAIN_ACCOUNTS : has
    ACCOUNTS ||--o{ ASSET_ACCOUNTS_NUMERIC : has
    ACCOUNTS ||--o{ ASSET_ACCOUNTS_STORE : has
    ASSETS ||--o{ DOMAIN_ASSETS : serves
    ASSETS ||--o{ ASSET_ACCOUNTS_NUMERIC : serves
    ASSETS ||--o{ ASSET_ACCOUNTS_STORE : serves

erDiagram
    %% parent tables
    ACCOUNTS {
        _ id PK
        _ metadata
    }
    PERMISSIONS {
        _ id PK
    }
    ROLES {
        _ id PK
    }
    %% child tables
    ACCOUNT_PERMISSIONS {
        _ account_id PK, FK
        _ permission_id PK, FK
    }
    ACCOUNT_ROLES {
        _ account_id PK, FK
        _ role_id PK, FK
    }
    ROLE_PERMISSIONS {
        _ role_id PK, FK
        _ permission_id PK, FK
    }
    %% relationships
    ACCOUNTS ||--o{ ACCOUNT_PERMISSIONS : has
    ACCOUNTS ||--o{ ACCOUNT_ROLES : has
    PERMISSIONS ||--o{ ACCOUNT_PERMISSIONS : serves
    PERMISSIONS ||--o{ ROLE_PERMISSIONS : serves
    ROLES ||--o{ ROLE_PERMISSIONS : has
    ROLES ||--o{ ACCOUNT_ROLES : serves

where:

if table P has table C, entity c in C that references p is also deleted when entity p in P is deleted (CASCADE)
if table P serves table C, entity p in P cannot be deleted as long as any entity c in C references p (RESTRICT)
P serves C when P administrates C

Jul 02 '24 05:07 s8sato

Interesting, when you visualized it this way i noticed that either removal of account or domain can have cascade a lot.

Like:

removal of domain owner would delete it's domain
which in turn delete every account in this domain
if any account in this domain was domain owner it trigger domain removal
...
Until domain is empty or every account in the domain is not owner of some other domain

Jul 02 '24 07:07 Erigara

As long as ACCOUNTS serves DOMAINS, as suggested, the removal of the domain owner would be restricted. But yeah, this is exactly the kind of discussion I wanted to have

Jul 02 '24 07:07 s8sato

updates:

Fixed DOMAINS:ACCOUNTS from N:N to 1:N
Fixed DOMAINS:ASSETS from N:N to 1:N
In response to https://github.com/hyperledger/iroha/issues/4087#issuecomment-2210348956, assuming that NUMERIC/STORE assets correspond to FT/NFT respectively, they have been separated as totally different entities:

erDiagram
    %% parent tables
    DOMAINS {
        _ id PK
        _ logo
        _ metadata
        _ admin FK
    }
    ACCOUNTS {
        _ id PK
        _ metadata
        _ domain_id FK
    }
    FASSETS {
        %% fungible assets
        _ id PK
        Numeric total_quantity
        _ is_mintable
        _ logo
        _ metadata
        _ admin FK
        _ domain_id FK
    }
    NFASSETS {
        %% non-fungible assets
        _ id PK
        Metadata datastore
        _ owner FK
        _ domain_id FK
    }
    %% child tables
    FASSET_ACCOUNTS {
        _ asset_id PK, FK
        _ account_id PK, FK
        Numeric balance
    }
    %% relationships
    DOMAINS ||--o{ ACCOUNTS : has
    DOMAINS ||--o{ FASSETS : has
    DOMAINS ||--o{ NFASSETS : has
    ACCOUNTS ||--o{ DOMAINS : administrates
    ACCOUNTS ||--o{ FASSETS : administrates
    ACCOUNTS ||--o{ FASSET_ACCOUNTS : has
    ACCOUNTS ||--o{ NFASSETS : has
    FASSETS ||--o{ FASSET_ACCOUNTS : serves

where:

FASSET is an abbreviation for FUNGIBLE ASSET
NFASSET is an abbreviation for NON FUNGIBLE ASSET
some additional views would be desired for reference e.g. DOMAIN_ACCOUNTS, DOMAIN_ASSETS

Jul 05 '24 09:07 s8sato

Improve World to be more RDB-like:

Account balances
Metadata
Ownerships
Multisig entries: #5229
Executables: #5147
Permissions: #5359

Mar 16 '25 19:03 s8sato

Emulate RDBMS data integrity.

Motivation

We have relied on ad-hoc solutions to remove orphaned entries when parents are deleted. These are error-prone and have introduced bugs such as #5244.
Certain deletions should be restricted. For example, domain owners must not be deleted while they still own domains. Today they can self-delete, leaving "null pointers" referring to them.

Therefore, we need a systematic approach akin to foreign-key constraints in RDBMS.

Restrictions

No entry may be implicitly deleted (for example, via a custom Drop trait).
Every deletion must be recorded as part of a transaction.

Solution

Model each field of World as either a lookup table Map<ID_a, VALUE_a> or a junction table Map<(Ref<ID_a>, Ref<ID_b>), _>.
Implement an RDBMS-style RESTRICT rule using reference counters in lookup tables.
Use a dedicated data trigger to remove orphans in CASCADE fashion.

May 21 '25 02:05 s8sato

This issue handles only unregister operations. For revoke, see #5360.

Reference

referential constraint definition, ISO/IEC 9075-2, pp.426-439

Possible Design

Implement the following aspects of referential constraints:

match type: FULL
ON DELETE
referential action: CASCADE | RESTRICT

Expected Behavior 1

Define the referential constraint on (asset, account) in ASSET_ACCOUNTS as follows:
- ON DELETE CASCADE when unregistering an account in ACCOUNTS
- ON DELETE RESTRICT when unregistering an asset in ASSETS
When unregistering account alice, all entries (*, alice) in ASSET_ACCOUNTS are removed.
When unregistering asset (definition) rose, the transaction rolls back if any entry (rose, *) exists in ASSET_ACCOUNTS.

Expected Behavior 2

Define the referential constraints as follows:
- domain in DOMAINS is referenced by account.domain_id in ACCOUNTS with ON DELETE CASCADE
- account in ACCOUNTS is referenced by domain.admin in DOMAINS with ON DELETE RESTRICT
When unregistering domain wonderland, all entries in ACCOUNTS where account.domain_id == wonderland are removed.
When unregistering account alice, the transaction rolls back if any entry in DOMAINS exists where domain.admin == alice.
Apply the CASCADE/RESTRICT rules recursively across all related tables. If a conflict arises, RESTRICT takes precedence and the transaction rolls back.

Jun 24 '25 12:06 s8sato