feat: keysets v2

[cloudsupper]

Keyset ID Version 2 Implementation

[!WARNING] This PR bumps Nutshell's version to 0.19.0

Overview

This PR implements the Keyset ID Version 2 specification, which changes how keyset IDs are derived and introduces support for short keyset IDs. This change improves determinism, security, and functionality of keysets while maintaining backward compatibility.

Key Changes

1. New Keyset ID Derivation

• Version-based ID derivation:

  • Version < 0.15.0: Base64 IDs (legacy format, 12 chars)
  • Version 0.15.0-0.18.1: V1 IDs (prefix 00, 16 chars)
  • Version ≥ 0.19.0: V2 IDs (prefix 01, 66 chars)

• V2 Keyset ID calculation:

  • Prefixes 01 to indicate V2 format
  • Includes unit information in the hash (unit:{unit_name})
  • Optionally includes expiration timestamp (final_expiry:{timestamp})
  • Uses full SHA256 hash (32 bytes) instead of truncated hash
  • Properly sorted keys for consistent results

2. Short Keyset ID Support

• Implemented short V2 keyset IDs (16 chars) for token efficiency
• Added ID expansion functionality to map short→full IDs
• Updated token redemption code to handle V2 short IDs
• Added comprehensive detection and validation of different ID formats

3. Keyset Rotation Improvements

• Enhanced rotate_next_keyset to support the final_expiry parameter
• Ensures V1 keysets rotate properly into V2 keysets based on version
• Maintains backward compatibility with existing keysets

4. DB Fetching Rearrangement

• Improved keyset fetching from DB to use derivation path instead of ID
• Added proper support for final_expiry field in keyset database tables
• Updated DB schema to store and retrieve the new field
• Fixed CRUD operations to handle V2 keyset format

5. Version-based Secret Derivation

• Implemented keyset version-based secret derivation mechanism (NUT-13)
• Different derivation methods are used based on keyset version:
  • Base64 keysets (pre-0.15.0) and V1 keysets (00): BIP32 derivation path
  • V2 keysets (01): HMAC-SHA256 with specific domain separation
• HMAC-SHA256 derivation for V2 keysets:
  • Uses formula: message = "Cashu_KDF_HMAC_SHA256" || keyset_id || counter
  • Secret = HMAC-SHA256(seed, message || 0x00)
  • Blinding factor = HMAC-SHA256(seed, message || 0x01)
• Ensures deterministic secret generation while improving security

Technical Details

Keyset ID Derivation Changes

The V2 keyset ID derivation now includes:

1. Sorting public keys by amount in ascending order
2. Concatenating all public key bytes without separators
3. Adding unit:{unit_name} to the byte array
4. Optionally adding final_expiry:{timestamp} if provided
5. Computing SHA256 hash of the entire byte array
6. Prefixing with 01 to indicate V2 format

This creates a more robust, deterministic ID that properly includes all relevant keyset information, unlike previous versions that only used public keys.

Secret Derivation Implementation

The wallet now detects keyset version and uses the appropriate secret derivation method:

1. For V2 keysets (version 01):

   # Derive message components
   keyset_id_bytes = bytes.fromhex(keyset_id)
   counter_bytes = counter.to_bytes(8, byteorder="big", signed=False)
   base = b"Cashu_KDF_HMAC_SHA256" + keyset_id_bytes + counter_bytes
   
   # Derive secret and blinding factor
   secret = hmac.new(seed, base + b"\x00", hashlib.sha256).digest()
   r = hmac.new(seed, base + b"\x01", hashlib.sha256).digest()
   

2. For V1 keysets (version 00) and Base64 keysets:

   # Convert keyset ID to integer
   keyest_id_int = int.from_bytes(bytes.fromhex(keyset_id), "big") % (2**31 - 1)
   
   # Generate BIP32 derivation paths
   token_derivation_path = f"m/129372'/0'/{keyest_id_int}'/{counter}'"
   secret_derivation_path = f"{token_derivation_path}/0"
   r_derivation_path = f"{token_derivation_path}/1"
   
   # Derive keys using BIP32
   secret = bip32.get_privkey_from_path(secret_derivation_path)
   r = bip32.get_privkey_from_path(r_derivation_path)
   

The secret derivation mechanism is critical for wallet restoration and token recovery. This change ensures compatibility with all keyset formats while introducing a more robust derivation method for V2 keysets.

Rotate-Next-Keyset Behavior

When rotate_next_keyset is called:

1. It finds the highest counter keyset for the specified unit
2. Creates a new derivation path, increasing the counter by one
3. Generates a new keyset with the latest ID v2 format
4. The new final_expiry parameter is passed through to the new keyset
5. The old keyset is deactivated

Keyset Fetching Rearrangement

Previously, keysets were fetched from DB by their ID, which could cause issues when trying to load a keyset whose ID calculation might have changed. Now:

1. Keysets are primarily fetched by derivation path in the activate_keyset method
2. This allows proper loading of keysets regardless of ID format changes
3. New keysets are created only when no matching derivation path is found

Backward Compatibility

• All existing keysets continue to work correctly
• Older version tokens can be redeemed without issues
• Automatic detection of different keyset ID formats
• V2 short IDs can be expanded to full IDs for operations

Testing

• Comprehensive test suite added for all new functionality
• Test vectors for all keyset ID formats and secret derivation
• Tests for rotation behavior between V1 and V2 keysets
• Validation of proper short ID expansion
• Verified secret derivation matches NUT-13 specification

[codecov[bot]]

Codecov Report

:x: Patch coverage is 70.65868% with 49 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 49.23%. Comparing base (4a4b7f7) to head (c929f1b). :warning: Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
cashu/mint/management_rpc/protos/management_pb2.py	0.00%	11 Missing :warning:
cashu/wallet/wallet_deprecated.py	0.00%	9 Missing :warning:
cashu/core/base.py	46.66%	8 Missing :warning:
cashu/wallet/helpers.py	33.33%	6 Missing :warning:
cashu/mint/management_rpc/cli/cli.py	0.00%	5 Missing :warning:
cashu/core/crypto/keys.py	90.32%	3 Missing :warning:
cashu/wallet/keyset_manager.py	90.00%	3 Missing :warning:
cashu/mint/management_rpc/management_rpc.py	0.00%	2 Missing :warning:
cashu/wallet/transactions.py	83.33%	2 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #798      +/-   ##
==========================================
+ Coverage   47.84%   49.23%   +1.39%     
==========================================
  Files          90       92       +2     
  Lines       10592    11026     +434     
==========================================
+ Hits         5068     5429     +361     
- Misses       5524     5597      +73     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.