BIP93: Generalize codex32 format for any hrp and fix typos

[BenWestgate]

Summary of Changes: Describe codex32 format for arbitrary human-readable parts not just "ms", specify master seed encoding standard, add new test vectors and enhance readability. This makes the document more like BIP-0173: proposing an encoding "codex32", then defining a standard for something using it.

See discussion on https://github.com/bitcoin/bips/pull/2023#issuecomment-3538570493.

Spec:

• fixed the threshold mistake in the abstract
• replaced "master seed" with "secret", prior to the "Master seed format" section and made descriptions hrp general
• updated the checksum reference code to produce valid checksums for any hrp
• change t to k to match the test vectors and book
• defined "ms" codex32 secrets:
  • using terms "secret seed" (as the book does) and "codex32-encoded master seed" to refer to "ms" codex32 secrets
  • recommended using first 4 characters of the bech32-encoded fingerprint as the identifier
  • recommended the padding bits be set with a CRC code for extra error detection. Provided reference code for this checksum.

Test Vectors:

• Fixed the cornucopia of naming conventions in the Test vectors
  • used mostly "secret seed", "codex32 secret", and "codex32-encoded X".
• Fixed test vector 5 which did not actually append a long checksum to "random" data as the text said it would.
• Added vector 6 encoding a "cl" prefix codex32-encoded HSM secret, then relabels the identifier (producing a new checksum and codex32-encoded HSM secret)
• Added vector 7 which parses a "cl" prefix codex32 secret and decodes the HSM secret
• Clarified why invalid prefix test vectors were bad (their checksum is for "ms" but their prefix is not "ms")
• We might want to add one that uses "cl" with the old "ms" checksum code as that will now fail with the updated ms32_verify_checksum function

[BenWestgate]

Fair enough, double counting it is. That's easy to remember and implement.

Packing the 7-bit values together wouldn't have helped much anyway because single ASCII errors would affect multiple data symbols especially when isolated.

I'll be adding your length of the string is 93 - length of the HRP.

And we're keeping the special rule for 96 where the HRP MUST equal "MS"?

Sent with Proton Mail secure email.

On Wednesday, November 26th, 2025 at 6:41 PM, roconnor @.***> wrote:

@roconnor commented on this pull request.

---

In bip-0093.mediawiki:

 polymod = ms32_polymod(values + [0] * 13) ^ MS32_CONST

 return [(polymod >> 5 * (12 - i)) & 31 for i in range(13)]

+This implements a [https://en.wikipedia.org/wiki/BCH_code BCH code] that +guarantees detection of '''any error affecting at most 8 characters''' +and has less than a 3 in 10¹⁹ chance of failing to detect more +errors. The human-readable part is processed by first +feeding the higher bits of each character's US-ASCII value into the +checksum calculation followed by a zero and then the lower bits of each'''Why are the high bits of the human-readable part processed first?''' +This results in the actually checksummed data being ''[high hrp] 0 [low hrp] [data]''. This means that under the assumption that errors to the

What do we need to restrict it to in order to allow every US-ASCII character? Keeping in mind it expands to two 5 bit values but only the upper 2 bits can change so we should have better detection ability than 10 bits per character.

The issue is that for BCH codes the data really needs to fit within their length restriction, so we cannot just count entropy. Even if we know some some bits are fixed, if the polynomial we extract has degree more that 93, everything falls apart because it can no longer distinguish errors on one side of the polynomial from errors on the other side of the polynomial. The rules that the the string length plus counting the hrp again must be less than 93 is the only thing that makes the HRP expanded polynomial concatenated with the data part fit in degree at most 93.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[roconnor]

Yes let's keep the silly exception for now for the sake of getting a agreeable PR. We should hammer out the master seed bit-size restrictions in a separate PR.

If you want to say that length 96 ms seeds are deprecated that's okay too. But I still want to argue for the merits of 160 bit master seeds.

[bosshaas13131313]

No.

On Thu, Nov 27, 2025, 2:20 AM Ben Westgate @.***> wrote:

@.**** commented on this pull request.

In bip-0093.mediawiki https://github.com/bitcoin/bips/pull/2040#discussion_r2567413699:

+def codex32_verify_checksum(hrp, data): if len(data) >= 96: # See Long codex32 Strings

•    return ms32_verify_long_checksum(data)
  

•    return codex32_verify_long_checksum(bech32_hrp_expand(hrp) + data)
  

  if len(data) <= 93:

•    return ms32_polymod(data) == MS32_CONST
  

•    return codex32_polymod(bech32_hrp_expand(hrp) + data) == CODEX32_CONST
  

  return False

Needs to become now to:

def codex32_verify_checksum(hrp, data): combined = bech32_hrp_expand(hrp) + data if len(combined) >= 96: return codex32_verify_long_checksum(combined) if len(combined) <= 93: return codex32_polymod(combined) == CODEX32_CONST return False

Missing:

• the thorny zero length "ms" rule.
• the check in codex32_decode() for the upper long codex32 length limit.

Because of this new max length rule rule we have the curious situation where valid "long codex32" strings can actually be shorter overall (and in data part characters) than regular codex32.

May want to rename that format any thoughts?

Ex: "long" codex32 format: 10 hrp characters + 1 + 6 header characters + 54 payload characters + 15 checksum characters = 86 codex32 format: "ms" hrp characters + 1 + 6 header characters + 74 payload characters + 13 checksum characters = 96

— Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/2040#pullrequestreview-3513818681, or unsubscribe https://github.com/notifications/unsubscribe-auth/BNFR33IQPDGIR4ICII5KOVT362Q2BAVCNFSM6AAAAACM4BCWD6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTKMJTHAYTQNRYGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[scgbckbone]

How I generate non-secret shares:

1. From current loaded secret (whether it is mnemonic, xprv, or codex32)
   • codex32: secret = master_seed (secret share with hrp MS)
   • others: secret = chaincode + privkey (64bytes) (secret share with hrp CC)
2. BIP-85 derive from above secret --> master secret for share 'a'
3. interpolate secret share with share 'a' while changing only index (c,d,d,e,f,g,h...) to generate new shares

[BenWestgate]

* HRP: `cc`

"bc" and "tb" for Bech32 addresses were an upgrade in human-readable prefix from the base58 encoding.

I consider it a regression if you use less characters to encode a human-readable prefix than the base58 extended key format did. "xpriv" is an option here.

* encodes chaincode + private key of BIP-32 master extended key (64 bytes)

Does your format need a 65th byte for the public key that is zero when encoding private keys?

There are many advantages to the strings needing disambiguation having the same byte length.

I only have one HRP, I do not differentiate between testnet/mainnet, ... DO you consider the lack of testnet/mainnet separation an issue?

Yes, this is a huge regression from the current bip32 extended key format we want to upgrade. Mostly that I can't tell by looking at the descriptor if it's for real funds or not.

Your addition of HRP into checksum definitely broke my tests wrt checksum for cc hrp secrets (not an issue, I haven't released yet - but I'm planning to in few weeks)

HRP was always in the checksum, it just was pre-computed for "ms" so the checksums for other HRP were wrong. I noticed when I tried to validate the CLN HSM secret examples in my python-codex32 package.

My implementation is not ECW.

@roconnor has a PR in codex32 that does ECC you could test.

I even provide generate support for secret share S. I only allow to generate 128 & 256 bit MS secrets (but allow to import also 512 bit).

I have a codex32 PR to update wallets.md guidance for generation, you may see something useful, especially in the HWW case.

In short:

1. TRNG 256 entropy bits

2. r = sha256(sha265(entropy))

3. x = r[:byte_len]

4. x is new master secret, and default ID is 20 MSB from master XFP (but user can change if he wishes to)

You can and probably should use the entropy bits directly. If they lack entropy, sha256d is an illusion of security.

What are the chances of this patch-set to be accepted? Is this spec stable enough to start releasing it ?

It will need wider community review than us. But there's comments by P. Wuille as far back as 2020 stating a 4 error correcting bech32 encoding of extended keys is needed. So high acceptance changes once it's correct and shiney.

This spec PR will not change anything that affects your encoding of ~78 bytes or whatever an extended key has.

We're mostly debating behavior at the limit between short and long checksums. Yours unambiguously use long codex32.

[BenWestgate]

How I generate non-secret shares:

2. BIP-85 derive from above secret --> master secret for share 'a'

3. interpolate secret share with share 'a' while changing only index (c,d,d,e,f,g,h...) to generate new shares

It is unsafe to child derive shares from the secret they recover. They should be independently random.

When part of the secret is compromised and an attacker tries to brute force the rest: the dependent relation between the secret and share A allows an attacker with k-1 shares or share A to check his guesses against this. This is far faster than checking an address.

[scgbckbone]

HRP was always in the checksum, it just was pre-computed for "ms" so the checksums for other HRP were wrong. I noticed when I tried to validate the CLN HSM secret examples in my python-codex32 package.

I see now...

It is unsafe to child derive shares from the secret they recover. They should be independently random.

I do not want to use randomness here, as I want to split existing secret, and I require the "split" to be deterministic, so that if user is splitting the exact same secret, uses same hrp, same threshold, same id, and same number of shares - application always produces the exact same shares. I could add an option to to choose, if random, or deterministic split, but deterministic is a hard requirement.

...also it is 5 hardened derivation steps plus hmac_sha512

When part of the secret is compromised and an attacker tries to brute force the rest: the dependent relation between the secret and share A allows an attacker with k-1 shares or share A to check his guesses against this. This is far faster than checking an address.

there are plenty other brute-force options if attacker has part of secret, I do not consider this scenario of yours to be something I should optimize for

Yes, this is a huge regression from the current bip32 extended key format we want to upgrade. Mostly that I can't tell by looking at the descriptor if it's for real funds or not.

I do not encode extended key (or full extended key), I only encode chaincode + privkey, without any other data as I just want to be able to restore naked xpriv from it, without any more meta extended keys carry. As I use it for both mnemonics and extended keys.

That is why I dismissed the idea of doing testnet/mainnet differentiation as I consider my 64bytes to be the "secret"

[BenWestgate]

It is unsafe to child derive shares from the secret they recover. They should be independently random.

I do not want to use randomness here, as I want to split existing secret, and I require the "split" to be deterministic, so that if user is splitting the exact same secret, uses same hrp, same threshold, same id, and same number of shares - application always produces the exact same shares. I could add an option to to choose, if random, or deterministic split, but deterministic is a hard requirement.

The best you could do here if you insist, is perform a KDF on the secret data to harden it before deriving child shares from that derived key. But it still reduces security from information theoretic to computational.

...also it is 5 hardened derivation steps plus hmac_sha512

Still significantly faster than address checking. The EC mult is the bottleneck for address checking is what Andrew told me.

When part of the secret is compromised and an attacker tries to brute force the rest: the dependent relation between the secret and share A allows an attacker with k-1 shares or share A to check his guesses against this. This is far faster than checking an address.

there are plenty other brute-force options if attacker has part of secret, I do not consider this scenario of yours to be something I should optimize for

My point is your standard should be harder to exploit than all other options or we lose security for nothing. Simply deriving child shares from an argon2id or scrypt derived key is probably enough protection.

That is why I dismissed the idea of doing testnet/mainnet differentiation as I consider my 64bytes to be the "secret"

It seems better to encode the recovery words and wordlist with a bip39_12w or bip39_24w human-readable part encoding standard than encode the resulting private key and chaincode bytes. A full bip32 codex32 encoding standard would be more useful than a neutered master xprv only edition.

[BenWestgate]

This table shows the undetectable errors, each row has 2-3 characters which cannot be distinguished since they differ only in the upper bits.

I found an 83 character Bech32 HRP with 3 substitutions that validates. In theory, some long HRP won't detect even 1-2 errors affecting high bits. We inherit this problem if we copy Bech32 max length rules.

The worse case is: a secret is transcribed wrong or damaged, user or heirs, application is forgotten, it validates or corrects to a different application and then is transmitted.

This is worse than a wrong HRP address validating.

We should guarantee to correct 2 HRP errors by covering the expanded characters. Now any wrong 2 character HRP for every seed length reveals it is "ms" secret data. For the more common errors affecting only the low (or high) bits two errors from the data can also be corrected.

So the correct 4 errors guarantee holds under the assumption the HRP errors affect only low (or high) bits. Same assumption as Bech32's detection guarantee, and it's a detection only standard. We store secrets so we need correction guarantees and this is how we get them.

[roconnor]

I'm this close to throwing in the towel. BIP-93's design was never intended to be generalized to arbitrary HRP, and it shows. If people want to reuse our polynomial for their own schemes, then more power to them. They can make their own BIP.

[apoelstra]

Sorry for being late to the party. I have read through this whole discussion except for the digression about deterministic share derivation and except for Russell's detailed code. As I understand it there are a few issues at play:

• Ben wants the HRP to be covered by the checksum, which has multiple problems
  • if you don't know the HRP you arguably don't know the checksum so how can you correct it?
  • but conversely Ben points out that we likely want xpub/xprv HRPs which are easy to mess up and would share a checksum
  • each HRP character contributes two characters to the checksum plus an extra "separator" character so length n takes away 2n+1 from your total length, which is surprising and weird
  • (There was a long discussion about restricting the character set of HRPs. As you have observed, I've already violated this with my bip32_24w HRPs. I'm skeptical this matters. If there is anybody except me doing this, they would have needed to do a comparable amount of insane off-spec work to accomplish it and "protecting them" by extending the spec to include them should not be a priority.)
• How do we determine the threshold at which to switch to "long codex32" which is a totally different checksum
  • Ben would like there to only be a few allowable lengths of long codex32 strings, which I directionally agree with, but I also note that I have violated this (I have 264-bit strings which are converted directly from BIP39 seed words).
• Some discussion on what the allowable HRPs should be. BIP-173 allows any ASCII string up to 83 characters.
  • ...but if you look at the registered list of BIP-173 prefixes, despite there being some pretty crazy crap in there, every single prefix is less than 12 characters, and except for one using : and one using @ every single one is alphanumeric
  • Ben initially proposed restricting the set of characters to ones that all have distinct low bits so that we could "ignore the high bits". But as seen in his above table, this is impossible if we allow both numbers and letters.

In the interest of moving forward I would kinda like Ben to make a new PR with the non-HRP changes, which it seems like everyone agrees with and would reduce the size of the diff of this one.

Then my opinions on the above:

I agree with Russell that in general we should not attempt to correct the HRP. This was outside of the design space for our codex32 SSSS application and among other things we (ab)used this fact to distinguish codex32 from long codex32 on length alone and not HRP. Having said this, if Ben wants to try to error-correct HRPs all the power to him and we should take some effort to avoid undermining that goal.

So for this BIP we should say:

• Users can register their own HRPs at [link] but they are only allowed to use ASCII 96 to 126, and their length can be at most 8, say. (These are the tightest restrictions I would support, and I'd also accept any looser ones up to the "83 ascii characters free for all" of BIP 173.) This gives us { | } and ~ as well as letters. People who want a separator should be happy to use ~.
• The HRP defines the checksum and SHOULD NOT be error-corrected, unless there is a separate specification describing how to do this. xpub/xprv I think needs to have its own BIP for this. Maybe there could be a general-purpose "bip93 with HRP correction" BIP that covers questions like "what if the user has a character outside of the allowable set" or "should we preferentially try to correct _ and - to ~ or just try random things" or "should we have a fixed set of supported HRPs and just try all of these". It seems that different answers make sense in different contexts.
• I'm happy with whatever length threshold we want for switching between codex32 and long codex32. I think "93 - length of HRP" is fine, along with an exception for ms. We should specify the maximum length in the table of registered HRPs so people don't have to know the formula if they don't want to.

I think this should make everyone happy, except that it leaves HRP correction underspecified and delegated to another future BIP. (I would also be open to bringing more text into BIP 93 itself, but let's try to accomplish the above before we do that.)

[jonatack]

@BenWestgate Do you plan to update here following the merge of #2052?

[BenWestgate]

• How do we determine the threshold at which to switch to "long codex32" which is a totally different checksum

It's best for this to depend only on length. Consistent with BIP-0173.

The "ms" exception for 93 data characters we can either:

• deprecate (let it decode but future len(hrp) + len(data) > 80 characters encodings will use long codex32
• count "ms" as zero characters so nothing changes

• Some discussion on what the allowable HRPs should be.

Prefer keeping what BIP-0173 allows to avoid redefinition.

• Ben initially proposed restricting the set of characters to ones that all have distinct low bits so that we could "ignore the high bits". But as seen in his above table, this is impossible if we allow both numbers and letters.

Not quite, I propose to restrict the registry so that every hrp in it has unique low bits from every other registered hrp.

Allow every US-ASCII character but applications should not register an hrp that is only unique in the high bits as it might be mistaken for another.

I think this maintains 8 character error detection guarantees as the low bits are always covered and unique among valid hrp.

...I would like Ben to make a new PR... [to] reduce the size of the diff of this one.

Done and merged.

Then my opinions on the above:

I agree with Russell that in general we should not attempt to correct the HRP.

I agree. Error detection guarantees on it are enough to avoid disasters when honest software detects data it should not have been given.

So for this BIP we should say:

• The HRP defines the checksum and SHOULD NOT be error-corrected, unless there is a separate specification describing how to do this.

Correction should try all registered hrp if rebroadcast is not an option. Assume the fewest edits is the valid hrp. According to the edit distance formula in wallets.md

xpub/xprv I think needs to have its own BIP for this.

Agreed. Someone may have volunteered to do this.

• I'm happy with whatever length threshold we want for switching between codex32 and long codex32. I think "93 - length of HRP" is fine, along with an exception for ms.

Agree. 93-len(hrp) is simplest. Is that "ms" rule a verify exception (deprecate) or create and verify exception?

specify the maximum length in the table of registered HRPs so people don't have to know the formula if they don't want to.

Unsure why string length for a given payload size and hrp belongs in the registry.

except that it leaves HRP correction underspecified and delegated to another future BIP.

I think that's fine, correction is application specific. Or at very least rules for: public/private/secret and always/sometimes/never retransmissable data. That's up to 9 hrp types and then correction guidance may include contexts expecting combinations of multiple of these.

I just don't think it can or should all be said here. Just enough to avoid disasters such IF hrp correction is attempted the least edits should be assumed as the correct hrp. And in applications that transmit data, probably MUST.

(I would also be open to bringing more text into BIP 93 itself, but let's try to accomplish the above before we do that.)

Agreed. Safety guidance only here

[BenWestgate]

@BenWestgate Do you plan to update here following the merge of #2052?

Yea. Enough consensus has formed to do another commit incorporating it this week.

[roconnor]

If it is helpful we could start with an intermediate amendment to BIP-93 seed sizes to only allow between 128 and 256 bit seeds (specifically 16 to 32 byte seeds) for short codex 32 strings, and only allow exactly 512 bit secrets for long codex32 strings. This would eliminate the 95 and 96 character special exceptions we are worried about. I think we all agree we want to restrict the valid size values to some subset of these values anyways, and there is only a small debate on how far we ought to go.

E.g. I would also not oppose going as far as restricting seed sizes to be of 128, 160, 192, 224, and 256, which are the entropy sizes listed in BIP-39. (And also keeping 512 bit long codex32 for compatibility with BIP-39 generated master seeds).

My only hesitation is that I know some folks want to restrict this list even further, and it would be somewhat annoying to make "breaking changes" twice. I use the word "breaking changes" loosely since, in practice people are using 128, 256 and 512 bit entropy sizes.

[BenWestgate]

I found this reply in my browser cache:

if you don't know the HRP you arguably don't know the checksum so how can you correct it?

Bech32/Bech32m checksum:

• length 90 or less
• 1st value is 0-16
• The seventh-from-last character has zero padding
• implementations SHOULD NOT implement correction...

Codex32 checksum:

• length 94 or less or 97 or more
• Data starts with digit
  • If "0", 6th character is "s"
• 14/16th (short/long) from last character may have a 1 in its padding
• MAY implement correction...

Only threshold 9, 8, 2, and 0 are valid segwit data[0] and 0 has "s" ruling 31:1 in codex32 favor. 61.25% of random 7th from last characters will have a 1 in their padding. 1 in 10^9 the Bech32 checksum validates

Any attempt at suggesting HRP corrections should assume codex32 after checking it's invalid:

• Bech32/Bech32m
• Base58Check
• Hex encoded public key

Then only suggest registered HRP.

If it is helpful we could start with an intermediate amendment to BIP-93 seed sizes to only allow between 128 and 256 bit seeds (specifically 16 to 32 byte seeds) for short codex 32 strings, and only allow exactly 512 bit secrets for long codex32 strings.

It is very helpful to lose the "ms" exceptions. These rules fit perfectly in the new "Master seed format" section. 16..32 for regular and 64 for long looks great if you're OK with no insert/delete correction outside 16, 24, 32, and 64. Do you want to write it, or should I?

Wallets.md says:

MAY attempt correction by deleting and/or inserting characters, as long as the resulting string has a valid length for a codex32 string. ECWs MAY assume the correct length is the closest of 48 or 74.

We can safely amend that to "48, 61, or 74." as the correctable lengths of each do not overlap.

I think we all agree we want to restrict the valid size values to some subset of these values anyways, and there is only a small debate on how far we ought to go.

If it's ready to test again, https://github.com/BlockstreamResearch/codex32/pull/70 could give objective data how much performance or accuracy insert/delete correction loses when it checks every length or lengths other than 48, 61, or 74.

E.g. I would also not oppose going as far as restricting seed sizes to be of 128, 160, 192, 224, and 256, which are the entropy sizes listed in BIP-39. (And also keeping 512 bit long codex32 for compatibility with BIP-39 generated master seeds).

Having just two lengths in correction range is like losing 1-bit of checksum, 160 and 224 may not significantly harm accuracy.

My only hesitation is that I know some folks want to restrict this list even further, and it would be somewhat annoying to make "breaking changes" twice. I use the word "breaking changes" loosely since, in practice people are using 128, 256 and 512 bit entropy sizes.

If testing shows a significant loss of insert/delete correction accuracy would that sway your opinion? If not, only 128, 160, 192, 224, and 256 has consensus.

P.S.: I checked the characters for descriptor key origin data and if it is prepended inside the HRP of xpub/tpub, it maintains the "low bits always unique" hrp registry rule for any master fingerprint hex and derivation path symbols which is a nice omen.