Make BEGIN and END tags easier to type

[bwbug]

trafficstars

For use-cases in which the shares will be distributed in hardcopy and manually typed for decryption, typing in the current BEGIN/END tags is unnecessarily cumbersome (especially with arbitrary requirements such as having to insert exactly one space character before S4 in the BEGIN tag but having no space after the ] bracket).

My suggestion is to simplify the tags, and to also allow some variability in the allowed use of whitespace within the tags (specifically, make all whitespace optional, by stripping whitespace before further parsing of the inputs). In addition, I would suggest not using the Base64 padding character (=) within the BEGIN/END tags, so that the line breaks can be made optional.

For example, I might suggest the following format:

[s4|v0.5|S4:YsVweIrrKfDotKXSTt4hGc2PZrwBJli1x8s25XkzlBkBLPc=]

If whitespace is stripped before processing, then optional line breaks (and/or indentation) could be inserted:

[s4|v0.5|S4:
    YsVweIrrKfDotKXS
    Tt4hGc2PZrwBJli1
    x8s25XkzlBkBLPc=
]

If you feel the need to keep the BEGIN and END keywords, then I would suggest making their use optional, as well:

[BEGIN|s4|v0.5|S4:
YsVweIrrKfDotKXSTt4hGc2PZrwBJli1x8s25XkzlBkBLPc=
|END]

[qgustavor]

Thinking about better UX for manual writing/reading, an encoding scheme made with this in mind could be used, like Base58, which omits similar characters like O and 0, l and I and so on. Even when printing, using Base64 requires a font that allows distinguishing those characters. Of course, a more lenient wrapping format would be useful when dealing with pen and paper users. Alternatively, using a word list to encode (like BIP39) seems useful.

Those are all options that could be integrated without too much difficulty editing this file and can be an option, current mode being "the shares will be transmitted online" and the new mode "the shares will be transmitted in paper". As someone that only knows a bit of Go from working with other repos, I'm amazed at all those ninja comments. :eyes:

[bwbug]

Those are all options that could be integrated without too much difficulty

The biggest difficulty in implementing these types of improvements will be coding it to be backwards compatible (although making this possible is presumably the entire reason why the version number is required in the wrapping tags to begin with).

[qgustavor]

The version number is more useful for cases like changing the used cypher or algorithm, not the formatting. But, since the current format is really strict I think it makes it easier to support, as there are fewer edge cases.

There are test cases, right? Use those to avoid any kind of regression. One way to make this more easy to implement would be restricting to the main points related to using it on paper: making it case insensitive and/or changing the base to one without similar characters.

[simonfrey]

Great ideas here. Thank you! Will see if I find time in the christmas break to work on this :)

[bwbug]

Making any progress?

[simonfrey]

Alrighty, I implemented both ideas in https://github.com/simonfrey/s4/pull/37 (plus some restructuring of the code, as I figured, that my go knowledge back then was a lot worse than now ;). Should be tiny bit more readable now)

@bwbug & @qgustavor I please need your help testing this major release. It has the potential to break backwards-compatability, hence please give it a thorough testing if all old codes still work. Can be found on https://simon-frey.com/s4/easier_format/

[bwbug]

I will do some testing and let you know what I find. My first impression is that this is a huge improvement, but if I would change one thing, it would be for the encoder/encryption tool to use all lowercase in the header it produces ([s4 v0.6 s4 instead of [s4 v0.6 S4). It is good that the decryption/decoding tool accepts the lowercase version, but it is confusing that there is a single uppercase S in the header).

Another minor suggestion would be to also offer Base58 as an option (to reduce the size of the encrypted shares, and still robust when hardcopy shares are printed using a printer — not handwritten).

[bwbug]

So far, the new decoder seems to be backwards compatible with v0.5. I set up the following shares using the default "S4" scheme (6 total, with 3 required for decryption), and tested around 10% of the possible permutations of shares.

Next, will do something similar with the "AES+S4 " encryption scheme.

Shares:

====== BEGIN [s4 v0.5 || S4]======
M8w/tsI2DIqBF0rov32B4QtGW++fDH64ajoKnGKYltVqls1G1cm5A8qKXIwQFiedwvRTU+2Up7J/QigTEzMHmgje2LY6owFRReiOIJfZJWe7YioaMuiyjQ==
====== END   [s4 v0.5||S4]======

====== BEGIN [s4 v0.5 || S4]======
bU/Z3TCPopoRl+dWsYp0vgj4SQrcvRK0u1Gbph0fGmgGNj6F9T21wNeAsUbVV6d0kygPu7xtGGZCOXoK847LPQufS7FsMFPXr3neXz6EIbYqkg8/5nty2A==
====== END   [s4 v0.5||S4]======

====== BEGIN [s4 v0.5 || S4]======
FYULoAyrRKc7YWETspOb+zevR9A3FC6MNHSWHbVX8WNf0uY0JmeoVW4/n72JBD7Qs09peUzmAh15IvmpE2Zx29Tr1EVZ8RApKH10FpfrzFNq3KNNleAzQA==
====== END   [s4 v0.5||S4]======

====== BEGIN [s4 v0.5 || S4]======
952zkrX47E9xGMnsB+a8On77a9/eOJ09B0eU79h/bCHj19GdrBdMsbic2+C1dbhk4JXax2oRmY/80Wygw8+ALV7/k74q52t+7qdOoC6tA+eK49Tfx8e1ZQ==
====== END   [s4 v0.5||S4]======

====== BEGIN [s4 v0.5 || S4]======
9o28W/UaqAubCOXxlRHf/Hz6PQEe7GJEaRdce4XzymBa9jGF6kP6kbxPlKVLVeJO8PP/sRNj9sgpx5YhScXrsRXlfqedlxHHJ2FDQ6Ge1cU0z0nWedslxQ==
====== END   [s4 v0.5||S4]======

====== BEGIN [s4 v0.5 || S4]======
t2DeiNnYn4zBSfCHVPZqJyo3GbVMniugNAmhCl3KVHN0VqTFuVIlvKsKJTjyYO6td4mg/2e8xPtwpC3LHubcLYlH4KmIK06gQExmB2UhTMAGSrC1zAbohQ==
====== END   [s4 v0.5||S4]======

[bwbug]

I did a few spot checks with "s4 v0.5 || AES+S4" shares (5 total, 4 required), and they also seemed to decrypt correctly using the v0.6 decryption tool.

I noticed an unrelated issue that is present in both versions, and will report it as a separate issue.

[simonfrey]

Another minor suggestion would be to also offer Base58 as an option (to reduce the size of the encrypted shares, and still robust when hardcopy shares are printed using a printer — not handwritten).

I would opt against this. I find it not intuitive to have too many configuration options, and having one "optimize for printing", "optimize for handwriting", "optimize for printing" seems overkill to me.

I mean when you print it, the font-quality should be good enough to OCR (what every phone by now does in their camera) it afterwards, so base64 should be fine, right?

[simonfrey]

but if I would change one thing, it would be for the encoder/encryption tool to use all lowercase in the header it produces ([s4 v0.6 s4 instead of [s4 v0.6 S4). It is good that the decryption/decoding tool accepts the lowercase version, but it is confusing that there is a single uppercase S in the header).

Good idea. Adapted. Now everything in new signature is lower-case

[bwbug]

I mean when you print it, the font-quality should be good enough to OCR (what every phone by now does in their camera) it afterwards, so base64 should be fine, right?

You are right. For some reason, I assumed that the improved headers were only available for the "handwriting" mode, but I see now that this is not the case.

One thing to potentially keep on your radar: Because whitespace is now stripped before pattern matching (which is great!), if you ever create an encryption format that ends with "s4" followed by any additional character(s) (e.g., "s4a"), then you may end up with a parsing ambiguity. Basically, to avoid this, not format code can be a prefix of any other format code (unless you introduce a separator character between the header and the cipher).

And as a final suggestion: Can the opening and closing brackets ([, ]) be made optional for the decryption algorithm? If they are not present in the input string, simply insert them before proceeding with pattern matching. They should still be there when the shares are generated, but I think it could be optional whether a user actually enters them when decrypting.

[simonfrey]

Can the opening and closing brackets ([, ]) be made optional for the decryption algorithm?

Done :)

[simonfrey]

Deployed to https://simon-frey.com/s4/