Replace protobuf client implementation with wire

[ligi]

Protobuf is still quite heavy - especially method-count wise - perhaps (opinions welcome - cc @prusnak) we should replace it with wire: https://github.com/square/wire

[prusnak]

Wire seems very nice, if you think we should use it I am fine with that!

[prusnak]

It seems they are not really friends with Gradle, though: https://github.com/square/wire/pulls

[ligi]

Think this is just a resource-allocation problem - square is doing gradle usually now. And this should also not be a blocker for using it here.

[prusnak]

OK :-)

[ligi]

this seems to be a bigger problem - think I will release a version based on the protobuf implementation for now:

⋊> ~/g/3/t/protob on master ⨯ java -jar /home/ligi/Downloads/wire-compiler-2.3.0-RC1-jar-with-dependencies.jar --proto_path=. --java_out=. messages.proto
Writing com.satoshilabs.trezor.protobuf.ClearSession to .
Writing com.satoshilabs.trezor.protobuf.Ping to .
Writing com.satoshilabs.trezor.protobuf.GetFeatures to .
Writing com.satoshilabs.trezor.protobuf.ApplySettings to .
Writing com.satoshilabs.trezor.protobuf.MessageType to .
Writing com.satoshilabs.trezor.protobuf.Initialize to .
Writing com.satoshilabs.trezor.protobuf.ChangePin to .
Writing com.satoshilabs.trezor.protobuf.Features to .
Writing com.satoshilabs.trezor.protobuf.Success to .
Writing com.satoshilabs.trezor.protobuf.ButtonAck to .
Writing com.satoshilabs.trezor.protobuf.Failure to .
Writing com.satoshilabs.trezor.protobuf.PinMatrixRequest to .
Writing com.satoshilabs.trezor.protobuf.ButtonRequest to .
Writing com.satoshilabs.trezor.protobuf.PinMatrixAck to .
Writing com.satoshilabs.trezor.protobuf.Cancel to .
Writing com.satoshilabs.trezor.protobuf.PassphraseRequest to .
Writing com.satoshilabs.trezor.protobuf.PassphraseAck to .
Writing com.satoshilabs.trezor.protobuf.GetEntropy to .
Writing com.satoshilabs.trezor.protobuf.GetPublicKey to .
Writing com.satoshilabs.trezor.protobuf.PublicKey to .
Writing com.satoshilabs.trezor.protobuf.GetAddress to .
Writing com.satoshilabs.trezor.protobuf.EthereumGetAddress to .
Writing com.satoshilabs.trezor.protobuf.EthereumAddress to .
Writing com.satoshilabs.trezor.protobuf.Entropy to .
Writing com.satoshilabs.trezor.protobuf.Address to .
Writing com.satoshilabs.trezor.protobuf.WipeDevice to .
Writing com.satoshilabs.trezor.protobuf.BackupDevice to .
Writing com.satoshilabs.trezor.protobuf.EntropyRequest to .
Writing com.satoshilabs.trezor.protobuf.EntropyAck to .
Writing com.satoshilabs.trezor.protobuf.LoadDevice to .
Writing com.satoshilabs.trezor.protobuf.WordRequest to .
Writing com.satoshilabs.trezor.protobuf.ResetDevice to .
Writing com.satoshilabs.trezor.protobuf.VerifyMessage to .
Writing com.satoshilabs.trezor.protobuf.RecoveryDevice to .
Writing com.satoshilabs.trezor.protobuf.MessageSignature to .
Writing com.satoshilabs.trezor.protobuf.WordAck to .
Writing com.satoshilabs.trezor.protobuf.SignMessage to .
Writing com.satoshilabs.trezor.protobuf.EncryptMessage to .
Writing com.satoshilabs.trezor.protobuf.EncryptedMessage to .
Writing com.satoshilabs.trezor.protobuf.DecryptedMessage to .
Writing com.satoshilabs.trezor.protobuf.DecryptMessage to .
Writing com.satoshilabs.trezor.protobuf.CipheredKeyValue to .
Writing com.satoshilabs.trezor.protobuf.EstimateTxSize to .
Writing com.satoshilabs.trezor.protobuf.SignTx to .
Writing com.satoshilabs.trezor.protobuf.SimpleSignTx to .
Writing com.satoshilabs.trezor.protobuf.EthereumTxRequest to .
Writing com.satoshilabs.trezor.protobuf.TxSize to .
Writing com.satoshilabs.trezor.protobuf.EthereumSignTx to .
Writing com.satoshilabs.trezor.protobuf.TxRequest to .
Writing com.satoshilabs.trezor.protobuf.CipherKeyValue to .
Writing com.satoshilabs.trezor.protobuf.TxAck to .
Writing com.satoshilabs.trezor.protobuf.SignedIdentity to .
Writing com.satoshilabs.trezor.protobuf.SetU2FCounter to .
Writing com.satoshilabs.trezor.protobuf.FirmwareErase to .
Writing com.satoshilabs.trezor.protobuf.FirmwareRequest to .
Writing com.satoshilabs.trezor.protobuf.FirmwareUpload to .
Writing com.satoshilabs.trezor.protobuf.SelfTest to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkDecision to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkGetState to .
Writing com.satoshilabs.trezor.protobuf.GetECDHSessionKey to .
Writing com.satoshilabs.trezor.protobuf.ECDHSessionKey to .
Writing com.satoshilabs.trezor.protobuf.EthereumTxAck to .
Writing com.satoshilabs.trezor.protobuf.SignIdentity to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkStop to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkMemory to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkLog to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkState to .
Writing com.satoshilabs.trezor.protobuf.OutputScriptType to .
Writing com.satoshilabs.trezor.protobuf.InputScriptType to .
Writing com.satoshilabs.trezor.protobuf.FailureType to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkMemoryWrite to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkFlashErase to .
Writing com.satoshilabs.trezor.protobuf.RecoveryDeviceType to .
Writing com.satoshilabs.trezor.protobuf.DebugLinkMemoryRead to .
Writing com.satoshilabs.trezor.protobuf.WordRequestType to .
Writing com.satoshilabs.trezor.protobuf.ButtonRequestType to .
Writing com.satoshilabs.trezor.protobuf.HDNodePathType to .
Writing com.satoshilabs.trezor.protobuf.MultisigRedeemScriptType to .
Writing com.satoshilabs.trezor.protobuf.HDNodeType to .
Writing com.satoshilabs.trezor.protobuf.TxOutputBinType to .
Writing com.satoshilabs.trezor.protobuf.RequestType to .
Writing com.satoshilabs.trezor.protobuf.PinMatrixRequestType to .
Writing com.satoshilabs.trezor.protobuf.TransactionType to .
Writing com.satoshilabs.trezor.protobuf.TxRequestDetailsType to .
Writing com.satoshilabs.trezor.protobuf.TxOutputType to .
Writing com.satoshilabs.trezor.protobuf.TxRequestSerializedType to .
Writing com.satoshilabs.trezor.protobuf.IdentityType to .
Exception in thread "main" java.io.IOException: java.lang.NumberFormatException
	at com.squareup.wire.WireCompiler.compile(WireCompiler.java:261)
	at com.squareup.wire.WireCompiler.main(WireCompiler.java:132)
Caused by: java.util.concurrent.ExecutionException: java.lang.NumberFormatException
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at com.squareup.wire.WireCompiler.compile(WireCompiler.java:258)
	... 1 more
Caused by: java.lang.NumberFormatException
	at java.math.BigDecimal.<init>(BigDecimal.java:494)
	at java.math.BigDecimal.<init>(BigDecimal.java:383)
	at java.math.BigDecimal.<init>(BigDecimal.java:806)
	at com.squareup.wire.java.JavaGenerator.fieldInitializer(JavaGenerator.java:1504)
	at com.squareup.wire.java.JavaGenerator.defaultValue(JavaGenerator.java:1466)
	at com.squareup.wire.java.JavaGenerator.defaultField(JavaGenerator.java:1018)
	at com.squareup.wire.java.JavaGenerator.generateMessage(JavaGenerator.java:544)
	at com.squareup.wire.java.JavaGenerator.generateType(JavaGenerator.java:358)
	at com.squareup.wire.WireCompiler$JavaFileWriter.call(WireCompiler.java:284)
	at com.squareup.wire.WireCompiler$JavaFileWriter.call(WireCompiler.java:267)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

[prusnak]

I finish even earlier:

$ java -jar wire-compiler-2.3.0-RC1-jar-with-dependencies.jar -proto_path=. --java_out=. messages.proto 
Exception in thread "main" java.lang.IllegalStateException: No sources added.
	at com.squareup.wire.schema.SchemaLoader.load(SchemaLoader.java:89)
	at com.squareup.wire.WireCompiler.compile(WireCompiler.java:217)
	at com.squareup.wire.WireCompiler.main(WireCompiler.java:132)

[ligi]

You have one "-" missing - "--proto_path"

[prusnak]

I see. I found the culprit in Wire, I'll post it to the other issue.

[prusnak]

Pushed the Wire fix to https://github.com/trezor/trezor-common/commit/224bdde39f23df2c08b3506a0763215ec99ffbbe

[ligi]

Nice!

[prusnak]

Does it still make sense to try to change this?

[ligi]

Yes - I was/am just busy with other things currently. This is more a reminder ticket to not forget about this - or if someone else in the mean-time has spare time could pick this up .. I would also do that at some point - but currently have some things with higher priority on my table.

[prusnak]

No worries, I was just wondering if this still makes sense. Let's keep this open for now ...

[ychescale9]

Any update on this? Wire now supports generating Kotlin files btw https://github.com/square/wire/pull/739

[ligi]

Thanks for the ping - but sorry I did not yet find time to do this. There was always something else on the table. Hope to find some time for this in the future - but would also be happy if someone else would care for this ;-)

[ligi]

Was giving this a look again and trezor support is a huge chunk (about 1/5th of the apk size) in WallETH - still not a lot of time to work on it - but priority just got higher. Perhaps also a good use-case for a bounty as this task is quite well defined (no more protobuf dependency by using it with wire) and having the same function set afterwards.

[ychescale9]

@ligi was that after proguarding?

Is this work already in-progress? If the remaining work is fairly mechanical I might be able to find some time to work on it.

[ligi]

@ychescale9 no it was before proguarding - but I would still prefer the size to be small before - makes a better developer experience - but will also try to see how proguard helps (but no obfuscation - really do not want this - I hate dealing with these stacktraces and the extra work associated with this) I also did not yet start work on it - it should be fairly mechanical

[robin-thomas]

@ligi Is the Gitcoin bounty still available? I was approved for this task, but I can see that the comments had been deleted. So just checking.

[ligi]

@robin-thomas I think it is still available - not sure why these comments where removed by @trezor - hope they will comment. Perhaps wait with starting the implementation until there will be some clarification.

[robin-thomas]

@ligi Sure. I'll wait for a confirmation from @trezor. Thanks!

[prusnak]

I don't want to have our Github issues spammed via bots. Feel free to use third party platforms, but don't spam the issues (and transitively my inbox), please.

[spm32]

Sorry about that @prusnak, we can mute the bot for this issue. @robin-thomas you're still good to work on this :) You can still find details on the issue here: https://gitcoin.co/issue/trezor/trezor-android/29/2052

[rsercano]

Could you please confirm that bounty is still available and noone is working on this ? I'm interested in this, but as I've specified in the express interest I'm new to wire, just wanted to give it a shot !

[ligi]

@rsercano I think unfortunately the bounty now timed out when looking at the gitcoin page - but @ceresstation might know more

Also now I think it would make sense to wait for: https://github.com/square/wire/issues/881 and target wire 3 with the gradle plugin directly