bitcoinfuzz icon indicating copy to clipboard operation
bitcoinfuzz copied to clipboard

feat: Add Eclair module for invoice deserialization

Open erickcestari opened this issue 8 months ago • 8 comments

This pr introduces support for deserializing Lightning Network invoices using the Eclair implementation.

Key changes include:

  • Added Eclair as a Git submodule (external/eclair).
  • Created a new C++ module (modules/eclair) that uses JNI (Java Native Interface) to interact with Eclair's Java/Scala code.
  • Implemented the deserialize_invoice function within the Eclair module. This function calls fr.acinq.eclair.payment.Bolt11Invoice .fromString via JNI to parse invoices and extract relevant fields (hash, amount, description, recipient, expiry, timestamp, routing hints, min CLTV).
  • Updated the main Makefile to include JVM linking flags and automatically detect JAVA_HOME.
  • Added a Makefile within the modules/eclair directory to handle building Eclair (using mvnw), extracting necessary JARs, and compiling the C++ JNI wrapper.
  • Modified the GitHub Actions workflow (workflow.yml) to:
    • Set up JDK 21 using actions/setup-java.
    • Add a build step for the Eclair module.
    • Include the -DECLAIR CXX flag in the deserialize_invoice fuzz test step.
  • Updated .gitignore to exclude *.jar and *.zip files.

Closes #94

erickcestari avatar Apr 25 '25 17:04 erickcestari

It's missing the instructions on README.

Added!

erickcestari avatar Apr 29 '25 11:04 erickcestari

Can you please rebase?

brunoerg avatar Apr 30 '25 20:04 brunoerg

It looks like the tests passed on macOS 13. Should we consider bumping to a more recent version, like macOS 14? Also, there are some strange errors in the logs. See this one, for example:

https://github.com/bitcoinfuzz/bitcoinfuzz/actions/runs/14775258814/job/41482277497#step:31:33

erickcestari avatar May 01 '25 13:05 erickcestari

It looks like the tests passed on macOS 13. Should we consider bumping to a more recent version, like macOS 14? Also, there are some strange errors in the logs. See this one, for example:

https://github.com/bitcoinfuzz/bitcoinfuzz/actions/runs/14775258814/job/41482277497#step:31:33

We can bump to a more recent version (I think it's better) or we could use MACOSX_DEPLOYMENT_TARGET to set the target according to the version we're using. Anyway, we can do it in another PR, feel free to ignore here for now.

brunoerg avatar May 01 '25 13:05 brunoerg

I could successfully build and run it. But everytime I give a CTRL+C to stop the execution, it causes a crash:

==99304== ERROR: libFuzzer: fuzz target exited
==99304==WARNING: Can't write to symbolizer at fd 150
    #0 0x0001033eacd4 in __sanitizer_print_stack_trace+0x28 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x5ecd4)
    #1 0x00010273ba74 in fuzzer::PrintStackTrace() FuzzerUtil.cpp:210
    #2 0x00010271d178 in fuzzer::Fuzzer::ExitCallback() FuzzerLoop.cpp:248
    #3 0x00018d795fa4 in __cxa_finalize_ranges+0x1e8 (libsystem_c.dylib:arm64+0x26fa4)
    #4 0xa37680018d795d1c  (<unknown module>)
    #5 0x136c8001044fdfb4  (<unknown module>)
    #6 0x000104248600 in vm_direct_exit(int)+0x1c (libjvm.dylib:arm64+0x388600)
    #7 0x0001046bf460 in VM_Exit::doit()+0x9c (libjvm.dylib:arm64+0x7ff460)
    #8 0x0001046be97c in VM_Operation::evaluate()+0xf0 (libjvm.dylib:arm64+0x7fe97c)
    #9 0x0001046ce794 in VMThread::evaluate_operation(VM_Operation*)+0xf0 (libjvm.dylib:arm64+0x80e794)
    #10 0x0001046cedd0 in VMThread::inner_execute(VM_Operation*)+0x1f8 (libjvm.dylib:arm64+0x80edd0)
    #11 0x0001046ce5c4 in VMThread::loop()+0x64 (libjvm.dylib:arm64+0x80e5c4)
    #12 0x0001046ce450 in VMThread::run()+0x80 (libjvm.dylib:arm64+0x80e450)
    #13 0x0001046615f4 in Thread::call_run()+0x88 (libjvm.dylib:arm64+0x7a15f4)
    #14 0x0001044f88e8 in thread_native_entry(Thread*)+0xc8 (libjvm.dylib:arm64+0x6388e8)
    #15 0x0001033dc1e4 in asan_thread_start(void*)+0x48 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x501e4)
    #16 0x00018d8da030 in _pthread_start+0x84 (libsystem_pthread.dylib:arm64+0x7030)
    #17 0x492580018d8d4e38  (<unknown module>)

SUMMARY: libFuzzer: fuzz target exited
MS: 4 ChangeBit-Custom-CMP-Custom- DE: "Java_jdk_internal_misc_ScopedMemoryAccess_registerNatives"-; base unit: b8f85389a2ecf1345c5adb0328e599dcdf9240dd
0x62,0x31,0x36,0x6c,0x74,0x61,0x30,0x34,0x37,0x70,0x70,0x35,0x68,0x36,0x6c,0x74,0x61,0x30,0x34,0x37,0x68,0x36,0x6c,0x74,0x61,0x30,0x34,0x37,0x68,0x36,0x6c,0x30,0x61,0x30,0x34,0x37,0x68,0x36,0x6c,0x74,0x61,0x30,0x34,0x37,0x68,0x36,0x6c,0x74,0x61,0x30,0x34,0x37,0x68,0x36,0x6c,0x74,0x61,0x30,0x34,0x37,0x68,0x36,0x6c,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x71,0x32,0x70,0x6b,0x70,0x6c,0x32,0x79,0x74,0x6c,0x66,0x77,0x35,0x39,0x6a,0x77,0x70,0x76,0x6c,0x64,0x66,0x6e,0x72,0x6c,0x6e,0x72,0x30,0x73,0x39,0x79,0x64,0x39,0x64,0x30,0x6a,0x65,0x70,0x72,0x72,0x39,0x6e,0x6e,0x6c,0x6a,0x39,0x38,0x66,0x6e,0x35,0x39,0x6a,0x77,0x70,0x35,0x66,0x6b,0x39,0x6e,0x71,0x71,0x32,0x70,0x39,0x37,0x37,0x75,
b16lta047pp5h6lta047h6lta047h6l0a047h6lta047h6lta047h6lta047h6lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq2pkpl2ytlfw59jwpvldfnrlnr0s9yd9d0jeprr9nnlj98fn59jwp5fk9nqq2p977u
artifact_prefix='./'; Test unit written to ./crash-d92dcb2a06ab7a77e8ac8a6420d7ddf0e37a30a4
Base64: YjE2bHRhMDQ3cHA1aDZsdGEwNDdoNmx0YTA0N2g2bDBhMDQ3aDZsdGEwNDdoNmx0YTA0N2g2bHRhMDQ3aDZscXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcTJwa3BsMnl0bGZ3NTlqd3B2bGRmbnJsbnIwczl5ZDlkMGplcHJyOW5ubGo5OGZuNTlqd3A1Zms5bnFxMnA5Nzd1

Fixed

The -Xrs JVM flag fixed Ctrl+C crash.

erickcestari avatar May 02 '25 17:05 erickcestari

Hello @t-bast,

I'd appreciate your review on this PR as it adds Eclair to our differential fuzzing for Lightning Network invoice deserialization. Since you have deep expertise with Eclair and the Lightning protocol, your insights would be valuable regarding the JNI implementation and how we're extracting the invoice fields for comparison testing.

Thanks!

erickcestari avatar May 05 '25 13:05 erickcestari

Nice, thanks for working on this! I'm not at all an expert on JNI, but @sstone should be able to provide some feedback on the approach!

t-bast avatar May 05 '25 15:05 t-bast

Needs rebase

brunoerg avatar May 15 '25 15:05 brunoerg

I'm drafting this for now, as I plan to submit another PR with a better approach. In the next iteration, I'll create a Java file to avoid repeatedly invoking the JVM in module.cpp. I'll call it just once to get the formatted result.

erickcestari avatar Jun 26 '25 18:06 erickcestari

I'm drafting this for now, as I plan to submit another PR with a better approach. In the next iteration, I'll create a Java file to avoid repeatedly invoking the JVM in module.cpp. I'll call it just once to get the formatted result.

It seems that this "another PR" is here, right? haha. I'm going to review it.

brunoerg avatar Jul 14 '25 20:07 brunoerg

I'm drafting this for now, as I plan to submit another PR with a better approach. In the next iteration, I'll create a Java file to avoid repeatedly invoking the JVM in module.cpp. I'll call it just once to get the formatted result.

It seems that this "another PR" is here, right? haha. I'm going to review it.

Yes, it is hahaha. It mutated and become a new PR

erickcestari avatar Jul 14 '25 20:07 erickcestari

Draft. I need to update for a new formatted result due to #227.

erickcestari avatar Jul 17 '25 19:07 erickcestari

This is open, again.

erickcestari avatar Jul 23 '25 19:07 erickcestari

needs rebase

brunoerg avatar Jul 24 '25 20:07 brunoerg

Needs rebase

brunoerg avatar Jul 28 '25 17:07 brunoerg