KeyManagement node randomly failed to start

[lacabra]

Describe the bug KeyManagement node randomly failed to start.

Is this the same as #98? @fredfortier, @elichai you can close it if it's a duplicate, but the backtrace looks different to me.

To Reproduce Steps to reproduce the behavior:

1. Launch the discovery-integration-tests network

This is not easily reproducible, as next time I restarted the network, it started just fine.

Expected behavior The KeyManagement node should have started.

Backtrace

principal_1  | CLI params: Opt { info: false, verbose: 0, debug_stdout: false, sign_address: false, register: false, set_worker_params: false, confirm_worker_params: false, get_state_keys: None, contract_address: None, reset_epoch_state: false, deploy: false, network: "http://.c/", mine: 0, time_to_live: 0, deploy_config: "../app/tests/principal_node/config/deploy_config.json", principal_config: "../app/tests/principal_node/config/principal_test_config.json" }
principal_1  | [+] Home dir is /root
principal_1  | [+] Home dir is /root
principal_1  | [-] Create .enigma folder => AlreadyExists
principal_1  | [-] Create .enigma/epoch folder => AlreadyExists
principal_1  | [-] Create .enigma/state-keys folder => AlreadyExists
principal_1  | The current directory is /root/enigma-core/enigma-principal/bin
principal_1  | [+] Open token file success! 
principal_1  | [+] Token file valid!
principal_1  | [+] Saved updated launch token!
principal_1  | [+] Init Enclave Successful 2!
principal_1  | Loading Principal config: "../app/tests/principal_node/config/principal_test_config.json"
principal_1  | [+] Home dir is /root
principal_1  | Back from Ocall: /root/.enigma
principal_1  | Succeeded reading key from file
principal_1  | [Mode:] run node NO DEPLOY.
principal_1  | Connected to the Enigma contract: "59d3631c86BbE35EF041872d502F218A39FBa150" with account: 0x1df62f291b2e969fb0849d99d9ce41e2f137006e
principal_1  | [+] Home dir is /root
principal_1  | No existing epoch state, starting with block 0
principal_1  | Fetching the signing address for account: 0x1df62f291b2e969fb0849d99d9ce41e2f137006e
contract_1   | eth_call
principal_1  | Using SGX_MODE: "HW"
principal_1  | Fetching quote with SPID: "B0335FD3BC1CCA8F804EB98A6420592D"
principal_1  | aesm_service[10]: [ADMIN]EPID Provisioning initiated
principal_1  | aesm_service[10]: [ADMIN]EPID Provisioning successful
principal_1  | Hardware mode, fetching report from the Attestation Service
principal_1  | thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', libcore/option.rs:355:21
principal_1  | stack backtrace:
principal_1  |    0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
principal_1  |              at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
principal_1  |    1: std::sys_common::backtrace::print
principal_1  |              at libstd/sys_common/backtrace.rs:71
principal_1  |              at libstd/sys_common/backtrace.rs:59
principal_1  |    2: std::panicking::default_hook::{{closure}}
principal_1  |              at libstd/panicking.rs:211
principal_1  |    3: std::panicking::default_hook
principal_1  |              at libstd/panicking.rs:227
principal_1  |    4: std::panicking::rust_panic_with_hook
principal_1  |              at libstd/panicking.rs:476
principal_1  |    5: std::panicking::continue_panic_fmt
principal_1  |              at libstd/panicking.rs:390
principal_1  |    6: rust_begin_unwind
principal_1  |              at libstd/panicking.rs:325
principal_1  |    7: core::panicking::panic_fmt
principal_1  |              at libcore/panicking.rs:77
principal_1  |    8: core::panicking::panic
principal_1  |              at libcore/panicking.rs:52
principal_1  |    9: <core::option::Option<T>>::unwrap
principal_1  |              at libcore/macros.rs:20
principal_1  |   10: enigma_tools_u::attestation_service::service::AttestationService::unwrap_result::{{closure}}
principal_1  |              at /root/enigma-core/enigma-tools-u/src/attestation_service/service.rs:161
principal_1  |   11: enigma_tools_u::attestation_service::service::AttestationService::unwrap_result
principal_1  |              at /root/enigma-core/enigma-tools-u/src/attestation_service/service.rs:161
principal_1  |   12: enigma_tools_u::attestation_service::service::AttestationService::unwrap_response
principal_1  |              at /root/enigma-core/enigma-tools-u/src/attestation_service/service.rs:176
principal_1  |   13: enigma_tools_u::attestation_service::service::AttestationService::send_request
principal_1  |              at /root/enigma-core/enigma-tools-u/src/attestation_service/service.rs:124
principal_1  |   14: enigma_tools_u::attestation_service::service::AttestationService::get_report::{{closure}}
principal_1  |              at /root/enigma-core/enigma-tools-u/src/attestation_service/service.rs:97
principal_1  |   18: <enigma_principal_app::boot_network::principal_manager::PrincipalManager as enigma_principal_app::boot_network::principal_manager::Sampler>::register::{{closure}}
principal_1  |              at src/boot_network/principal_manager.rs:166
principal_1  |   19: <enigma_principal_app::boot_network::principal_manager::PrincipalManager as enigma_principal_app::boot_network::principal_manager::Sampler>::register
principal_1  |              at src/boot_network/principal_manager.rs:164
principal_1  |   20: <enigma_principal_app::boot_network::principal_manager::PrincipalManager as enigma_principal_app::boot_network::principal_manager::Sampler>::verify_identity_or_register::{{closure}}
principal_1  |              at src/boot_network/principal_manager.rs:193
principal_1  |   21: <enigma_principal_app::boot_network::principal_manager::PrincipalManager as enigma_principal_app::boot_network::principal_manager::Sampler>::verify_identity_or_register
principal_1  |              at src/boot_network/principal_manager.rs:184
principal_1  |   22: <enigma_principal_app::boot_network::principal_manager::PrincipalManager as enigma_principal_app::boot_network::principal_manager::Sampler>::run::{{closure}}
principal_1  |              at src/boot_network/principal_manager.rs:212
principal_1  |   23: <enigma_principal_app::boot_network::principal_manager::PrincipalManager as enigma_principal_app::boot_network::principal_manager::Sampler>::run
principal_1  |              at src/boot_network/principal_manager.rs:209
principal_1  |   24: enigma_principal_app::cli::app::start::{{closure}}
principal_1  |              at src/cli/app.rs:82
principal_1  |   25: enigma_principal_app::cli::app::start
principal_1  |              at src/cli/app.rs:14
principal_1  |   26: enigma_principal_app::main
principal_1  |              at src/main.rs:66
principal_1  |   27: std::rt::lang_start::{{closure}}
principal_1  |              at libstd/rt.rs:74
principal_1  |   28: std::panicking::try::do_call
principal_1  |              at libstd/rt.rs:59
principal_1  |              at libstd/panicking.rs:310
principal_1  |   29: __rust_maybe_catch_panic
principal_1  |              at libpanic_unwind/lib.rs:102
principal_1  |   30: std::rt::lang_start_internal
principal_1  |              at libstd/panicking.rs:289
principal_1  |              at libstd/panic.rs:392
principal_1  |              at libstd/rt.rs:58
principal_1  |   31: std::rt::lang_start
principal_1  |              at libstd/rt.rs:74
principal_1  |   32: main
principal_1  |   33: __libc_start_main
principal_1  |   34: _start

[elichai]

It does sounds like they're the same, one thing I'm not sure about is that it looks like the url returns a success and then it fails unwrapping. and that's weird, if that's the case then I'm not sure what I should retry for, does the proxy server have logs that we can check for? it shouldn't return 200 if intel didn't return 200