bellperson icon indicating copy to clipboard operation
bellperson copied to clipboard

Published 20 hours ago • verified publisher icon jackoelv

lotus-bench:failed to create ColumnTreeBuilder: ClError(BusIdNotAvailable)

Open bugxzhu opened this issue 4 years ago • 1 comments

2021-01-26T20:16:44.626 INFO storage_proofs_porep::stacked::vanilla::cache > parent cache: opening /home/lotus_user/filecoin_proofs_parent_cache/v28-sdr-parent-7ba215a1d2345774ab90b8cb1158d296e409d6068819d7b8c7baf0b25d63dc34.cache, verify enabled: false 2021-01-26T20:16:44.626 INFO storage_proofs_porep::stacked::vanilla::proof > multi core replication 2021-01-26T20:16:44.626 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > create labels 2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > Cores: 32, Shared Caches: 8, cores per cache (group_size): 4 2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > checked out core group 0 2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in main thread 2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 0,32 2021-01-26T20:16:44.689 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 0 2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::memory_handling > initializing cache 2021-01-26T20:16:44.692 WARN storage_proofs_porep::stacked::vanilla::memory_handling > failed to lock map Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }, falling back 2021-01-26T20:16:44.692 WARN storage_proofs_porep::stacked::vanilla::memory_handling > failed to lock map Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }, falling back 2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Layer 1 2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Creating labels for layer 1 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 2 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 3,35 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 3 2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 0 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 1,33 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 1 2021-01-26T20:16:44.692 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner 2021-01-26T20:16:44.692 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 1 2021-01-26T20:16:44.700 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 2,34 2021-01-26T20:16:44.700 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 2 2021-01-26T20:16:44.700 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner 2021-01-26T20:16:45.944 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1399425 2021-01-26T20:16:49.873 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 5777665 2021-01-26T20:16:51.161 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 7204993 2021-01-26T20:16:55.505 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 12001921 2021-01-26T20:16:59.855 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > storing labels on disk 2021-01-26T20:17:00.001 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > generated layer 1 store with id layer-1 2021-01-26T20:17:00.001 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Layer 2 2021-01-26T20:17:00.001 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > Creating labels for layer 2 2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1 2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 2 2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 3 2021-01-26T20:17:00.001 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 3 2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 1 2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 2 2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 2 2021-01-26T20:17:00.002 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner 2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > binding core in producer thread 0 2021-01-26T20:17:00.002 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner 2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > allowed cpuset: 1 2021-01-26T20:17:00.002 DEBUG storage_proofs_porep::stacked::vanilla::cores > binding to 1 2021-01-26T20:17:00.002 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > created label runner 2021-01-26T20:17:00.625 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 702465 2021-01-26T20:17:00.627 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 702849 2021-01-26T20:17:01.638 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 1843073 2021-01-26T20:17:05.746 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 6434817 2021-01-26T20:17:11.633 DEBUG storage_proofs_porep::stacked::vanilla::create_label::multi > PRODUCER NOT READY! 12956289 2021-01-26T20:17:15.112 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > storing labels on disk 2021-01-26T20:17:15.256 INFO storage_proofs_porep::stacked::vanilla::create_label::multi > generated layer 2 store with id layer-2 2021-01-26T20:17:15.333 INFO filecoin_proofs::api::seal > seal_pre_commit_phase1:finish: SectorId(0) 2021-01-26T20:17:15.333 INFO filcrypto::proofs::api > seal_pre_commit_phase1: finish 2021-01-26T20:17:15.333+0800 INFO lotus-bench lotus-bench/main.go:576 [0] Running replication(2)... 2021-01-26T20:17:15.333 INFO filcrypto::proofs::api > seal_pre_commit_phase2: start 2021-01-26T20:17:15.333 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:start 2021-01-26T20:17:15.333 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:finish 2021-01-26T20:17:15.333 INFO filecoin_proofs::api::seal > seal_pre_commit_phase2:start 2021-01-26T20:17:15.333 TRACE filecoin_proofs::api::seal > seal phase 2: base tree size 33554431, base tree leafs 16777216, rows to discard 7 2021-01-26T20:17:15.334 INFO storage_proofs_porep::stacked::vanilla::proof > replicate_phase2 2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > transform_and_replicate_layers 2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > nodes count 16777216, data len 536870912 2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > is_merkle_tree_size_valid(16777216, BINARY_ARITY) = true 2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > is_merkle_tree_size_valid(16777216, 8) = true 2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > tree_r_last using rows_to_discard=2 2021-01-26T20:17:15.334 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree c using the GPU 2021-01-26T20:17:15.334 INFO storage_proofs_porep::stacked::vanilla::proof > Building column hashes 2021-01-26T20:17:15.334 INFO neptune::triton::cl > getting default futhark context 2021-01-26T20:17:15.334 TRACE storage_proofs_porep::stacked::vanilla::proof > processing config 1/1 with column nodes 800000 2021-01-26T20:17:15.334 INFO neptune::triton::cl > getting context for ~Index(0) 2021-01-26T20:17:15.512 TRACE storage_proofs_porep::stacked::vanilla::proof > node index 800000/800000/16777216 2021-01-26T20:17:25.336 DEBUG rust_gpu_tools::opencl::utils > loaded devices: {} thread '' panicked at 'failed to create ColumnTreeBuilder: ClError(BusIdNotAvailable)', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/storage-proofs-porep-5.4.0/src/stacked/vanilla/proof.rs:516:23 stack backtrace: 0: rust_begin_unwind at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:483 1: core::panicking::panic_fmt at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/panicking.rs:85 2: core::option::expect_none_failed at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/option.rs:1234 3: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once 4: std::panicking::try 5: <rayon_core::job::HeapJob<BODY> as rayon_core::job::Job>::execute 6: rayon_core::registry::WorkerThread::wait_until_cold 7: rayon_core::scope::ScopeBase::steal_till_jobs_complete 8: rayon_core::scope::scope::{{closure}} 9: std::panicking::try 10: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute 11: rayon_core::registry::WorkerThread::wait_until_cold 12: rayon_core::registry::ThreadBuilder::run note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace. thread '' panicked at 'failed to send columns: "SendError(..)"', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/storage-proofs-porep-5.4.0/src/stacked/vanilla/proof.rs:502:34 stack backtrace: 0: rust_begin_unwind at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/std/src/panicking.rs:483 1: core::panicking::panic_fmt at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/panicking.rs:85 2: core::option::expect_none_failed at ./rustc/beb5ae474d2835962ebdf7416bd1c9ad864fe101/library/core/src/option.rs:1234 3: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once 4: std::panicking::try 5: <rayon_core::job::HeapJob<BODY> as rayon_core::job::Job>::execute 6: rayon_core::registry::WorkerThread::wait_until_cold 7: rayon_core::registry::ThreadBuilder::run note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace. 2021-01-26T20:17:25.659+0800 WARN lotus-bench lotus-bench/main.go:116 failed to run seals: main.glob..func3 /home/bugx/lotus/cmd/lotus-bench/main.go:277

  • commit: main.runSeals.func1.1 /home/bugx/lotus/cmd/lotus-bench/main.go:579
  • presealing sector 0 (): github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper.(*Sealer).SealPreCommit2 /home/bugx/lotus/extern/sector-storage/ffiwrapper/sealer_cgo.go:486
  • Rust panic: no unwind information github.com/filecoin-project/filecoin-ffi.SealPreCommitPhase2 /home/bugx/lotus/extern/filecoin-ffi/proofs.go:321 github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper.(*Sealer).SealPreCommit2 /home/bugx/lotus/extern/sector-storage/ffiwrapper/sealer_cgo.go:484 main.runSeals.func1.1 /home/bugx/lotus/cmd/lotus-bench/main.go:577 main.runSeals.func1 /home/bugx/lotus/cmd/lotus-bench/main.go:688 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1374

bugxzhu avatar Jan 26 '21 12:01 bugxzhu

1.4.0优化版编译 环境 cpu 7542 显卡3090

bugxzhu avatar Jan 26 '21 12:01 bugxzhu