pesnet icon indicating copy to clipboard operation
pesnet copied to clipboard

Published 20 hours ago • verified publisher icon n-gao

Failed in plotting

Open xiazhuozhao opened this issue 1 year ago • 6 comments

After installing the latest version of JAX, I used the pip install -e . command to install PESnet. The network runs successfully, but it seems there's no correct output for the plotting. I'm unable to pinpoint where the issue is. Could you help me, thanks!

2023-12-09 00:02:45 (INFO): Running command 'func'
2023-12-09 00:02:45 (INFO): Started
2023-12-09 00:02:46 (INFO): Unable to initialize backend 'rocm': NOT_FOUND: Could not find registered platform with name: "rocm". Available platform names are: CUDA
2023-12-09 00:02:46 (INFO): Unable to initialize backend 'tpu': INTERNAL: Failed to open libtpu.so: libtpu.so: cannot open shared object file: No such file or directory
2023-12-09 00:02:50.408321: W external/xla/xla/service/gpu/nvptx_compiler.cc:698] The NVIDIA driver's CUDA version is 12.0 which is older than the ptxas CUDA version (12.3.103). Because the driver is older than the ptxas version, XLA is disabling parallel compilation, which may slow down compilation. You should update your NVIDIA driver or use the NVIDIA-provided CUDA forward compatibility packages.  
2023-12-09 00:03:02 (INFO): creating RunStatusReporter for e9feee6d91754a878e2f5a31
2023-12-09 00:03:02 (INFO): starting from: {}
2023-12-09 00:03:02 (INFO): starting writer thread for <aim.sdk.reporter.RunStatusReporter object at 0x14714c03ca30>
log_dir: /home/shhgroup/shanghui/logs/pesnet/H2__09-12-23_00:02:50:760536
name: H2
aim_hash: e9feee6d91754a878e2f5a31
Initialization
CG precision: float32
converged SCF energy = -0.957081335258677
converged SCF energy = -1.06810203295004
converged SCF energy = -1.11904988746119
converged SCF energy = -1.12485012825092
converged SCF energy = -1.11685689709792
converged SCF energy = -1.09217978938209
converged SCF energy = -1.08209327618322
converged SCF energy = -1.04169183529242
converged SCF energy = -1.02684230384466
converged SCF energy = -0.993078569432703
converged SCF energy = -0.949031242350657
converged SCF energy = -0.924239239786162
converged SCF energy = -0.886451415542872
converged SCF energy = -0.86337080411078
converged SCF energy = -0.84716056352074
converged SCF energy = -0.834458913313593
MO initial loss: 0.011822241796392522; final loss: 0.011822241796392522
MO initial loss: 0.00943993208689221; final loss: 0.00943993208689221
MO initial loss: 0.006633517984115942; final loss: 0.006633517984115942
MO initial loss: 0.004138748404110831; final loss: 0.004138748404110831
MO initial loss: 0.00290062683801973; final loss: 0.00290062683801973
MO initial loss: 0.0012568684013349455; final loss: 0.0012568684013349455
MO initial loss: 0.0008599610176328052; final loss: 0.0008599610176328052
MO initial loss: 5.554254636501985e-05; final loss: 5.554254636501985e-05
MO initial loss: 0.00026591531105711684; final loss: 0.00026591531105711684
MO initial loss: 0.0013454877996926162; final loss: 0.0013454877996926162
MO initial loss: 0.0022861422131632654; final loss: 0.0022861422131632654
MO initial loss: 0.00414431091105927; final loss: 0.00414431091105927
MO initial loss: 0.0055136435914820685; final loss: 0.0055136435914820685
MO initial loss: 0.006573633496650197; final loss: 0.006573633496650197
MO initial loss: 0.007457328172754619; final loss: 0.007457328172754619
Pretraining
100%|██████████| 10000/10000 [04:16<00:00, 38.92it/s, MSE=2.2454947e-07, pmove=0.53344727]
Thermalizing
Training
  0%|          | 0/60000 [00:00<?, ?it/s]2023-12-09 00:09:57 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".   
  0%|          | 0/60000 [01:17<?, ?it/s, E=-.993, E_std=0.585, E_var=0.369, pmove=0.5372742][0] creating checkpoint
  0%|          | 100/60000 [03:10<16:11:07,  1.03it/s, E=-1.1, E_std=0.0255, E_var=0.000687, pmove=0.5363648]2023-12-09 00:11:52 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  0%|          | 200/60000 [04:50<16:16:43,  1.02it/s, E=-1.1, E_std=0.0181, E_var=0.00041, pmove=0.5311035]2023-12-09 00:13:31 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  0%|          | 300/60000 [06:29<16:11:30,  1.02it/s, E=-1.1, E_std=0.0158, E_var=0.000281, pmove=0.5317566]2023-12-09 00:15:11 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  1%|          | 400/60000 [08:09<16:12:11,  1.02it/s, E=-1.1, E_std=0.0173, E_var=0.000426, pmove=0.5332947]2023-12-09 00:16:50 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  1%|          | 500/60000 [09:48<16:09:12,  1.02it/s, E=-1.1, E_std=0.0277, E_var=0.00425, pmove=0.5339844]2023-12-09 00:18:29 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  1%|          | 600/60000 [11:27<16:09:12,  1.02it/s, E=-1.1, E_std=0.0102, E_var=0.000126, pmove=0.5315796]2023-12-09 00:20:09 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  1%|          | 700/60000 [13:12<15:54:04,  1.04it/s, E=-1.1, E_std=0.0191, E_var=0.00117, pmove=0.5337219]2023-12-09 00:21:53 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  1%|▏         | 800/60000 [14:51<16:02:14,  1.03it/s, E=-1.1, E_std=0.0129, E_var=0.000386, pmove=0.532489]2023-12-09 00:23:32 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  2%|▏         | 900/60000 [16:30<16:02:25,  1.02it/s, E=-1.1, E_std=0.0111, E_var=0.000151, pmove=0.5359986]2023-12-09 00:25:11 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  2%|▏         | 1000/60000 [18:08<16:00:20,  1.02it/s, E=-1.11, E_std=0.00919, E_var=0.000111, pmove=0.533606]2023-12-09 00:26:50 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  3%|▎         | 2000/60000 [32:34<4:10:41,  3.86it/s, E=-1.11, E_std=0.00491, E_var=2.67e-5, pmove=0.53570557]2023-12-09 00:41:14 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  5%|▌         | 3000/60000 [35:46<2:49:56,  5.59it/s, E=-1.1, E_std=0.00601, E_var=4.02e-5, pmove=0.53441167]2023-12-09 00:44:27 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  7%|▋         | 4000/60000 [38:48<2:47:37,  5.57it/s, E=-1.1, E_std=0.00541, E_var=3.36e-5, pmove=0.53153074]2023-12-09 00:47:28 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  8%|▊         | 5000/60000 [41:49<2:45:09,  5.55it/s, E=-1.1, E_std=0.00635, E_var=4.95e-5, pmove=0.5348511]2023-12-09 00:50:30 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
  8%|▊         | 5000/60000 [41:50<2:45:09,  5.55it/s, E=-1.11, E_std=0.00665, E_var=5.37e-5, pmove=0.5355103][5000] creating checkpoint
 10%|█         | 6000/60000 [44:52<2:42:03,  5.55it/s, E=-1.1, E_std=0.00576, E_var=3.7e-5, pmove=0.5304138]2023-12-09 00:53:33 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 12%|█▏        | 7000/60000 [47:57<2:38:48,  5.56it/s, E=-1.1, E_std=0.00658, E_var=7.58e-5, pmove=0.5318177]2023-12-09 00:56:37 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 13%|█▎        | 8000/60000 [51:03<2:36:34,  5.54it/s, E=-1.1, E_std=0.00487, E_var=2.86e-5, pmove=0.5316162]2023-12-09 00:59:44 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 15%|█▌        | 9000/60000 [54:07<2:32:31,  5.57it/s, E=-1.11, E_std=0.00504, E_var=4.03e-5, pmove=0.5337403]2023-12-09 01:02:48 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 17%|█▋        | 10000/60000 [57:14<2:30:25,  5.54it/s, E=-1.1, E_std=0.00567, E_var=4.15e-5, pmove=0.5334778]2023-12-09 01:05:54 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 17%|█▋        | 10000/60000 [57:14<2:30:25,  5.54it/s, E=-1.11, E_std=0.00633, E_var=8.39e-5, pmove=0.5336121][10000] creating checkpoint
 18%|█▊        | 11000/60000 [1:00:20<2:26:04,  5.59it/s, E=-1.11, E_std=0.00466, E_var=2.65e-5, pmove=0.5334228]2023-12-09 01:09:01 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 20%|██        | 12000/60000 [1:03:29<2:23:55,  5.56it/s, E=-1.1, E_std=0.00591, E_var=6.19e-5, pmove=0.5322876]2023-12-09 01:12:09 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 22%|██▏       | 13000/60000 [1:06:36<2:20:52,  5.56it/s, E=-1.09, E_std=0.0039, E_var=1.75e-5, pmove=0.53063357]2023-12-09 01:15:17 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 23%|██▎       | 14000/60000 [1:09:41<2:16:56,  5.60it/s, E=-1.1, E_std=0.00453, E_var=3.38e-5, pmove=0.53336185]2023-12-09 01:18:22 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 25%|██▌       | 15000/60000 [1:12:47<2:14:16,  5.59it/s, E=-1.1, E_std=0.00351, E_var=1.34e-5, pmove=0.5331665]2023-12-09 01:21:27 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 25%|██▌       | 15000/60000 [1:12:47<2:14:16,  5.59it/s, E=-1.1, E_std=0.00417, E_var=2.14e-5, pmove=0.53515625][15000] creating checkpoint
 27%|██▋       | 16000/60000 [1:15:57<2:12:11,  5.55it/s, E=-1.1, E_std=0.00386, E_var=1.8e-5, pmove=0.5310974]2023-12-09 01:24:37 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 28%|██▊       | 17000/60000 [1:18:58<2:09:28,  5.54it/s, E=-1.11, E_std=0.00328, E_var=1.14e-5, pmove=0.53376466]2023-12-09 01:27:39 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 30%|███       | 18000/60000 [1:22:03<2:05:13,  5.59it/s, E=-1.11, E_std=0.00448, E_var=3.08e-5, pmove=0.5347473]2023-12-09 01:30:44 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 32%|███▏      | 19000/60000 [1:25:06<2:02:21,  5.58it/s, E=-1.1, E_std=0.00396, E_var=2.06e-5, pmove=0.5323364]2023-12-09 01:33:47 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 33%|███▎      | 20000/60000 [1:28:10<1:59:44,  5.57it/s, E=-1.1, E_std=0.00335, E_var=1.31e-5, pmove=0.5312073]2023-12-09 01:36:51 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 33%|███▎      | 20000/60000 [1:28:11<1:59:44,  5.57it/s, E=-1.1, E_std=0.00401, E_var=2.12e-5, pmove=0.5314636][20000] creating checkpoint
 35%|███▌      | 21000/60000 [1:31:16<1:56:16,  5.59it/s, E=-1.1, E_std=0.00371, E_var=1.6e-5, pmove=0.5339905]2023-12-09 01:39:56 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 37%|███▋      | 22000/60000 [1:34:18<1:53:02,  5.60it/s, E=-1.1, E_std=0.00294, E_var=9.08e-6, pmove=0.53037715]2023-12-09 01:42:59 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 38%|███▊      | 23000/60000 [1:37:24<1:50:42,  5.57it/s, E=-1.1, E_std=0.00272, E_var=8.24e-6, pmove=0.53443605]2023-12-09 01:46:05 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 40%|████      | 24000/60000 [1:40:37<1:48:06,  5.55it/s, E=-1.1, E_std=0.00283, E_var=9.42e-6, pmove=0.53353274]2023-12-09 01:49:17 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 42%|████▏     | 25000/60000 [1:43:40<1:44:34,  5.58it/s, E=-1.1, E_std=0.00318, E_var=1.17e-5, pmove=0.53358155]2023-12-09 01:52:21 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 42%|████▏     | 25000/60000 [1:43:41<1:44:34,  5.58it/s, E=-1.1, E_std=0.00298, E_var=9.85e-6, pmove=0.5335877] [25000] creating checkpoint
 43%|████▎     | 26000/60000 [1:46:53<1:41:07,  5.60it/s, E=-1.1, E_std=0.00309, E_var=1.11e-5, pmove=0.53308105]2023-12-09 01:55:33 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 45%|████▌     | 27000/60000 [1:49:55<1:38:27,  5.59it/s, E=-1.1, E_std=0.00346, E_var=1.76e-5, pmove=0.53447264]2023-12-09 01:58:36 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 47%|████▋     | 28000/60000 [1:52:58<1:35:30,  5.58it/s, E=-1.09, E_std=0.0028, E_var=9.79e-6, pmove=0.53217167]2023-12-09 02:01:39 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 48%|████▊     | 29000/60000 [1:56:02<1:32:26,  5.59it/s, E=-1.1, E_std=0.00322, E_var=1.22e-5, pmove=0.53204346]2023-12-09 02:04:43 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 50%|█████     | 30000/60000 [1:59:06<1:29:23,  5.59it/s, E=-1.1, E_std=0.00244, E_var=6.71e-6, pmove=0.53277594]2023-12-09 02:07:47 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 50%|█████     | 30000/60000 [1:59:07<1:29:23,  5.59it/s, E=-1.1, E_std=0.00401, E_var=1.99e-5, pmove=0.5334656] [30000] creating checkpoint
 52%|█████▏    | 31000/60000 [2:02:10<1:26:24,  5.59it/s, E=-1.1, E_std=0.00501, E_var=5e-5, pmove=0.5336304]2023-12-09 02:10:50 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 53%|█████▎    | 32000/60000 [2:05:14<1:23:04,  5.62it/s, E=-1.1, E_std=0.00385, E_var=2.88e-5, pmove=0.53297734]2023-12-09 02:13:55 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 55%|█████▌    | 33000/60000 [2:08:18<1:20:15,  5.61it/s, E=-1.1, E_std=0.00263, E_var=7.37e-6, pmove=0.5326904]2023-12-09 02:16:58 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 57%|█████▋    | 34000/60000 [2:11:19<1:17:42,  5.58it/s, E=-1.1, E_std=0.00363, E_var=2.9e-5, pmove=0.53212893]2023-12-09 02:20:00 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 58%|█████▊    | 35000/60000 [2:14:24<1:14:33,  5.59it/s, E=-1.11, E_std=0.00338, E_var=1.83e-5, pmove=0.53356326]2023-12-09 02:23:04 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 58%|█████▊    | 35000/60000 [2:14:24<1:14:33,  5.59it/s, E=-1.11, E_std=0.00299, E_var=1.07e-5, pmove=0.53430784][35000] creating checkpoint
 60%|██████    | 36000/60000 [2:17:30<1:11:05,  5.63it/s, E=-1.1, E_std=0.00321, E_var=1.75e-5, pmove=0.5345215]2023-12-09 02:26:10 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 62%|██████▏   | 37000/60000 [2:20:31<1:08:33,  5.59it/s, E=-1.1, E_std=0.00332, E_var=1.98e-5, pmove=0.53378296]2023-12-09 02:29:12 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 63%|██████▎   | 38000/60000 [2:23:32<1:05:31,  5.60it/s, E=-1.1, E_std=0.00312, E_var=1.41e-5, pmove=0.53267825]2023-12-09 02:32:13 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 65%|██████▌   | 39000/60000 [2:26:35<1:02:43,  5.58it/s, E=-1.1, E_std=0.00303, E_var=1.25e-5, pmove=0.53477174]2023-12-09 02:35:16 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 67%|██████▋   | 40000/60000 [2:29:36<59:41,  5.58it/s, E=-1.1, E_std=0.00293, E_var=1.35e-5, pmove=0.53344727]2023-12-09 02:38:17 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 67%|██████▋   | 40000/60000 [2:29:37<59:41,  5.58it/s, E=-1.1, E_std=0.00346, E_var=2.41e-5, pmove=0.53345335][40000] creating checkpoint
 68%|██████▊   | 41000/60000 [2:32:42<56:40,  5.59it/s, E=-1.1, E_std=0.00282, E_var=9.83e-6, pmove=0.5316833]2023-12-09 02:41:23 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 70%|███████   | 42000/60000 [2:35:46<53:52,  5.57it/s, E=-1.1, E_std=0.00365, E_var=2.69e-5, pmove=0.53312993]2023-12-09 02:44:27 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 72%|███████▏  | 43000/60000 [2:38:48<50:51,  5.57it/s, E=-1.1, E_std=0.00229, E_var=5.73e-6, pmove=0.5336548]2023-12-09 02:47:29 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 73%|███████▎  | 44000/60000 [2:41:50<47:45,  5.58it/s, E=-1.1, E_std=0.00327, E_var=1.94e-5, pmove=0.53303224]2023-12-09 02:50:31 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 75%|███████▌  | 45000/60000 [2:44:51<44:46,  5.58it/s, E=-1.1, E_std=0.00242, E_var=7.22e-6, pmove=0.53007203]2023-12-09 02:53:32 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 75%|███████▌  | 45000/60000 [2:44:52<44:46,  5.58it/s, E=-1.1, E_std=0.00328, E_var=1.57e-5, pmove=0.53167117][45000] creating checkpoint
 77%|███████▋  | 46000/60000 [2:47:57<41:51,  5.57it/s, E=-1.1, E_std=0.00235, E_var=6.81e-6, pmove=0.53195804]2023-12-09 02:56:37 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 78%|███████▊  | 47000/60000 [2:51:00<38:59,  5.56it/s, E=-1.1, E_std=0.00217, E_var=5.32e-6, pmove=0.5342041]2023-12-09 02:59:40 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 80%|████████  | 48000/60000 [2:54:02<35:37,  5.61it/s, E=-1.1, E_std=0.0023, E_var=6.82e-6, pmove=0.5297302]2023-12-09 03:02:42 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 82%|████████▏ | 49000/60000 [2:57:05<32:54,  5.57it/s, E=-1.09, E_std=0.00212, E_var=5.36e-6, pmove=0.53096926]2023-12-09 03:05:46 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 83%|████████▎ | 50000/60000 [3:00:09<29:38,  5.62it/s, E=-1.1, E_std=0.00231, E_var=6.53e-6, pmove=0.53167725]2023-12-09 03:08:49 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 83%|████████▎ | 50000/60000 [3:00:09<29:38,  5.62it/s, E=-1.1, E_std=0.00287, E_var=1.2e-5, pmove=0.53486943] [50000] creating checkpoint
 85%|████████▌ | 51000/60000 [3:03:13<26:53,  5.58it/s, E=-1.1, E_std=0.00248, E_var=8.79e-6, pmove=0.5317627]2023-12-09 03:11:53 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 87%|████████▋ | 52000/60000 [3:06:15<30:37,  4.35it/s, E=-1.1, E_std=0.00212, E_var=5.34e-6, pmove=0.532367]2023-12-09 03:14:55 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 88%|████████▊ | 53000/60000 [3:09:17<20:59,  5.56it/s, E=-1.1, E_std=0.00304, E_var=2.22e-5, pmove=0.5289795]2023-12-09 03:17:58 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 90%|█████████ | 54000/60000 [3:12:21<17:56,  5.57it/s, E=-1.1, E_std=0.0021, E_var=4.94e-6, pmove=0.53134155]2023-12-09 03:21:01 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 92%|█████████▏| 55000/60000 [3:15:23<14:55,  5.58it/s, E=-1.1, E_std=0.00197, E_var=4.73e-6, pmove=0.5349182]2023-12-09 03:24:03 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 92%|█████████▏| 55000/60000 [3:15:23<14:55,  5.58it/s, E=-1.1, E_std=0.00236, E_var=9.55e-6, pmove=0.53149414][55000] creating checkpoint
 93%|█████████▎| 56000/60000 [3:18:26<12:00,  5.55it/s, E=-1.1, E_std=0.00369, E_var=2.58e-5, pmove=0.53092045]2023-12-09 03:27:07 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 95%|█████████▌| 57000/60000 [3:21:29<08:58,  5.57it/s, E=-1.1, E_std=0.00265, E_var=1.14e-5, pmove=0.53120124]2023-12-09 03:30:09 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 97%|█████████▋| 58000/60000 [3:24:30<06:01,  5.54it/s, E=-1.11, E_std=0.00233, E_var=6.36e-6, pmove=0.5334717]2023-12-09 03:33:11 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
 98%|█████████▊| 59000/60000 [3:27:32<02:58,  5.59it/s, E=-1.1, E_std=0.00273, E_var=1.37e-5, pmove=0.5321838]2023-12-09 03:36:13 (WARNING): Tracking a matplotlib object using "aim.Figure" might not behave as expected.In such cases, consider tracking with "aim.Image".
100%|██████████| 60000/60000 [3:30:37<00:00,  4.75it/s, E=-1.11, E_std=0.00244, E_var=8.71e-6, pmove=0.5347779]  
Evaluating final energy
Computing Energy: 100%|██████████| 123/123 [01:59<00:00,  1.03it/s, E=-1.012566(6)]
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.114847(7)]49, E_err=6.47e-6]
Computing Energy: 100%|██████████| 123/123 [01:53<00:00,  1.09it/s, E=-1.159060(4)]5, E_err=7.32e-6] 
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.173584(4)]7, E_err=3.86e-6]
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.1723436(32)] E_err=4.11e-6]
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.1626763(35)] E_err=3.22e-6]
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.148697(4)]E_err=3.49e-6]   
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.1328236(35)]err=3.79e-6]
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.1164778(31)]_err=3.49e-6]
Computing Energy: 100%|██████████| 123/123 [01:52<00:00,  1.09it/s, E=-1.1005371(29)]err=3.09e-6] 
2023-12-09 05:10:38 (INFO): Result: {'E_final': [-1.012566089630127, -1.11484694480896, -1.1590598821640015, -1.173583745956421, -1.1723438501358032, -1.1626760959625244, -1.148697018623352, -1.1328238248825073, -1.1164777278900146, -1.100536823272705, -1.0855448246002197, -1.071805715560913, -1.0594877004623413, -1.048685073852539, -1.039363980293274, -1.0314862728118896], 'E_final_std': [0.006494690198451281, 0.00734989857301116, 0.0038714748807251453, 0.004126369953155518, 0.0032370712142437696, 0.0035024748649448156, 0.0038031628355383873, 0.003507445566356182, 0.0031044657807797194, 0.0029472357127815485, 0.0026276602875441313, 0.0028136828914284706, 0.0028574015013873577, 0.002692325972020626, 0.002870911965146661, 0.004232349805533886], 'E_final_err': [6.470098795458199e-06, 7.322069021139649e-06, 3.856815983062527e-06, 4.110745924400338e-06, 3.224814413639831e-06, 3.4892131437163674e-06, 3.788762593636266e-06, 3.4941650241343023e-06, 3.0927110755111616e-06, 2.936076341215851e-06, 2.617710951842896e-06, 2.8030292023742407e-06, 2.846582276807499e-06, 2.68213178708745e-06, 2.8600415847381517e-06, 4.216324496166491e-06], 'E_gnn': [-1.0126067399978638, -1.1148062944412231, -1.1589748859405518, -1.17353355884552, -1.1723034381866455, -1.1626331806182861, -1.1486608982086182, -1.1327881813049316, -1.1164401769638062, -1.1004892587661743, -1.0854833126068115, -1.071752905845642, -1.0594615936279297, -1.0486533641815186, -1.0393187999725342, -1.0314669609069824], 'GNN_MAE': 4.3332576751708984e-05}
2023-12-09 05:10:38 (INFO): Completed after 5:07:53
2023-12-09 05:10:38 (ERROR): Traceback (most recent call last):
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/sacred/run.py", line 429, in _final_call
    getattr(observer, method)(**kwargs)
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/seml/observers.py", line 364, in completed_event
    requests.post(self.webhook_url, data=json.dumps(data), headers=headers)
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/home/shhgroup/shanghui/miniconda3/envs/pesnet2/lib/python3.10/site-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL 'YOUR_WEBHOOK': No scheme supplied. Perhaps you meant https://YOUR_WEBHOOK?```

xiazhuozhao avatar Dec 09 '23 07:12 xiazhuozhao