SVision
SVision copied to clipboard
"Empty output in the score file!!! Program exit"
So, the error message is completely accurate. The score files in the "predict_results" folder are all zero bytes, but it doesn't give me an idea why.
first 1000 lines of log (with the SVision call):
[Wed Jul 26 23:59:33 2023]
rule svision:
input: /ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/HG00514.clr_reads.chm13.pb_aligned_sorted.bam, /ceph/project/davieslab/jdalglei/reference/chm13v2.0.fa
output: /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision, /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/HG00514.clr_reads_svision_cnn_model.ckpt
jobid: 0
reason: Missing output files: /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/HG00514.clr_reads_svision_cnn_model.ckpt, /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision
wildcards: sample=HG00514.clr_reads
resources: mem_mb=871293, mem_mib=830930, disk_mb=871293, disk_mib=830930, tmpdir=/var/scratch/jdalglei/86987
mkdir -p /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/;
/ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/HG00514.clr_reads.chm13.pb_aligned_sorted.bam -m /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/HG00514.clr_reads_svision_cnn_model.ckpt -g /ceph/project/davieslab/jdalglei/reference/chm13v2.0.fa -n HG00514.clr_reads -s 5 --graph --qname;
singularity run -B /ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/:/ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/,/ceph/project/davieslab/jdalglei/code/sm/snakemake-pacbio/:/ceph/project/davieslab/jdalglei/code/sm/snakemake-pacbio/,/var/scratch/jdalglei:/var/scratch/jdalglei,/usr/lib/locale/:/usr/lib/locale/,/ceph/project/Wellcome_Discovery/jdalglei/:/ceph/project/Wellcome_Discovery/jdalglei/,/ceph/project/davieslab/jdalglei/:/ceph/project/davieslab/jdalglei/ docker://jiadongxjtu/svision:latest SVision -o /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/ -b /ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/HG00514.clr_reads.chm13.pb_aligned_sorted.bam -m /ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/HG00514.clr_reads_svision_cnn_model.ckpt -g /ceph/project/davieslab/jdalglei/reference/chm13v2.0.fa -n HG00514.clr_reads -s 5 --graph --qname -t 60;
INFO: Using cached SIF image
[W::hts_idx_load3] The index file is older than the data file: /ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/HG00514.clr_reads.chm13.pb_aligned_sorted.bam.bai
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
[W::hts_idx_load3] The index file is older than the data file: /ceph/project/Wellcome_Discovery/jdalglei/data/pacbio/bam/HG00514.clr_reads.chm13.pb_aligned_sorted.bam.bai
last part of the of the log:
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2023-07-27 08:20:07.285730: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285738: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285738: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285737: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285739: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285738: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285766: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285742: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285760: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285739: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285740: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285790: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285751: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285744: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285743: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285749: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.289618: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.285785: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.293747: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.293697: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-07-27 08:20:07.690458: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.690458: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.690459: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.690458: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.690494: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.690465: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.690464: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.699477: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa562090 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699483: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa573c20 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699495: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa5875b0 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699510: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa51b600 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699510: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa523d50 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699554: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.699511: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa544a60 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699563: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.699583: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.699511: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa562120 executing computations on platform Host. Devices:
2023-07-27 08:20:07.699618: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.699637: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.699652: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.699675: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.709629: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.709630: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.709635: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.709635: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.709630: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.709635: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.709646: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713605: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713614: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713624: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713609: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713604: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713704: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996205000 Hz
2023-07-27 08:20:07.713750: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa538950 executing computations on platform Host. Devices:
2023-07-27 08:20:07.713841: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa523f20 executing computations on platform Host. Devices:
2023-07-27 08:20:07.713863: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.713993: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.714102: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa552cb0 executing computations on platform Host. Devices:
2023-07-27 08:20:07.714147: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa513b70 executing computations on platform Host. Devices:
2023-07-27 08:20:07.714312: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.714292: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.714932: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa552a30 executing computations on platform Host. Devices:
2023-07-27 08:20:07.714946: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa5389e0 executing computations on platform Host. Devices:
2023-07-27 08:20:07.715020: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.715032: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa51b560 executing computations on platform Host. Devices:
2023-07-27 08:20:07.715111: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.715147: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.719175: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa5623a0 executing computations on platform Host. Devices:
2023-07-27 08:20:07.719150: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa544b20 executing computations on platform Host. Devices:
2023-07-27 08:20:07.719228: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa544af0 executing computations on platform Host. Devices:
2023-07-27 08:20:07.719290: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.719279: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.719313: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.719392: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa523d20 executing computations on platform Host. Devices:
2023-07-27 08:20:07.719487: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.719756: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa51b1b0 executing computations on platform Host. Devices:
2023-07-27 08:20:07.719888: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:07.720017: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa544c60 executing computations on platform Host. Devices:
2023-07-27 08:20:07.720141: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2023-07-27 08:20:08.001815: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.001831: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.001867: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002162: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002186: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002198: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002198: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002201: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002206: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002225: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002212: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002249: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.002864: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.003190: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.003306: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.003325: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.003381: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.003381: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.005717: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2023-07-27 08:20:08.006535: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Empty output in the score file!!! Program exit
Waiting at most 5 seconds for missing files.
MissingOutputException in rule svision in file /ceph/project/davieslab/jdalglei/code/sm/snakemake-pacbio/Snakefile, line 1560:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/ceph/project/Wellcome_Discovery/jdalglei/results/sm_pacbio/HG00514.clr_reads/svision/HG00514.clr_reads_svision_cnn_model.ckpt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
The complete log:svision.sample=HG00514.clr_reads-86987.out.gz
Do you have outputs in the segments.bed file?
Do you have outputs in the segments.bed file? Hello, I have "segments.bed file" in the output, and some have content and some do not, but the same error is reported. May I ask why?
These two files are output log files. bed file in my "segments" folder has content, so does "graphs" folder, but files in "predict_results" folder are empty.
First: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor 2023-11-07 08:51:10.627888: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2023-11-07 08:51:11.417364: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2394710000 Hz 2023-11-07 08:51:11.493230: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x71236b0 executing computations on platform Host. Devices: 2023-11-07 08:51:11.493307: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , 2023-11-07 08:51:12.600218: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile. 2023-11-07 08:51:12.693543: W tensorflow/core/framework/allocator.cc:107] Allocation of 150994944 exceeds 10% of system memory. 2023-11-07 08:51:12.704326: W tensorflow/core/framework/allocator.cc:107] Allocation of 67108864 exceeds 10% of system memory. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. 2023-11-07 08:51:13.956801: W tensorflow/core/framework/allocator.cc:107] Allocation of 67108864 exceeds 10% of system memory. 2023-11-07 08:51:13.956895: W tensorflow/core/framework/allocator.cc:107] Allocation of 150994944 exceeds 10% of system memory. 2023-11-07 08:51:14.538817: W tensorflow/core/framework/allocator.cc:107] Allocation of 150994944 exceeds 10% of system memory. Empty output in the score file!!! Program exit
Second:
2023-11-06 15:45:43,133 [INFO ] ******************** Start SVision, version 1.3.8 ********************
2023-11-06 15:45:43,133 [INFO ] CMD: /usr/local/bin/SVision -o /container/path/output -b /container/path/data/merge_01.pbmm2.sorted.bam -m /T02Data/haoxp/trio/data/svision-cnn-model.ckpt -g /container/path/data/Homo_sapiens_assembly38.fasta -n TGS0731-1 -s 5 --graph --qname
2023-11-06 15:45:43,133 [INFO ] WORKDIR DIR: /container/path/output
2023-11-06 15:45:43,134 [INFO ] CNN MODEL: /T02Data/haoxp/trio/data/svision-cnn-model.ckpt
2023-11-06 15:45:43,567 [INFO ] INPUT BAM: /container/path/data/merge_01.pbmm2.sorted.bam
2023-11-06 15:45:43,681 [INFO ]
****************** Step1 Image coding and segmentation ******************
2023-11-06 15:46:31,466 [INFO ] Processing chr1:0-10000000, 4569 segments write to: chr1.segments.0.bed
2023-11-06 15:47:44,705 [INFO ] Processing chr1:10000000-20000000, 1346 segments write to: chr1.segments.1.bed
2023-11-06 15:48:20,442 [INFO ] Processing chr1:20000000-30000000, 1750 segments write to: chr1.segments.2.bed
2023-11-06 15:48:52,369 [INFO ] Processing chr1:30000000-40000000, 1217 segments write to: chr1.segments.3.bed
2023-11-06 15:49:20,989 [INFO ] Processing chr1:40000000-50000000, 1011 segments write to: chr1.segments.4.bed
2023-11-06 15:49:49,122 [INFO ] Processing chr1:50000000-60000000, 1345 segments write to: chr1.segments.5.bed
2023-11-06 15:50:19,710 [INFO ] Processing chr1:60000000-70000000, 986 segments write to: chr1.segments.6.bed
2023-11-06 15:50:50,045 [INFO ] Processing chr1:70000000-80000000, 1267 segments write to: chr1.segments.7.bed
Too many, omit.......
2023-11-07 08:49:39,422 [INFO ] Processing HLA-DRB115:01:01:01:0-11080, 0 segments write to: HLA-DRB115:01:01:01.segments.0.bed
2023-11-07 08:49:39,477 [INFO ] Processing HLA-DRB115:01:01:02:0-11571, 0 segments write to: HLA-DRB115:01:01:02.segments.0.bed
2023-11-07 08:49:39,531 [INFO ] Processing HLA-DRB115:01:01:03:0-11056, 0 segments write to: HLA-DRB115:01:01:03.segments.0.bed
2023-11-07 08:49:39,585 [INFO ] Processing HLA-DRB115:01:01:04:0-11056, 0 segments write to: HLA-DRB115:01:01:04.segments.0.bed
2023-11-07 08:49:39,639 [INFO ] Processing HLA-DRB115:02:01:0-10313, 0 segments write to: HLA-DRB115:02:01.segments.0.bed
2023-11-07 08:49:39,693 [INFO ] Processing HLA-DRB115:03:01:01:0-11567, 0 segments write to: HLA-DRB115:03:01:01.segments.0.bed
2023-11-07 08:49:39,747 [INFO ] Processing HLA-DRB115:03:01:02:0-11569, 0 segments write to: HLA-DRB115:03:01:02.segments.0.bed
2023-11-07 08:49:39,801 [INFO ] Processing HLA-DRB116:02:01:0-11005, 0 segments write to: HLA-DRB116:02:01.segments.0.bed
2023-11-07 08:51:06,286 [INFO ] [Coding finished]: Collect segment signatures, Cost time: 61522
2023-11-07 08:51:06,406 [INFO ]
****************** Step2 CNN prediction ******************
2023-11-07 08:51:09,092 [WARNING] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2023-11-07 08:51:13,040 [WARNING] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2023-11-07 09:15:42,123 [INFO ] [Prediction finished]: Predicting types, Cost time: 1475
Sorry for the delayed reply. I think the reason for 'empty score file ' is due to the loss of the deep learning model files of SVision. There are in total 3 model files: (1) svision-cnn-model.ckpt.data-00000-of-00001, (2) svision-cnn-model.ckpt.index and (3) svision-cnn-model.ckpt.meta.
It is crucial that all three of these files share the same prefix, such as 'svision-cnn-model.ckpt', and they must be located in the same folder, otherwise, the predicting process of SVision will terminate.
If you changed the model prefix, such as 'HG00514.clr_reads_svision_cnn_model', make sure you have changed all the three files
Sorry for the delayed reply. I think the reason for 'empty score file ' is due to the loss of the deep learning model files of SVision. There are in total 3 model files: (1) svision-cnn-model.ckpt.data-00000-of-00001, (2) svision-cnn-model.ckpt.index and (3) svision-cnn-model.ckpt.meta.
It is crucial that all three of these files share the same prefix, such as 'svision-cnn-model.ckpt', and they must be located in the same folder, otherwise, the predicting process of SVision will terminate.
If you changed the model prefix, such as 'HG00514.clr_reads_svision_cnn_model', make sure you have changed all the three files
Hello, I did not change the prefixes of the model files and made sure they were in a folder.
Sorry for the delayed reply. I think the reason for 'empty score file ' is due to the loss of the deep learning model files of SVision. There are in total 3 model files: (1) svision-cnn-model.ckpt.data-00000-of-00001, (2) svision-cnn-model.ckpt.index and (3) svision-cnn-model.ckpt.meta.
It is crucial that all three of these files share the same prefix, such as 'svision-cnn-model.ckpt', and they must be located in the same folder, otherwise, the predicting process of SVision will terminate.
If you changed the model prefix, such as 'HG00514.clr_reads_svision_cnn_model', make sure you have changed all the three files
Hello, what do you think is the problem? I tried again and got the same result. It should not be the network problem, so please help solve it.
The score file is generated in the prediction step. If the score file is empty, the most direct cause would be the model. Thus, would you please send us a part of your data. We are also wondering where is the issue.
Thanks for helping us debugging the program.
I'm experiencing the same problem. Are there any updates yet?
@Rickrk4 Could you please share your data and attach your log as well? Please list all your package (pip list
) under the python environment.
We know it should be the problem of loading the deep learning model, while we did not receive the data from user to debug. Thanks for helping us debug!