mindyolo
mindyolo copied to clipboard
mindspore-lab
mindspore/core/ir/func_graph_extends.cc:139 GenerateKwParams
完整的日志如下：
(MindSpore) [ma-user mindyolo]$python train.py --config ./configs/yolov8/yolov8l.yaml
2023-07-13 16:47:25,258 [INFO] parse_args:
2023-07-13 16:47:25,258 [INFO] device_target Ascend
2023-07-13 16:47:25,258 [INFO] save_dir ./runs/2023.07.13-16.47.25
2023-07-13 16:47:25,258 [INFO] device_per_servers 8
2023-07-13 16:47:25,258 [INFO] log_level INFO
2023-07-13 16:47:25,258 [INFO] is_parallel False
2023-07-13 16:47:25,258 [INFO] ms_mode 0
2023-07-13 16:47:25,258 [INFO] ms_amp_level O0
2023-07-13 16:47:25,258 [INFO] keep_loss_fp32 True
2023-07-13 16:47:25,258 [INFO] ms_loss_scaler static
2023-07-13 16:47:25,258 [INFO] ms_loss_scaler_value 1024.0
2023-07-13 16:47:25,258 [INFO] ms_grad_sens 1024.0
2023-07-13 16:47:25,258 [INFO] ms_jit True
2023-07-13 16:47:25,258 [INFO] ms_enable_graph_kernel False
2023-07-13 16:47:25,258 [INFO] ms_datasink False
2023-07-13 16:47:25,258 [INFO] overflow_still_update True
2023-07-13 16:47:25,258 [INFO] ema True
2023-07-13 16:47:25,258 [INFO] weight
2023-07-13 16:47:25,258 [INFO] ema_weight
2023-07-13 16:47:25,258 [INFO] freeze []
2023-07-13 16:47:25,258 [INFO] epochs 500
2023-07-13 16:47:25,258 [INFO] per_batch_size 1
2023-07-13 16:47:25,258 [INFO] img_size 640
2023-07-13 16:47:25,258 [INFO] nbs 64
2023-07-13 16:47:25,258 [INFO] accumulate 1
2023-07-13 16:47:25,258 [INFO] auto_accumulate False
2023-07-13 16:47:25,258 [INFO] log_interval 100
2023-07-13 16:47:25,258 [INFO] single_cls False
2023-07-13 16:47:25,258 [INFO] sync_bn False
2023-07-13 16:47:25,258 [INFO] keep_checkpoint_max 100
2023-07-13 16:47:25,258 [INFO] run_eval False
2023-07-13 16:47:25,258 [INFO] conf_thres 0.001
2023-07-13 16:47:25,258 [INFO] iou_thres 0.7
2023-07-13 16:47:25,258 [INFO] conf_free True
2023-07-13 16:47:25,258 [INFO] rect False
2023-07-13 16:47:25,258 [INFO] nms_time_limit 20.0
2023-07-13 16:47:25,258 [INFO] recompute False
2023-07-13 16:47:25,258 [INFO] recompute_layers 0
2023-07-13 16:47:25,258 [INFO] seed 2
2023-07-13 16:47:25,258 [INFO] summary True
2023-07-13 16:47:25,258 [INFO] profiler False
2023-07-13 16:47:25,258 [INFO] profiler_step_num 1
2023-07-13 16:47:25,258 [INFO] opencv_threads_num 0
2023-07-13 16:47:25,258 [INFO] enable_modelarts False
2023-07-13 16:47:25,258 [INFO] data_url
2023-07-13 16:47:25,258 [INFO] ckpt_url
2023-07-13 16:47:25,258 [INFO] multi_data_url
2023-07-13 16:47:25,258 [INFO] pretrain_url
2023-07-13 16:47:25,258 [INFO] train_url
2023-07-13 16:47:25,258 [INFO] data_dir /cache/data/
2023-07-13 16:47:25,258 [INFO] ckpt_dir /cache/pretrain_ckpt/
2023-07-13 16:47:25,258 [INFO] data.dataset_name coco
2023-07-13 16:47:25,258 [INFO] data.train_set /home/ma-user/work/data0713-2/train.txt
2023-07-13 16:47:25,258 [INFO] data.val_set /home/ma-user/work/data0713-2/val.txt
2023-07-13 16:47:25,258 [INFO] data.test_set /home/ma-user/work/data0713-2/tval.txt
2023-07-13 16:47:25,258 [INFO] data.nc 5
2023-07-13 16:47:25,258 [INFO] data.names ['crack', 'crul', 'dent', 'material', 'nick']
2023-07-13 16:47:25,258 [INFO] train_transforms.stage_epochs [490, 10]
2023-07-13 16:47:25,258 [INFO] train_transforms.trans_list [[{'func_name': 'mosaic', 'prob': 1.0, 'degrees': 0.0, 'translate': 0.1, 'scale': 0.9, 'shear': 0.0, 'copy_paste_prob': 0.3}, {'func_name': 'mixup', 'prob': 0.15, 'alpha': 32.0, 'beta': 32.0, 'needed_mosaic': True}, {'func_name': 'label_norm', 'xyxy2xywh_': True}, {'func_name': 'albumentations'}, {'func_name': 'hsv_augment', 'prob': 1.0, 'hgain': 0.015, 'sgain': 0.7, 'vgain': 0.4}, {'func_name': 'fliplr', 'prob': 0.5}, {'func_name': 'label_pad', 'padding_size': 160, 'padding_value': -1}, {'func_name': 'image_norm', 'scale': 255.0}, {'func_name': 'image_transpose', 'bgr2rgb': True, 'hwc2chw': True}], [{'func_name': 'letterbox', 'scaleup': True}, {'func_name': 'random_perspective', 'prob': 1.0, 'degrees': 0.0, 'translate': 0.1, 'scale': 0.9, 'shear': 0.0}, {'func_name': 'label_norm', 'xyxy2xywh_': True}, {'func_name': 'albumentations'}, {'func_name': 'hsv_augment', 'prob': 1.0, 'hgain': 0.015, 'sgain': 0.7, 'vgain': 0.4}, {'func_name': 'fliplr', 'prob': 0.5}, {'func_name': 'label_pad', 'padding_size': 160, 'padding_value': -1}, {'func_name': 'image_norm', 'scale': 255.0}, {'func_name': 'image_transpose', 'bgr2rgb': True, 'hwc2chw': True}]]
2023-07-13 16:47:25,258 [INFO] data.test_transforms [{'func_name': 'letterbox', 'scaleup': False}, {'func_name': 'label_norm', 'xyxy2xywh_': True}, {'func_name': 'label_pad', 'padding_size': 160, 'padding_value': -1}, {'func_name': 'image_norm', 'scale': 255.0}, {'func_name': 'image_transpose', 'bgr2rgb': True, 'hwc2chw': True}]
2023-07-13 16:47:25,258 [INFO] data.num_parallel_workers 4
2023-07-13 16:47:25,258 [INFO] optimizer.optimizer momentum
2023-07-13 16:47:25,258 [INFO] optimizer.lr_init 0.01
2023-07-13 16:47:25,258 [INFO] optimizer.momentum 0.937
2023-07-13 16:47:25,258 [INFO] optimizer.nesterov True
2023-07-13 16:47:25,258 [INFO] optimizer.loss_scale 1.0
2023-07-13 16:47:25,258 [INFO] optimizer.warmup_epochs 3
2023-07-13 16:47:25,258 [INFO] optimizer.warmup_momentum 0.8
2023-07-13 16:47:25,258 [INFO] optimizer.warmup_bias_lr 0.1
2023-07-13 16:47:25,258 [INFO] optimizer.min_warmup_step 1000
2023-07-13 16:47:25,258 [INFO] optimizer.group_param yolov8
2023-07-13 16:47:25,258 [INFO] optimizer.gp_weight_decay 0.0005
2023-07-13 16:47:25,258 [INFO] optimizer.start_factor 1.0
2023-07-13 16:47:25,258 [INFO] optimizer.end_factor 0.01
2023-07-13 16:47:25,258 [INFO] optimizer.epochs 500
2023-07-13 16:47:25,258 [INFO] optimizer.nbs 64
2023-07-13 16:47:25,258 [INFO] optimizer.accumulate 1
2023-07-13 16:47:25,258 [INFO] optimizer.total_batch_size 1
2023-07-13 16:47:25,258 [INFO] loss.name YOLOv8Loss
2023-07-13 16:47:25,258 [INFO] loss.box 7.5
2023-07-13 16:47:25,258 [INFO] loss.cls 0.5
2023-07-13 16:47:25,258 [INFO] loss.dfl 1.5
2023-07-13 16:47:25,258 [INFO] loss.reg_max 16
2023-07-13 16:47:25,258 [INFO] network.model_name yolov8
2023-07-13 16:47:25,258 [INFO] network.nc 5
2023-07-13 16:47:25,258 [INFO] network.reg_max 16
2023-07-13 16:47:25,258 [INFO] network.stride [8, 16, 32]
2023-07-13 16:47:25,258 [INFO] network.backbone [[-1, 1, 'ConvNormAct', [64, 3, 2]], [-1, 1, 'ConvNormAct', [128, 3, 2]], [-1, 3, 'C2f', [128, True]], [-1, 1, 'ConvNormAct', [256, 3, 2]], [-1, 6, 'C2f', [256, True]], [-1, 1, 'ConvNormAct', [512, 3, 2]], [-1, 6, 'C2f', [512, True]], [-1, 1, 'ConvNormAct', [1024, 3, 2]], [-1, 3, 'C2f', [1024, True]], [-1, 1, 'SPPF', [1024, 5]]]
2023-07-13 16:47:25,258 [INFO] network.head [[-1, 1, 'Upsample', ['None', 2, 'nearest']], [[-1, 6], 1, 'Concat', [1]], [-1, 3, 'C2f', [512]], [-1, 1, 'Upsample', ['None', 2, 'nearest']], [[-1, 4], 1, 'Concat', [1]], [-1, 3, 'C2f', [256]], [-1, 1, 'ConvNormAct', [256, 3, 2]], [[-1, 12], 1, 'Concat', [1]], [-1, 3, 'C2f', [512]], [-1, 1, 'ConvNormAct', [512, 3, 2]], [[-1, 9], 1, 'Concat', [1]], [-1, 3, 'C2f', [1024]], [[15, 18, 21], 1, 'YOLOv8Head', ['nc', 'reg_max', 'stride']]]
2023-07-13 16:47:25,258 [INFO] network.depth_multiple 1.0
2023-07-13 16:47:25,258 [INFO] network.width_multiple 1.0
2023-07-13 16:47:25,258 [INFO] network.max_channels 512
2023-07-13 16:47:25,258 [INFO] config ./configs/yolov8/yolov8l.yaml
2023-07-13 16:47:25,258 [INFO] rank 0
2023-07-13 16:47:25,258 [INFO] rank_size 1
2023-07-13 16:47:25,258 [INFO] total_batch_size 1
2023-07-13 16:47:25,258 [INFO] callback []
2023-07-13 16:47:25,258 [INFO]
2023-07-13 16:47:25,261 [INFO] Please check the above information for the configurations
2023-07-13 16:47:26,263 [WARNING] Parse Model, args: nearest, keep str type
2023-07-13 16:47:26,355 [WARNING] Parse Model, args: nearest, keep str type
2023-07-13 16:47:26,801 [INFO] number of network params, total: 43.680162M, trainable: 43.633679M
2023-07-13 16:47:48,740 [WARNING] Parse Model, args: nearest, keep str type
2023-07-13 16:47:48,833 [WARNING] Parse Model, args: nearest, keep str type
2023-07-13 16:47:49,293 [INFO] number of network params, total: 43.680162M, trainable: 43.633679M
2023-07-13 16:47:57,842 [INFO] ema_weight not exist, default pretrain weight is currently used.
2023-07-13 16:47:58,212 [INFO] Dataset cache file hash/version check fail.
2023-07-13 16:47:58,212 [INFO] Datset caching now...
Scanning '/home/ma-user/work/data0713-2/train.cache' images and labels... 1
2023-07-13 16:48:06,279 [INFO] New cache created: /home/ma-user/work/data0713-2/train.cache.npy
2023-07-13 16:48:06,280 [INFO] Dataset caching success.
2023-07-13 16:48:06,521 [INFO] Dataloader num parallel workers: [4]
2023-07-13 16:48:06,991 [INFO] Dataset Cache file hash/version check success.
2023-07-13 16:48:06,991 [INFO] Load dataset cache from [/home/ma-user/work/data0713-2/train.cache.npy] success.
Scanning '/home/ma-user/work/data0713-2/train.cache.npy' images and labels.
2023-07-13 16:48:07,722 [INFO] Dataloader num parallel workers: [4]
2023-07-13 16:48:34,348 [INFO] Registry(name=callback, total=4)
2023-07-13 16:48:34,348 [INFO] (0): YoloxSwitchTrain in mindyolo/utils/callback.py
2023-07-13 16:48:34,348 [INFO] (1): EvalWhileTrain in mindyolo/utils/callback.py
2023-07-13 16:48:34,348 [INFO] (2): SummaryCallback in mindyolo/utils/callback.py
2023-07-13 16:48:34,348 [INFO] (3): ProfilerCallback in mindyolo/utils/callback.py
2023-07-13 16:48:34,348 [INFO]
2023-07-13 16:48:37,155 [INFO] got 1 active callback as follows:
2023-07-13 16:48:37,155 [INFO] SummaryCallback()
2023-07-13 16:48:37,155 [WARNING] The first epoch will be compiled for the graph, which may take a long time; You can come back later :).
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
[WARNING] package not installed, albumentations load failed
Traceback (most recent call last):
File "/home/ma-user/work/mindyolo/train.py", line 309, in
- The Traceback of Net Construct Code:
The function call stack (See file '/home/ma-user/work/mindyolo/rank_0/om/analyze_fail.dat' for more details. Get instructions about analyze_fail.dat at https://www.mindspore.cn/search?inputValue=analyze_fail.dat):
0 In file /home/ma-user/work/mindyolo/mindyolo/utils/train_step_factory.py:72
return train_step_func(*args)
^
1 In file /home/ma-user/work/mindyolo/mindyolo/utils/train_step_factory.py:57
if optimizer_update:
2 In file /home/ma-user/work/mindyolo/mindyolo/utils/train_step_factory.py:52
(loss, loss_items), grads = grad_fn(x, label)
^
3 In file /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/ops/composite/base.py:504
return grad_(fn, weights)(*args)
^
4 In file /home/ma-user/work/mindyolo/mindyolo/utils/train_step_factory.py:44
pred = network(x)
^
5 In file /home/ma-user/work/mindyolo/mindyolo/models/yolov8.py:36
return self.model(x)
^
6 In file /home/ma-user/work/mindyolo/mindyolo/models/model_factory.py:65
for i in range(len(self.model)):
7 In file /home/ma-user/work/mindyolo/mindyolo/models/yolov8.py:36
return self.model(x)
^
8 In file /home/ma-user/work/mindyolo/mindyolo/models/model_factory.py:65
for i in range(len(self.model)):
9 In file /home/ma-user/work/mindyolo/mindyolo/models/yolov8.py:36
return self.model(x)
^
10 In file /home/ma-user/work/mindyolo/mindyolo/models/model_factory.py:65
for i in range(len(self.model)):
11 In file /home/ma-user/work/mindyolo/mindyolo/models/yolov8.py:36
return self.model(x)
^
12 In file /home/ma-user/work/mindyolo/mindyolo/models/model_factory.py:81
x = m(x) # run
^
13 In file /home/ma-user/work/mindyolo/mindyolo/models/layers/bottleneck.py:86
x_tuple = ops.split(x, axis=1, split_size_or_sections=_c)
^
- C++ Call Stack: (For framework developers)
mindspore/core/ir/func_graph_extends.cc:139 GenerateKwParams
Mind2.0 ---- Replied Message ---- | From | @.> | | Date | 07/17/2023 09:37 | | To | mindspore-lab/mindyolo @.> | | Cc | lixinyu @.>, Author @.> | | Subject | Re: [mindspore-lab/mindyolo] mindspore/core/ir/func_graph_extends.cc:139 GenerateKwParams (Issue #170) |
What environment does the problem occur in?
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
You can try to run the following command to check the MindSpore version and verify whether it is installed properly
pip show mindspore
cat /path_to/mindspore/.commit_id
python
>>> import mindspore as ms
>>> ms.run_check()