GPU配置

[wangfeiyu-zerobug]

一机多卡怎么配置device啊 好像没看到相关介绍

[zhangjiajin]

@wangfeiyu-zerobug

参考如下配置： https://github.com/huawei-noah/vega/blob/master/docs/cn/user/config_reference.md#2-%E5%85%AC%E5%85%B1%E9%85%8D%E7%BD%AE%E9%A1%B9

将如下配置项配置为true后，使用本机所有GPU搜索模型：

parallel_search: True

若不希望使用所有的GPU，设置parallel_search: True后，还需要设置环境变量CUDA_VISIBLE_DEVICES，如：export CUDA_VISIBLE_DEVICES=0,1,2，使用三块GPU。

[wangfeiyu-zerobug]

似乎是并行计算库不能启动？ INFO:root:------------------------------------------------ INFO:root: Step: serial INFO:root:------------------------------------------------ INFO:root:master ip and port: 127.0.0.1:28703 INFO:root:Initializing cluster. Please wait. INFO:root:Dask-scheduler not start. Start dask-scheduler in master 127.0.0.1 ERROR:vega.core.pipeline.pipeline:Failed to run pipeline, message: [Errno 2] No such file or directory: 'dask-scheduler': 'dask-scheduler' ERROR:vega.core.pipeline.pipeline:Traceback (most recent call last): File "/root/.local/lib/python3.7/site-packages/vega/core/pipeline/pipeline.py", line 84, in run pipestep = PipeStep(name=step_name) File "/root/.local/lib/python3.7/site-packages/vega/core/pipeline/search_pipe_step.py", line 45, in init self.master = create_master(update_func=self.generator.update) File "/root/.local/lib/python3.7/site-packages/vega/core/scheduler/master_ops.py", line 44, in create_master master_instance = Master(**kwargs) File "/root/.local/lib/python3.7/site-packages/vega/core/scheduler/master.py", line 65, in init status = self.dask_env.start() File "/root/.local/lib/python3.7/site-packages/vega/core/scheduler/dask_env.py", line 119, in start self._start_dask() File "/root/.local/lib/python3.7/site-packages/vega/core/scheduler/dask_env.py", line 155, in _start_dask scheduler_p = run_scheduler(ip=master_ip, port=master_port, tmp_file=scheduler_file) File "/root/.local/lib/python3.7/site-packages/vega/core/scheduler/run_dask.py", line 56, in run_scheduler env=os.environ File "/opt/conda/lib/python3.7/subprocess.py", line 800, in init restore_signals, start_new_session) File "/opt/conda/lib/python3.7/subprocess.py", line 1551, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'dask-scheduler': 'dask-scheduler'

[wangfeiyu-zerobug]

pip install dask Requirement already satisfied: dask in /root/.local/lib/python3.7/site-packages (2022.2.0) Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.7/site-packages (from dask) (21.3) Requirement already satisfied: partd>=0.3.10 in /root/.local/lib/python3.7/site-packages (from dask) (1.3.0) Requirement already satisfied: toolz>=0.8.2 in /root/.local/lib/python3.7/site-packages (from dask) (0.12.0) Requirement already satisfied: fsspec>=0.6.0 in /root/.local/lib/python3.7/site-packages (from dask) (2022.8.2) Requirement already satisfied: cloudpickle>=1.1.1 in /opt/conda/lib/python3.7/site-packages (from dask) (2.1.0) Requirement already satisfied: pyyaml>=5.3.1 in /opt/conda/lib/python3.7/site-packages (from dask) (5.4.1) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging>=20.0->dask) (2.4.7) Requirement already satisfied: locket in /root/.local/lib/python3.7/site-packages (from partd>=0.3.10->dask) (1.0.0)

[zhangjiajin]

@wangfeiyu-zerobug

dask组件路径未生效。 两种办法，添加路径到$PATH中，或者重新登录到你所在系统，命令生效。 https://github.com/huawei-noah/vega/blob/master/docs/cn/user/faq.md#14-%E5%BC%82%E5%B8%B8-permissionerror-errno-13-permission-denied-dask-scheduler-filenotfounderror-errno-2-no-such-file-or-directory-dask-scheduler-dask-scheduler-%E6%88%96%E8%80%85-vega-command-not-found

[wangfeiyu-zerobug]

已解决！ 提个建议 :这个路径采用os.environ这个接口导入，我是在docker起的容器中运行的，所以这块解决方法可能不太一样，这块是不是可以做一些补充 另外 https://github.com/huawei-noah/vega/blob/master/docs/cn/user/config_reference.md#2-%E5%85%AC%E5%85%B1%E9%85%8D%E7%BD%AE%E9%A1%B9 -2-2.1中general的pytorch打错了

[zhangjiajin]

@wangfeiyu-zerobug

感谢你的建议。 请问你在容器中运行，解决方法是怎样的？

[wangfeiyu-zerobug]

jupyter命令行：%env PATH=/root/.local/bin:

[zhangjiajin]

@wangfeiyu-zerobug

谢谢，我们及时刷新。

[wangfeiyu-zerobug]

如果想利用已经搜索出的网络测试另一批数据，该怎么做呢？还需要通过pipline进行fulltrain嘛

[zhangjiajin]

是的，需要fullytrain，看下精度。

[wangfeiyu-zerobug]

那是需要重新训练整个搜索好的网络？ 目前没有提供yaml配置选项去调用模型参数只进行测试数据嘛

[zhangjiajin]

若搜索时的pipeline中包含了fullytrain，就不需要重新训练。 测试的代码可参考https://github.com/huawei-noah/vega/blob/39741b5ddd9623f0984599d7f52ea38ef6f253c1/vega/tools/inference.py

[wangfeiyu-zerobug]

File "testcode.py", line 147, in main() File "testcode.py", line 141, in main result = _infer(args, loader, model) File "testcode.py", line 50, in _infer return _infer_pytorch(args, model, loader) File "testcode.py", line 70, in _infer_pytorch infer_result = model(**batch) TypeError: FasterRCNN object argument after ** must be a mapping, not list 我之前采用SP-NAS搜索的网络 !python testcode.py -c '/workspace/wfyexp/vega/vega-master/examples/nas/sp_nas/tasks/0928.183347.496/output/fullytrain/desc_4.json' -m '/workspace/wfyexp/vega/vega-master/examples/nas/sp_nas/tasks/0928.183347.496/output/fullytrain/model_4.pth' -df "COCO" -dp '/workspace/data/upper/added_dataset_COCO_format' 数据集就是更换了test部分的数据，然后这里显示dataload的输出结果放入mode时出错 打印batch结果： [[tensor([[[0.5255, 0.5255, 0.5255, ..., 0.3216, 0.3216, 0.3216], [0.5255, 0.5255, 0.5255, ..., 0.3216, 0.3216, 0.3216], [0.5255, 0.5255, 0.5255, ..., 0.3216, 0.3216, 0.3216], ..., [0.8078, 0.8078, 0.8078, ..., 0.2824, 0.2824, 0.2824], [0.8078, 0.8078, 0.8078, ..., 0.2824, 0.2824, 0.2824], [0.8078, 0.8078, 0.8078, ..., 0.2824, 0.2824, 0.2824]],

    [[0.5647, 0.5647, 0.5647,  ..., 0.3373, 0.3373, 0.3373],
     [0.5647, 0.5647, 0.5647,  ..., 0.3373, 0.3373, 0.3373],
     [0.5647, 0.5647, 0.5647,  ..., 0.3373, 0.3373, 0.3373],
     ...,
     [0.8588, 0.8588, 0.8588,  ..., 0.2902, 0.2902, 0.2902],
     [0.8588, 0.8588, 0.8588,  ..., 0.2902, 0.2902, 0.2902],
     [0.8588, 0.8588, 0.8588,  ..., 0.2902, 0.2902, 0.2902]],

    [[0.6000, 0.6000, 0.6000,  ..., 0.3333, 0.3333, 0.3333],
     [0.6000, 0.6000, 0.6000,  ..., 0.3333, 0.3333, 0.3333],
     [0.6000, 0.6000, 0.6000,  ..., 0.3333, 0.3333, 0.3333],
     ...,
     [0.9216, 0.9216, 0.9216,  ..., 0.2784, 0.2784, 0.2784],
     [0.9216, 0.9216, 0.9216,  ..., 0.2784, 0.2784, 0.2784],
     [0.9216, 0.9216, 0.9216,  ..., 0.2784, 0.2784, 0.2784]]])], [{'boxes': tensor([[190.9997, 410.9996, 250.9997, 470.9997],
    [412.9995, 312.0001, 477.9995, 365.0001]]), 'labels': tensor([1, 1]), 'masks': tensor([[[0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0],
     ...,
     [0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0]],

    [[0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0],
     ...,
     [0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0],
     [0, 0, 0,  ..., 0, 0, 0]]], dtype=torch.uint8), 'image_id': tensor([1]), 'area': tensor([3600.0029, 3445.0051]), 'iscrowd': tensor([0, 0])}]]

为什么dataload出来会是map呢？

[zhangjiajin]

数据格式是否是COCO格式的？

另外对于检测，需要参考这个代码：https://github.com/huawei-noah/vega/blob/39741b5ddd9623f0984599d7f52ea38ef6f253c1/vega/tools/detection_inference.py