PaddleRec
PaddleRec copied to clipboard
Published
20 hours ago •
PaddlePaddle
PaddlePaddle
tdm的示例跑不通
快速开始的step1就出错了，提示 ImportError: cannot import name 'index_dataset_pb2' from 'paddle.distributed.fleet.proto' (/Users/xxx/miniforge3/envs/paddle_env/lib/python3.9/site-packages/paddle/distributed/fleet/proto/init.py) 求帮忙看看@wangzhen38
我这边用的develop最新的包没复现问题，你试下Paddle 2.4.0 ，之前应该有人拿这个跑通过
试了一下还是不可以。会不会是因为我的机器是mac m1，编译的时候WITH_PSCORE参数是OFF，所以对应的文件没能copy过去，缺少index_dataset相关的文件
我今天也在跑这个，现在已经跑通了，其实没你们想象的那么复杂。 这就是罪魁祸首，这个proto里的定义没被写进去index_dataset_pb2.py里 https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/distributed/index_dataset/index_dataset.proto
解决思路：
- 下载index_dataset.proto, 安装protoc编译器
- 编译index_dataset.proto,
protoc index_dataset.proto --python_out .
我附上我编译好的代码：
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: index_dataset.proto
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x13index_dataset.proto\x12\x12paddle.distributed\"P\n\tIndexNode\x12\n\n\x02id\x18\x01 \x02(\x04\x12\x0f\n\x07is_leaf\x18\x02 \x02(\x08\x12\x13\n\x0bprobability\x18\x03 \x02(\x02\x12\x11\n\titem_name\x18\x04 \x01(\t\"*\n\x08TreeMeta\x12\x0e\n\x06height\x18\x01 \x02(\x05\x12\x0e\n\x06\x62ranch\x18\x02 \x02(\x05\"$\n\x06KVItem\x12\x0b\n\x03key\x18\x01 \x02(\x0c\x12\r\n\x05value\x18\x02 \x02(\x0c')
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'index_dataset_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:
DESCRIPTOR._options = None
_INDEXNODE._serialized_start=43
_INDEXNODE._serialized_end=123
_TREEMETA._serialized_start=125
_TREEMETA._serialized_end=167
_KVITEM._serialized_start=169
_KVITEM._serialized_end=205
# @@protoc_insertion_point(module_scope)
