FATE
FATE copied to clipboard
关于bind一个csv格式的数据提交任务后报错
您好 bind table和upload table在使用上有什么不同吗? 我通过v1/table/bind这个接口绑定了一个csv格式的文件 然后通过
dsl:
{
"components":{
"featurescale_2":{
"output":{
"data":[
"data"
],
"model":[
"model"
]
},
"input":{
"data":{
"data":[
"datatransform_1.data"
]
}
},
"module":"FeatureScale"
},
"reader_0":{
"output":{
"data":[
"data"
]
},
"input":{},
"module":"Reader"
},
"datatransform_1":{
"output":{
"data":[
"data"
],
"model":[
"model"
]
},
"input":{
"data":{
"data":[
"reader_0.data"
]
}
},
"module":"DataTransform"
},
"homolr_3":{
"output":{
"data":[
"data"
],
"model":[
"model"
]
},
"input":{
"data":{
"train_data":[
"featurescale_2.data"
]
}
},
"module":"HomoLR"
}
}
}
conf:
{
"dsl_version": 2,
"initiator": {
"role": "guest",
"party_id": "10000"
},
"role": {
"guest": [
"10000"
],
"arbiter": [
"10000"
],
"host": [
"9999"
]
},
"component_parameters": {
"common": {
"datatransform_1": {
"with_label": true,
"output_format": "dense"
},
"featurescale_2": {
"method": "min_max_scale"
},
"homolr_3": {
"alpha": 0.01,
"optimizer": "sgd",
"batch_size": 320,
"learning_rate": 0.15,
"max_iter": 3,
"decay": 1.0,
"decay_sqrt": true
}
},
"role": {
"guest": {
"0": {
"reader_0": {
"table": {
"name": "breast_homo_guest",
"namespace": "experiment"
}
}
}
},
"host": {
"0": {
"reader_0": {
"table": {
"name": "breast_homo_host",
"namespace": "experiment"
}
}
}
}
}
}
}
提交训练任务 但是在datatransform模块报错:
[ERROR] [2024-04-17 14:13:28,198] [202404171401157397360] [28618:139967649003264] - [job_saver.update_task] [line:93]: role host party id 9999 task 202404171401157397360_datatransform_1 error report: Traceback (most recent call last):
File "xxx/fate_flow/worker/task_executor.py", line 210, in _run_
cpn_output = run_object.run(cpn_input)
File "xxx/federatedml/model_base.py", line 239, in run
self._run(cpn_input=cpn_input)
File "xxx/federatedml/model_base.py", line 315, in _run
this_data_output = func(*real_param)
File "xxx/federatedml/util/data_transform.py", line 1052, in fit
data_inst = self.transformer.read_data(data, "fit")
File "xxx/federatedml/util/data_transform.py", line 137, in read_data
abnormal_detection.empty_table_detection(input_data)
File "xxx/federatedml/util/abnormal_detection.py", line 29, in empty_table_detection
raise ValueError(f"Count of data_instance is 0: {data_instances}")
ValueError: Count of data_instance is 0: <fate_arch.computing.non_distributed.LocalData object at 0x7f9ddc04de20>
这种csv的数据是必须要通过upload的方式才可以在联邦学习里使用吗
bind绑定一个本地路径,是用的时候透传这个路径给到相应的算法组件,需要算法组件本身可以处理这种情况,当前应该只有homo_nn可以 upload是将本地数据转化成分布式的表,fate的所有算法组件都支持这个模式