FATE
FATE copied to clipboard
fate求交算法在1.9和1.11版本下的运行时间差异
我们测试了fate在1.92和1.11版本下的求交算法的运行时间 (本来准备测raw的,但是后面发现不支持raw,会默认改成ecdh算法) 但是测试结果显示fate1.11求交的时间是1.92的接近三倍,想请问下这是由于什么原因所导致的呢?
这是我们的作业配置
# host端
docker exec -it confs-9999_client_1 bash
cat > /tmp/upload.json <<EOF
{
"file": "/data/projects/fate/examples/data/10m.csv",
"id_delimiter": ",",
"head": 1,
"partition": 8,
"namespace": "test",
"table_name": "data_10m"
}
EOF
flow data upload -c /tmp/upload.json
# guest端
docker exec -it confs-10000_client_1 bash
cat > /tmp/upload.json <<EOF
{
"file": "/data/projects/fate/examples/data/5m.csv",
"id_delimiter": ",",
"head": 1,
"partition": 8,
"namespace": "test",
"table_name": "data_5m"
}
EOF
flow data upload -c /tmp/upload.json
cat > /tmp/dsl.json <<EOF
{
"components": {
"reader_0": {
"module": "Reader",
"output": {
"data": [
"data"
]
}
},
"data_transform_0": {
"module": "DataTransform",
"input": {
"data": {
"data": [
"reader_0.data"
]
}
},
"output": {
"data": [
"data"
],
"model": [
"model"
]
}
},
"intersection_0": {
"module": "Intersection",
"input": {
"data": {
"data": [
"data_transform_0.data"
]
}
},
"output": {
"data": [
"data"
]
}
}
}
}
EOF
cat > /tmp/conf.json <<EOF
{
"dsl_version": "2",
"initiator": {
"role": "guest",
"party_id": 10000
},
"role": {
"guest": [
10000
],
"host": [
9999
]
},
"job_parameters": {
"common": {
"task_parallelism": 1,
"auto_retries": 0,
"computing_partitions": 8,
"task_cores": 32
}
},
"component_parameters": {
"common": {
"intersection_0": {
"intersect_method": "raw",
"sync_intersect_ids": true,
"join_role": "host"
}
},
"role": {
"guest": {
"0": {
"reader_0": {
"table": {
"name": "data_5m",
"namespace": "test"
}
},
"data_transform_0": {
"with_label": false,
"output_format": "dense"
}
}
},
"host": {
"0": {
"reader_0": {
"table": {
"name": "data_10m",
"namespace": "test"
}
},
"data_transform_0": {
"with_label": false,
"output_format": "dense"
}
}
}
}
}
}
EOF
flow job submit -c /tmp/conf.json -d /tmp/dsl.json
1.11哪个版本呢?从数据表名看是guest: 10m vs host: 5m?
1.11哪个版本呢?从数据表名看是guest: 10m vs host: 5m?
是kube-fate docker 1.11 数据量是guest: 5m vs host: 10m
这是对应1.11.1?
这是对应1.11.1?
是的,是1.11.1
在测试样例的里面你也出现这个情况吗?
这是对应1.11.1?
是的,是1.11.1
想问一下你用的机子是什么配置
我也测试了下,发现两个版本任务时间确实差很多,这个是结果:
版本 | 任务时间 |
---|---|
1.11.1 | 01:07:13 |
1.9.2 | 00:26:14 |
不同版本的配置没有太大区别,不确定是哪里配置还是是代码更新的原因
请问不用flowboard的话,在哪里查看求交的结果呢