hpx icon indicating copy to clipboard operation
hpx copied to clipboard

Published 20 hours ago • verified publisher icon STEllAR-GROUP

Optimization on LCI parcelport: uses LCI_putva

Open JiakunYan opened this issue 2 years ago • 4 comments

Background

LCI_putva: put iovec allocate

  • LCI_putva transfers an iovec, which includes one medium message and multiple long messages.
  • It is a one-sided put operation, which means the target side doesn't need to post a recv. Instead, a completion entry will be pushed to a preconfigured LCI completion queue when all messages arrive.
  • Unlike RDMA, the sender side doesn't need to specify a remote registered buffer. LCI runtime will allocate and register the receive buffer.

Changes on HPX parcelport layer

  • No tag management on HPX layer, so no mutex lock.
  • Only one "posting send and testing" on the sender side. (It can be further optimized.)
  • No "posting receive and testing" on the receive side, so receiver_connection can be removed.

Status

Ongoing

  • Implementing a naive version: it uses std::memcpy to copy data from LCI buffers to HPX buffers.

JiakunYan avatar May 04 '22 03:05 JiakunYan

@JiakunYan since we have finally finished the release, is there anything you'd still like to do here before merging it?

hkaiser avatar May 25 '22 21:05 hkaiser

Performance test report

HPX Performance

Comparison

BENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTOR
For Each(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2022-05-31T12:57:29+00:002022-08-25T22:12:48+00:00
HPX Commitd5655f89770ba953e8de3ceb349567d5fcb74a95b578e682619cf00e63d51f444fed64d56a51c1db
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Clusternamedaintdaint
Datetime2022-05-31T15:13:01.357969+02:002022-08-26T00:33:53.397124+02:00
Envfile
Hostnamenid01193nid00041

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch(=)

Info

PropertyBeforeAfter
HPX Datetime2022-05-31T12:57:29+00:002022-08-25T22:12:48+00:00
HPX Commitd5655f89770ba953e8de3ceb349567d5fcb74a95b578e682619cf00e63d51f444fed64d56a51c1db
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Clusternamedaintdaint
Datetime2022-05-31T15:13:18.026239+02:002022-08-26T00:34:10.013849+02:00
Envfile
Hostnamenid01193nid00041

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add--(=)
Stream Benchmark - Scale(=)-(=)
Stream Benchmark - Triad-(=)-
Stream Benchmark - Copy(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2021-11-10T19:14:21+00:002022-08-25T22:12:48+00:00
HPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01ab578e682619cf00e63d51f444fed64d56a51c1db
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Clusternamedaintdaint
Datetime2021-11-10T20:28:18.266961+01:002022-08-26T00:34:25.534244+02:00
Envfile
Hostnamenid00120nid00041

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

StellarBot avatar Aug 25 '22 22:08 StellarBot

Performance test report

HPX Performance

Comparison

BENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTOR
For Each(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2022-05-31T12:57:29+00:002022-09-13T17:41:07+00:00
HPX Commitd5655f89770ba953e8de3ceb349567d5fcb74a958375ca8b9b604db0e7147ceb72a5cba92ded1a46
Envfile
Datetime2022-05-31T15:13:01.357969+02:002022-09-13T19:56:25.735256+02:00
Hostnamenid01193nid00875
Clusternamedaintdaint
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch(=)

Info

PropertyBeforeAfter
HPX Datetime2022-05-31T12:57:29+00:002022-09-13T17:41:07+00:00
HPX Commitd5655f89770ba953e8de3ceb349567d5fcb74a958375ca8b9b604db0e7147ceb72a5cba92ded1a46
Envfile
Datetime2022-05-31T15:13:18.026239+02:002022-09-13T19:56:42.186507+02:00
Hostnamenid01193nid00875
Clusternamedaintdaint
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add(=)(=)(=)
Stream Benchmark - Scale(=)(=)(=)
Stream Benchmark - Triad-(=)(=)
Stream Benchmark - Copy(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2021-11-10T19:14:21+00:002022-09-13T17:41:07+00:00
HPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01a8375ca8b9b604db0e7147ceb72a5cba92ded1a46
Envfile
Datetime2021-11-10T20:28:18.266961+01:002022-09-13T19:56:57.653483+02:00
Hostnamenid00120nid00875
Clusternamedaintdaint
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

StellarBot avatar Sep 13 '22 17:09 StellarBot

@hkaiser Do you have any idea why the test "jenkins/lsu/gcc-12-release" failed?

JiakunYan avatar Sep 13 '22 20:09 JiakunYan

Performance test report

HPX Performance

Comparison

BENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTOR
For Each(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2022-05-31T12:57:29+00:002022-10-02T20:00:15+00:00
HPX Commitd5655f89770ba953e8de3ceb349567d5fcb74a95bbdf01c51a38f3911b5b8067820927e849a891ad
Envfile
Clusternamedaintdaint
Datetime2022-05-31T15:13:01.357969+02:002022-10-02T22:16:54.620267+02:00
Hostnamenid01193nid00457
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch(=)

Info

PropertyBeforeAfter
HPX Datetime2022-05-31T12:57:29+00:002022-10-02T20:00:15+00:00
HPX Commitd5655f89770ba953e8de3ceb349567d5fcb74a95bbdf01c51a38f3911b5b8067820927e849a891ad
Envfile
Clusternamedaintdaint
Datetime2022-05-31T15:13:18.026239+02:002022-10-02T22:17:11.365256+02:00
Hostnamenid01193nid00457
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add(=)-(=)
Stream Benchmark - Scale(=)(=)(=)
Stream Benchmark - Triad(=)(=)(=)
Stream Benchmark - Copy(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2021-11-10T19:14:21+00:002022-10-02T20:00:15+00:00
HPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01abbdf01c51a38f3911b5b8067820927e849a891ad
Envfile
Clusternamedaintdaint
Datetime2021-11-10T20:28:18.266961+01:002022-10-02T22:17:26.762919+02:00
Hostnamenid00120nid00457
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

StellarBot avatar Oct 02 '22 20:10 StellarBot

bors merge

hkaiser avatar Oct 08 '22 23:10 hkaiser

Build succeeded:

bors[bot] avatar Oct 09 '22 00:10 bors[bot]