hpx icon indicating copy to clipboard operation
hpx copied to clipboard

Published 20 hours ago • verified publisher icon STEllAR-GROUP

Adding feature, Range Based Lock

Open Johan511 opened this issue 1 year ago • 12 comments

Adding feature, Range Based lock to HPX

API: range_lock rl; lockId = rl.lock(rangeBegin, rangeEnd); rl.unlock(lockId);

Pending Task: Adding tests

Johan511 avatar Nov 01 '23 20:11 Johan511

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
:white_check_mark: -84.56% :white_check_mark: ∅
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (89f342bc9267b89ceaa8a5f71775f90983b8068c) 0 0 84.59%
Head commit (14e182e0cd62086eae4fecec279b93a3c5379897) 153777 (+153777) 51 (+51) 0.03% (-84.56%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6379) 0 0 ∅ (not applicable)

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

codacy-production[bot] avatar Nov 01 '23 23:11 codacy-production[bot]

Performance test report

HPX Performance

Comparison

BENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTOR
For Each(=)??(=)

Info

PropertyBeforeAfter
HPX Commitdcb541576898d370113946ba15fb58c20c8325b29da61de2f3f1673886cbc916056d19a3ac2bc167
HPX Datetime2023-05-10T12:07:53+00:002023-11-02T04:22:07+00:00
Datetime2023-05-10T14:50:18.616050-05:002023-11-01T23:30:22.792853-05:00
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Clusternamerostamrostam
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1
Envfile

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch(=)

Info

PropertyBeforeAfter
HPX Commitdcb541576898d370113946ba15fb58c20c8325b29da61de2f3f1673886cbc916056d19a3ac2bc167
HPX Datetime2023-05-10T12:07:53+00:002023-11-02T04:22:07+00:00
Datetime2023-05-10T14:52:35.047119-05:002023-11-01T23:32:35.779661-05:00
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Clusternamerostamrostam
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1
Envfile

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add(=)(=)(=)
Stream Benchmark - Scale(=)(=)(=)
Stream Benchmark - Triad(=)(=)(=)
Stream Benchmark - Copy(=)-(=)

Info

PropertyBeforeAfter
HPX Commitdcb541576898d370113946ba15fb58c20c8325b29da61de2f3f1673886cbc916056d19a3ac2bc167
HPX Datetime2023-05-10T12:07:53+00:002023-11-02T04:22:07+00:00
Datetime2023-05-10T14:52:52.237641-05:002023-11-01T23:32:52.616083-05:00
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Clusternamerostamrostam
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1
Envfile

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

StellarBot avatar Nov 02 '23 04:11 StellarBot

More comprehensive tests on the way @hkaiser. Will add them soon. Was planning something similar to what we have for shared_mutex.

Johan511 avatar Nov 02 '23 17:11 Johan511

@hkaiser do we have a concurrent hash map like the one by boost or the one by folly?

Johan511 avatar Dec 08 '23 00:12 Johan511

@hkaiser do we have a concurrent hash map like the one by boost or the one by folly?

No we don't have something like that.

hkaiser avatar Dec 08 '23 00:12 hkaiser

@Johan511 I know that currently the reporting of the build logs for the HPX CIs is broken (we're working on fixing this). For this reason, here is the error message that makes the builders fail:

/.../hpx/libs/core/synchronization/include/hpx/synchronization/range_mutex.hpp:71:25: error: no member named 'get' in 'hpx::synchronization::detail::range_mutex<hpx::detail::spinlock<true>, std::lock_guard>'

hkaiser avatar Jan 27 '24 20:01 hkaiser

Performance test report

HPX Performance

Comparison

BENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTOR
For Each(=)??-

Info

PropertyBeforeAfter
HPX Commitdcb541576898d370113946ba15fb58c20c8325b26834baab450742da0b0694281ff770008ca9ec91
HPX Datetime2023-05-10T12:07:53+00:002024-02-07T09:11:33+00:00
Envfile
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1
Clusternamerostamrostam
Datetime2023-05-10T14:50:18.616050-05:002024-02-07T03:20:02.039903-06:00

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch=

Info

PropertyBeforeAfter
HPX Commitdcb541576898d370113946ba15fb58c20c8325b26834baab450742da0b0694281ff770008ca9ec91
HPX Datetime2023-05-10T12:07:53+00:002024-02-07T09:11:33+00:00
Envfile
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1
Clusternamerostamrostam
Datetime2023-05-10T14:52:35.047119-05:002024-02-07T03:22:15.846511-06:00

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add(=)(=)(=)
Stream Benchmark - Scale+(=)=
Stream Benchmark - Triad(=)=(=)
Stream Benchmark - Copy(=)(=)(=)

Info

PropertyBeforeAfter
HPX Commitdcb541576898d370113946ba15fb58c20c8325b26834baab450742da0b0694281ff770008ca9ec91
HPX Datetime2023-05-10T12:07:53+00:002024-02-07T09:11:33+00:00
Envfile
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1
Clusternamerostamrostam
Datetime2023-05-10T14:52:52.237641-05:002024-02-07T03:22:32.733787-06:00

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

StellarBot avatar Feb 07 '24 09:02 StellarBot

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
:white_check_mark: +0.02% :white_check_mark: 98.02%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (17bdd0d04b7d06be59e017cdcb125b1078d31c81) 206733 176202 85.23%
Head commit (600b4a29e8e621932e935e15811d5076bab2e562) 207519 (+786) 176912 (+710) 85.25% (+0.02%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6379) 101 99 98.02%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

codacy-production[bot] avatar Feb 07 '24 12:02 codacy-production[bot]

I have not looked at the implementation (I will examine it later), but I'm curious as to the use case for the range-lock. Is this work inspired by the GSoC project I proposed a few years ago (with an astrophysics particle code in mind), or do you have another use case that requires it? (or perhaps it was "just for fun" as an interesting project to try?)

biddisco avatar Feb 08 '24 13:02 biddisco

Hi @biddisco, this is a fun piece of work inspired by the GSoC project you had proposed. I didn't have an use-case in mind, I had tested out it's performance based on the amount of time it took to execute the test. It'd be great if I could test the impact of the code on a more real life scenario.

Johan511 avatar Feb 08 '24 19:02 Johan511

@hkaiser can you have a look at the CI? I am not able to view the logs on the jenkins CI. Also is there any reference I could use to write documentation? Maybe some sort of specialized mutex somewhere in HPX.

Johan511 avatar Feb 08 '24 19:02 Johan511

@hkaiser can you have a look at the CI? I am not able to view the logs on the jenkins CI. Also is there any reference I could use to write documentation? Maybe some sort of specialized mutex somewhere in HPX.

Sorry for the CI issues. We're currently working on a solution for this problem.

hkaiser avatar Feb 10 '24 22:02 hkaiser