FEAT: scalable metered reactivity

[hiiamboris]

TL;DR: ~10x faster reactivity with O(1) complexity and multiple fixes here.

Rationale.

Since hash lookups are very cheap compared to Red token evaluation time, I decided to scratch the idea of putting relations into on-change* and use a central reactors index (hashtable). As I expected it's even faster than select body-of :on-change* relations approach I used before. Tests show further 20% speedup, but the number is not reliable as I'm on another PC which may have better optimized CPU cache.

Performance

Main source of speedup in this PR is avoiding O(n) lookup times by splitting the unstructured index into 2:

• reactor [relations ...]
• reaction [source-objects ...]

All operations become O(1) and it really scales. You can see the benchmark results here: for reactors, for faces . I expect ~20-30% better timings with the fix for https://github.com/red/red/issues/4505 (it was fixed in this PR's snapshot).

There's still slight linearity O(n) for block reactions due to https://github.com/red/red/issues/4466 - once it is fixed I expect it to be nearly O(1) up to huge loads.

Other notes

Shallow reactors do not take ownership of assigned series anymore, so if some series is watched by a deep reactor, it can now safely be assigned to any number of shallow reactors

Objects not based on reactor! template can still use reactivity/check to embed themselves into reactivity framework (this part was also true in the original design)

I've also come to realize that reactivity/check should return blazingly fast for objects without relations. E.g. in my Spaces project I'd like to make objects possibly reactive, but there may be thousands of objects, 99% of which won't be reactive, only a few top-level ones. And I need performance cost of adding reactivity on top of it to be negligible. This implementation satisfies that need.

Bonus features are: better debugging output and the ability to collect statistics.

Debugging summary: system/reactivity/debug? now can be:

• off/none to turn debug output off
• true or any other truthy value to turn it on for most interesting output, namely added, removed, skipped (most valuable!) and queued reactions
• 'full (word) adds to yes a more extensive output: each fired reaction, checks for reactions, on-change events. Note: part of this output won't be shown in GUI console (as it'll crash with stack overflow). Use CLI console to see all the details.
• when it can find an object inside the global context, it will refer to it by word, not by the massive object [....] output, so it's much easier to tell what's what.

Metering summary:

• system/reactivity/metrics?: true turns on stats collection, false turns it off. Note: metering (when on) slows it down by ~20%, so use only when you need it.
• system/reactivity/metrics/reset zeroes all stats for a fresh start
• system/reactivity/metrics/show prints the full statistics, e.g.:

***** REACTIVITY METRICS REPORT *****
Metrics collection enabled?: true
Statistical counts:
    events triggered:    80015
    reactions fired:     50000 (immediately: 25005 , queued: 24995 )
    reactions skipped:   0
Time spent in reactions: 0:00:00
Time spent in reactivity:
    total:               0:00:09.55855
    adding relations:    0:00:03.80822 (preparations: 0:00:00.901052 )
    removing relations:  0:00:00.843048
    dispatching:         0:00:04.90728
    longest queue flush: 0:00:00.0020001
Peak values:
    maximum queue size:  5000
    maximum index size:  5000
    biggest relation:    2 reactors
    most used reactor:   5000 relations

I also made is much smarter: https://github.com/red/red/blob/a77d72c37f55805b1122ccfee0c2046552081358/environment/reactivity.red#L302-L309

I have a lot of notes on pitfalls, tricks. If this PR gets approved, I will write them down to a wiki.

And along the way: Fixes #4510, fixes #4507, fixes #4471, fixes #4176, fixes #4166, grants wish REP#75

[hiiamboris]

~~Another side effect of this PR - less GC load during face creation.~~ disregard this, it seems totally random (varies by 500% from time to time)

[hiiamboris]

Last commit makes tight (r/x: r/x + 1) loops ~5% slower but reduces reactor's memory consumption by a factor of four: from 5.7kB to 1.4kB (on master branch it's 4.5kB).

[greggirwin]

Thanks for continuing work on this. I hope @dockimbel will be able to review soon.

[hiiamboris]

I'd like to make Spaces reactive, but this has been ignored for almost two years already.

[greggirwin]

I think @dockimbel has looked at it, but wasn't sure about the design. Please weigh in when you can @dockimbel.