math/rand: reconsider lock-free source

[oakad]

trafficstars

Hereby, I would like to petition to reopen and salvage #18514. There are several areas where having a global, performant and lock free random source is of very high utility.

1. Development/testing of concurrent apps/algorithms (and Go is all about concurrency). One of the most potent techniques in this area requires placing random delays all over the code to emulate uneven progress of concurrent goroutines. Present implementation of rand.Int() with a global lock in its path all but negates the usefulness of this approach.

2. Zero communication "unfair" sharing. Go is commonly used to implement various proxies, load balancers and other similar gadgets, which require "unfair" sharing of resources across multiple instances (example applications include AB testing, canary deployments, various types of hot failovers and so on). This can be efficiently achieved by means of random selection with custom probability distribution and very efficient algorithms were developed for this purpose (such as Walker-Vose O(1) uneven sampling algorithm). Unfortunately, "unfair" selection can only be as performant as the underlying uniform random source is.

3. "Locally distributed" data structures. A good, simple example of those is Java's java.util.concurrent.atomic.LongAccumulator and friends (however, same technique can be applied to other similarly constructed objects, such as concurrent pools, multiple producer queues, and so on). sync.Pool cunningly uses the private fastrand(), we, the users, want one too! :-)

In most other languages these problems are resolved by means of thread local PRNGs. In my opinion, Go should also expose one, or, rather, make its global PRNG instance behave like one. It does not even requires any changes to the language, because at present users have no ability to affect the private rand.globalRand object in any way and thus can have no preference on what sort of pseudo-random sequence is returned from gloabal rand functions.

[DeedleFake]

Go purposefully has no thread-local storage system, and it even purposefully avoids exposing anything that would make it easy to implement one, like IDs. One could be implemented for something like this, but it would be preferable for people to just instantiate their own rand.Rands and keep track of them themselves, I think.

That being said, as you point out it wouldn't exactly be terrible for the global rand.Rand instance to behave that way. It could be considered a breaking change, however, and especially since it can be relatively easily worked around, I don't think this is likely to get accepted. I could be wrong, though.

[oakad]

Of course, I don't want this to be about thread local, it's a long and difficult topic (just mentioned it for the context).

All I personally need is exported and inlineable runtime.Fastrand(), but it may be called rand.Uint32() just the same. Even though having a similar 64b version will be nice too.

The previous proposal was accepted. And not, it's not easy to achieve the same behavior with rand.Rand, you're welcome to try. It's not for the fainthearted though. :-)

[DeedleFake]

In that case, while I would certainly not be against exporting fastrand() in some way, it should be noted that you can use a //go:linkname directive to get direct access to it. Maybe the math/rand package can just do something like

//go:linkname Runtime runtime.fastrand
func Runtime() uint32

And I very much know how difficult it is to deal with performance and concurrency with rand.Rand. I had to once and it caused me all sorts of problems. I wound up faking it, as it didn't really matter that much for what I was doing. Speaking of which, I still need to go clean that mess up... I haven't touched it in years.

[oakad]

I know about the go:linkname hack. Everybody who's doing the sort of thing I enumerated in my OP does. :-)

My point here that time has come to make it nice and official. In fact, this was the case even back in 2017, pretty unfortunate that the previous proposal was closed, with implementation almost ready and reviewed.

[robpike]

See https://github.com/golang/go/issues/21835

The exp/rand package might help you today.

[cespare]

Using exp/rand makes it cheaper to create per-goroutine sources, but sometimes it is not convenient to structure the code this way.

I still think the best solution here is to implement #18802 in some form. Then people can create their own CPU-local RNGs easily enough.

Failing that, I think adding a CPU-local RNG in the stdlib would be good. It doesn't have the API concerns of a more general sharding mechanism: the per-CPU-ness isn't user-observable except insofar that the global functions don't see a huge slowdown in highly parallel contexts.

I have an experimental demonstration of one #18802 solution at github.com/cespare/percpu and it includes a wrapper around exp/rand as github.com/cespare/percpu/clrand, in case anyone would like to see how these ideas could look and perform. (But beware that there's a go:linkname hack under the hood.)

[oakad]

We can either make sharded values with fast random sources or we can make fast random sources with sharded values. Sort of like chickens and eggs. :-)

[ianlancetaylor]

There are lots of things we could shared on a per-goroutine or per-thread or per-P basis (a P is the entity counted by GOMAXPROCS). For example, we've built magic into sync.Pool to shard it across P's (but that's not so bad since sync.Pool is also magic with respect to garbage collection). It's not clear to me that random numbers are important enough to shard in this away. Or, perhaps the desire to shard them suggests that there should be some general mechanism to support sharding. While there are natural concerns with sharding across goroutines, as that can easily lead to a complicated programming model, sharding across threads or P's is perhaps less bad; the lack of a persistent connection between a goroutine and a thread or P may sufficiently limit the effect on the programming model. Maybe.

[oakad]

What's wrong with defining user visible rand.Uint32() in terms of internal runtime.fastrand()? No fancy stuff, just that and call it a day.

This is the de-facto situation today with the linkname thing.

[josharian]

See also a slightly more general discussion of the interaction between concurrency and reproducibility guarantees at https://github.com/golang/go/issues/26263.

[ianlancetaylor]

@oakad I think that in effect you are suggesting that random numbers are important enough that we should shard them across P's. Why is that?

(The fact that the current implementation happens to have an existing mechanism for doing that isn't an argument for why it should be done in user space. We shouldn't in general reason from a particular implementation.)

[oakad]

(Original comment replaced with this. Sorry.)

I have went on to see what people do on Github in general. Right now, there are several common techniques to obtain fast randoms:

1. Linkname to runtime.fastrand. Generally, appears to be a favorite approach, but requires unsafe and generally suboptimal, because fastrand is not inlined.
2. sync.Pool of PRNGs seeded from something. This approach feels totally dubious to me, especially considering how sync.Pool is implemented. Sort of like always driving in reverse.
3. Fake "thread local" by means of syscall.Gettid. Various approaches possible, but again, a strong feeling of "driving in reverse".
4. Assembler (rdrand or rdtsc sometimes). We are at the mercy of the hardware platform, especially considering all the evils Intel had done to rdrand (on some CPUs it's actually very slow these days).

[oakad]

Somehow it feels that newly added maphash.MakeSeed could solve the present problem.

The users are allowed to call it at will affecting the generator state, but god forbid mere mortals from using that puny random value directly without jumping through hoops. :-)

[aclements]

Now that #54880 has been accepted and the implementation has landed, it should be possible to make the global source lock-free.

[rsc]

Now that math/rand defaults to a pre-seeded random generator, this is not an API change and does not need to be a proposal. (Of course, any implementation needs to respect the GODEBUG to get back to the old behavior, and it needs to disable the lock-free generator when rand.Seed has been called.)

[rsc]

Removed from the proposal process. This was determined not to be a “significant change to the language, libraries, or tools” or otherwise of significant importance or interest to the broader Go community. — rsc for the proposal review group

[ianlancetaylor]

My understanding is that the specific suggestion is that we change the functions that currently use globalRand such that if

• the GODEBUG setting randautoseed=0 is not used, and
• nothing has called rand.Seed

then those functions will use a more efficient lock-free random number generator.

The affected functions would be: Int63, Uint32, Uint64, Int31, Int, Int63n, Int31n, Intn, Float64, Float32, Perm, Shuffle, Read, NormFloat64, ExpFloat64.

Those functions will become some sort of atomic load to see if we can call the fast random number generator, followed by that call.

[oakad]

"it should be possible to make the global source lock-free"

How creating another level of PRNG indirection in some unspecified future is any better than simply exposing fastrand?

[flyingmutant]

If you just want to access runtime.fastrand64, it is possible even without unsafe and with relatively low overhead. I've used this trick in my take on rand:

import "hash/maphash"

func rand64() uint64 {
	return maphash.Bytes(maphash.MakeSeed(), nil)
}

[gopherbot]

Change https://go.dev/cl/465037 mentions this issue: math/rand: use fastrand64 if possible