Run multiple MCMC chains with NumPyro without flags

[null-a]

See pyro-ppl/numpyro#364.

[fehiepsi]

@neerajprad Should we set num_host_device_count to 4 by default? If users run MCMC with a higher num_chains, then we still have a warning message which says that parallel will fall back to sequential and points to the right place numpyro.set_host_device_count for users. At least I have no knowledge about the side effects of having a higher default device count than 1, and having num_host_device_count=4 seems to work even for 1 CPU system (we can test this in Travis).

[neerajprad]

@fehiepsi - Do you mean setting it to 4 in brmp or in NumPyro? I think setting it to 4 for brmp makes sense (for CPU). In any case, I think we should also try to develop a better understanding of XLA's behavior when we change this environment flag. Even if it seems safe now, it is possible that XLA starts using this flag for some optimization in the future and users will be pushed into a regime of poor performance simply because the default value for the environment flag is different than the actual device count or how XLA expects this flag to be used.

[fehiepsi]

I meant in both libraries. :D

we should also try to develop a better understanding of XLA's behavior when we change this environment flag

Agreed! We should talk about this when we meet its developers. For now, let's keep the default behavior.