til icon indicating copy to clipboard operation
til copied to clipboard

How to make the non-production Postgres planner behave like in production

Open xluffy opened this issue 1 year ago • 0 comments

Notes:

  • The planner's behavior doesn't depend on the actual resources available such as CPU or RAM. Nor does it depend on OS, FS or their settings.
  • The value of shared_buffers doesn't matter(!) – it will only affect the executor's behavior and buffer pool's hit/read ratio. What does matter for the planner is effective_cache_size and you can set it to a value that significantly exceed the actual RAM available, "fooling" the planner in a good sense, achieving the goal to match the production planner behavior. So you can have, say, 1 TiB of RAM and shared_buffers = '250GB' in production and effective_cache_size = '750GB', and be able to effectively analyze and optimize queries on a small 8-GiB machine with shared_buffers = '2GB' and effective_cache_size = '750GB'. The planner will assume you have a lot of RAM when choosing the optimal plans.
  • Similarly, you can have fast SSDs in production and use random_page_cost = 1.1 there, and very slow cheap magnetic disks in your testing environments. With random_page_cost = 1.1 being used there, the planner will consider random access to be not costly. The execution will be still slow, but the plan chosen and the data volumes (actual rows, buffer numbers) will be the same or very close to production.

https://twitter.com/samokhvalov/status/1726887992340738324

xluffy avatar Nov 21 '23 11:11 xluffy