cmdstanr icon indicating copy to clipboard operation
cmdstanr copied to clipboard

possibly speed up `save_object`

Open stemangiola opened this issue 1 year ago • 2 comments

save_object is slow for big models (~10K parameters)

Possible improvement 1

qs2 is a very fast format for saving and reading data. For huge models, fit$save_object() is slow in the rds format.

It would be great to ass the qs2 (or qs) option to save_object.

Possible improvement 2

allow save_object to save a set of parameters. Would this speed up the function?

Possible improvement 3

If we had the option of saving or reading draws as DuckDB, making the summary of one parameter would not require the loading of the whole posterior draws.

stemangiola avatar Nov 20 '24 04:11 stemangiola

Yeah, it's definitely slow for big models. We do have a section in the vignette (contributed by @wlandau) showing how to do it with qs::qsave(), but there's not a built-in option at the moment:

https://mc-stan.org/cmdstanr/articles/cmdstanr.html#saving-fitted-model-objects

jgabry avatar Nov 20 '24 17:11 jgabry

For now https://github.com/stan-dev/cmdstanr/pull/1042 will point to this section of the vignette from the documentation of save_object. But I agree that these are good suggestions!

jgabry avatar Nov 20 '24 18:11 jgabry