New error: Unable to calculate quantile regression for quantile .... Possibly to few (unique) data points / predictions.

[KuepferA]

Dear Florian,

I have been using DHARMa to do the model validation for my GLMMs run in glmmTMB.

The function simulationOutput <- simulateResiduals(fittedModel = m1, plot = T) always used to give me an output without any warning messages. I recently had to rerun my models for some additional plots. I reran the same models, and the model validation outputs were the same, but I am now suddenly getting the following warning message, for all my models.

Unable to calculate quantile regression for quantile 0.25. Possibly to few (unique) data points / predictions. Will be ommited in plots and significance calculations. Unable to calculate quantile regression for quantile 0.5. Possibly to few (unique) data points / predictions. Will be ommited in plots and significance calculations. Unable to calculate quantile regression for quantile 0.75. Possibly to few (unique) data points / predictions. Will be ommited in plots and significance calculations.

I am mostly confused as to why this warning come up now, when it didn't before. I reinstalled DHARMa in case there was some bug, but I am still getting the same message.

Any ideas?

Many thanks

[florianhartig]

Hi,

if the same code produces suddenly a different result, the most likely explanation is that you updated DHARMa or a package DHARMa depends on.

I suppose you don't have records of the R system before and after the error? Because of this type of issues, it's always good to save a session.info() (I do this in every Rmd I run)

The error itself is nothing to worry about, it is basically what it says, the qgam package didn't manage to calculate a reliable quantile line, the only thing that is surprising is that it wasn't there before.

F

[KuepferA]

Thank you for your very speedy reply. So am I right to assume that the actual outputs of KS test, Dispersion and outliers are still reliable for nbinom1 and nbinom2 models?

Many thanks.

This email and any files transmitted with it are confidential and intended solely for the use of the individual(s) or entity named above. If you have received this email in error you must not use, copy, distribute, disseminate or disclose the information contained in it. Please notify us immediately, delete the email from your computer system, and destroy any hard copies you may have created.

[florianhartig]

From the DHARMa side, nothing much has changed in this regard in the last months. Of course, I have no idea which version you were using before. Outlier test has seen some improvements recently. In any case, the latest DHARMa version will be the most reliable.

[KuepferA]

Excellent thank you.

[Sammie-Jo]

I had this problem as well after updating to 0.3.3.0. Dropped back down to DHARMA 0.2.4 and problem went away.

[florianhartig]

OK, that makes sense - I switched the function for calculating the quantile regressions to qgam with DHARMa 0.3.0, see NEWS.

It is true that qgam more easily gets problems with determining a line for a small number of data points, which is the error you are getting here, but on the other hand, unlike the previous function, qgam calculates also an uncertainty / p-value on the line.

From my perspective, it is better to know how uncertain the quantile line is, and if an uncertainty can't be calculated, to not calculate it at all (which is the behavior >= 0.3.0 ), rather than calculating a line which, however, may be very dubious. If the there is a small number of data points, the lines can get quite erratic / misleading, and I received a large number of questions here on the forum based on such pattern which seemed to me just overinterpretation of the lines.

So, in my mind, using the latest version of DHARMa is highly preferable, also because of a bunch of other changes. You can still eye-ball the residuals. If you have doubts about this, maybe post a picture of your residuals (0.3.3 and 0.2.4) here.