vespa icon indicating copy to clipboard operation
vespa copied to clipboard

Sort ASC by a field should give those results without that field value

Open nehajatav opened this issue 1 year ago • 7 comments

Describe the bug Currently none of sort ascending and sort descending are giving the right result

To Reproduce Add doc1 with field f1 as a, add doc2 with field f1 as b, add doc3 with field f1 missing

Expected behavior Ascending should give order doc3,doc1, doc2

nehajatav avatar May 03 '24 20:05 nehajatav

You have to provide the schema definition, 3 documents and the query and the result you get. This is ancient functionality that see extensive use, so you must provide something reproducible. If the value is missing behavior depends on type of field. Fx for strings, if the field is not set set it will be treated as an empty string.

baldersheim avatar May 03 '24 21:05 baldersheim

What is the behaviour if the field is of type long and some docs are missing that field?

nehajatav avatar May 08 '24 20:05 nehajatav

If the field is unset for a document, the sort value for that document will be lower than the value for any other document with a value set. See Quick Start, using Docker, for an easy way to experiment with Vespa at a small scale.

If you need another behaviour you can use a document processor to set a value when it is missing when ingesting data. See Fields

A field can not be defined with a default value. Use a document processor to assign a default to document put/update operations.

bjormel avatar May 09 '24 12:05 bjormel

Correct, if you sort ascending unset document will come first, descending it will come last.

baldersheim avatar May 09 '24 13:05 baldersheim

This is not something that we can change without a major version, and it is not clear what is correct either.

baldersheim avatar May 09 '24 13:05 baldersheim

Or is it ? @bratseth could we change this so sorting on unset values always sorted last, similar to what we do for NaN rank ?

baldersheim avatar May 09 '24 14:05 baldersheim

I would say people shouldn't reasonably depend on this. But that said, lots of people ask for ways to find documents with an unset value of some field. We have never told them about this way of doing it (as far as I know), but maybe some found it by themselves?

bratseth avatar May 09 '24 15:05 bratseth