OrderBy string is case sensitive - Needs unit tests

[mahgo]

We need unit tests written and working for both case sensitve and insensitive searching.

Original description:

Is there a way to order by a string case insensitively?

[Shazwazza]

I think this would probably be based on the analyzer used. What are you using?

[mahgo]

I'm using a custom analyzer - I needed to do this to be able to search for words with special characters in them:

public class CustomAnalyzer : StandardAnalyzer
    {

        private Util.Version matchVersion;

        public CustomAnalyzer() : base(Util.Version.LUCENE_29)
        {
            matchVersion = Util.Version.LUCENE_29;
        }

        public override TokenStream TokenStream(string fieldName, System.IO.TextReader reader)
        {
            TokenStream result = new StandardTokenizer(this.matchVersion, reader);
            result = new StandardFilter(result);
            result = new LowerCaseFilter(result);
            result = new StopFilter(true, result, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
            result = new ASCIIFoldingFilter(result);
            return result;
        }
    }

[mahgo]

Woops, I didn't mean to close this one.

[mahgo]

I noticed if you save the _Sort field with ToLower() on line 1278 of LuceneIndexer, this achieves this.

default:
                            field =
                                new Field(x.Key,
                                    x.Value,
                                    Field.Store.YES,
                                    lucenePolicy,
                                          Equals(lucenePolicy, Field.Index.NO) ? Field.TermVector.NO : Field.TermVector.YES
                                );
                            sortedField = new Field(SortedFieldNamePrefix + x.Key,
                                                    x.Value.ToLower(),
                                                    Field.Store.NO, //we don't want to store the field because we're only using it to sort, not return data
                                                    Field.Index.NOT_ANALYZED,
                                                    Field.TermVector.NO
                                );
                            break;

Is this the best way to do this? Maybe we could do something like this? <add Name="nodeName" EnableSorting="true" Type="STRING" CaseInsensitive="true" />

[Shazwazza]

Where are you seeing this code? This is the code for LuceneIndexer and it doesn't do this: https://github.com/Shazwazza/Examine/blob/master/src/Examine/LuceneEngine/Providers/LuceneIndexer.cs#L1283

Even in the history of this class I cannot see where that ToLower() would be.

[mahgo]

Sorry I don't think I was clear - I added ToLower myself as a temporary solution.

[Shazwazza]

I still feel like this could be achieved with a custom analyzer/filter for the fields you want to be case insensitive for sorting

[mahgo]

Do you have an example of this? Everything I've tried has not worked.

[PiotrKlys]

@mahgo if this topic still interests you, I strongly recommend you last Ismail's response from this thread https://our.umbraco.com/forum/extending-umbraco-and-using-the-api/88592-examine-ordering-results In my case adding field _Sort_myfield dedicated for sorting and sorting on myfield in search criteria solves the problem

[Shazwazza]

I know this is super old but yes there should be a unit test created for case insensitive and sensitive searching in the code. There's probably a few ways to acheive that. I will update the task