ObjectRepository.find(ObjectFilter) returning null entries

[Doc1faux]

Hi @anidotnet :)

I faced an issue today on a search with a filtered index which returned me null values while expecting only objects causing a crash on my side. I'm currently using v3.4.2 on Android. I investigated a bit and while the library is creating the FindResult in the SearchService.find(Filter) method, the Set<NitriteId> result was containing 17 entries while the NitriteMap<NitriteId, Document> underlyingMap was containing only 15. Then the toList method iterate on the 17 entries and add null entries for each entry it could not find in the underlyingMap. It is like the index was not up-to-date :/ I dived into the code and I could not understand beyond the IndexedSearchService.findEqual(String, Object) method where the NitriteMap<Comparable, ConcurrentSkipListSet<NitriteId>> indexMap is instantiated which retrieves me the 17 index entries while there are only 15 in DB.

For the moment, the only choice I have is to drop null entries after the toList method :/

[Doc1faux]

After applying this dirty fix, I've faced another issue when I wanted to remove objects with the same filtered index. DataService.remove(Filter, RemoveOptions) method searching objects to remove in the same way via SearchService.find(Filter), it returns the 17 indexed entries instead of the existing 15 and when it tries to get the document.getId(), obviously it falls in an NPE :/

[anidotnet]

Can you please post a reproducible code here which can successfully recreate the issue you are facing?

[Doc1faux]

I @anidotnet,

I'm trying to write some sample code for it but struggle to reproduce the issue as it happens rarely :/ But once it happens, I could reproduce it each time because the index is corrupted pointing deleted entries.

I will update the issue when I will have a 100% reproducible use case. Thanks for your patience.

[dveyarangi]

I have also encountered similar issue. It seems to be due to discrepancy between index and the underlyingMap. I could not reproduce this in a clean environment, because I do not understand how the db got to this shape. Attaching the existing DB and the code that yields nulls from find() Windows 11, Nitrite 3.4.4, JDK 8 NitriteNullPointerIssue.zip

I actually have encountered this issue when trying to call for collection.remove(Filter), which is rising a NullPointerException from within the Nitrite code, which makes it quite complicated to recover this db java.lang.NullPointerException: null at org.dizitart.no2.internals.DataService.remove(DataService.java:218) at org.dizitart.no2.internals.NitriteService.remove(NitriteService.java:459) at org.dizitart.no2.internals.DefaultNitriteCollection.remove(DefaultNitriteCollection.java:360) at org.dizitart.no2.internals.DefaultNitriteCollection.remove(DefaultNitriteCollection.java:349) at midas.core.catalog.store.NitriteIndexDataBase.removeBySubstrate(NitriteIndexDataBase.java:143) at midas.core.catalog.DBCatalogService.removeBySubstrate(DBCatalogService.java:393) at midas.core.store.MDStore.processSubstrate(MDStore.java:277) at midas.core.store.MDStore.watch(MDStore.java:187) at midas.core.util.WatchdogThread.run(WatchdogThread.java:60) at java.lang.Thread.run(Thread.java:748) PS: still an issue with Nitrite 4

[anidotnet]

Closed due to inactivity