Cannot add index twice for off-heap and disk persistence

[bluedoo]

Using on-heap I can:

 this.collection = new ConcurrentIndexedCollection<>(OnHeapPersistence.onPrimaryKey(Foo.ID_INDEX));
 this.collection.addIndex(NavigableIndex.onAttribute(Foo.ID_INDEX));

However, if I try the same thing with disk persistence or off-heap then I get:

java.lang.IllegalStateException: An identity index for persistence has already been added, and no additional non-heap indexes are allowed, on attribute: Attribute{objectType=class Foo, attributeType=class java.lang.String, attributeName='<Unnamed attribute, class Foo$2>'}

Off-Heap attempt:

 this.collection= new ConcurrentIndexedCollection<>(OffHeapPersistence.onPrimaryKey(Foo.ID_INDEX));
 this.collection.addIndex(OffHeapIndex.onAttribute(Foo.ID_INDEX));

Disk attempt:

        this.collection= new ConcurrentIndexedCollection<>(DiskPersistence.onPrimaryKeyInFile(Foo.ID_INDEX, file));
        this.collection.addIndex(DiskIndex.onAttribute(Foo.ID_INDEX));

Can you please explain why it works like this?

[npgall]

This is expected, but I agree it's confusing! I think the CQEngine documentation should explain this better.

Basically, when the collection itself is stored off-heap, it is implicitly stored in a clustered index on the primary key. So effectively, if you try to add another index on the primary key, you will get this exception which says there already is such an index. For more information on clustered versus non-clustered indexes see http://stackoverflow.com/questions/1251636/what-do-clustered-and-non-clustered-index-actually-mean

When the collection is stored on the heap, the objects are not actually stored in any particular index by default (because they can be accessed via object references into the heap instead).

Anyway sorry that the situation is confusing. I'll keep this issue open until I've had a chance to explain it better in the documentation. Thanks!

[bluedoo]

So ideally we don't want to add the index again on the on-heap collection? It sounds like this will result in a decrease in performance

[npgall]

If the collection is on the heap: you can add that index and doing so might be beneficial If the collection is not on the heap (it's off-heap or on disk): you don't need to add that index, as there will be an equivalent index already

[bluedoo]

Okay that makes sense... I'll leave this open for you to close since you mentioned you wanted to add some info to the docs...

[npgall]

Yes I will close it when the docs are updated. Thanks bluedoo