elassandra icon indicating copy to clipboard operation
elassandra copied to clipboard

Published 20 hours ago • verified publisher icon strapdata

Unexpected error while indexing document with geo_shape field type

Open a-sergeichik opened this issue 4 years ago • 2 comments

Elassandra version: 6.8.4.10

Plugins installed: []

JVM version (java -version): Version 8 Update 251

OS version (uname -a if on a Unix-like system): WIN10

docker image: strapdata/elassandra:6.8.4.10

Description of the problem including expected versus actual behavior:

Unexpected error while indexing document with geo_shape field type

Steps to reproduce:

  1. Create a new table
CREATE TABLE test.example (
	id uuid,
	location text,
	PRIMARY KEY (id)
);
  1. Create an index in ElasticSearch
{
   "settings":{
      "keyspace":"test",
      "analysis":{
         "normalizer":{
            "keyword_lowercase":{
               "type":"custom",
               "filter":[
                  "lowercase"
               ]
            }
         }
      }
   },
   "mappings":{
      "example":{
         "_source":{
            "enabled":true
         },
         "properties":{
            "id":{
               "type":"keyword",
               "cql_collection":"singleton"
            },
            "location":{
               "type":"geo_shape",
               "cql_collection":"singleton"
            }
         }
      }
   }
}
  1. Insert a record into the table
insert into example (id, location) values (uuid(), '{"type":"Point","coordinates":[27.59765625,53.904338156274704]}')
  1. Unexpected error occurs
elassandra    | 2020-09-14 11:21:06,903 ERROR [MutationStage-2] org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.write(ElasticSecondaryIndex.java:2245) Unexpected error
elassandra    | org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [location] of type [geo_shape]
elassandra    |         at org.elasticsearch.index.mapper.GeoShapeFieldMapper.parse(GeoShapeFieldMapper.java:116)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex.addField(ElasticSecondaryIndex.java:367)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex.addField(ElasticSecondaryIndex.java:342)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.buildContext(ElasticSecondaryIndex.java:2187)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.buildContext(ElasticSecondaryIndex.java:1737)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.index(ElasticSecondaryIndex.java:2287)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.index(ElasticSecondaryIndex.java:2257)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.index(ElasticSecondaryIndex.java:1737)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.write(ElasticSecondaryIndex.java:2240)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.write(ElasticSecondaryIndex.java:1737)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer.update(ElasticSecondaryIndex.java:1778)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer.finish(ElasticSecondaryIndex.java:1958)
elassandra    |         at org.apache.cassandra.index.SecondaryIndexManager$WriteTimeTransaction.commit(SecondaryIndexManager.java:1435)
elassandra    |         at org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:167)
elassandra    |         at org.apache.cassandra.db.Memtable.put(Memtable.java:282)
elassandra    |         at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1356)
elassandra    |         at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:631)
elassandra    |         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:475)
elassandra    |         at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
elassandra    |         at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
elassandra    |         at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
elassandra    |         at org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1408)
elassandra    |         at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2705)
elassandra    |         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
elassandra    |         at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
elassandra    |         at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
elassandra    |         at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:113)
elassandra    |         at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
elassandra    |         at java.lang.Thread.run(Thread.java:748)
elassandra    | Caused by: java.lang.IllegalArgumentException: invalid shape type found [class org.locationtech.spatial4j.shape.jts.JtsPoint] while indexing shape
elassandra    |         at org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:160)
elassandra    |         at org.elasticsearch.index.mapper.GeoShapeFieldMapper.parse(GeoShapeFieldMapper.java:112)
elassandra    |         ... 28 common frames omitted

a-sergeichik avatar Sep 14 '20 11:09 a-sergeichik

A shape cannot be defined by one point, isn’t it ? => invalid shape type found [class org.locationtech.spatial4j.shape.jts.JtsPoint] while indexing shape

On 14 Sep 2020, at 13:24, a-sergeichik [email protected] wrote:

Elassandra version: 6.8.4.10

Plugins installed: []

JVM version (java -version): Version 8 Update 251

OS version (uname -a if on a Unix-like system): WIN10

docker image: strapdata/elassandra:6.8.4.10

Description of the problem including expected versus actual behavior:

Unexpected error while indexing document with geo_shape field type

Steps to reproduce:

Create new table CREATE TABLE test.example ( id uuid, location text, PRIMARY KEY (id) ); Create index in ElasticSearch { "settings": { "keyspace": "test", "analysis": { "normalizer": { "keyword_lowercase": { "type": "custom", "filter": [ "lowercase" ] } } } }, "mappings": { "example": { "_source": { "enabled": true }, "properties": { "id": { "type": "keyword", "cql_collection": "singleton" }, "location": { "type": "geo_shape", "cql_collection": "singleton" } } } } } Insert a record into the table insert into example (id, location) values (uuid(), '{"type":"Point","coordinates":[27.59765625,53.904338156274704]}') Unexpected error occurs elassandra | 2020-09-14 11:21:06,903 ERROR [MutationStage-2] org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.write(ElasticSecondaryIndex.java:2245) Unexpected error elassandra | org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [location] of type [geo_shape] elassandra | at org.elasticsearch.index.mapper.GeoShapeFieldMapper.parse(GeoShapeFieldMapper.java:116) elassandra | at org.elassandra.index.ElasticSecondaryIndex.addField(ElasticSecondaryIndex.java:367) elassandra | at org.elassandra.index.ElasticSecondaryIndex.addField(ElasticSecondaryIndex.java:342) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.buildContext(ElasticSecondaryIndex.java:2187) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.buildContext(ElasticSecondaryIndex.java:1737) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.index(ElasticSecondaryIndex.java:2287) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.index(ElasticSecondaryIndex.java:2257) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.index(ElasticSecondaryIndex.java:1737) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.write(ElasticSecondaryIndex.java:2240) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.write(ElasticSecondaryIndex.java:1737) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer.update(ElasticSecondaryIndex.java:1778) elassandra | at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer.finish(ElasticSecondaryIndex.java:1958) elassandra | at org.apache.cassandra.index.SecondaryIndexManager$WriteTimeTransaction.commit(SecondaryIndexManager.java:1435) elassandra | at org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:167) elassandra | at org.apache.cassandra.db.Memtable.put(Memtable.java:282) elassandra | at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1356) elassandra | at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:631) elassandra | at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:475) elassandra | at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) elassandra | at org.apache.cassandra.db.Mutation.apply(Mutation.java:232) elassandra | at org.apache.cassandra.db.Mutation.apply(Mutation.java:241) elassandra | at org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1408) elassandra | at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2705) elassandra | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) elassandra | at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165) elassandra | at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137) elassandra | at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:113) elassandra | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) elassandra | at java.lang.Thread.run(Thread.java:748) elassandra | Caused by: java.lang.IllegalArgumentException: invalid shape type found [class org.locationtech.spatial4j.shape.jts.JtsPoint] while indexing shape elassandra | at org.elasticsearch.index.mapper.GeoShapeFieldMapper.indexShape(GeoShapeFieldMapper.java:160) elassandra | at org.elasticsearch.index.mapper.GeoShapeFieldMapper.parse(GeoShapeFieldMapper.java:112) elassandra | ... 28 common frames omitted — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/strapdata/elassandra/issues/371, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOMPGJDI3UVAANXEPVO3X3SFX4N7ANCNFSM4RLPKQXA.

vroyer avatar Sep 14 '20 20:09 vroyer

ES 6.8.4 supports Point shape type https://www.elastic.co/guide/en/elasticsearch/reference/6.8/geo-shape.html#point Moreover it throws the same error for Polygon and other shape types

insert into example (id, location) values (uuid(), '{
        "type" : "polygon",
        "coordinates" : [
            [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]
        ]
    }');

--->

elassandra    | 2020-09-15 05:24:27,979 ERROR [Native-Transport-Requests-1] org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.write(ElasticSecondaryIndex.java:2245) Unexpected error
elassandra    | org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [location] of type [geo_shape]
elassandra    |         at org.elasticsearch.index.mapper.GeoShapeFieldMapper.parse(GeoShapeFieldMapper.java:116)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex.addField(ElasticSecondaryIndex.java:367)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex.addField(ElasticSecondaryIndex.java:342)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.buildContext(ElasticSecondaryIndex.java:2187)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.buildContext(ElasticSecondaryIndex.java:1737)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.index(ElasticSecondaryIndex.java:2287)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.index(ElasticSecondaryIndex.java:2257)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.index(ElasticSecondaryIndex.java:1737)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer$Rowcument.write(ElasticSecondaryIndex.java:2240)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer$SkinnyRowcument.write(ElasticSecondaryIndex.java:1737)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$SkinnyRowcumentIndexer.update(ElasticSecondaryIndex.java:1778)
elassandra    |         at org.elassandra.index.ElasticSecondaryIndex$ImmutableMappingInfo$RowcumentIndexer.finish(ElasticSecondaryIndex.java:1958)

At the same time, Elassandra 6.2.3.31 works well on the same data.

a-sergeichik avatar Sep 15 '20 05:09 a-sergeichik