gisgraphy icon indicating copy to clipboard operation
gisgraphy copied to clipboard

Datastore optimizer

Open lucidlynx opened this issue 4 years ago • 2 comments

We almost finished importing the map, settled on the Datastore optimizer item. In the import logs this is:

[ERROR] [com.gisgraphy.importer.DatastoreOptimizer] [05 Mar 2021 19: 13: 30,335] (importBackgroundThread,): an error occured during optimization, we ignore it but you have to manually run it to have
 good performances: could not execute native bulk manipulation query; SQL [CREATE INDEX locationindexopenstreetmap ON openstreetmap USING GIST (location)]; nested exception is org.hibernate.except
ion.SQLGrammarException: could not execute native bulk manipulation query
org.springframework.dao.InvalidDataAccessResourceUsageException: could not execute native bulk manipulation query; SQL [CREATE INDEX locationindexopenstreetmap ON openstreetmap USING GIST (location
)]; nested exception is org.hibernate.exception.SQLGrammarException: could not execute native bulk manipulation query
        at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException (SessionFactoryUtils.java:643)
        at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException (HibernateAccessor.java:416)
        at org.springframework.orm.hibernate3.HibernateTemplate.doExecute (HibernateTemplate.java:416)
        at org.springframework.orm.hibernate3.HibernateTemplate.execute (HibernateTemplate.java:342)
        at com.gisgraphy.domain.repository.OpenStreetMapDao.createSpatialIndexes (OpenStreetMapDao.java:283)
        at com.gisgraphy.importer.DatastoreOptimizer.tearDown (DatastoreOptimizer.java:143)
        at com.gisgraphy.importer.AbstractSimpleImporterProcessor.process (AbstractSimpleImporterProcessor.java:314)
        at com.gisgraphy.importer.ImporterManager.importAll (ImporterManager.java:138)
        at com.gisgraphy.webapp.action.ImportAction.doImport (ImportAction.java:130)
        at com.gisgraphy.webapp.action.ImportAction.execute (ImportAction.java:165)
        at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke (Method.java:498)
        at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction (DefaultActionInvocation.java:441)
        at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly (DefaultActionInvocation.java:280)
        at org.apache.struts2.interceptor.BackgroundProcess $ 1.run (BackgroundProcess.java:57)
        at java.lang.Thread.run (Thread.java:748)
Caused by: org.hibernate.exception.SQLGrammarException: could not execute native bulk manipulation query
        at org.hibernate.exception.SQLStateConverter.convert (SQLStateConverter.java:92)
        at org.hibernate.exception.JDBCExceptionHelper.convert (JDBCExceptionHelper.java:66)
        at org.hibernate.engine.query.NativeSQLQueryPlan.performExecuteUpdate (NativeSQLQueryPlan.java:219)
        at org.hibernate.impl.SessionImpl.executeNativeUpdate (SessionImpl.java:1310)
        at org.hibernate.impl.SQLQueryImpl.executeUpdate (SQLQueryImpl.java:396)
        at com.gisgraphy.domain.repository.OpenStreetMapDao $ 3.doInHibernate (OpenStreetMapDao.java:299)
        at org.springframework.orm.hibernate3.HibernateTemplate.doExecute (HibernateTemplate.java:411)
        ... 15 more
Caused by: org.postgresql.util.PSQLException: ERROR: relation "locationindexopenstreetmap" already exists
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse (QueryExecutorImpl.java:1525)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults (QueryExecutorImpl.java:1309)
        at org.postgresql.core.v3.QueryExecutorImpl.execute (QueryExecutorImpl.java:188)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute (AbstractJdbc2Statement.java:452)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags (AbstractJdbc2Statement.java:354)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate (AbstractJdbc2Statement.java:308)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate (DelegatingPreparedStatement.java:102)
        at org.hibernate.engine.query.NativeSQLQueryPlan.performExecuteUpdate (NativeSQLQueryPlan.java:210)
        ... 19 more 

lucidlynx avatar Mar 09 '21 08:03 lucidlynx

|if you try to run the sql command by hand, do you get an error ?|

|the list of index created is also available in createGISTindex.sql in the sql directory |

gisgraphy avatar Mar 10 '21 09:03 gisgraphy

Indexes were created using the createGISTindex.sql file. Indexes are present in the database.

Tell me how to restart the fill?

lucidlynx avatar May 07 '21 09:05 lucidlynx