ckanext-spatial
ckanext-spatial copied to clipboard
Published
20 hours ago •
ckan
ckan
Searching on widget fails
We want to include this great extension into our website, but searching with a bounding box on the widget returns no results even if the search area completely contains the dataset extent area.
This is an example of a search over all of Norway returning no results even though a dataset lies just outside the coast:
Following the docs, we have selected the solr backend search by writing
ckanext.spatial.search_backend = solr
in ckan's configuration file (called production.ini)
and updated the Solr schema with
<field name="bbox_area" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />
<field name="minx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />
When rebuilding the indexing with
ckan-paster --plugin=ckan search-index rebuild --config=/etc/ckan/production.ini
the debug messages report
DEBUG [ckanext.spatial.plugin] Setting up the spatial model
2019-04-09 09:05:12,801 DEBUG [ckanext.spatial.model.package_extent] Spatial tables defined in memory
2019-04-09 09:05:12,809 DEBUG [ckanext.spatial.model.package_extent] Spatial tables already exist
2019-04-09 09:05:13,098 DEBUG [ckanext.spatial.plugin] Setting up the spatial model
2019-04-09 09:05:13,105 DEBUG [ckanext.spatial.model.package_extent] Spatial tables already exist
Any help would be appreciated.
Having a similar issue on https://data.amerigeoss.org but get an error:
Error There was an error while searching. Please try again.
Any help would be appreciated.
~~Same here.~~
~~Here a snapshot of the search page:~~
~~Here the URL that includes the spatial filter:~~
http://.../dataset?q=&sort=score+desc%2C+metadata_modified+desc&ext_bbox=-121.28906250000001%2C-30.751277776257812%2C2.4609375%2C49.38237278700955&ext_prev_extent=-161.71874999999997%2C-50.28933925329178%2C42.890625%2C63.23362741232569
~~Here the Solr log:~~
2019-10-22 11:47:37.066 INFO (qtp1791930789-16) [ x:ckan] o.a.s.c.S.Request [ckan] webapp=/solr path=/select params={mm=2<-1+5<80%25&facet.field=my_owner_org&facet.field=my_trial&facet.field=my_platform&facet.field=my_sensor&facet.field=my_experiment&facet.field=my_classification&facet.field=my_dimension&facet.field=tags&facet.field=res_format&facet.field=license_id&bf=div(mul(mul(max(0,sub(min(2.4609375,maxx),max(-121.2890625,minx))),max(0,sub(min(49.382372787,maxy),max(-30.7512777763,miny)))),2),add(9916.5392572,mul(sub(maxy,miny),sub(maxx,minx))))&fl=id+validated_data_dict&start=0&sort=score+desc,+metadata_modified+desc&fq=+-dataset_type:harvest&fq={!frange+incl%3Dfalse+l%3D0+u%3D1}div(mul(mul(max(0,sub(min(2.4609375,maxx),max(-121.2890625,minx))),max(0,sub(min(49.382372787,maxy),max(-30.7512777763,miny)))),2),add(9916.5392572,mul(sub(maxy,miny),sub(maxx,minx))))&fq=%2Bsite_id:"default"&fq=%2Bstate:active&rows=21&fq_list={!frange+incl%3Dfalse+l%3D0+u%3D1}div(mul(mul(max(0,sub(min(2.4609375,maxx),max(-121.2890625,minx))),max(0,sub(min(49.382372787,maxy),max(-30.7512777763,miny)))),2),add(9916.5392572,mul(sub(maxy,miny),sub(maxx,minx))))&facet.limit=50&q=*:*&tie=0.1&defType=edismax&qf=name^4+title^4+tags^2+groups^2+text&facet.mincount=1&wt=json&facet=true} status=500 QTime=3
2019-10-22 11:47:37.067 ERROR (qtp1791930789-16) [ x:ckan] o.a.s.s.HttpSolrCall null:java.lang.UnsupportedOperationException
at org.apache.lucene.queries.function.FunctionValues.floatVal(FunctionValues.java:44)
at org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:42)
at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
at org.apache.solr.search.ValueSourceParser$18$1.func(ValueSourceParser.java:298)
at org.apache.lucene.queries.function.valuesource.DualFloatFunction$1.floatVal(DualFloatFunction.java:60)
at org.apache.lucene.queries.function.valuesource.MaxFloatFunction.func(MaxFloatFunction.java:42)
at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
at org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:39)
at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
at org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:39)
at org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:82)
at org.apache.lucene.queries.function.valuesource.DivFloatFunction.func(DivFloatFunction.java:40)
at org.apache.lucene.queries.function.valuesource.DualFloatFunction$1.floatVal(DualFloatFunction.java:60)
at org.apache.lucene.queries.function.FunctionValues$5.matches(FunctionValues.java:203)
at org.apache.lucene.queries.function.ValueSourceScorer$1.matches(ValueSourceScorer.java:53)
at org.apache.lucene.search.TwoPhaseIterator$TwoPhaseIteratorAsDocIdSetIterator.doNext(TwoPhaseIterator.java:89)
at org.apache.lucene.search.TwoPhaseIterator$TwoPhaseIteratorAsDocIdSetIterator.nextDoc(TwoPhaseIterator.java:77)
at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:244)
at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:197)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
at org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:152)
at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:141)
at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1603)
at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1262)
at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1451)
at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:2066)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1840)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:609)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:547)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
~~Using:~~ ~~- CKAN 2.8.3~~ ~~- ckanext-spatial master [f9379473f25c53fcb8ec9a376fb59608c3d7013c]~~ ~~- SOLR 6.5.1~~
EDIT: a custom script removed the spatial fields from the Solr schema configuration, causing the reported error.
I was facing the same issue as above comment in a project of mine. In my case, I found out that it was because of this tag in solrconfig.xml:
<schemaFactory class="ManagedIndexSchemaFactory"> <bool name="mutable">true</bool> <str name="managedSchemaResourceName">managed-schema</str> </schemaFactory>
This overrode the 'schema.xml' with the 'managed-schema' file available in the directory, which did not contain the geospatial fields.
I replaced the above tag with:
<schemaFactory class="ClassicIndexSchemaFactory"/>
and the rebuilt the search-index in CKAN, which fixed the issue.
Hope this helps anyone else who might be facing this issue.
I am having the same issue. Unfortunately the workaround by @alpha2303 doesn't work for me as I cam not able to rebuild the search index (ckan.lib.search.common.SearchIndexError: Solr responded with an error (HTTP 500): [Reason: SolrCore 'ckan' is not available due to init failure: Could not load conf for core ckan: Error loading schema resource schema.xml] ). I am using ckan 2.9.8 with solr 8.11.2.
It's fixed I added these fields (https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html#choosing-a-backend-for-the-spatial-search) in the wrong file before.
<fields>
<!-- ... -->
<field name="minx" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />
</fields>
Now I added the fields to /var/solr/data/ckan/conf/managed-schema, rebuild the search indexes (ckan search-index rebuild) and restarted ckan. The error is gone.
Th error is gone yay! But unfortunately the search returns no results, although it should. This seems to be the exact problem @filipporemonato opened this issue with. And it might be related to #224 and #287.
I have also tried using the pre-configured Solr Docker images for CKAN with solr-spatial. Same issue. Spatial search via bbox doesnt return anything.
By the way, there's a discrepancy between the instructions on how to modify the solr-schema between the ckanext-spatial and what is implemented in the pre-configured Solr Docker images.
Documentation adds:
<field name="minx" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />
Added in Docker file:
<field name="bbox_area" type="float" indexed="true" stored="true" />
<field name="maxx" type="float" indexed="true" stored="true" />
<field name="maxy" type="float" indexed="true" stored="true" />
<field name="minx" type="float" indexed="true" stored="true" />
<field name="miny" type="float" indexed="true" stored="true" />
@amercader - do you have an idea how to fix it or troubleshoot this issue? Cheers
@ChristianF88 for the spatial search to work you need to check two things:
- That Solr is properly configured with the custom fields needed. This is already done if you are using one of the
-spatialDocker images. - That geometries are properly indexed. A dataset needs to have a geometry attached in order to get spatially indexed in Solr. The most common way to do that is to add a custom field or extra named
spatialthat contains a valid GeoJSON geometry. ckanext-spatial will take care of indexing these contents on the relevant Solr fields. Alternatively you can use your own indexing logic in a custombefore_dataset_index()hook to set the Solr fields manually but I'll recommend using thespatialfield until you can troubleshoot the issue.
Things to try:
- If you are using your own Solr install and can access the admin UI, go to http://localhost:8983/solr/#/ckan/schema (replace the host to match your setup) and look for the needed fields (
minx,maxxetc if usingsolr-bboxorspatial_geomif usingsolr-spatial-field) - Create a dataset with a valid GeoJSON polygon in a
spatialextra. Check in Solr if the field was indexed. You can see the indexed fields for a dataset using the Solr UI (http://127.0.0.1:8990/solr/#/ckan/query?q=:) or theckan search-index show <dataset-id> command. There should be anextras_spatialfield shown. - Make sure there is no other plugin that implements
IPackageController.before_dataset_index()that might be messing with the dict sent to Solr (ie turn off other plugins)
Hope this helps
@amercader - thank you for you input.
Using my Solr source install I can confirm that the schema and indexed test packages within Solr contain the fields required by solr-bbox:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"q":"id:\"94f8749a-86ed-4677-afd1-6056ad67a0be\"",
"indent":"true"}},
"response":{"numFound":1,"start":0,"numFoundExact":true,"docs":[
{
"id":"94f8749a-86ed-4677-afd1-6056ad67a0be",
"name":"test_package",
"name_ngram":"test_package",
"title":"Test_Package",
"title_ngram":"Test_Package",
"author":"[\"ckadm\"]",
"maintainer":"ckadm",
"notes":"some_note",
"metadata_created":"2023-07-04T12:30:50.073Z",
"metadata_modified":"2023-07-04T13:38:52.307Z",
"state":"active",
"organization":"test_organization",
"validated_data_dict":"{\"author\": [\"ckadm\"], \"author_email\": null, \"creator_user_id\": \"f3cab305-ccab-4bfa-a64e-0dbeb0a4e267\", \"geographic_name\": [], \"has_part\": \"\", \"id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"id_external\": \"\", \"is_part_of\": \"\", \"isopen\": false, \"license_id\": null, \"license_title\": null, \"maintainer\": \"ckadm\", \"maintainer_email\": null, \"metadata_created\": \"2023-07-04T12:30:50.073651\", \"metadata_modified\": \"2023-07-04T13:38:52.307870\", \"name\": \"test_package\", \"notes\": \"some_note\", \"notes-2\": \"\", \"num_resources\": 1, \"num_tags\": 1, \"organization\": {\"id\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"name\": \"test_organization\", \"title\": \"Test_Organization\", \"type\": \"organization\", \"description\": \"This is my organization.\", \"image_url\": \"https://www.techrepublic.com/wp-content/uploads/2017/03/meme05.jpg\", \"created\": \"2023-07-04T12:30:49.696158\", \"is_organization\": true, \"approval_status\": \"approved\", \"state\": \"active\"}, \"owner_org\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"private\": false, \"publicationlink\": \"\", \"review_level\": \"none\", \"reviewed_by\": \"\", \"spatial\": \"{\\\"type\\\": \\\"Point\\\", \\\"coordinates\\\": [8.609776496939471, 47.40384502816517]}\", \"state\": \"active\", \"status\": \"incomplete\", \"substances\": [], \"substances_generic\": [], \"systems\": [], \"tags_string\": \"some_tag\", \"taxa\": [], \"taxa_generic\": [], \"timerange\": [\"2014 TO 2016\"], \"title\": \"Test_Package\", \"type\": \"dataset\", \"url\": null, \"usage_contact\": \"ckadm\", \"variables\": [\"none\"], \"version\": null, \"resources\": [{\"cache_last_updated\": null, \"cache_url\": null, \"created\": \"2023-07-04T12:30:50.816697\", \"datastore_active\": false, \"datastore_contains_all_records_of_source_file\": false, \"description\": null, \"format\": \"JPEG\", \"hash\": \"\", \"id\": \"cc0a99b3-5903-4d24-9770-3c52946031db\", \"last_modified\": null, \"metadata_modified\": \"2023-07-04T12:30:50.812879\", \"mimetype\": \"image/jpeg\", \"mimetype_inner\": null, \"name\": \"test_resource\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"position\": 0, \"resource_type\": \"Dataset\", \"restricted_level\": \"public\", \"size\": null, \"state\": \"active\", \"url\": \"https://static.demilked.com/wp-content/uploads/2021/07/60ed37b256b80-it-rage-comics-memes-reddit-60e6fee1e7dca__700.jpg\", \"url_type\": null}], \"tags\": [{\"display_name\": \"some_tag\", \"id\": \"e87204c3-d8e0-4647-9f23-f4a96ce179e1\", \"name\": \"some_tag\", \"state\": \"active\", \"vocabulary_id\": null}], \"groups\": [], \"relationships_as_subject\": [], \"relationships_as_object\": []}",
"extras_geographic_name":"[]",
"extras_review_level":"none",
"extras_reviewed_by":"[]",
"extras_status":"incomplete",
"extras_substances":"[]",
"extras_substances_generic":"[]",
"extras_systems":"[]",
"extras_tags_string":"some_tag",
"extras_taxa":"[]",
"extras_taxa_generic":"[]",
"extras_timerange":"[\"[2014 TO 2016]\"]",
"extras_usage_contact":"ckadm",
"extras_variables":"[\"none\"]",
"tags":["some_tag"],
"capacity":"public",
"res_name":["test_resource"],
"res_format":["JPEG"],
"res_url":["https://static.demilked.com/wp-content/uploads/2021/07/60ed37b256b80-it-rage-comics-memes-reddit-60e6fee1e7dca__700.jpg"],
"res_type":["Dataset"],
"entity_type":"package",
"dataset_type":"dataset",
"site_id":"default",
"index_id":"da4066ef659c3df1bd1be08136c47bae",
"minx":8.6097765,
"miny":47.403847,
"maxx":8.6097765,
"maxy":47.403847,
"_version_":"1770497436439019520",
"indexed_ts":"2023-07-04T13:38:52.392Z",
"data_dict":"{\"id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"name\": \"test_package\", \"title\": \"Test_Package\", \"version\": null, \"url\": null, \"author\": \"[\\\"ckadm\\\"]\", \"author_email\": null, \"maintainer\": \"ckadm\", \"maintainer_email\": null, \"notes\": \"some_note\", \"license_id\": null, \"type\": \"dataset\", \"owner_org\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"creator_user_id\": \"f3cab305-ccab-4bfa-a64e-0dbeb0a4e267\", \"metadata_created\": \"2023-07-04T12:30:50.073651\", \"metadata_modified\": \"2023-07-04T13:38:52.307870\", \"private\": false, \"state\": \"active\", \"resources\": [{\"id\": \"cc0a99b3-5903-4d24-9770-3c52946031db\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"url\": \"https://static.demilked.com/wp-content/uploads/2021/07/60ed37b256b80-it-rage-comics-memes-reddit-60e6fee1e7dca__700.jpg\", \"format\": \"JPEG\", \"description\": null, \"hash\": \"\", \"position\": 0, \"name\": \"test_resource\", \"resource_type\": \"Dataset\", \"mimetype\": \"image/jpeg\", \"mimetype_inner\": null, \"size\": null, \"created\": \"2023-07-04T12:30:50.816697\", \"last_modified\": null, \"metadata_modified\": \"2023-07-04T12:30:50.812879\", \"cache_url\": null, \"cache_last_updated\": null, \"url_type\": null, \"state\": \"active\", \"restricted_level\": \"public\", \"datastore_active\": false, \"datastore_contains_all_records_of_source_file\": false}], \"num_resources\": 1, \"tags\": [{\"id\": \"e87204c3-d8e0-4647-9f23-f4a96ce179e1\", \"name\": \"some_tag\", \"vocabulary_id\": null, \"state\": \"active\", \"display_name\": \"some_tag\"}], \"num_tags\": 1, \"extras\": [{\"id\": \"8ba28b3a-8140-4aab-8f3f-0211c813538c\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"geographic_name\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"4955ee0d-1590-4d31-9c36-8a8f9a64d1c1\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"has_part\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"1a952b3c-0b86-4c47-a1d6-0f930fb3e7c1\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"id_external\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"fd1d8f44-771c-4c52-93a3-f144667a2a49\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"is_part_of\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"44f58c51-9ebe-4c19-9286-8b749b93bc52\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"notes-2\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"88d79983-d1de-4fe5-a55f-1ec14e931fd6\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"publicationlink\", \"value\": \"\", \"state\": \"active\"}, {\"id\": \"4b6a09fd-86b7-4de1-9be7-a41682cded59\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"review_level\", \"value\": \"none\", \"state\": \"active\"}, {\"id\": \"9179426d-6bef-46ba-91eb-563b76d936ea\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"reviewed_by\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"44c05ac7-6cd4-4fb1-b8b9-efce164e89ac\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"spatial\", \"value\": \"{\\\"type\\\": \\\"Point\\\", \\\"coordinates\\\": [8.609776496939471, 47.40384502816517]}\", \"state\": \"active\"}, {\"id\": \"b845e7e6-597e-4661-8a70-bb4b48f2b057\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"status\", \"value\": \"incomplete\", \"state\": \"active\"}, {\"id\": \"460906a1-55eb-42b0-982c-23208a017d0d\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"substances\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"1b515e6b-0674-4b9a-9c0d-3509ccd3d5e0\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"substances_generic\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"eb0a33bb-5cb6-4b2a-aa1c-2e73cb87bd04\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"systems\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"a9e39b26-816a-4767-bc99-4c3ce7e193fa\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"tags_string\", \"value\": \"some_tag\", \"state\": \"active\"}, {\"id\": \"bce11a67-e468-497c-b336-71584909e852\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"taxa\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"79dc34d4-6439-4f98-90a0-9b55e8da303e\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"taxa_generic\", \"value\": \"[]\", \"state\": \"active\"}, {\"id\": \"e970f507-268f-47e6-8d2f-47de9836dcd8\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"timerange\", \"value\": \"[\\\"[2014 TO 2016]\\\"]\", \"state\": \"active\"}, {\"id\": \"1c404146-371a-4d87-980c-2db95fe60a8e\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"usage_contact\", \"value\": \"ckadm\", \"state\": \"active\"}, {\"id\": \"6d19cf68-77b4-49e3-b682-5635d15def34\", \"package_id\": \"94f8749a-86ed-4677-afd1-6056ad67a0be\", \"key\": \"variables\", \"value\": \"[\\\"none\\\"]\", \"state\": \"active\"}], \"groups\": [], \"organization\": {\"id\": \"226b4978-03eb-4b15-85cd-f9c449881076\", \"name\": \"test_organization\", \"title\": \"Test_Organization\", \"type\": \"organization\", \"description\": \"This is my organization.\", \"image_url\": \"https://www.techrepublic.com/wp-content/uploads/2017/03/meme05.jpg\", \"created\": \"2023-07-04T12:30:49.696158\", \"is_organization\": true, \"approval_status\": \"approved\", \"state\": \"active\"}, \"relationships_as_subject\": [], \"relationships_as_object\": [], \"isopen\": false, \"license_title\": null}"}]
}
So I assume of the things to try number 1 and 2 function as intended.
Running grep to see if before_dataset_index is implemented in any other plugins shows that only ckanext-spatial is implementing it.
(default) root@bookworm:/usr/lib/ckan/default/src# grep -R before_dataset_index ckanext-*
ckanext-spatial/doc/spatial-search.rst:In this case you need to implement the ``before_dataset_index()`` method of the `IPackageController <https://docs.ckan.org/en/latest/extensions/plugin-interfaces.html#ckan.plugins.interfaces.IPackageController.before_dataset_index>`_ interface::
ckanext-spatial/doc/spatial-search.rst:* Check the default implementation of ``before_dataset_index()`` in `ckanext/spatial/plugins/__init__.py <https://github.com/ckan/ckanext-spatial/blob/master/ckanext/spatial/plugin/__init__.py>`_ for extra useful checks and validations.
ckanext-spatial/ckanext/spatial/plugin/__init__.py: return self.before_dataset_index(pkg_dict)
ckanext-spatial/ckanext/spatial/plugin/__init__.py: def before_dataset_index(self, pkg_dict):
grep: ckanext-spatial/ckanext/spatial/plugin/__pycache__/__init__.cpython-39.pyc: binary file matches
ckanext-spatial/ckanext/spatial/tests/test_plugin/plugin.py: return self.before_dataset_index(pkg_dict)
ckanext-spatial/ckanext/spatial/tests/test_plugin/plugin.py: def before_dataset_index(self, pkg_dict):
Which confirms that also the 3rd point to try cannot cause the issue.
Are there any pointers from your side regarding the config?
Currently I am loading these plugins:
ckan.plugins = ... spatial_metadata spatial_query resource_proxy geo_view ...
and am applying these other settings:
ckan.views.default_views = image_view text_view recline_view pdf_view webpage_view geo_view
ckanext.spatial.search_backend = solr-bbox
ckanext.geoview.ol_viewer.formats = geojson
ckanext.spatial.common_map.type = custom
ckanext.spatial.common_map.custom.url = ...
ckanext.spatial.common_map.attribution = ...
# setting the default extend in theming extension
ckanext.eaw_theme.spatial_query_default_extent = [[45.80, 5.84], [47.84, 10.50]]
@ChristianF88 looks like you are trying to index a point ("spatial\": \"{\\\"type\\\": \\\"Point\\\", \\\"coordinates\\\": [8.609776496939471, 47.40384502816517]}). As stated in the docs, the solr-bbox backend only supports Bounding boxes and Polygons. If you want to index points you'll need to use the solr-spatial-field backend.
I added a polygon example package and it works! I think this issue can be closed. Possibly also the related issues from my previous comment .