nextcloud
Not working after the last update to v26
After updating to the v26 I found that the search results are neither shown in UI, nor in the desktop client. There is no more context search block in the search results at all.
Not working here:
`lltextsearch:check Full text search 26.0.0
-
Search Platform: Elasticsearch 26.0.0 (Selected) { "elastic_host": [ "http://internesuche:********@127.0.0.1:9200" ], "elastic_index": "internesuche", "fields_limit": "10000", "es_ver_below66": "0", "analyzer_tokenizer": "" }
-
Content Providers: Deck 1.9.0 [] Files 26.0.0 { "files_local": "1", "files_external": "0", "files_group_folders": "0", "files_encrypted": "0", "files_federated": "0", "files_size": "10", "files_pdf": "1", "files_office": "1", "files_image": "0", "files_audio": "0", "files_chunk_size": "2", "files_fulltextsearch_tesseract": { "version": "25.0.0", "enabled": "1", "psm": "4", "lang": "de,eng", "pdf": "1", "pdf_limit": "0" } }`
I hope anyone can maintain these App again.
`lars@livesystem-debian:~$ sudo -uwww-data php8.2 /var/www/nextcloud/occ fulltextsearch:test
.Testing your current setup: Creating mocked content provider. ok Testing mocked provider: get indexable documents. (2 items) ok Loading search platform. (Elasticsearch) ok Testing search platform. fail In Test.php line 304:
Search platform (Elasticsearch) down ?`
Not sure if this is the same issue - with occ fulltextsearch:test I'm getting "No alive nodes. All the 1 nodes seem to be down." Have verified that I can browse to elasticsearch from externally and curl to it from server (I'm using a docker deployment and can curl from inside container).
I have the same issue as the original author. I can curl elasticsearch instance and index, but NC Test does not work. With the 25 Fulltext Search / ES-Plugin on a 26 Nextcloud it worked (i added a manual exception to install), but since I upgraded the app to 26 it broke. I tried to manually downgrade by replacing the 26 code by the 25 code but failed to get Nextcloud running again. If this hints someone to a working workaround please let me know how you did it :)
I tried again and got a working workaround. I got it running with the following versions:
Full text search - Elasticsearch Platform 25.0.0 (downgraded) Full text search 26.0.0 Full text search - Files 26.0.0 Full text search - Files - Tesseract OCR 25.0.0 (26 not yet released)
How I did it step by step:
- deactivated ES Platform 26 App on NC webinterface
- deleting / moving out the 26 App files like: mv ./nextcloud/apps/fulltextsearch_elasticsearch fulltextsearch_elasticsearch-26.back
- Downloading v25: wget https://github.com/nextcloud-releases/fulltextsearch_elasticsearch/releases/download/25.0.0/fulltextsearch_elasticsearch-25.0.0.tar.gz (btw: this release is not listed on github, i just guessed the url based on the old ones)
- unextract it: tar -xf fulltextsearch_elasticsearch-25.0.0.tar.gz
- moving the v25 version in: mv fulltextsearch_elasticsearch ./nextcloud/apps/
- activating the v25 app in the webinterface (you might have to confirm that you really want to activate it)
Hope it helps someone!
Thanks - downgrading to v25 with these instructions worked for me also
Meanwhile, what to do to make the platform v26 working?
Is this actually an issue for fulltextsearch_elasticsearch?
I have it (temporarily) working by reverting to v25 and then reinstalling v26 through the app. Based on log files, I'm wondering if there's an issue with the files populating /var/www/html/custom_apps/fulltextsearch_elasticsearch/vendor/psr in v26?
That fixed it for me ☺️https://github.com/R0Wi/elasticsearch-nextcloud-docker
That fixed it for me ☺️https://github.com/R0Wi/elasticsearch-nextcloud-docker
How? Is it working with the Full Text Search Platform version 26.0.0 in your case? As in my case it said failed. Which ES client version do you use?
i use this Version.
Great! As far as I understand your app version is 26.0.0 and the ES Client id ocker is v.8.6.1, am I right? Meanwhile, how did you manage to migrate your index?
Yes, correct @ostasevych. I had no index anymore. I index the files for the first time.
Yes, correct @ostasevych. I had no index anymore. I index the files for the first time.
Meanwhile, what about security concerns? Is it so safe to trust to dockerised solution comparing to the legacy installed on the server side? I mean, might the index be accessible from outside in this case? What is your opinion?
@ostasevych no security issue. Like the Version installed as service, both Versions listen to 127.0.01:9200. This means only to localhost. You can block this port for outgoing traffic with UFW.
Yes, correct @ostasevych. I had no index anymore. I index the files for the first time.
Need some help here. I have a lot of files, and the indexing process is quite long, so I've started it from remote ssh terminal several times due to drop/downs.
So, the curl shows that the data are indexed
curl -X GET "localhost:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"query_string" : {
"query" : "test",
"default_field" : "title"
}
}
}
'
But the reply of the command:
sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:search userA test
is just
search
and no data in the webUI :(
So, how to check whether it works?
UPD After restarting the indexing I am getting tons of unknown errors:
Are any tools or methods to debug fulltextsearch indexing and querying?
this could be the underling error: https://github.com/nextcloud/fulltextsearch_elasticsearch/issues/256
@Happyfeet01 are you mayben not using basic auth with your docker setup? i did before i downgraded (also did not use docker)
No basic Auth. Only Cluster Name and Address. But your linked issue seems the reason.
Before I tried this Docker Method I using the auth for Elasticsearch.
Again about reindexing. I am still struggling to make the full text search working on my instance:
- NC 26.0.3 (bare metal)
- All fulltextsearch apps are update
- Using groufolders app
- Elasticsearch via dockerised container: 8.6.1
Reindexing has been done with the following commands:
$ sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:stop
$ curl -X DELETE localhost:9200/my-index
$ sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:reset
$ sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:index
Searching with curl in index "my-index" for the word "Ivanova" (our member) gives (user data are stripped and scratched):
$ curl -X GET "localhost:9200/my-index/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"match": {
"title": "Ivanova"
}
}
}
'
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 6,
"relation" : "eq"
},
"max_score" : 9.4341955,
"hits" : [
{
"_index" : "my-index",
"_id" : "files:6811",
"_score" : 9.4341955,
"_ignored" : [
"content.keyword"
],
"_source" : {
"owner" : "user1",
"groups" : [
"GroupA",
"GroupB",
"GroupC",
"GroupD"
],
"circles" : [ ],
"metatags" : [
"files_group_folders"
],
"source" : "files_group_folders",
"title" : "GroupA/Scanned/Ivanova.pdf",
"users" : [ ],
"content" : "Ivanova document"
"tags" : [ ],
"attachment" : {
"date" : "2022-04-06T06:29:11Z",
"content_type" : "application/pdf",
"format" : "application/pdf; version=\"A-2b\"",
"modified" : "2022-04-06T06:29:11Z",
"language" : "uk",
"creator_tool" : "ABBYY FineReader 14",
"content_length" : 1612
},
"provider" : "files",
"subtags" : [ ],
"parts" : {
"comments" : ""
},
"links" : [ ],
"share_names" : {
"user0" : "GroupA/Scanned/Ivanov.pdf",
"user1" : "Scanned/Ivanov.pdf",
"user2" : "GroupA/Scanned/Ivanov.pdf",
"user3" : ""
},
"hash" : "aeca335860b2f59954q5e7fd34b174a1"
}
},
{
"_index" : "my-index",
"_id" : "files:6812",
"_score" : 9.4341955,
"_ignored" : [
"content.keyword"
],
"_source" : {
"owner" : "user0",
"groups" : [
"GroupA",
"GroupB",
"GroupC",
"GroupD"
],
"circles" : [ ],
"metatags" : [
"files_group_folders"
],
"source" : "files_group_folders",
"title" : "GroupA/Scanned/Ivanova2.pdf",
"users" : [ ],
"content" : "Ivanova........................................................... ...................",
"tags" : [ ],
"attachment" : {
"date" : "2022-04-01T08:00:50Z",
"content_type" : "application/pdf",
"format" : "application/pdf; version=\"A-2b\"",
"modified" : "2022-04-01T08:00:50Z",
"language" : "uk",
"creator_tool" : "ABBYY FineReader 14",
"content_length" : 1260
},
"provider" : "files",
"subtags" : [ ],
"parts" : {
"comments" : ""
},
"links" : [ ],
"share_names" : {
"user0" : "GroupA/Scanned/Ivanova2.pdf",
"user1" : "Scanned/Ivanova2.pdf",
"user2" : "GroupA/Scanned/Ivanova2.pdf",
"user3" : "",
},
"hash" : "e09e889376ebe62b907b8023f37d21a9"
}
},
{
"_index" : "my-index",
"_id" : "files:1576",
"_score" : 9.30954,
"_source" : {
"owner" : "user0",
"groups" : [
"GroupA",
"GroupB",
"GroupC",
"GroupD"
],
"circles" : [ ],
"metatags" : [
"files_group_folders"
],
"source" : "files_group_folders",
"title" : "GroupA/Scanned/Ivaniva CV.pdf",
"users" : [ ],
"content" : "",
"tags" : [ ],
"attachment" : {
"date" : "2022-06-02T07:57:28Z",
"keywords" : "Scanned image",
"content_type" : "application/pdf",
"author" : "NAPS2",
"format" : "application/pdf; version=1.4",
"modified" : "2022-06-02T07:57:28Z",
"language" : "lt",
"title" : "Scanned image",
"creator_tool" : "NAPS2",
"content_length" : 4
},
"provider" : "files",
"subtags" : [ ],
"parts" : {
"comments" : ""
},
"links" : [ ],
"share_names" : {
"user0" : "GroupA/Scanned/Ivanova CV.pdf",
"user5" : "GroupA/Scanned/Ivanova CV.pdf",
"user3" : ""
},
"hash" : "53f4a218a7ac0f31648efc0834c35199"
}
},
{
"_index" : "my-index",
"_id" : "files:518673",
"_score" : 8.218001,
"_ignored" : [
"content.keyword"
],
"_source" : {
"owner" : "user0",
"groups" : [
"GroupD"
],
"circles" : [ ],
"metatags" : [
"files_group_folders"
],
"source" : "files_group_folders",
"title" : "GroupA/Scanned/Ivanova Bulletin.docx",
"users" : [
"user8"
],
"content" : "Ivanova Bulletin text text",
"tags" : [ ],
"attachment" : {
"date" : "2023-06-13T09:19:00Z",
"content_type" : "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"author" : "Administrator",
"modifier" : "User6",
"modified" : "2023-06-14T13:32:00Z",
"language" : "uk",
"content_length" : 1701,
"print_date" : "2023-06-14T13:30:00Z"
},
"provider" : "files",
"subtags" : [ ],
"parts" : {
"comments" : ""
},
"links" : [ ],
"share_names" : {
"user0" : "GroupA/Bulls/Ivanova Bull.docx",
"user9" : "GroupA/Bulls/Ivanova Bull.docx"
},
"hash" : "99a498dbb1db6f68a6be3793e30a9476"
}
},
{
"_index" : "my-index",
"_id" : "files:6806",
"_score" : 7.622202,
"_source" : {
"share_names" : {
"user0" : "GroupA/Scanned/Photos/Ivanova",
"user2" : "Scanned/Photos/Ivanova",
"user3" : ""
},
"owner" : "user0",
"users" : [ ],
"groups" : [
"GroupA",
"GroupB",
"GroupC",
"GroupD"
],
"circles" : [ ],
"links" : [ ],
"metatags" : [
"files_group_folders"
],
"subtags" : [ ],
"tags" : [ ],
"hash" : "",
"provider" : "files",
"source" : "files_group_folders",
"title" : "GroupA/Scanned/Ivanova",
"parts" : [ ],
"content" : ""
}
},
{
"_index" : "my-index",
"_id" : "files:466569",
"_score" : 5.747198,
"_ignored" : [
"content.keyword"
],
"_source" : {
"owner" : "user0",
"groups" : [
"GroupD"
],
"circles" : [ ],
"metatags" : [
"files_group_folders"
],
"source" : "files_group_folders",
"title" : "GroupA/Applications/Ivanova.docx",
"users" : [
"user4",
"user5",
"user6",
"user7"
],
"content" : "Application of Ivanova text text text",
"tags" : [ ],
"attachment" : {
"date" : "2023-03-10T12:25:00Z",
"content_type" : "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"author" : "user5",
"modifier" : "user10",
"modified" : "2023-05-17T17:55:09Z",
"language" : "uk",
"content_length" : 1550
},
"provider" : "files",
"subtags" : [ ],
"parts" : {
"comments" : ""
},
"links" : [ ],
"share_names" : {
"user0" : "GroupA/Applications/Ivanova.docx",
"user6" : "GroupA/Applications/Ivanova.docx",
"user3" : "",
},
"hash" : "f8c12747f9b240ed696385d2aff7f0fe"
}
}
]
}
}
Next, I search for Ivanova on behind of a user0
$ sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:search user0 Ivanova
search
> Files
- 518673 score:0
- 480179 score:0
- 514585 score:0
- 527182 score:0
- 531276 score:0
- 363692 score:0
The documents are stored in the groupfolder, which is accessible to user0, user1, user2, user3, user10, but that's what I see when searching with the help of NC fulltextsearch app:
$ sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:search user10 Ivanova
search
> Files
$ sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:search user3 Ivanova
search
> Files
- 97091 score:0
Obviously, users in webUI cannot see the proper output.
Would you be so kind to explain, why it is not working as expected? What should I do to make it working? Thanks!
Are you aware that Elasticsearch 8 is not offical supported? I would start there. see https://github.com/nextcloud/fulltextsearch_elasticsearch
Or is it? https://github.com/nextcloud/fulltextsearch_elasticsearch/wiki#installation-of-elasticsearch-via-docker points to https://github.com/R0Wi/elasticsearch-nextcloud-docker, which suggests elasticsearch version 8.6.1.
Or is it? https://github.com/nextcloud/fulltextsearch_elasticsearch/wiki#installation-of-elasticsearch-via-docker points to https://github.com/R0Wi/elasticsearch-nextcloud-docker, which suggests elasticsearch version 8.6.1.
Exactly!
Are you aware that Elasticsearch 8 is not offical supported? I would start there. see https://github.com/nextcloud/fulltextsearch_elasticsearch
There was an update to the Readme. Seems like ES-plugin 26+ supports ES 8. If the 26 plugin does not work that means you still need to use 7, otherwise 8 is (newly) working.
Also the update to ES 8 might solve some issues people had here when going up to ES 26 plugin
Tried it yesterday, it did not work with ES plugin 26. and elasticsearch 8.6.1 or newer. Maybe it changed with nc version 27.0.2, which I downloaded a couple hours ago. not tested yet.
