fulltextsearch icon indicating copy to clipboard operation
fulltextsearch copied to clipboard

fulltextsearch 0.6.1 not working with object storage (S3) as primary storage

Open fa-ribeiro opened this issue 6 years ago • 5 comments

I looks like fultextsearch can't add documents when using object storage as primary storage :-(

When I run:

sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:live

the output is:

- RAM: 16.301345825195
Exception: {"_index":"nextcloud","_type":"standard","_id":"files:127","_version":2,"result":"not_found","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":5,"_primary_term":1}
- RAM: 18.080711364746
Exception: {"_index":"nextcloud","_type":"standard","_id":"files:127","_version":3,"result":"not_found","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":6,"_primary_term":1}
- RAM: 18.080772399902
[...]

and search results always displays no result.


Installation on a DigitalOcean droplet:

  • Debian 9.4
  • php7.0-fpm: PHP 7.0.27-0+deb9u1
  • MariaDB Ver 15.1 Distrib 10.1.26-MariaDB

My Nextcloud config.php:

<?php
$CONFIG = array (
  'instanceid' => 'XXXXXXXX',
  'passwordsalt' => 'XXXXXXXXXXXXXXXX',
  'secret' => 'XXXXXXXXXXXXXXXX',
  'trusted_domains' =>
  array (
    0 => 'cloud.example.com',
  ),
  'datadirectory' => '/home/nextcloud/data',
  'overwrite.cli.url' => 'https://cloud.example.com',
  'dbtype' => 'mysql',
  'version' => '13.0.2.1',
  'dbname' => 'nextcloud',
  'dbhost' => '127.0.0.1',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'XXXXXXXXXXXXXXXX',
  'installed' => true,
  'updater.release.channel' => 'stable',
  'tempdirectory' => '/home/nextcloud/data/temp',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
  ),
  'log_rotate_size' => 52428800,
  'logtimezone' => 'Europe/Lisbon',
  'maintenance' => false,
  'objectstore' =>
  array (
    'class' => 'OC\\Files\\ObjectStore\\S3',
    'arguments' =>
    array (
      'bucket' => 'MyNextcloudBucket',
      'key' => 'XXXXXXXXXXXXXXXX',
      'secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
      'hostname' => 'ams3.digitaloceanspaces.com',
      'use_ssl' => true,
    ),
  ),
);

Fulltext search apps config:

sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:check
Full text search 0.6.1

- Search Platform:
Elasticsearch 0.6.1
{
    "elastic_host": "http:\/\/localhost:9200",
    "elastic_index": "nextcloud"
}

- Content Providers:
Files 0.6.0
{
    "files_local": "1",
    "files_external": "1",
    "files_encrypted": "0",
    "files_federated": "0",
    "files_size": "20",
    "files_pdf": "1",
    "files_office": "1",
    "files_image": "0",
    "files_audio": "0"
}

Any hints?

fa-ribeiro avatar May 07 '18 17:05 fa-ribeiro

I have the same kind of problem using Swift as primary storage,

occ fulltextsearch:index does nothing

Fulltextsearch is supposed to work with object storage as primary?

theninjadev avatar May 11 '18 09:05 theninjadev

I also need this function.

chinalujw avatar Jun 14 '18 06:06 chinalujw

A workaround: mount s3 as partition and use this partition as primary storage with 's3fs'.

But the s3 integration with primary storage will be great!

Tab3r avatar Jul 25 '18 13:07 Tab3r

@daita as this ticket has >1yr, is there any plans to extend full-text search to objectStorage? More and more providers migrate content storage to objectStorage ans its benefits.

Thanks.

solracsf avatar Sep 10 '19 21:09 solracsf

same here, not working (well, index filenames, not content)

vpecinka avatar Apr 12 '21 16:04 vpecinka