not working with arabic language

[Kareimovich]

[ivanvermeyen]

Hello,

Can you try if #13 fixes your problem (solution at the end)? If not, can you give some more context on the issue?

[Kareimovich]

My DB column

and this is how i save translations

       $model->$field = $data[$field];
       $model->save();

My Saved Value

{"ar":"\u062a\u062c\u0631\u0628\u0647","en":"test"}

i have reviewed your fix but didn't work for me @ivanvermeyen

[Kareimovich]

Arabic is working with search but not with validation

[ivanvermeyen]

Hello,

I've been looking into this and the following test indeed fails when using another database column type than json.

Validation should fail because there is a duplicate value, but it doesn't.

/** @test */
public function it_handles_arabic_language()
{
    Model::create([
        'slug' => ['ar' => 'جديد'],
    ]);

    $rules = [
        'slug.*' => "{$this->rule}:{$this->table}",
    ];

    $validation = Validator::make([
        'slug' => ['ar' => 'جديد'],
    ], $rules);

    $this->assertTrue($validation->fails());
}

The only way to make it pass is to change the database column type to json (text and longtext doesn't seem to work). I don't know if this problem can be handled in the code. I'll do some research about it.

[ivanvermeyen]

This is probably the culprit: https://stackoverflow.com/questions/18616682/searching-arabic-words-that-have-diacritics-in-mysql

Their solution is to add a FULLTEXT index to the column, specify the correct charset and collation and use BOOLEAN MODE in the query:

CREATE TABLE articles (
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
title VARCHAR(200),
body TEXT,
FULLTEXT (title,body)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;


INSERT INTO articles (title,body) VALUES
('الْجِنَّةِ DCKIEW', 'DAVADV الْجِنَّةِ AVADV')


SELECT * FROM articles
WHERE MATCH (title,body)
AGAINST ('الجنة' IN BOOLEAN MODE);

Now, I have tried to use fulltext search, but Laravel has no built-in option to specify the BOOLEAN MODE in the query.

return DB::connection($connection)->table($table)
    ->where(function ($query) use ($column, $locale, $value) {
        $query->whereFullText($column, "\"{$locale}\": \"{$value}\"")
            ->orWhereFullText($column, "\"{$locale}\":\"{$value}\"");
        //$query->where($column, 'LIKE', "%\"{$locale}\": \"{$value}\"%")
        //    ->orWhere($column, 'LIKE', "%\"{$locale}\":\"{$value}\"%");
    });

So it didn't work.

Also, it tripped a few other tests, so I'm not sure if fulltext is even a possible solution. The easiest way would be to use a json column.