Slugs generation mismatch between frontend and backend on restore

[antonioribeiro]

Description

When adding a new record for Russia's War in Ukraine, using the create form, the frontend generates this slug:

russia-s-war-in-ukraine

But if we use Laravel's Str::slug("Russia's War in Ukraine") helper we get this:

russias-war-in-ukraine

On Laravel 5.8 and 9.52.

So if this Twill code is executed for some reason:

public function updateOrNewSlug($slugParams, $restoring = false)
{
    if (in_array($slugParams['locale'], config('twill.slug_utf8_languages', []))) {
        $slugParams['slug'] = $this->getUtf8Slug($slugParams['slug']);
    } else {
        $slugParams['slug'] = Str::slug($slugParams['slug']);
    }
    ...
}

It may update the slug to a different one. I cannot personally reproduce this, but we just had a client reporting this change on a title that didn't change and a slug that was not directly updated by a user using the CMS.

Update

I've been able to reproduce it by restoring a record.

[ifox]

Was the record restored?

[antonioribeiro]

@ifox Possibly, I just restored it myself locally and I can now reproduce it

[antonioribeiro]

Here's a solution to make Twill generate slugs on PHP the same way the frontend does:

A new helper:

if (!function_exists('twill_js_slugify')) {
    function twill_js_slugify($title) {
        // Convert to lowercase
        $title = strtolower($title);

        // Make it only ascii characters
        $chars = [',','/',"'",';','_','©','·','ß','à','á','â','ã','ä','å','æ','ç','è','é','ê','ë','ì','í','î','ï','ð','ñ','ò','ó','ô','õ','ö','ø','ù','ú','û','ü','ý','þ','ÿ','ā','ă','ą','ć','č','ď','ē','ę','ě','ğ','ģ','ī','ı','ķ','ļ','ł','ń','ņ','ň','ő','œ','ŕ','ř','ś','ş','š','ť','ū','ů','ű','ź','ż','ž','ǘ','ǵ','ǹ','ș','ț','ΐ','ά','έ','ή','ί','ΰ','α','β','γ','δ','ε','ζ','η','θ','ι','κ','λ','μ','ν','ξ','ο','π','ρ','ς','σ','τ','υ','φ','χ','ψ','ω','ϊ','ϋ','ό','ύ','ώ','а','б','в','г','д','е','ж','з','и','й','к','л','м','н','о','п','р','с','т','у','ф','х','ц','ч','ш','щ','ъ','ы','ь','э','ю','я','ё','є','і','ї','ґ','ḧ','ḿ','ṕ','ẃ','ẍ','ә','ғ','қ','ң','ө','ұ','&'];

        $replacements = ['-','-','-','-','-','(c)','-','ss','a','a','a','a','a','a','ae','c','e','e','e','e','i','i','i','i','d','n','o','o','o','o','o','o','u','u','u','u','y','th','y','a','a','a','c','c','d','e','e','e','g','g','i','i','k','l','l','n','n','n','o','oe','r','r','s','s','s','t','u','u','u','z','z','z','u','g','n','s','t','i','a','e','h','i','y','a','b','g','d','e','z','h','8','i','k','l','m','n','3','o','p','r','s','s','t','y','f','x','ps','w','i','y','o','y','w','a','b','v','g','d','e','zh','z','i','j','k','l','m','n','o','p','r','s','t','u','f','h','c','ch','sh','sh','','y','','e','yu','ya','yo','ye','i','yi','g','h','m','p','w','x','a','g','q','n','o','u','-and-'];

        $title = str_replace($chars, $replacements, $title);

        // Replace all non-word chars with -
        $title = preg_replace('![^\w-]+!u', '-', $title);

        // Replace multiple - with single -
        $title = preg_replace('!--+!u', '-', $title);

        // Remove leading and traling -
        $title = preg_replace('~(?<!\S)-|-(?!\S)~', '', $title);

        // Return without leading and trailing whitespaces
        return trim($title);
    }
}

A trait to replace updateOrNewSlug implementation:

<?php

namespace App\Models\Behaviours;

use Illuminate\Support\Str;

trait JsSlugify
{
    /**
     * @param array $slugParams
     * @param bool $restoring
     * @return void
     */
    public function updateOrNewSlug($slugParams, $restoring = false)
    {
        if (in_array($slugParams['locale'], config('twill.slug_utf8_languages', []))) {
            $slugParams['slug'] = $this->getUtf8Slug($slugParams['slug']);
        } else {
            $slugParams['slug'] = twill_js_slugify($slugParams['slug']);
        }

        //active old slug if already existing or create a new one
        if (
            (($oldSlug = $this->getExistingSlug($slugParams)) != null)
            && ($restoring ? $slugParams['slug'] === $this->suffixSlugIfExisting($slugParams) : true)
        ) {
            if (!$oldSlug->active && ($slugParams['active'] ?? false)) {
                $this->getSlugModelClass()::where('id', $oldSlug->id)->update(['active' => 1]);
                $this->disableLocaleSlugs($oldSlug->locale, $oldSlug->id);
            }
        } else {
            $this->addOneSlug($slugParams);
        }
    }
}

And we need to use the trait but tell PHP to use the new implementation:

class Post extends Model implements Sortable
{
    use HasSlug;

    use JsSlugify {
        JsSlugify::updateOrNewSlug insteadof HasSlug;
    }

    ...

Note that this a quick and dirty fix to just to make the backend generate the same URLs as the frontend does and not break a database with thousands of slugs already generated by the frontend.

Because users can create custom slugs on the CMS, ideally, when restoring a record, we should keep the current slug and restore only the older contents, but this seems like a bigger change.

Also ideally, on a new application with zero records, it would be better to use PHP's implementation, but this also seems to be a bit complex, as we would need to do a call to backend to slugify slugs in real time.

[Tofandel]

Or maybe we just add a flag to the slug editor, if the slug has been edited manually we send the slug, if not we send null to let php take care of it

But having the js algo match the php algo would definitely be better as well