`nodeToString` result doesn't match with react-i18n

[techird]

Version

• i18next: 13.1.1
• react-i18next: 9.0.2
• i18next-scanner: 2.9.1

Configuration

{ // i18next-scanner options
  // the config has no matters to this issue
}

The behavior

Checkout this simple jsx source fragment:

<Trans>
  You click
  <Button
    className="m"
    onClick={() => setCounter(counter + 1)}>
    this button
  </Button>
  {{counter}} 
  times
</Trans>

The i18n-scanner parse the Trans component into defaultValue in nodes-to-string.js with this memo:

You click<1>this button</1><2>{{counter}}</2>  times

But in react-i18next implementation, it generate memo like this:

You click<1>this button</1>{{counter}}times

Please notice the tag mark was not included in interpolations.

The diffencene break the usage of the resource.

[cheton]

This is probably made by this line:

https://github.com/i18next/i18next-scanner/blob/ee0535/src/nodes-to-string.js#L58

memo += `<${nodeIndex}>{{${expression.properties[0].key.name}}}</${nodeIndex}>`;

The above code will add extra nodeIndex for {{var}}.

Can someone help confirm if removing element key (<${nodeIndex}>) for {{var}} is the expected behaviour to generate the defaultValue?

[char0n]

I can confirm this issue:

      <Trans count={count} ns="feature1">
        Hello <Strong>World</Strong>, you have {{count}} unread message.
      </Trans>

is scanned as Hello <1>World</1>, you have <3>{{count}}</3> unread message. but the react-i18next search for it as Hello <1>World</1>, you have {{count}} unread message..

[cheton]

In react-i18next v7.13.0, an element key was added as element tag to wrap the child in object type:

// https://github.com/i18next/react-i18next/blob/v7.13.0/src/Trans.js#L28-L37
} else if (typeof child === 'object') {
  const clone = { ...child };
  const format = clone.format;
  delete clone.format;

  const keys = Object.keys(clone);
  if (format && keys.length === 1) {
    mem = `${mem}<${elementKey}>{{${keys[0]}, ${format}}}</${elementKey}>`;
  } else if (keys.length === 1) {
    mem = `${mem}<${elementKey}>{{${keys[0]}}}</${elementKey}>`;

There is a major version bump in react-i18next 8.0.0 that made a change to simplify the translation string with interpolated values:

// https://github.com/i18next/react-i18next/blob/v8.0.0/src/Trans.js#L29-L37
} else if (typeof child === 'object') {
  const clone = { ...child };
  const format = clone.format;
  delete clone.format;

  const keys = Object.keys(clone);
  if (format && keys.length === 1) {
    mem = `${mem}{{${keys[0]}, ${format}}}`;
  } else if (keys.length === 1) {
    mem = `${mem}{{${keys[0]}}}`;

[cheton]

Hi @jamuhl,

In the CHANGELOG, it stated that:

---

'Hello <1><0>{{name}}</0></1>, you have <3>{{count}}</3> message. Open <5>hear</5>.'

can be written as:

'Hello <1>{{name}}</1>, you have {{count}} message. Open <5>hear</5>.'

=> there is no need to add <0>...</0> around interpolated values any longer => your old files having those extra pseudo tags will still work without needing to change them

---

May I know if react-i18next v8~v10 will still work if extra pseudo tags are added to old files?

[techird]

Hi all,

I figure out a workaround to make a match hash with i18next:

parser.parseTransFromString(content, (transKey, options) => {
      let sentence = options.defaultValue;
      // remove <Tag> surrounding interopations to match i18next simpilied result
      // @see https://github.com/i18next/react-i18next/blob/master/CHANGELOG.md#800
      sentence = sentence.replace(/<(\d+)>{{(\w+)}}<\/\1>/g, '{{$2}}');
      sentence = sentence.replace(/\s+/g, ' ');
      transKey = transKey || hashKey(sentence);
      options.defaultValue = sentence;

      parser.set(transKey, options);
    });

[char0n]

@techird how do you incorporate your workaround to the scanner config ? Can you provide a POC if you already managed to do it ? THank you

[techird]

@char0n Here is my steps:

First, use a script scan.js to scan source file and generate translation files.

Example of scan.js

const run = (appPath) => {
  vfs.src('src/**/*.{js,jsx,ts,tsx}', { cwd: appPath })
    .pipe(sort())
    .pipe(scanner())
    .pipe(map(writer(appPath)))
    .pipe(vfs.dest(appPath));
};

function scanner() {
  const options = {
    lngs: Object.keys(lngs),
    ns: ['translation'],
    defaultLng: 'zh',
    defaultNs: 'translation',
    resource: {
      savePath: 'i18n/{{ns}}/{{lng}}'
    }
  };
  /**
   * @param {import('vinyl')} file
   * @param {string} encoding
   * @param {Function} done
   */
  function transform(file, encoding, done) {
    const { parser } = this;
    const extname = path.extname(file.path);

    // scan source only
    if (!['.js', '.jsx', '.ts', '.tsx'].includes(extname)) {
      return done();
    }

    const content = file.contents.toString();

    parser.parseFuncFromString(content, { list: ['t', 'i18n.t'] }, (sentence, options) => {
      const key = hashKey(sentence);
      options.defaultValue = sentence;
      parser.set(key, options);
    });

    parser.parseTransFromString(content, (transKey, options) => {
      let sentence = options.defaultValue;
      // remove <Tag> surrounding interopations to match i18next simpilied result
      // @see https://github.com/i18next/react-i18next/blob/master/CHANGELOG.md#800
      sentence = sentence.replace(/<(\d+)>{{(\w+)}}<\/\1>/g, '{{$2}}');
      sentence = sentence.replace(/\s+/g, ' ');
      transKey = transKey || hashKey(sentence);
      options.defaultValue = sentence;

      parser.set(transKey, options);
    });
    done();
  }
  return i18nScanner.createStream(options, transform);
}

Then init i18next with the following options:

i18next options

i18next.use(reactI18nextModule).init({
  lng,
  resources: {
    // import translation from translation files
    [lng]: { translation: translation }
  },

  ns: "translation",
  defaultNS: "translation",

  interpolation: {
    escapeValue: false, // not needed for react as it escapes by default
  },

  react: {
    // here the `hashKey()` should be same with the one in `scan.js`
    hashTransKey: hashKey
  }
});

Finally, export an wrapped t method to mark the words:

function t(sentence: string, options?: I18NTranslationOptions {
    const key = hashKey(sentence);
    return i18next.t(key, { ...(options || {}), defaultValue: sentence }) as string;
  }

[Harukisatoh]

Since this repo looks unmaintained I made a fork of it that solves this issue, and I'm also publishing this fix into npm as i18next-scanner-fix.

Currently, the solution implemented into the fix was removing the enclosing node indexes tags from interpolated values, but this approach has a downside that is breaking translations when used with react-i18next with a version before v8.0.0.

Regardless of that, I would love to see this being implemented into the official lib, and I would certainly help develop a more backward-compatible approach if needed. For now, feel free to contribute to the fork by fixing known bugs.