feat: Jinja template to translate contents dynamically

[AddouIsmail]

SUMMARY

Problematic to solve:

To manage translation for dimensions, we need to create various calculated dimensions for each language. If you want your dashboard to be translated in 5 languages, let's say because your organisation is present in 5 countries. So in this case you may create 5 calculated dimensions (for each column) that corresponds to the number of translations. And you need to create 1 dashboard / language using the appropriate translated column. The total number of translation is the product of distinct values count for all columns x languages

Feature description

This PR introduce a new function to Jinja context that allows translation ability for features with the least effort. The used library is translate == 3.6.1, it's quite simple as it calls api of translation services such as : deepl, mymemory, microsoft or also libre translate. This brings the ability to translate and have all in 1 dashboard and reduce number of calculated columns.

BEFORE

to handle translations for SQL we have to handle it value by value, so we create as many custom dimensions as the number of languages that we want to handle

SELECT 
    CASE 
     WHEN value = 'Good morning' THEN 'Bonjour'
     WHEN value = 'Good evening' THEN 'Bonsoir'
     ELSE 'No translation found'
    END AS greeting_fr

AFTER

Things could get much simpler, by creating 1 custom dimension or write it directly inside your dataset SQL query.

SELECT {{translate(value, 'fr')}} AS greeting_fr

or

SELECT {{translate(value, from_language='en', to_language='fr', add_to_cache_keys=True, escape_result=True)}} AS greeting_fr

TESTING INSTRUCTIONS

1. Set "ENABLE_TEMPLATE_PROCESSING": True on the config
2. Optional: set TRANSLATION_PROVIDER to one of the available providers deepl, mymemory, microsoft, libre.
3. Test on SQL Lab or create your dynamically translated dimension

ADDITIONAL INFORMATION

• [ ] Has associated issue:
• [ ] Required feature flags:
• [ ] Changes UI
• [ ] Includes DB Migration (follow approval process in SIP-59)
  • [ ] Migration is atomic, supports rollback & is backwards-compatible
  • [ ] Confirm DB migration upgrade and downgrade tested
  • [ ] Runtime estimates and downtime expectations provided
• [x] Introduces new feature or API
• [ ] Removes existing feature or API

This PR includes a fix of the error: error: Unused "type: ignore" comment that happens during the pre-commit.

Some next steps doable after this PR:

If this PR is accepted as a feature for superset, there is some next step features that could be implemented, or created on the issue tab.

• feature 1: Create translation table that maps values to their various translations in other languages. That could be enriched through the interface or through file imports. Therefore the current translate template will mainly use the translation mappings if exists and fallback on the api if the translation is not found.
• feature 2: Link the translation to the selected language on Superset
• feature 3: Send the translation language on the body of the guest_token for embedding

[codecov[bot]]

Codecov Report

Merging #23252 (8c4d9da) into master (9a4839f) will decrease coverage by 1.77%. The diff coverage is 59.19%.

:exclamation: Current head 8c4d9da differs from pull request most recent head 9746e29. Consider uploading reports for the commit 9746e29 to get more accurate results

@@            Coverage Diff             @@
##           master   #23252      +/-   ##
==========================================
- Coverage   67.50%   65.74%   -1.77%     
==========================================
  Files        1898     1907       +9     
  Lines       72978    73445     +467     
  Branches     7899     7973      +74     
==========================================
- Hits        49262    48283     -979     
- Misses      21691    23114    +1423     
- Partials     2025     2048      +23     

Flag	Coverage Δ
hive	?
mysql	78.40% <69.70%> (-0.08%)	:arrow_down:
postgres	78.46% <69.70%> (-0.08%)	:arrow_down:
presto	?
python	78.57% <69.70%> (-3.76%)	:arrow_down:
sqlite	76.93% <68.07%> (-0.08%)	:arrow_down:
unit	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...set-frontend/src/components/Table/VirtualTable.tsx	72.30% <0.00%> (-1.13%)	:arrow_down:
...t-frontend/src/dashboard/actions/dashboardState.js	55.76% <0.00%> (+1.77%)	:arrow_up:
superset-frontend/src/dashboard/actions/hydrate.js	2.06% <0.00%> (+0.19%)	:arrow_up:
...ard/components/FiltersBadge/DetailsPanel/index.tsx	82.22% <ø> (ø)
.../components/FiltersBadge/FilterIndicator/index.tsx	87.50% <ø> (ø)
...nd/src/dashboard/components/FiltersBadge/index.tsx	86.11% <0.00%> (ø)
...Filters/FilterBar/FilterControls/FilterControl.tsx	28.57% <ø> (ø)
...veFilters/FilterBar/FilterControls/FilterValue.tsx	5.71% <0.00%> (-0.29%)	:arrow_down:
...Filters/FilterBar/FiltersDropdownContent/index.tsx	20.00% <0.00%> (-5.00%)	:arrow_down:
...s/FilterBar/FiltersOutOfScopeCollapsible/index.tsx	16.66% <0.00%> (-3.34%)	:arrow_down:
... and 186 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[rusackas]

Hi @AddouIsmail - any intent to follow up on this PR? Just making sure questions get addressed, and that the PR gets a rebase if you want to merge it. Thanks!