community
community copied to clipboard
documize
Rendering Hindi(Indian) Language Wrong in Kivy for Android App
Software Versions
- Python:3.7
- OS:Android
- Kivy:1.11
- Kivy installation method: P4A
When I render the following text with the google NotoSansfont for Devanagari script in kivy android, I get it rendered wrongly. I have seen similar messages in SO for other languages and it says use pango text provider but it also mentions it is not available for android. Is there a possibility to render the font correctly in android in a kivy App. This will be a make or break for using kivy as the app development platform for us as we need to support Indian languages in our mobile app.
Example Text Expected output ->
Example Text Rendered Output ->
This happens in all the controls - Label and TextInputs and we have tried different ttf files - NotoSans, Akshar, atleast 10 other font files.
The only font which works correctly is GNU Unifont for us but that is because it has all the glyphs within and the problem we have is it does not look good and readable although that is now our current and only option.
Can we get some help here please?
example code: from kivy.app import App from kivy.uix.image import Image from kivy.uix.label import Label from kivy.uix.boxlayout import BoxLayout from kivy.uix.button import Button
class MainApp(App): def build(self): label = Label(text="अंतरिक्ष", font_name="notosans.ttf", size_hint=(0.5,0.5),pos_hint={'center_x':.5,'center_y':.5}) layout = BoxLayout(padding =10) layout.add_widget(label) return layout
if name=='main': app = MainApp() app.run()
Font file link : https://fonts.google.com/specimen/Noto+Sans?subset=devanagari
Let me know if I understand it correctly, the notosans font has the 'क्ष' glyph but is not displayed when you use it for the kivy label? I will try looking at this issue if that is the case.
That is a minor problem or a problem we can live with as the 'क्ष' glyph is not used very frequently in words. The major problem is the रि being rendered out of sequence as र and the ि glyph coming after the letter. The sequence is the problem. This is happening for other Indian languages as well like Tamil and Bengali . Example issue -> https://stackoverflow.com/questions/63047020/kivy-isnt-showing-bengali-unicode-character-properly
Thanks, that stack overflow question is helpful. There is long discussion on this here: https://github.com/kivy/kivy/issues/2669. This issue is still open.
A possible solution is here: https://github.com/kivy/kivy/issues/4902
According to #4902 , it suggests using pango textprovider which is possible in linux but I do not think it is possible with Android or the kivy packaging provides a means to use the renderer in Android using Python for Android. So do we conclude the Kivy does not support Indian languages in Android and move ahead?
From an immediate requirement point of view yes, but the problem of wrong sequencing of glyphs is happening in many of the Indian languages as mentioned above.
Can you try Arabic text reshaper ? https://github.com/mpcabd/python-arabic-reshaper I think I tried that once and it seemed to solve a lot of issues for me with Hindi too. Can not remember of it works with re specifically. Trying it out now to see if it works
If arabic text reshaper works then you can simply include it as is cause both arabic text reshaper and python-bidi are pure python and should work as is if you include them in your app dir
I tried your suggestion . Here is the code with the Arabic reshaper. It seems to render the text in the same way. Hope I used it in the right way, please check from your side as well.
from kivy.app import App
from kivy.uix.image import Image
from kivy.uix.label import Label
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.button import Button
import arabic_reshaper
class MainApp(App):
def build(self):
text = "अंतरिक्ष"
reshaped_text = arabic_reshaper.reshape(text)
label = Label(text=reshaped_text, font_name="notosans.ttf", size_hint=(0.5,0.5),pos_hint={'center_x':.5,'center_y':.5})
layout = BoxLayout(padding =10)
layout.add_widget(label)
return layout
if __name__=='__main__':
app = MainApp()
app.run()
btw the issue is not only with "re" it is with all alphabets with the ि extension. Examples Words below - In effect the language becomes unreadable overall because of this problem. कवि | तकिया | मिलन कविता | तिनका | मिला कितना | तिल | मिलाना
ok, I tried
import kivy.app
import kivy.uix.label
import arabic_reshaper
import bidi.algorithm
class TestApp(kivy.app.App):
def build(self):
reshaped_text = arabic_reshaper.reshape("अंतरिक्ष")
bidi_text = bidi.algorithm.get_display(reshaped_text)
return kivy.uix.label.Label(text=bidi_text, font_name="/Users/quanon/Library/Android/sdk/platforms/android-28/data/fonts/NotoSansDevanagari-Bold.ttf")
testApp = TestApp()
testApp.run()
Got the output:
From what I understand the problem is of wrong shape/positioning rather than of font fallback, correct? If so, we should be able to adapt arabic text reshaper or python-bidi for our purpose? Looking into it a bit ...
Yes, positioning primarily is wrong. If that can be done it would mean the world to us in terms of moving forward.
I am looking at https://github.com/python-pillow/Pillow/pull/2576/files this seems to have added support for reshaping to pillow directly which would mean we should be able to just use pillow text provider, though just using KIVY_TEXT=pil does not seem to fix anything for me… needs further investigation.
Update: related issue for reference https://github.com/python-pillow/Pillow/issues/1089
Yes, I tried the pil textprovider as well on Android, the text provider changes succesfully but the rendering remains the same.
While I am totally a noob with respect to font rendering aspect and just an application developer, I read in other posts that harfbuzz has capability to render complex fonts. I see when the p4a build is running for android there is now a freetype dependency for my application after I added the arabic-shaper and in the build log I see something like compiling freetype with harfbuzz=No . Would changing it somehow to harfbuzz=yes help us? Just a random observation, you can ignore it if it does not make sense.
some more reference https://forums.libsdl.org/viewtopic.php?p=48243 This patch seems like a good thing to integrate in p4a sdl2_ttf as a patch….
Update, p4a already includes harfbuzz. So we should just need to update sdl2_ttf to latest and to enable harfbuzz at compile time and have font reshaping. Theoretically this should then be able to be enabled by default on all platforms as sdl2 is the default text provider.
This is a good discussion. I am also leaving a reference to Pillow issue about this: https://github.com/python-pillow/Pillow/issues/3191
Any suggestions for a quick fix? Can I do something in my dev system which is ubuntu where my p4a build is running to try things out? To me p4a is a blackbox - so if you can help me with some instructions I can try it out in my system.
Any update on this please? We would need to make a decision based on this by end this week. Kivy meets all our requirements and is a pleasure to develop on, but this can be a blocker from our business perspective. So any help and exploration ideas will be useful.
I asked on the Sanskrit programmers Google group: https://groups.google.com/g/sanskrit-programmers/c/I57qkxqT-q0/m/EQ69f_TKAAAJ. Hoping to get some more response there.
Looks like this is not priority for the Kivy support or development. We shall be moving to native development this week. Multi Language support is now a de-facto requirement for software frameworks these days. Would be good if you can prioritize it in the future for developers to use it more.
@kriyanation it's important to keep in mind that none of the kivy developers are paid to work on Kivy and that we work on Kivy in our free time and according to our own interests. So generally it's considered rude to push the developers, who are gifting you their time, to meet any business deadlines as it will probably be counter productive.
If a feature has not been implemented, it means no one has so far found the time or interest to implement it, whether devs or users. The fastest way to get a feature, especially if you need quickly it for business purposes is to either implement it yourself and find a core-dev also interested in the issue to help you get it in, or to wait if a core-dev shows interest. But if that doesn't work for you then open source software may not match your requirements.
@akshayaurora @kriyanation
I don't know much about font rendering but interested in making kivy support indian language. Can we write a method to re-configure unicode characters like vowel sign for इ For example: 'किव' will give 'कवि' as per current situation.
A method that will automatically do this type of workaround for us. Also, according to hindi 'क्' + 'ष' = 'क्ष'
Rendering speed might be slow but it may do the work. If my idea is too innocent, then forgive me as a child.
@darpan5552 would it then work for all combinations of इ like बल्कि or किंतु ? We should be careful to not to have case by case implementation. That said, the focus for Kivy framework does not seem to be on language support. Not sure if this issue will get traction.
This can be a standalone project, based on kivy, but not a part of kivy, I
mean extension.
Think of you as a chini speaker, for example. Would you consider a
foreign language like hindi to donate important time to it?
Moreover, only a native speaker of language can create a text engine for
it. Kivy developers might not have familiarity with indian languages.
There are 3 ways to accomplish this task:
- Create a method (Suggested by me) Can't work for all fonts. Just a workaround.
- Connect to Harfbuzz shaping library (officialy suggested by kivy team) Covers all font types & styles
- Access Android api (backup plan by kivy team) Best way but does not guarantee success
Kivy docs mentioned some people to get help from.
On Fri, 25 Dec, 2020, 9:10 pm kriyanation, [email protected] wrote:
@darpan5552 https://github.com/darpan5552 would it then work for all combinations of इ like बल्कि or किंतु ? We should be careful to not to have case by case implementation. That said, the focus for Kivy framework does not seem to be on language support. Not sure if this issue will get traction.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kivy/kivy/issues/7149#issuecomment-751264523, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANZGQCEXNQ2LI2PERMA7TLDSWSXAJANCNFSM4SQ3WDDA .
Can you point me to some more details of 3. Access Android api (backup plan by kivy team) Best way but does not guarantee success? Your point 2 suggestion seems to be the best way, do let me know if there are some ideas or docs helping to implement the same. That said, this is a unicode support issue, it is not only causing problem for one language but multiple Indian languages and maybe other unicodes as well. When most of other renderers on the web and mobile including native Android support these languages, not sure if a contemporary framework should be not supporting it.
With all this I am still a big fan of Kivy - please do not get me wrong.....
Android api are actually rendering devnagari fonts properly. Best means perfection here. Does not guarantee success means what if future versions of android break those api for other languages, as kivy is pythonic.
I also appreciate your view that harfbuzz provides low level api for better chances of development.
On Fri, 25 Dec, 2020, 9:33 pm kriyanation, [email protected] wrote:
Can you point me to some more details of 3. Access Android api (backup plan by kivy team) Best way but does not guarantee success? Your point 2 suggestion seems to be the best way, do let me know if there are some ideas or docs helping to implement the same. That said, this is a unicode support issue, it is not only causing problem for one language but multiple Indian languages and maybe other unicodes as well. When most of other renderers on the web and mobile including native Android support these languages, not sure if a contemporary framework should be not supporting it.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kivy/kivy/issues/7149#issuecomment-751266522, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANZGQCDYBNG3VN2UBWY5S4TSWSZTJANCNFSM4SQ3WDDA .
About harfbuzz and SDL_ttf, this one recently got merged into SDL_ttf, so seems that We're really near to support you use case :
https://hg.libsdl.org/SDL_ttf/rev/b357aefce885
I'm currently busy, but if someone would like to update the SDL recipes on python-for-android, I'm sure it's a game-changer for you all.
