pdfplumber icon indicating copy to clipboard operation
pdfplumber copied to clipboard

Error While extracting Non-English tables (Arabic) reversed Text

Open iiAmeer opened this issue 1 year ago • 8 comments

I have a problem, when i use extract_tables function, i get a reversed arabic text. Like the actual name is joe, but i get eoj. (it's arabic but i wrote english so u can understand), there is no need for screenshots its clear.

Environment

  • pdfplumber version: latest
  • Python version: 3.11
  • OS: Windows (later linux)

iiAmeer avatar Jun 23 '24 11:06 iiAmeer

Thanks for noting this. Could you share the PDF? That will make it easier to diagnose the issue and suggest solutions.

jsvine avatar Jun 25 '24 13:06 jsvine

Thanks for noting this. Could you share the PDF? That will make it easier to diagnose the issue and suggest solutions.

087_ثانوية الإمام علي بن ابي طالب الأهلية للبنات (1).pdf

This is the file i used, if you want my code & run preview wait me until tomorrow .

IMG_20240626_092555.jpg

iiAmeer avatar Jun 25 '24 17:06 iiAmeer

Thanks for providing the PDF. Ignoring the numbers for now, and instead focusing on the text ... does the output of this look more like what you want?:

import pandas as pd
table = page.extract_table(dict(text_char_dir_render="rtl"))
pd.DataFrame(table, columns=None).fillna("")
Screenshot 2024-07-01 at 11 09 20 PM

jsvine avatar Jul 02 '24 03:07 jsvine

Thanks for providing the PDF. Ignoring the numbers for now, and instead focusing on the text ... does the output of this look more like what you want?:

import pandas as pd
table = page.extract_table(dict(text_char_dir_render="rtl"))
pd.DataFrame(table, columns=None).fillna("")
Screenshot 2024-07-01 at 11 09 20 PM

yes, exactlly. BUT numbers are IMPORTANT and can't be reversed. Note second col (13 in ur viewer) refer to index , so it can't be reversed. Please apply reverstion in RTL only langs like Arabic, Aramaic. Azeri. Dhivehi/Maldivian. Hebrew. Kurdish (Sorani) Persian/Farsi. Urdu. etc...

iiAmeer avatar Jul 16 '24 13:07 iiAmeer

@jsvine im waiting....

iiAmeer avatar Aug 10 '24 08:08 iiAmeer

Thanks for your patience, @iiAmeer, and thank your July 16 response above. This is indeed something I'd like pdfplumber to handle well. It just happens to be a bit tricky.

jsvine avatar Aug 10 '24 19:08 jsvine

@jsvine it's actually easy (i think) , as the module loops on every character, just try to convert str to int. so when its an integer it won't be reversed. Pdfplumber is my favourite pdf management tool So please do your best <3.

iiAmeer avatar Aug 14 '24 12:08 iiAmeer

I encountered the same issue with Arabic content, where some strings contain both text and numbers. While the numbers are displayed correctly, the text appears reversed. To address this, I created a function as a workaround.

import re

def fix_arabic_with_numbers(text):
    """
    First reverses characters in each word, then reverses word order.
    Numbers remain unchanged.
    """
    if not isinstance(text, str):
        return text
    
    # Convert multiple spaces to single space and strip
    text = ' '.join(text.split())
    
    # Split into words
    words = text.split(' ')
    
    # Process each word - reverse characters but keep numbers
    fixed_words = []
    for word in words:
        # Split word into number and non-number parts
        parts = re.findall(r'\d+|[^\d]+', word)
        fixed_parts = []
        for part in parts:
            if part.isdigit():
                fixed_parts.append(part)  # Keep numbers as is
            else:
                fixed_parts.append(part[::-1])  # Reverse characters in text
        fixed_words.append(''.join(fixed_parts))
    
    # Reverse the order of words
    fixed_words = fixed_words[::-1]
    
    # Join words back together with spaces
    return ' '.join(fixed_words)

I hope that languages with RTL (Right-to-Left) will have support in the future.

moamen270 avatar Nov 09 '24 00:11 moamen270