Jiapeng Wang

Hi, I just created our organization: https://huggingface.co/SCUT-DLVCLab.

My email is [email protected]. Thanks for your effort :)

Hi folks， Happy to share that LiLT has been added to HuggingFace/transformers library：https://huggingface.co/docs/transformers/main/en/model_doc/lilt ! Thank @NielsRogge for his great efforts!

Hi, sorry for that I haven't used colab...

Hi, since LiLT does not introduce image information yet, it cannot directly output the probabilities of each pixel. However, maybe you can first use the OCR engine to get the...

Hi, due to the limitation of computing resources, we haven't trained the large LiLT model yet. It is considered for future work.

Hi, 1. Yes. 2. During pretraining, ```Python layout_attention_scores = tmp_layout_attention_scores + tmp_attention_scores ``` is changed to ```Python layout_attention_scores = tmp_layout_attention_scores + tmp_attention_scores.detach() ```

“its overall consistency” said in the paper means the **optimization consistency** of the text flow. We don't want the gradients to back-propagate from **layout_attention_scores** to **tmp_attention_scores**, to influce the optimization...

The layout flow needs to influence the text flow in inference but not in optimziaiton; The text flow needs to influence the layout flow in both inference and optimziaiton.

```layout_attention_scores = tmp_layout_attention_scores + tmp_attention_scores.detach() ``` means 1) text influences layout in inference calculation and 2) layout **does not** influence text in back-propagation; ```attention_scores = tmp_attention_scores + tmp_layout_attention_scores ``` means...