[$500] [SMARTSCAN] HIGH: Show PDF receipt thumbnails

[trjExpensify]

Coming from here. CC: @sobitneupane @pradeepmdk @Gonals

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Version Number: v1.3.98-5 Reproducible in staging?: Y Reproducible in production?: Y If this was caught during regression testing, add the test name, ID and link from TestRail: Email or phone of affected tester (no customers): Logs: https://stackoverflow.com/c/expensify/questions/4856 Expensify/Expensify Issue URL: Issue reported by: @trjExpensify Slack conversation: https://github.com/Expensify/App/pull/30747#issuecomment-1802321587

Action Performed:

1. Request Money > Scan > Upload a PDF receipt > choose a participant
2. Observe there isn't a preview of the PDF receipt in the thumbnail on the confirmation page
3. Click confirm to create the request
4. Observe there isn't a preview of the PDF receipt in the thumbnail on the report preview component in the chat
5. Click the report preview component
6. Observe there isn't a preview of the PDF receipt in the thumbnail on the request preview component in the expense/iouReport
7. Click the request preview component
8. Observe there isn't a preview of the PDF receipt in the thumbnail on the receipt preview component in the request view

Expected Result:

PDF receipts should show a preview in the relevant thumbnails throughout the app.

P.s this also includes in the split preview component, so let's make sure it works there.

Actual Result:

Generic "empty" thumbnail previews are displayed for PDF receipts.

Workaround:

Yes, they can tap the thumbnail to see the PDF but they'd really not know to do that.

Platforms:

Which of our officially supported platforms is this issue occurring on?

• [ ] Android: Native
• [ ] Android: mWeb Chrome
• [ ] iOS: Native
• [ ] iOS: mWeb Safari
• [ ] MacOS: Chrome / Safari
• [ ] MacOS: Desktop

Screenshots/Videos

Confirmation screen:

Report preview on the chat:

Request preview on the expense/iouReport:

Receipt preview on the request:

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01f90fbd2f6a7c49f2
• Upwork Job ID: 1725113274629943296
• Last Price Increase: 2023-11-23

[melvin-bot[bot]]

Current assignee @trjExpensify is eligible for the Bug assigner, not assigning anyone new.

[melvin-bot[bot]]

Job added to Upwork: https://www.upwork.com/jobs/~01f90fbd2f6a7c49f2

[melvin-bot[bot]]

Bug0 Triage Checklist (Main S/O)

• [ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• [ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• [ ] This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• [ ] This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• [ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[melvin-bot[bot]]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @0xmiroslav (External)

[tienifr]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Show PDF receipt thumbnails

What is the root cause of that problem?

We're getting thumbnail for receipt here

https://github.com/Expensify/App/blob/69a33583877f26d7c446ba231af6fc2c477cca5b/src/libs/ReceiptUtils.ts#L31

We do not have the logic to get the thumbnail of pdf

-> Default image is shown

What changes do you think we should make in order to solve the problem?

We can lean on the source to get the thumbnail for pdf.

Here's the sample source https://www.expensify.com/receipts/{id}.pdf -> the thumbnail should be https://www.expensify.com/receipts/{id}.jpg.1024.jpg

    const isReceiptPDF = Str.isPDF(filename);
...

 if(isReceiptPDF){
            return {thumbnail: `${path.substring(0, path.length-4)}.jpg.1024.jpg`, image: path};
        }

What alternative solutions did you explore? (Optional)

BE will return the thumbnail and use it to shown on FE side

Result

https://github.com/Expensify/App/assets/113963320/c7dff2d8-4bfe-45c2-b1fe-ddefd2f3c0be

[trjExpensify]

@0xmiroslav can you review this please? Also, @tienifr confirming in the results we're solving for all the thumbnails noted in the OP?

[eh2077]

Proposal

Please re-state the problem that we are trying to solve in this issue.

We want to show PDF thumbnail in the process of uploading pdf receipt.

What is the root cause of that problem?

The root cause of this issue is that, currently, we don't have an implementation of previewing PDF source by thumbnail. The PDF source here means expensify source like https://www.expensify.com/chat-attachments/7486855856104260927/w_1c2aef0dea2f6e30acd3ca31c50286af09a4d3c9.pdf or local file source starts with blob:// or file://.

Currently, uploading PDF attachment from chat can be previewed before upload and the PDF thumbnail is only available after it's been successfully uploaded and the backend will create an image source url for the PDF, like, https://www.expensify.com/chat-attachments/7486855856104260927/w_1c2aef0dea2f6e30acd3ca31c50286af09a4d3c9.jpg.1024.jpg

Intuitively, we should be able to preview the PDF thumbnail before uploading it to backend because we're able to preview PDF pages before uploading.

The new feature to have a ThumbnailPDF which works on both server and local uri like what Image component supports is great complement to the App, especially for the use case like creating an IOU request and the file is not yet been uploaded to backend and the offline use case.

What changes do you think we should make in order to solve the problem?

We can add an implementation of ThumbnailPDF which previews pdf uri (both server and local uri) in thumbnail. On the web platform, we can refer the implementation of PDFView by only rendering the first page of PDF file as the thumbnail. We can convert the first page to image, like this example.

We can also update react-pdf to 7.x.x and then use the thumbnail feature provided. There's some extra rendering issue need to be solved if we choose to upgrade.

On native platform, we can make use of the singlePage property of https://github.com/wonday/react-native-pdf.

This solution will need more effort to implement.

What alternative solutions did you explore? (Optional)

[melvin-bot[bot]]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[trjExpensify]

@0xmiroslav bump!

[trjExpensify]

Same. @0xmiroslav can you review these proposals today, please? They've been hanging out for 2 weeks.

[0xmiroslav]

reviewing today

[trjExpensify]

Perfect, thanks.

[trjExpensify]

@0xmiroslav did you review? 2 weeks is a pretty long time, and PDFs are pretty common receipt types. Thanks!

[0xmiroslav]

We definitely need thumbnail from backend like chat pdf attachment. But do we need to show thumbnail for local pdf (before uploading) as well? cc: @trjExpensify

[0xmiroslav]

When show local thumbnail, need to match getting thumbnail logic with backend for consistency Going to pull up engineer directly before proposal approval, since this requires help from backend 🎀 👀 🎀

[melvin-bot[bot]]

Triggered auto assignment to @luacmartins, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[trjExpensify]

But do we need to show thumbnail for local pdf (before uploading) as well?

Yeah, we should show you a preview of the PDF like we do a preview of the local image attachment prior to upload.

[0xmiroslav]

But do we need to show thumbnail for local pdf (before uploading) as well?

Yeah, we should show you a preview of the PDF like we do a preview of the local image attachment prior to upload.

It's actually full preview, not thumbnail. So there's no pre-existing solution to use.

[luacmartins]

@0xmiroslav should I review any specific proposal?

[0xmiroslav]

@0xmiroslav should I review any specific proposal?

There's no acceptable proposal yet since we agreed to show thumbnail for local pdf as well. But I need your help sharing backend logic to get thumbnail for pdf so that we can apply same to frontend.

[luacmartins]

We simply replace the extension .pdf with .1024.jpg, so the resulting url is filename.1024.jpg. Is that what you needed?

[melvin-bot[bot]]

@trjExpensify @luacmartins @0xmiroslav this issue was created 2 weeks ago. Are we close to approving a proposal? If not, what's blocking us from getting this issue assigned? Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!

[0xmiroslav]

We simply replace the extension .pdf with .1024.jpg, so the resulting url is filename.1024.jpg. Is that what you needed?

ok 2 more things:

• always shows first page as thumbnail
• max size is 1024, keep same ratio as original pdf size. i.e. 649x1024

[luacmartins]

I think we should show the first page as thumbnail, just like we do for regular pdf attachments. As for the ratio, I'm not sure what you mean. We should show the preview in the same format/dimensions we show other receipt file types, no?

[0xmiroslav]

As for the ratio, I'm not sure what you mean. We should show the preview in the same format/dimensions we show other receipt file types, no?

It's preview size in the app. I meant the real size of generated image. If you see pdf in chat attachment, the size will be something like w x 1024 even though preview in report action item is much smaller.

[luacmartins]

Yes, we scale the thumbnail to fit a bounding box of 1024x1024 keeping its aspect ratio and only scaling down if the image is larger than 1024px.

[melvin-bot[bot]]

@trjExpensify @luacmartins @0xmiroslav this issue was created 2 weeks ago. Are we close to approving a proposal? If not, what's blocking us from getting this issue assigned? Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!

[0xmiroslav]

We're waiting proposals based on expected behavior discussed

[melvin-bot[bot]]

@trjExpensify @luacmartins @0xmiroslav this issue was created 2 weeks ago. Are we close to approving a proposal? If not, what's blocking us from getting this issue assigned? Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!

[tienifr]

@luacmartins Can you share the logic to generate the thumbnail of pdf

We simply replace the extension .pdf with .1024.jpg, so the resulting url is filename.1024.jpg. Is that what you needed?

we have to generate the filename.1024.jpg before replace the extension .pdf with .1024.jpg.

Or maybe we can use the library to do that like react-native-pdf-thumbnail. cc @0xmiroslav