OCR to clipboard hook for selections

[kaihendry]

Sometimes I'm working with screenshots and would be awesome to make a selection around some text and send it off to say https://aws.amazon.com/rekognition/ for it analyse and return the text to set my clipboard.

[MyriaCore]

OCR would be great honestly!

[nsa]

There is a powerful OCR engine from Google called https://github.com/tesseract-ocr/tesseract and it's open-sourced. It would be super cool if the maintainers could integrate tesseract to flameshot as the author suggested, by doing so there would no need for external API connections such as Amazon's, because tesseract works offline.

[JScriber]

There's a project called dpscreenocr that does the job, yet the selection is not as convenient and user-friendly as flameshot's. OCR integrated to flameshot with a dpscreenocr inspired configuration would be really great.

[enoy19]

I implemented that feature and created a pull request: #1239

[xeaon]

@enoy19 after fixing you typo/c&p error in 8432d12960c89730efbc3096b90f677cc3a97fa9, this compiles in Manjaro and the OCR works really great, thanks!

edit: you have to change the header to baseapi.h in src/tools/ocr/ocrtool.cpp#L22 as suggested in https://github.com/flameshot-org/flameshot/pull/1239#discussion_r582078087

[kaihendry]

https://news.ycombinator.com/item?id=27242392 discusses the topic in a MacOS context

[Pheggas]

Anything new around this feature request? It would be super cool to me tho. Oftentimes i'm working with text copying and this feature would be absolute killer!

[xeaon]

@Pheggas If you are comfortable with compiling yourself, check my comment above.

I'm using this feature since then and it works great.

[borgmanJeremy]

I am okay to merge this feature if we can:

• Fix the merge request to it passes CI
• Explain how to package / distribute the language packs on all platforms.

[Pheggas]

@Pheggas If you are comfortable with compiling yourself, check my comment above.

I'm using this feature since then and it works great.

I'd like to. But it would be my first compilation. I'll try to look at it. In case of failure, I'll write you 🙂

[leoneivaw]

I am anxious waiting for this feature! Thanks!

[irfan798]

For anybody waiting for the functionality I found this blog post: https://slint.github.io/blog/ocr-screenshot.html

If you install tesseract, it is basically a one liner

flameshot gui --raw | tesseract stdin stdout | xclip -in -selection clipboard

[ek1ng]

For anybody waiting for the functionality I found this blog post: https://slint.github.io/blog/ocr-screenshot.html

If you install tesseract, it is basically a one liner

flameshot gui --raw | tesseract stdin stdout | xclip -in -selection clipboard

It's an amazing solution! Thanks!

[siva-sub]

For anybody waiting for the functionality I found this blog post: https://slint.github.io/blog/ocr-screenshot.html

If you install tesseract, it is basically a one liner

flameshot gui --raw | tesseract stdin stdout | xclip -in -selection clipboard

For those using wayland use this command instead:

flameshot gui --raw | tesseract stdin stdout | wl-copy

it requires the installation of wl-clipboard, together with tesseract and the respective tesseract language data package.

For me in arch linux the packages are : tesseract tesseract-data-eng wl-clipboard

in arch linux it can be installed with: sudo pacman -S tesseract tesseract-data-eng wl-clipboard

[xeaon]

Last year, I've spent a view hours to adapt @enoy19's pull request #1239 to the newer codebase but rm -rf'ed the directory by accident.

Yesterday I've done it again. You can find my fork here: xeaon/flameshot. After taking a screenshot, you can choose the OCR tool to copy the tesseract output to your clipboard. If successful there is a notification.

As of now, you need to install the dependency tesseract-ocr and leptonica manually. It won't build in CI but I'm trying to fix it for Linux builds.

As mentioned before, I'm no C++/Qt dev, so this is pretty challenging. Feel free to contribute to my fork, if you have any skills. This feature can be merged, if we can

• pass CI
• Explain how to package / distribute the language packs on all platforms

as @borgmanJeremy mentioned earlier.

[pitfiend]

recently there's a new kid on the block for OCR, https://github.com/TheJoeFin/Text-Grab, it's open source and seems to be a very nice solution. maybe it could be added as a feature.

[mmahmoudian]

@pitfiend Thanks for your comment, but I am going to hide your comment, and here is few points to clarify my reasoning:

• The software you suggested it platform specific (Windows)
• If we implement a feature, our emphasis would be to implement it in a cross-platform way. There are some features that are Linux and macOS specific (like support of CLI), but Windows will never be our main focus, unless the maintainers of the project collectively agree otherwise. For now, the best-case scenario for the C# software you suggested is via a Windows-specific plugin.
• The OCR can easily be implemented with 3 lines of shell script (detailed explanation here: https://mehrad.ai/posts/20210702-extracting-payment-info-in-rasterized-invoice/#ocr ).

[Nasreddine]

This is my version in Ubuntu using custom shortcuts.

I've created a shell script named shot.sh and placed it in /home/me/apps/. [change me to your username] The script contains the following line:

flameshot gui --raw | tesseract stdin stdout | tr -d '\n' | xclip -in -selection clipboard

The addition of tr -d '\n' is specifically to remove line breaks from the output.

To utilize this script in Ubuntu, you can set up a custom keyboard shortcut as follows:

1. Open Settings.
2. Navigate to Keyboard > Keyboard Shortcuts.
3. Click on View and Customize Shortcuts.
4. Under Custom Shortcuts, click the + sign to add a new shortcut.
5. In the new shortcut setup, enter the following details:
   • Name: flameshot_tesseract
   • Command: sh /home/me/apps/shot.sh
   • Shortcut: [Choose your preferred key combination]

This will enable you to use the script conveniently with a keyboard shortcut of your choice.

Linux makes your digital life easier : )

[Sheldonimo]

Hi guys, this is my custom version to linux mint.

I have been exploring methods to enhance the OCR output, and my first step was to define the language settings, which can also involve combinations. Through varying the --psm parameters in Tesseract, I discovered that the value of 6, corresponding to "Assume a single uniform block of text," yields the most favorable results. Below is the complete command for your experimentation:

flameshot gui --raw | tesseract stdin stdout -l eng+spa --psm 6 | xclip -in -selection clipboard

It's noteworthy that the optimal performance is achieved using Tesseract version 5. There are various ways to download or even compile it. I'm attaching the simplest method, which is through the PPA:

sudo add-apt-repository -y ppa:alex-p/tesseract-ocr-devel

This PPA was sourced from the following link, which originates from the official Tesseract page:

• https://notesalexp.org/tesseract-ocr/#tesseract_5.x
• https://tesseract-ocr.github.io/tessdoc/#5xx

[erfanium]

+1 tesseract ocr should be a built-in functionality

[GrabbenD]

+1 tesseract ocr should be a built-in functionality

Here's a really good PR by @rsrdesarrollo :tada: https://github.com/flameshot-org/flameshot/pull/3074

It checks for Tesseract in PATH meaning the feature is opt-in and uses upstream updates rather than a statically linked library (which would inevitably become outdated)

---

In the meantime, I found @DO-Ui's Tesseract workaround for small letters to work really good in Sway (Wayland)

bindsym --no-repeat $MOD+PRINT exec flameshot gui --raw | convert -resize 400% png:- png:- | tesseract -l eng stdin stdout | wl-copy