[Mouse Jump] - screenshot performance (#24607)

[mikeclayton]

Summary of the Pull Request

• #24607

Composite Screenshot

Reduces the time required for Mouse Jump to display the initial desktop thumbnail image - instead of copying the entire bounding desktop rectangle (pink in image below) with CopyFromScreen it now uses multiple calls to native StretchBlt to capture the individual active areas of screens 1-6 and composite them into a thumbnail image.

Progressive Activation

If the form takes >250ms to capture screenshots it trips over into a "progressive" mode where it shows the form with solid rectangles instead of screenshots so that the user sees something happening, and then continues to capture the screenshots in the background and displays them when they're ready:

https://user-images.githubusercontent.com/1193763/224168931-4bf5a2ad-687d-45a3-a97c-85330194bf85.mp4

Background Texture

I've also added a slightly-graduated blue background to make it "pop" a bit more than the previous black background. This wasn't on the original ticket so I can take it out again if you prefer.

PR Checklist

• [X] Closes: #24607
• [X] Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
• [X] Tests: Added/updated and all pass (no new tests)
• [X] Localization: All end user facing strings can be localized (no new strings)
• [X] Dev docs: Added/updated (no new dev info)
• [X] New binaries: Added on the required places - no new binaries
• [X] Documentation updated: If checked, please file a pull request on [our docs repo] (no new documentation) (https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys) and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

See #24607 for full details

Validation Steps Performed

Manual tests of updated Mouse Jump feature with 2 screens at different dpi scale, and 6 screens in a jagged diagonal layout per image above. Preview display time is noticeably improved for the 6 screen layout and no slower for the 2 screen layout.

[github-actions[bot]]

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view or the :scroll:action log for details.

Unrecognized words (23)

ANDSCANS
BLACKONWHITE
Blt
CAPTUREBLT
COLORONCOLOR
DCs
DELETESCANS
DSTINVERT
MERGECOPY
MERGEPAINT
NOMIRRORBITMAP
NOTSRCCOPY
NOTSRCERASE
ORSCANS
PATCOPY
PATINVERT
PATPAINT
rop
SRCAND
SRCERASE
SRCINVERT
SRCPAINT
WHITEONBLACK

Previously acknowledged words that are now absent

Akrotiri Aktobe Andreanof appxpackage Aqtobe Atikokan Atyrau Avanc Bashkortostan Bayan Belarus Bson Burkina Buryatia Cabo Caiguna Choibalsan Chukotka Chuuk Clipperton Cocklebiddy Comoros corewebview createcompatibledc Cunha Danmarkshavn datatemplate DONTRESOLVEDLLREFERENCES enumerationoptions Eswatini Eucla Faroe Futuna getancestor getasynckeystate globalassemblycache handlerroutine Heure Hovd HWHEEL inputdev ipreviewhandler Ittoqqortoormiit ivirtualdesktopmanager Ivoire Khakassia Khanty Khovd Kitts Krai Kwango Kwilu Kyrgyzstan Kyzylorda LEFTDOWN LEFTUP LOADLIBRARYASDATAFILE Luhansk Maarten Macquarie Magadan Mangere Mangystau Mansi Marquesas Mato MCDT MCST menurc MIDDLEDOWN MIDDLEUP Mishkeegogamang Moldova Mongala Mundrabilla MYTZ Navassa navigatetostring Ndombe NOCOALESCE nonpackaged Noronha Nunavut Nusa oledbcommand oledbconnection Pitcairn Pohnpei Primorsky queryfocus registerhotkey resourcemanager RIGHTDOWN RIGHTUP Roamable Rothera Schd secauthz securityoverview shemptyrecyclebina SPACEBAR Srednekolymsk Sul Suri sysinfo Tajikistan Tenggara timezones tostring Transnistria Tshuapa Tuva TValue uapmanifestschema Udmurtia uiauto Ulaanbaatar UMsg Unstub Urville Uvs VIRTUALDESK virtualkey winauto wingdi winmsg winuser wtypes WVk XControl XDOWN XUP Yamalia Zabaykalsky Zonev :arrow_right:

To accept :heavy_check_mark: these unrecognized words as correct and remove the previously acknowledged and now absent words, run the following commands

... in a clone of the [email protected]:mikeclayton/PowerToys.git repository on the dev/mclayton/stretchblt-mouse-jump branch (:information_source: how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/v0.0.21/apply.pl' |
perl - 'https://github.com/microsoft/PowerToys/actions/runs/4358530256/attempts/1'

Available :books: dictionaries could cover words not in the :blue_book: dictionary

This includes both expected items (2200) from .github/actions/spell-check/expect.txt and unrecognized words (23)

Dictionary	Entries	Covers
cspell:cpp/src/cpp.txt	30216	119
cspell:win32/src/win32.txt	53509	116
cspell:python/src/python/python-lib.txt	3873	29
cspell:php/php.txt	2597	16
cspell:node/node.txt	1768	13
cspell:typescript/typescript.txt	1211	12
cspell:python/src/python/python.txt	453	10
cspell:java/java.txt	7642	10
cspell:aws/aws.txt	218	8
cspell:python/src/common/extra.txt	741	7

Consider adding them using (in .github/workflows/spelling2.yml):

      with:
        extra_dictionaries:
          cspell:cpp/src/cpp.txt
          cspell:win32/src/win32.txt
          cspell:python/src/python/python-lib.txt
          cspell:php/php.txt
          cspell:node/node.txt
          cspell:typescript/typescript.txt
          cspell:python/src/python/python.txt
          cspell:java/java.txt
          cspell:aws/aws.txt
          cspell:python/src/common/extra.txt

To stop checking additional dictionaries, add:

      with:
        check_extra_dictionaries: ''

If the flagged items are :exploding_head: false positives

If items relate to a ...

• binary file (or some other file you wouldn't want to check at all).

  Please add a file path to the excludes.txt file matching the containing file.

  File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

  ^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

• well-formed pattern.

  If you can write a pattern that would match it, try adding it to the patterns.txt file.

  Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

  Note that patterns can't match multiline strings.

[mikeclayton]

@crutkas @jaimecbernardo - this PR is ready for review I think (assuming the build passes).

I need to re-add some unit tests that broke during this PR, but any thoughts about the core functionality welcome...

[jaimecbernardo]

Hi @mikeclayton , Thanks for opening this PR. I've given it a test but it seems that it's failing to teleport the mouse pointer to the right screen in my case. Displays are 3 horizontal 1920x1080 side by side, where the middle one is the main display. I suspect this might be related to the left screen having negative coordinates.

Basically what I'm observing on Mouse Jump is that if I try to jump to screen 3, it jumps to screen 1. If I try to jump to screen 1, it jumps to screen 2. If I try to jump to screen 2, it seems to jump to screen 1 at its right edge, which is weird.

[mikeclayton]

@jaimecbernardo - ah, yeah at a guess that's a regression on the primary screen not being the top-left-most again.

I'll take a look later today and try to get my unit tests working again as they would have caught that if they were actually working :-S

[mikeclayton]

@jaimecbernardo - I think I've fixed the issue with the primary monitor (I rearranged my screens and it worked fine this time), and I've fixed the broken unit tests as well, so it's ready for another look when you get a chance.

Cheers,

M

[mikeclayton]

Btw, I've since found a rounding issue which means the preview form fails to load on some combinations of screen layouts. I've got a fix which I'll push this evening, if you want to hold off reviewing until that's ready...

[mikeclayton]

I've fixed the rounding issue - that's the last problem I know about so this is ready for review again now...

[mikeclayton]

There are conflicts here after merging this fix https://github.com/microsoft/PowerToys/pull/24527/files. Can you resolve the conflicts? it can be merged after that

@stefansjfw - I suspected there would be :-).

I’ll merge from main into here and resolve the conflict tonight.

[mikeclayton]

I've merged main into this PR and fixed the conflicts.

Currently waiting for the build to finish - if it works then it's good to go, or if it fails I'll take a look tomorrow...