Handle minimized windows

[lwouis]

It is not possible to display thumbnails for minimized windows, but we could display a message instead.

Listing the minimized windows would be done by removing .optionOnScreenOnly in CGWindowListCopyWindowInfo([.excludeDesktopElements], kCGNullWindowID)

[lwouis]

Need to remove .optionOnScreenOnly to confirm window is not closed

I tried to implement this today but I keep hitting a wall. Without .optionOnScreenOnly, CGWindowListCopyWindowInfo doesn't list minimized windows, so we have to remove it. Otherwise how do we know that the window is not closed? Ok so we remove .optionOnScreenOnly. Now the number of windows returned is x10. The new windows are all bogus not-real-windows.

How to distinguish minimized windows from not-actually-windows?

I looked at all attributes of windows from the CGWindowListCopyWindowInfo call, but nothing is useful to discriminate. I thought the title could be a good way since most not-actually-windows don't have titles. However, it is legit for a window to have no title, so deadend. Then I tried to call CGWindowListCreateImage and notice that the not-actually-windows have width=height=1. However the minimized windows do to, so deadend.

Not sure how to proceed from here :/

How are similar apps doing?

• HyperSwitch doesn't support minimized windows at all
• WindowSwitcher supports it but I'm guessing they cache windows on their side because if I minimize a window then start WS, it doesn't show the minimized window. Also interesting, if I kill the app the windows belong to, WS correctly removes the app from its list

[lwouis]

Note this interesting approach to still getting a screenshot of a minimized window.

[lwouis]

This is the best approach I can imagine to implement given the terrible OS APIs:

• Call CGWindowListCreateImage without .optionOnScreenOnly. Cache the open windows in a map [ID: NSImageView]
• On further invocations, make a list of windows that left the list. Then call CGWindowListCreateImage without .optionOnScreenOnly. If the ID is in the list then it means the window is minimized, so show the cached image

[lwouis]

There may a call to the AX* APIs that would help us here. Need to review the options

[gcbw]

would just showing the icon in the thumbnail region suffice?

sometimes (specially when i option-click a inactive window because of muscle memory on other system) osx makes some window vanishes to who knows where... the only way to bring up the window again is to either original-alt-tab to the icon, or execute the app again. (there is no icon on the minimized window area or anything else that will work with the suggestions here, even original-alt-tab behaves weird if you press up/down on the icon it will show the desktop instead of the window)

[lwouis]

would just showing the icon in the thumbnail region suffice?

At the moment the issue lies in the technical implementation. I don't know how to do it. The OS APIs are super messy.

For the UX, it's either gonna be no thumbnail just title and app icon, or we keep the thumbnail from before. Or maybe still have a fresh thumbnail using the python trick but i doubt the performance is gonna be good enough to allow for that one

[mfn]

HyperSwitch doesn't support minimized windows at all

I though it did?

To be clear: what exactly do we call a minimized window? Pressing the yellow (-) or CMD-m, is it that or something else? Because such windows are shown on the CMD~ (i.e. application windows only), but not when viewing all windows over all applications.

And it's thumbnail is updated as anything else (delayed, when you hold CMD-~ and wait).

[lwouis]

To be clear: what exactly do we call a minimized window? Pressing the yellow (-) or CMD-m, is it that or something else?

Window goes in the dock. There is a setting in Mission Control which if you check makes the windows go in their app's icon instead of next to the trash icon in the dock with their own miniature. I mean it used to. I tried checking it now, and it doesn't change anything. Not sure what happened with this since I haven't used that settings in years

Because such windows are shown on the CMD``~

My god you're right. Why would they only show minimized window in that mode? Seems like a bug to me.

And it's thumbnail is updated as anything else

I tried a youtube video, and I don't see any update. I think they store the picture right before it's minimized and keep showing it.

[mfn]

Group windows by application

I see; I don't use this mission control and this setting is, as in your screenshot, not activated. However maybe I tweaked the settings somewhen in the past, can't say:

My god you're right. Why would they only show minimized window in that mode? Seems like a bug to me.

I never thought about this but obviously I'm used to it after so many years.

It keeps the list of all windows clean to some extend but if you want a specific one, you can find it within the apps list of windows. I find this useful.

I tried a youtube video, and I don't see any update. I think they store the picture right before it's minimized and keep showing it.

Regarding HyperSwitch, this is how I tested:

• terminal window
• hold cmd-~ and wait until thumbnail is updated
• change content in terminal window (I started mc :-)
• hold cmd-~ and wait until thumbnail is updated

Basically it works like anything else 🤷‍♀️

[koekeishiya]

@lwouis

I found this particular problem to be quite interesting, so I wanted to give this a test myself. The function I mentioned earlier does indeed allow you to perform a snapshot of a minimized window.

Some sample C code I used to test this (no error checking etc is being performed here, just the minimal amount of code to run a test):

// declaration of external function to retrieve the running process' connection to the WindowServer
extern int CGSMainConnectionID(void);

// declaration of external function for grabbing a snapshot from the WindowServer
extern CGError CGSCaptureWindowsContentsToRectWithOptions(int cid, uint32_t *wid, bool window_only, CGRect rect, uint32_t options, CGImageRef *image);

// Ask the system return a snapshot of the full rect of the given window
CGImageRef image = NULL;
CGSCaptureWindowsContentsToRectWithOptions(CGSMainConnectionID(), &window_id, true, CGRectZero, (1 << 8), &image);

// If we got a valid snapshot we store it to disk as a .png
if (image) {
    CFStringRef string = CFStringCreateWithCString(NULL, "file:////Users/Koe/Documents/test.png", kCFStringEncodingUTF8);
    CFURLRef url = CFURLCreateWithString(NULL, string, NULL);
    CGImageDestinationRef destination = CGImageDestinationCreateWithURL(url, kUTTypePNG, 1, NULL);
    CGImageDestinationAddImage(destination, image, NULL);
    CGImageDestinationFinalize(destination);
}

I have not verified which versions of macOS have this function available, but I highly suspect this to work all the way back to macOS El Capitan.

Edit:

As far as performance go, running the snippet took roughly ~34-38 milliseconds on my machine (i7 2.20GHz 8core CPU).

[lwouis]

@koekeishiya i implemented this private API in #78. It seems to work just fine, with even better performance than the public CG API that was used previously

Regarding compatibility, do you know if there are compat tables for different versions of macOS for the private APIs?

[koekeishiya]

Regarding compatibility, do you know if there are compat tables for different versions of macOS for the private APIs?

Not that I know about. The only way I can think of to check this would be to get a version of the framework for the macOS version you are wondering about, and then inspect the symbols for that version of macOS.

[lwouis]

I'm wondering for the UX part what to do with the minimized windows since they are not ordered from most-recently-used like the other windows we get from the CG API call.

I'm been thinking about different designs:

Segregated section

Separator

Marker (could be an icon, lower opacity, plied corner, etc)

[gingerr]

Thanks for the visual mocks.

I would go for the marker for now since its easier to implement as its just an overlay in every Cell (some flat icon from fontawesome or other possible sources). In the Preferences the user could maybe choose if minimized windows should be orderered in ThumbnailPanel first or last.

[gcbw]

nice work! ...this reminds me when doing anything on windows required documentation from the unofficial "win api" site, because everyone used the proprietary APIs for so long microsoft gave up on making them official

Apple have really replaced Microsoft on every front 😬

[lwouis]

This ticket and a bunch of others are closed in v2 released today. Feel free to test that new version out and give feedback here! Hopefully you experience better performance, can interact with minimized windows, and interact with windows from other spaces and displays. Cheers!