[Anchors] Improve the auto position system to support all cases #194

gileli121 · 2022-01-14T20:36:54Z

Sometimes the auto-position system will fail to move the anchors away from texts or images.
Examples of such cases (reported by @cpasoft):

The reason is that the existing UI Automation API failed to detect the elements on the window below the anchors.
When the OS does not know what the window shows, it can't report to WindowTop where the texts are.
As a result, the system in WindowTop will not able to calculate available positions because it has no information about where the texts/images are located.

Until now, it was impossible to fix this issue.
But thanks to what found here: microsoft/Windows.UI.Composition-Win32-Samples#97 (comment)

It should be possible to screenshot the window silently and then use fallback logic that will try to map the location of empty spaces" (where there is no text/image) using image processing.
After the mapping is done, the system will know where to move the anchors

cpasoft · 2022-01-15T09:48:00Z

I've tested the "auto position system" on others app, and not working either...
Typora is another example of this wrong behaviour.
Improve this system would be great, but I don't know how will it affect in processing time.
I've disabled the "auto position system" and set the right side position for the anchors. I think this is enough for me by now.
Anyway, I'll do any test you need if you finally decide to improve this system...

And as always, thank you for your efforts and dedication...

My best regards...

gileli121 · 2022-01-15T11:05:20Z

but I don't know how will it affect in processing time

I also don't know. I have some tricks.
You probably think that because it will be based image processing it will work slower but you may be wrong.
I know how to write very efficient algorithm for this job.
In addition, I will add roles for when and how to reduce the processing

I've disabled the "auto position system" and set the right side position for the anchors

This is what I don't like. If you disabled it, it mean that it is not good enough so need to be fixed

cpasoft · 2022-01-15T12:06:45Z

This is what I don't like. If you disabled it, it mean that it is not good enough so need to be fixed

Then, let's go for it... i will help you in any way i can!!! 👍
:-)

gileli121 · 2022-01-17T22:24:22Z

After about 3 days I finally start to get it right

But it is still not working perfect. There is more work to do here.
At least there is good POC

cpasoft · 2022-01-18T18:52:42Z

Great news... It sound promising... Keep doing!!!
👍

gileli121 · 2022-01-18T19:18:15Z

This is extremely complex.
I will have to rewrite it 2-3 times until it is finalized.
But this is why I so like it.. It is better than playing games

gileli121 · 2022-01-21T12:29:51Z

@cpasoft
Finally I got it right now

Do you use Windows 10 or 11?
It may not work in Windows 10 because the new API missing some feature in Windows 10

cpasoft · 2022-01-21T19:13:33Z

Uooohhh... It's look impressive!

I've use both OS, but mainly Windows 10.

gileli121 · 2022-01-21T19:30:10Z

We will have today or tomorrow first beta version.
Just need now to move all the code to interface so that I can create one implementation for Windows 10 and one for Windows 11.
It will use the new implementation in Windows 11. In Windows 10 it will use the old implementation so you will not see the improvement in Windows 10. This is until the new API will be fully available in Windows 10 also.

I also found that the new implementation using 10% GPU.
Probably this can be fixed by reducing the capture frame rate...
The auto-position algorithm do it's job every 1 second.
But there are multiple threads that in each thread it capturing non-covered window.. These threads capturing each window in very high frame rate so I think that reducing the capture rate will fix it

gileli121 · 2022-01-21T22:06:22Z

I fixed the 10% GPU usage. Now it will use up to 1.5% according to my tests.
Indeed - reducing the capture frame rate solve it.

cpasoft · 2022-01-22T12:06:12Z

Great news... Waiting for test it... 😃

gileli121 · 2022-01-22T12:18:38Z

I just finished the algorithm for Windows 11, including fixing all bugs that I found.
But did not moved yet the algorithm to an interface. So it will use the same algorithm in Windows 10 and 11 (and this will break in Windows 10)

I can release for now version that this will work only on Windows 11

gileli121 · 2022-01-22T12:36:25Z

@cpasoft
Try this on Windows 11:

This will work only on Windows 11 for now or in Windows 10 insider preview version.

Let me know how it works.
Thanks

gileli121 · 2022-01-22T12:54:41Z

The auto-move of the PiP/Shrink was also improved

Now the PiP window should work better with auto-move. It is less likely to cover texts/images

gileli121 · 2022-01-23T20:50:40Z

Today I moved both algorithms to interface IAutoMoveManager
First one is AutoMoveManagerGCImpl : IAutoMoveManager implementation - the new one that based on API https://docs.microsoft.com/en-us/uwp/api/windows.graphics.capture?view=winrt-22000
Second one is AutoMoveManagerUIAutImpl : IAutoMoveManager implementation that based on https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-overview API

Tomorrow I will implement logic for choosing which implementation to use based on the OS build and maybe also by user settings

AutoMoveManagerUIAutImpl : IAutoMoveManager:

AutoMoveManagerGCImpl : IAutoMoveManager:

gileli121 · 2022-01-23T21:09:05Z

gileli121 · 2022-01-23T21:50:34Z

@cpasoft
New beta:

cpasoft · 2022-01-25T18:02:00Z

Thanks @gileli121 ... I have been very busy this last days. I'll install the new beta2 on Windows11 tomorrow. I'll let you know how it works to me... Great work my friend!!! :-D

gileli121 · 2022-01-26T00:31:49Z

@cpasoft
Beta 2 still come with algorithmic bugs... Sometimes the algorithm moved the icons without valid reason... I fixed these cases now.

In addition, I optimized the code. Now instead of 1%-2% CPU usage, it uses 0% CPU in my computer.
And I also added logic that it will automatically choose the auto-position implementation based on available API in the OS

Try this version:

cpasoft · 2022-01-27T12:00:21Z

Hello @gileli121 ....
I've tested the three beta version in a computer without dedicated GPU (Integrated Intel UHD Graphics 630).
Here the CPU Usage of each version:

Beta1: 4~2% CPU
Beta2: 1~0,40% CPU
Beta3: 0,2-0% CPU

It works very well to me... Tested with Notepad++. I'll do more test with others app, like Typora , but it seems to work very well...
Congrats, very good work!!!

cpasoft · 2022-01-27T14:49:13Z

Tested in other aps, like typora... Everything works right by now...
I'd like to test it in Windows 10. Let me know when it will be ready for this... ;-)

Thanks!

gileli121 · 2022-01-27T17:16:42Z

@cpasoft
Thanks.
We are almost done here!
But there is more work to do.

Seems that the auto-position algorithm has a small bug when it tries to move the icons to the left side...
It may calculate wrong the position in this case (of moving left). I will try to fix it today or tomorrow.

In addition, there is a memory leak issue. If you will resize windows, the system in windowtop will reallocate memory for the capture and during the memory allocation, it did not clean the old used RAM. I will fix this bug also.

Need also to see how it works when using multiple monitors.
I did not test it yet.

About Windows 10, it will take time. It depends on when the next Windows 10 update will be released.

gileli121 · 2022-01-28T22:39:05Z

@cpasoft
I finished to develop beta 4.

Implemented: the anchors will try to move lowest distance to the left or to the right and if no such found, it will try to move lowest distance to the top or bottom
Fixed bug with 2 monitors that there is crash when moving window to second monitor
Fixed multiple bugs with the auto move on second monitor (not main monitor)
Fixed memory leak issue when resizing windows / when enabling/disabling anchors
Fixed issue that sometimes the app crash due to memory issue
Fixed multiple minor bugs in the auto-position algorithm

Download beta 4

gileli121 · 2022-01-28T23:14:10Z

@cpasoft
There was regression in the auto-position algorithm that the algorithm sometimes returned before finish... So sometimes anchors did not moved.

Try beta 5. It should be very close to final:

Let me know how it works.
Thanks.

gileli121 · 2022-01-30T07:48:52Z

@cpasoft any update please?
I created now beta 6.

Fixed: Bug when combining Shrink/PiP + Interact mode with Anchors. When clicking inside the PiP window, WindowTop showed anchor for the hidden window (due to the PiP)
Fixed potential issue that DWM may crash

gileli121 · 2022-03-11T14:00:34Z

I implemented logic to show the full error information also in these cases

Now it will show more dump from the failure.

It now should be fixed (you should not get error anymore).
But if you will still get error, it is expected that you will see full dump like in the screenshot.
So we should see better error information in worst case and I will able to understand the failure cause better.

Please try this version:

gileli121 · 2022-03-11T14:01:15Z

@cpasoft
See my last message

gileli121 · 2022-03-11T16:12:46Z

@cpasoft
In this version I added the toolbar settings to the wizard.
This change is not related to this feature but I anyway send it for testing in case it caused some other bug

Download:

gileli121 · 2022-03-11T23:14:30Z

I implemented logic to show the full error information also in these cases Now it will show more dump from the failure.

It now should be fixed (you should not get error anymore). But if you will still get error, it is expected that you will see full dump like in the screenshot. So we should see better error information in worst case and I will able to understand the failure cause better.

Please try this version: [WindowTop 5.11.0-beta6 - setup.exe.zip]

@cpasoft
I made small mistake. I forgot to cover more cases in the code so sometimes it still could show undetailed error like in your screenshot.

Please try this beta:

cpasoft · 2022-03-14T17:35:16Z

Hello @gileli121 ...
I was out for the weekend. I'm testing the beta8 right now... I'll let you know in some minutes how it works... ;-)

cpasoft · 2022-03-14T17:41:59Z

@gileli121 , here you have the full error:

Disabling anchors, everything seems to work right...

cpasoft · 2022-03-14T17:43:52Z

let me know if you want me to try something

gileli121 · 2022-03-14T17:58:22Z

@cpasoft
Thanks.
This is definitely much more detailed error!
Now I know from where it comes.

I will check it

cpasoft · 2022-03-14T18:02:32Z

Great!... :-)

gileli121 · 2022-03-14T19:09:41Z

@cpasoft
The error helps me to know what method broke it but I still can't tell what is the broken code in that method.

I need that you will do these steps:

Download and install this version:
Download this file:
NLog.zip
Extract from there the file NLog.config
Put the NLog.config in C:\Program Files\WindowTop
Start WindowTop
Reproduce the problem and right after you got the error, exit the app
Go to %appdata%/windowtop
Send me the file WindowTopLog.log that created there.

Thanks.

cpasoft · 2022-03-14T19:48:22Z

With this version the error window not appear...
I'll send you the WindowTopLog.log by email, because there is some personal information on it...

;-)

gileli121 · 2022-03-15T18:36:49Z

@cpasoft
Good news!
The logs that you sent finally provided enough information to reproduce the bug on my side.

All it takes to reproduce it is to disable this option:

and restart the app with the anchors feature enabled.

I will make a fix now

gileli121 · 2022-03-15T18:54:32Z

@cpasoft
Please try this beta:

Let me know if it fixed.
Please also try in on Windows 11 with the auto-move enabled when you can.
Thank you

cpasoft · 2022-03-16T18:59:01Z

Perfect @gileli121 ... I'm downloading right now and I'll test it on Windows 10.
Tomorrow I'll test it on Windows 11.
I'll give you reports

cpasoft · 2022-03-16T19:20:20Z

On Windows 10 it's working flawlessly right now. No errors at all...
I'll test it on Windows 11 tomorrow, with and without "stay away" feature...
Great job my friend :-)

cpasoft · 2022-03-17T07:26:37Z

I'm testing right now under windows 11. There are not error shown. I don't know if the orange error windows has been restored in this version, or the errors are only shown on the log. I'll enable the error log and see if there is something relevant there...

cpasoft · 2022-03-17T07:32:20Z

Checking the windowtoplog.log, no errors has been found...
CONGRATS!... Great job!!!...

gileli121 · 2022-03-17T17:22:36Z

@cpasoft
Thanks!
Great news

So I will soon release new version

gileli121 · 2022-03-17T22:05:42Z

@cpasoft
Released
https://github.com/BiGilSoft/WindowTop/releases/tag/v5.11.0
This version also contains few changes that should prevent cases that the new algorithm freeze sometimes.
It will be great if you check it also in WIndows 11.

I released because this time I am sure that it should be fine but I still ask to test just in case.
Thanks

cpasoft · 2022-03-18T08:11:14Z

Perfect... downloading right now in both OS (Windows 10 & 11)...
Is the NLog working in this version as well?

cpasoft · 2022-03-18T08:32:54Z

Is the NLog working in this version as well?

Answering to myself... it is still working in this version. No errors has been found in Windows 10 neither Windows 11. All seems to work right, with & without the "stay away from text" option.
I'll be testing all the day... If I found something I'll let you know...

gileli121 · 2022-03-18T08:45:51Z

@cpasoft
Thanks!
Thanks to people like you I keep working on this project

gileli121 · 2022-03-22T20:21:37Z

@cpasoft
I am closing this task.
If you still get issues, please report here.

Thanks for your effort in testing it!

cpasoft · 2022-03-23T06:31:31Z

I'm using the last beta without any errors. Everything working right by now...
Thanks for all the improvements. 👍

gileli121 added the enhancement New feature or request label Jan 14, 2022

This was referenced Jan 14, 2022

Implement an option to hide text for anchors if icons are selected. #181

Closed

[Opacity] Add option in opacity to change back to 100% when the mouse is above the window #190

Closed

gileli121 added the in-progress label Jan 15, 2022

gileli121 pinned this issue Jan 22, 2022

gileli121 mentioned this issue Mar 15, 2022

[PiP] Video Aspect Ratio Enquiry #210

Open

gileli121 added done and removed in-progress labels Mar 17, 2022

gileli121 closed this as completed Mar 22, 2022

gileli121 removed the testing... label Mar 22, 2022

gileli121 unpinned this issue May 20, 2022

[Anchors] Improve the auto position system to support all cases #194

[Anchors] Improve the auto position system to support all cases #194

Comments

gileli121 commented Jan 14, 2022 • edited Loading

cpasoft commented Jan 15, 2022

gileli121 commented Jan 15, 2022 • edited Loading

cpasoft commented Jan 15, 2022

gileli121 commented Jan 17, 2022

cpasoft commented Jan 18, 2022

gileli121 commented Jan 18, 2022

gileli121 commented Jan 21, 2022

cpasoft commented Jan 21, 2022

gileli121 commented Jan 21, 2022

gileli121 commented Jan 21, 2022

cpasoft commented Jan 22, 2022

gileli121 commented Jan 22, 2022 • edited Loading

gileli121 commented Jan 22, 2022 • edited Loading

gileli121 commented Jan 22, 2022

gileli121 commented Jan 23, 2022 • edited Loading

gileli121 commented Jan 23, 2022

gileli121 commented Jan 23, 2022 • edited Loading

cpasoft commented Jan 25, 2022

gileli121 commented Jan 26, 2022 • edited Loading

cpasoft commented Jan 27, 2022 • edited Loading

cpasoft commented Jan 27, 2022 • edited Loading

gileli121 commented Jan 27, 2022

gileli121 commented Jan 28, 2022 • edited Loading

gileli121 commented Jan 28, 2022 • edited Loading

gileli121 commented Jan 30, 2022 • edited Loading

gileli121 commented Mar 11, 2022 • edited Loading

gileli121 commented Mar 11, 2022

gileli121 commented Mar 11, 2022 • edited Loading

gileli121 commented Mar 11, 2022 • edited Loading

cpasoft commented Mar 14, 2022

cpasoft commented Mar 14, 2022 • edited Loading

cpasoft commented Mar 14, 2022

gileli121 commented Mar 14, 2022

cpasoft commented Mar 14, 2022

gileli121 commented Mar 14, 2022 • edited Loading

cpasoft commented Mar 14, 2022

gileli121 commented Mar 15, 2022 • edited Loading

gileli121 commented Mar 15, 2022 • edited Loading

cpasoft commented Mar 16, 2022

cpasoft commented Mar 16, 2022

cpasoft commented Mar 17, 2022

cpasoft commented Mar 17, 2022

gileli121 commented Mar 17, 2022

gileli121 commented Mar 17, 2022

cpasoft commented Mar 18, 2022 • edited Loading

cpasoft commented Mar 18, 2022 • edited Loading

gileli121 commented Mar 18, 2022

gileli121 commented Mar 22, 2022

cpasoft commented Mar 23, 2022

gileli121 commented Jan 14, 2022 •

edited

Loading

gileli121 commented Jan 15, 2022 •

edited

Loading

gileli121 commented Jan 22, 2022 •

edited

Loading

gileli121 commented Jan 22, 2022 •

edited

Loading

gileli121 commented Jan 23, 2022 •

edited

Loading

gileli121 commented Jan 23, 2022 •

edited

Loading

gileli121 commented Jan 26, 2022 •

edited

Loading

cpasoft commented Jan 27, 2022 •

edited

Loading

cpasoft commented Jan 27, 2022 •

edited

Loading

gileli121 commented Jan 28, 2022 •

edited

Loading

gileli121 commented Jan 28, 2022 •

edited

Loading

gileli121 commented Jan 30, 2022 •

edited

Loading

gileli121 commented Mar 11, 2022 •

edited

Loading

gileli121 commented Mar 11, 2022 •

edited

Loading

gileli121 commented Mar 11, 2022 •

edited

Loading

cpasoft commented Mar 14, 2022 •

edited

Loading

gileli121 commented Mar 14, 2022 •

edited

Loading

gileli121 commented Mar 15, 2022 •

edited

Loading

gileli121 commented Mar 15, 2022 •

edited

Loading

cpasoft commented Mar 18, 2022 •

edited

Loading

cpasoft commented Mar 18, 2022 •

edited

Loading