wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Do you have a typical platform dependent issue you're battling with ? Ask it here. Make sure you mention your platform, compiler, and wxWidgets version.
Post Reply
Mick P.
Earned some good credits
Earned some good credits
Posts: 145
Joined: Thu Jun 06, 2019 3:41 am
Contact:

wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by Mick P. »

Has anyone seen this (read Subject text) before? I do experience eventual crashes (exception, probably segfault) when I open two (or more) identical windows in my app that both have identical menus and share some of the menus (I have modified the classes in simple ways to accommodate this sharing, which is normal practice in Win32, but I don't want to discuss this fact) that's probably related.

Right now (Windows 10) the menus are corrupted (in all applications, i.e. every Windows app using standard menus) as soon as I create the second Window. I don't believe this was always the case. What this looks like is when I pop-up the menu it's initially all white, then when I drag the mouse over it starts to draw its items underneath the cursor. Sometimes the menus regress to more "classic" drawing style (Visual Styles) instead of using the native style, and they stay this way. I don't suppose wxWidgets is responsible for this since it happens in unrelated apps too.

When clicking in a menu bar the first menu that pops up behaves this way, but if I then click another item in the bar that menu will be fully drawn. I'm just describing the symptoms in case this is a well known (bizarre) behavior in Windows that could give me a lead on where to look. Obviously I need to change something to be able to host multiple app instances (same processes) on Windows, but I don't have a very good lead. (I have looked at the code at the exception, but I can only remember off hand it didn't give me any ideas, but if I trigger it again I will share the surrounding code for context.)

Thanks! Sorry for long posting, I want to give a thorough description [-o<
User avatar
doublemax
Moderator
Moderator
Posts: 19118
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by doublemax »

that both have identical menus and share some of the menus (I have modified the classes in simple ways to accommodate this sharing, which is normal practice in Win32, but I don't want to discuss this fact)
As you say you don't want to discuss this, i'll just state that this sounds so hackish that it's a good candidate to be responsible for your issues.
Use the source, Luke!
ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 7461
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by ONEEYEMAN »

Hi,
Is there any reason for such design?
Can you post some {pseudo}code for us to look at?

Thank you.
Mick P.
Earned some good credits
Earned some good credits
Posts: 145
Joined: Thu Jun 06, 2019 3:41 am
Contact:

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by Mick P. »

I just want to know if anyone's ever seen this behavior in Windows. I'd suspect it was GDI object exhaustion but I used Process Explorer and everything looked fine. It cuts across the entire system. I guess Windows 10 shares menus somehow.

I'm kind of curious what could make one program leak bugs across all of Windows. I don't know if Microsoft would consider this a flaw in Windows or not. It kind of spoils the entire Windows session. I don't know if I remember this happening last year, it may be a new Windows Update thing.
As you say you don't want to discuss this, i'll just state that this sounds so hackish that it's a good candidate to be responsible for your issues.
In some sense I guess, but I have to hack wxWidgets about 40 ways to make it usable for my purposes.
Is there any reason for such design?
It's because that's how the API is, which is not surprising since all system menu systems work this way (i.e. Win32) except for GTK's. For GTK I just move the menus when a top-level window is activated. In theory Windows should have the least issues. It used to work fine last year, but it did sometimes crash. In my experience wxWidgets is pretty fragile around its objects pointers.

A small hack is much easier than managing multiple instances of logical menus. Every change to a checkbox state or something would have to be broadcasted across every instance for example. I'd rather not have to write that code if small hacks can suffice. It's more to maintain and looks illogical.

P.S. Thanks you two for double-handedly making this forum act like a response center even if your advice isn't always constructive, it's appreciated!
User avatar
doublemax
Moderator
Moderator
Posts: 19118
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by doublemax »

I just want to know if anyone's ever seen this behavior in Windows.
Not exactly the same, but yes, if an exe runs out of GDI handles, i saw it affect the whole system. Where controls or whole windows of other applications don't redraw properly or are just black.

How do you switch the menus from one frame to another? With pure wxWidgets code, pure Win32 api, or a mix of both?
Use the source, Luke!
Mick P.
Earned some good credits
Earned some good credits
Posts: 145
Joined: Thu Jun 06, 2019 3:41 am
Contact:

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by Mick P. »

How do you switch the menus from one frame to another? With pure wxWidgets code, pure Win32 api, or a mix of both?
Well I do that for GTK since it doesn't have a concept of shared menus. Win32 doesn't parent menus at all, so (it's been a while since I worked with the code) so I've just overridden some virtual methods in the wxWidgets classes to make the menus act like they don't care about parents. I just return the current active window for their window parent when it's needed by wxWidgets code.

Windows didn't have this problem last year but wxMSW would crash sometimes deep its generic looking code. It's a problem I'll have to fix one way or another. I wonder if Windows 10 has stopped using GDI internally but it still has the same pathological problems. I thought Task Manager used to have a GDI column but it's been removed, so I had to use a Microsoft program called Process Explorer instead. It shows tens of GDI objects but I don't know they include menus or not.

I agree I seem to remember times when the whole system gummed up. Maybe more so in Classic mode that's removed from Windows 10 (maybe high-contrast mode can still access it) I think. Fortunately I never had to reboot when I was exploring this problem since I was shutting down for the day anyway but I wonder if killing Explorer or anything else can restore things after this kind of bug wreaks its havoc. It seems like a strange vulnerability to persist in Windows when everything else is firmly isolated to individual processes. I've spent so much time with Win32 part of me is just genuinely fascinated with what could be happening.

I don't recommend it but the offending software can be tried here (https://github.com/mick-p1982/mm3d/rele ... n32-demo2b) by making a New window on current Windows 10 systems. It's definitely something that happens inside wxWidgets, not that I'm holding its feet to the fire or anything. At some point I will try to debug it.
ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 7461
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by ONEEYEMAN »

Hi,
Are you trying to make some kind of MDI/SDI-like application?
Maybe you can take a look there and see how menus are handled...

Thank you.
Mick P.
Earned some good credits
Earned some good credits
Posts: 145
Joined: Thu Jun 06, 2019 3:41 am
Contact:

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by Mick P. »

EDITED/FOREWORD: This is a solution to the problem. I failed to open my salvo by making that obvious, sorry #-o

Okay, the really weird behavior went away yesterday morning. That same morning Windows 10 started with the new Windows Update style blue screen (with infantile cartoons and gamer like iconography) that was weird because I have auto update disabled because I have a 10GB data plan on satellite Internet. This made me think it was a bug that was quickly patched out. I tested my app and it seemed fine.

But today I figured out how to cause the problem the old way, that was to make the two windows (with shared menus) and then close one window, and then in that case the other window's shared menus reverted from Windows 10 style to older style. Maybe something like you might get with SetWindowTheme if it affects menus (I'm not sure.)

But I had a spot of luck because out of the corner of my eye one time I saw in the output (in Visual Studio) this from wxWidgets:

Code: Select all

C:\wxWidgets-3.1.2\src\msw\menu.cpp(174): 'DestroyMenu' failed with error 0x00000579 (Invalid menu handle.).
So I tried to overload the virtual destructor and that didn't fix it. Then I saw something in the DestroyMenu documentation:
Win32 docs wrote: DestroyMenu is recursive, that is, it will destroy the menu and all its submenus.
I didn't understand really why this was a problem. But at a very late stage I noticed wxMenuItemBase::SetSubmenu is just implemented like this:

Code: Select all

void SetSubMenu(wxMenu *menu) { m_subMenu = menu; }
This is the classic kind of reason wxWidgets is broken all over the place. It just has complete disregard for basic correctness.

So that meant my strategy of disconnecting the submenus before destroying the menus was amounting to nothing for the Win32 layer.

Then I struggled for what seemed like hours until I could reach no further conclusion than SetMenuItemInfo wasn't working. I had to actually count the DestroyMenu errors.

Instead you have to use a different Win32 API called RemoveMenu to dissociate submenus. This is a classic reason why you have to pay close attention to Win32 to implement wrappers around it.

***I know this post is going long at this point****

My main mistake was not thinking DestroyMenu was recursive and applied to the main menu bar itself. But to be fair the menus actually worked, which they shouldn't have. Probably the submenus are ref-counted, but it doesn't explain why they revert to the old-fashioned rendering style. Honestly aside from the really crazy behavior from the OP (I don't want to be a conspiracy theorist but the behavior was totally different those two days and I wonder if it got patched or what was the other factor) it all would've worked fine except for the theme reversion. I haven't experienced any crashes after tens of sessions but I didn't change anything that would fix a crash either.

All in all the only main adaptation I had done for wxMSW was to override the ~wxMenuBar destructor to empty its child array. But that seemed not to help since the HMENU remembered them, but still by all rights it should've done ref-counting, so I'm not sure what its issue is unless it stupidly doesn't do ref-counting but still permits zombie menus to operate. Now I have to override ~wxMenu too and do RemoveMenu and DestroyMenu manually. Luckily there was a helper to expose the "private" HMENU member called MSWDetachHMENU otherwise I'd be screwed (well not entirely since it would just generate that error in the debugger output.)
Last edited by Mick P. on Thu Dec 24, 2020 9:24 am, edited 3 times in total.
ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 7461
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by ONEEYEMAN »

Hi,
Can you create a minimal, compilable sample to see the problem and post it here?

Thank you
Mick P.
Earned some good credits
Earned some good credits
Posts: 145
Joined: Thu Jun 06, 2019 3:41 am
Contact:

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by Mick P. »

ONEEYEMAN wrote: Mon Dec 21, 2020 5:06 pm Hi,
Can you create a minimal, compilable sample to see the problem and post it here?

Thank you
You know, sometimes you seem like a troll or a bot ONEEYEMAN :^o EDITED: Also, for the record, I solved the problem in my prior reply/post, I probably should've made that more clear #-o
ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 7461
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by ONEEYEMAN »

Hi,
You may solve it, but as you said - it was surprising it went away.

And in SD there should be no surprises.

Thank you.
Mick P.
Earned some good credits
Earned some good credits
Posts: 145
Joined: Thu Jun 06, 2019 3:41 am
Contact:

Re: wxMSW: Menus corrupted systemwide, even after app quits, GDI object count looks normal?

Post by Mick P. »

SD? Well the behavior I described originally was in fact different. It's what brought my attention back to the problem. I really think it was a bug in Windows that got patched out the next day. If not it was a system wide phenomenon that depended on outside factors. (But because it happened across reboots, and I could trigger it methodically, I'm inclined to think it was an uncaught bug in Windows itself. I think Microsoft is struggling right now to keep the old Win32 infrastructure functioning while trying to find a way to pivot away from it. Most of its developers don't seem to be familiar with it since MS has been trying--unsuccessfully--to move away from it for decades, so probably its main staff is using the newer alternatives to Win32 in their day to day work. They're constantly breaking Win32 these days. And the bugs aren't noticed and there's no way for outsiders to report them. I've actually solved a few by contacting MS employees through their blogs.)

EDITED: The original behavior was the menus were popping up as empty boxes, and this happened even in other programs after the first occurrence of the glitch. It happened always after triggering it. So I'm guessing after MS learned some programs could make this happen to the entire system a patch was rolled out. (After that went away I was back to just having the problem of the menus reverting to not having the system visual style applied to them, but otherwise stable. I think this was because the menus were actually destroyed--zombies--but somehow they remain functional when this happens since there's probably a table of fixed sized menu fields somewhere and the handles must not be checked for validity. I wouldn't be surprised if the hypothetical/possible patch was to prevent these zombies from corrupting the entire system. I have a feeling menus are shared, global handles, just like HWND.)
Post Reply