Mac 10.10+ Accessibility "Protocols" (NSAccessibility)

Do you have a typical platform dependent issue you're battling with ? Ask it here. Make sure you mention your platform, compiler, and wxWidgets version.
Post Reply
Forbin
Earned a small fee
Earned a small fee
Posts: 21
Joined: Mon Oct 31, 2016 7:26 pm

Mac 10.10+ Accessibility "Protocols" (NSAccessibility)

Post by Forbin »

Platform: Mac OSX 10.11.6,10.12 (and also maybe 10.9!)
Compiler: Xcode 7.3.1 (7D1014)
wxWidgets: OSX/cocoa 3.0.2

Colleagues:

We're a shop of MS Windows programmers with only minimal Mac OSX knowledge... but we have an application we must field and support on both platforms. We also are required to support "accessibility" functions on both Mac and Windows, in order to support things such as UI testing automation tools.

We have recently added some classes that derive from wxOwnerDrawnComboBox (yes, that again!), and are trying to connect it into the Mac native accessibility mechanism. The NSAccessibility "protocols" are giving us fits! We're able to get the Mac Accessibility Inspector to recognize our controls as AXComboBox, but, for example, we cannot seem to tell it what our current "value" attribute is. But if the inspector descends into the popup control, it can see the value. That's different from a native combo box.

Does anyone here have experience wiring non-native wxWidgets controls into NSAccessibility?

Instead of posting a lot of details here, not knowing what's useful / pertinent, and what's not, I'll provide information as requested.

Er... if requested. My co-workers have a bet going that nobody else knows this stuff, either! :(
User avatar
doublemax
Moderator
Moderator
Posts: 19103
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Mac 10.10+ Accessibility "Protocols" (NSAccessibility)

Post by doublemax »

There are not many wxWidgets users who work under OS X here, so your colleagues might win that bet...

I don' t know much about OS X, but searching the wx sources for "NSAccessibility" finds exactly two locations, both for wxNSComboBoxControl and they look remotely related to your problem:

Code: Select all

void wxNSComboBoxControl::Popup()
{
    id ax = NSAccessibilityUnignoredDescendant(m_comboBox);
    [ax accessibilitySetValue: [NSNumber numberWithBool: YES] forAttribute: NSAccessibilityExpandedAttribute];
}

void wxNSComboBoxControl::Dismiss()
{
    id ax = NSAccessibilityUnignoredDescendant(m_comboBox);
    [ax accessibilitySetValue: [NSNumber numberWithBool: NO] forAttribute: NSAccessibilityExpandedAttribute];
}
I think if you ask in an Apple related forum how to implement NSAccessibility for a custom combo control and get an answer, it should be possible to add that solution to your wxWidgets code.
Use the source, Luke!
ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 7449
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: Mac 10.10+ Accessibility "Protocols" (NSAccessibility)

Post by ONEEYEMAN »

Hi,
Moreover, IIRC, accessibility is implemented on MSW only.
GTK/OSX Cocoa does not have this feature.

Thank you.
Forbin
Earned a small fee
Earned a small fee
Posts: 21
Joined: Mon Oct 31, 2016 7:26 pm

Re: Mac 10.10+ Accessibility "Protocols" (NSAccessibility)

Post by Forbin »

@OneEyeMan:
I assume you mean "in wxWidgets" since OSX seems to have a whole, complicated, Objective-C based accessibility framework.

@DoubleMax:
That helped, but it turns out there are all sorts of other things that are necessary to "plug a control into" Mac accessibility.

Progress:
We have some template classes to help add this stuff to different controls, but none of them would work for owner-drawn combo boxes, so we've had to build another template from scratch. It required our developer who is the closest thing we have resembling a Mac expert, and he had to do a LOT of trial-and-error experimentation. [For others who may have to travel this road in the future, be forewarned: even Apple's own OSX documentation is not entirely correct! Be skeptical when reading.]

_______________________________

Now we are wrestling with a problem of whether or not to call DontCreatePeer() in the Create() methods of our class that derives from wxOwnerDrawnComboBox (which I'll abbeviate WODCB here for short).

It's a chicken-and-egg problem. If we call DontCreatePeer(), then call up to WODCB's Create(), then deeper in the inheritance hierarchy comes the step where it creates the text control sub-component. During that creation, wxTextCtrlBase (or code that it uses) does a GetParent(), which gets a pointer to the instance of our class in the process of being created. It tries to use GetPeer() on that, to get location and dimension information so it knows where to put the text control. But of course the instance of our class doesn't have a peer yet! So it crashes.

But the alternative doesn't seem helpful either. Skipping the call to DontCreatePeer(), the call up the Create()-chain works. But then we have a peer that I can't figure out how to pry loose. If we just create the real peer that we want and use SetPeer(), the original peer seems to still be partially wired in to the rest of the control, so that we crash when our destructor runs. I tried deleting the original peer after doing the swap. That causes a crash too. I also tried GetPeer()->RemoveFromParent(), which didn't help, either alone or in combination with deleting the peer.

Any clues as to how to pry loose the old peer so it can be replaced with the proper one?

Or... is there a way to force the creation of the text control to be deferred, so we can let the chain of Create() functions do its thing, then we create our peer, then follow up by letting WODCB (or its parent hierarchy) create the text control?

[Let me know if I need to post this as a separate question... and whether "Platform Related" would be better, or "Component Writing."]


Thanks!!
-- forbin
Forbin
Earned a small fee
Earned a small fee
Posts: 21
Joined: Mon Oct 31, 2016 7:26 pm

Re: Mac 10.10+ Accessibility "Protocols" (NSAccessibility)

Post by Forbin »

.
Follow-up:

For future readers... if you ever have this kind of problem with DontCreatePeer(), the answer is that you cannot skip that call. If you're going to replace the peer, you have to call this. Try the straightforward solution first, calling DontCreatePeer(), then either creating your own peer before or after calling up to the parent class's Create(). In some cases, it works, and honestly, one of those should always work. But if you encounter a case like ours where it crashes, you'll have to "hoist" various chunks of code from your classes ancestors into your own Create() function, rather than calling the parent Create(). This is better than editing wxWidgets library code directly. (We didn't have that option anyway, because we have a separate engineering group that maintains our build of the library, and they only change library code as a last resort.)

To figure out what code to hoist, we traced the execution through the debugger until we hit the location of the crash -- in our case, a null dereference within the wxWidgets library. We copied the code from our parent's Create() up to that point into our own. (You might also have to hoist some even lower level code to follow that, but we didn't for our case.) Next, you splice in your own code following it, which will usually be the creation and setup of your custom peer, and a call to SetPeer(). You follow that by hoisting the rest of the creation code starting with the location of the crash. Your Create() function will end up looking pretty messy, but it will work. If you want to simplify it, you can then test in the debugger and strip out code you don't need.

For our specialized owner-drawn combo box, the Create() ends up looking something like this.
(Note that on Mac, the inheritance goes: myclass -> wxOwnerDrawnComboBox -> wxComboCtrl -> wxGenericComboCtrl -> wxComboCtrlBase -> wxControl...)

Code: Select all

bool MySpecializedComboBox::Create( wxWindow* parent,
                                    wxWindowID id,
                                    const wxString& value,
                                    const wxPoint& pos,
                                    const wxSize& size,
                                    int numChoices,
                                    const wxString choices[],
                                    long style,
                                    const wxValidator& validator,
                                    const wxString& name )
{
#ifndef __WXOSX_COCOA__
    // BASECLASS = wxOwnerDrawnComboBox
    bool bResult = BASECLASS::Create(parent, id, value, pos, size, numChoices, choices, style, validator, name);
    return bResult;
#else
    DontCreatePeer();

    // Mac has problems with readonly flag; can't paint the editcontrol area correctly
    if ( style & wxCB_READONLY )
    {
        style &= ~ wxCB_READONLY;
        m_fIsReadOnly = true;
    }

    // CANNOT call our parent's Create() directly here, because its parents create the text control
    // and try to ask for the [Mac OSX] peer before it has been created.  So we hoist the code
    // before and after the text control creation and try reordering things so our manually
    // created peer is in place before the text control.

//From wxGenericComboCtrl:

    // Note that technically we only support 'default' border and wxNO_BORDER.
    long border = style & wxBORDER_MASK;

    #define UNRELIABLE_TEXTCTRL_BORDER

    if ( style & wxCB_READONLY )
        m_widthCustomBorder = 1;
    else
        m_widthCustomBorder = 0;

    // Because we are going to have button outside the border,
    // let's use wxBORDER_NONE for the whole control.
    border = wxBORDER_NONE;

    Customize( wxCC_BUTTON_OUTSIDE_BORDER|wxCC_NO_TEXT_AUTO_SELECT|wxCC_BUTTON_STAYS_DOWN );

    style = (style & ~ (wxBORDER_MASK)) | border;
    if ( style & wxCC_STD_BUTTON )
        m_iFlags |= wxCC_POPUP_ON_MOUSE_UP;

    if ( ! wxComboCtrlBase::Create(parent, id, value, pos, size, style | wxFULL_REPAINT_ON_RESIZE, validator, name) )
    {
        return false;
    }

    //----------------------------------------------------------------------------------------------
    // Here's where we spliced in a step:
    //----------------------------------------------------------------------------------------------
    // This creates a custom NSView that properly supports control level tooltips
    //    instead of "view" level tooltips.
    //----------------------------------------------------------------------------------------------
    SetPeer(MacAppKitHelper::CreateCustomViewCombo(this, pos, size));
    GetPeer()->SetNeedsFocusRect(false);

//From wxOwnerDrawnComboBox:
    for ( int i = 0; i < numChoices; ++i )
    {
        m_initChs.Add(choices[i]);
    }

//From wxGenericComboCtrl:

    // Create textctrl, if necessary
    CreateTextCtrl(wxNO_BORDER);

    // Add keyboard input handlers for main control and textctrl
    InstallInputHandlers();

    // Set background style for double-buffering, when needed
    // (cannot use when system draws background automatically)
    if ( ! HasTransparentBackground() )
        SetBackgroundStyle(wxBG_STYLE_PAINT);

    // SetInitialSize should be called last
    SetInitialSize(size);

    if ( m_fIsReadOnly )
        this->SetEditable(false);

    return true;
#endif
}
 
If you now go read the wxOSX code (v3.0.2) for the ancestor classes where we hoisted the code from, you can see clearly what we did and why it works.

Hope this helps anyone unfortunate enough to follow in my footsteps!
--forbin
Post Reply