Any method to build a big wxTreeCtrl progressively in GUI thread?

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
ollydbg23
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 202
Joined: Fri Dec 12, 2008 10:31 am

Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by ollydbg23 » Mon Mar 13, 2017 1:56 am

Hi, for a wxTreeCtrl which may have many nodes and labels, I would like to build the wxTreeCtrl progressively. So a sub-tree may constructed when user expand a node, and we may only shows the nodes which is visible in the window, if user scroll down, then we may dynamically adding more nodes.

Any ideas how to do this?

PS: As a developer for the Code::Blocks IDE, the big picture of this problem is that we need to show a symbol tree in the Code::Blocks' symbol browser(that's all the symbols in a c++ project's source files), but a project may have too many symbols, so we need to construct the tree progressively. The tree was build in a worker thread when we using wx 2.8.12, but it looks like it is not a good idea to construct a tree in a worker thread, so moving the code to the GUI thread is the only way, but building a symbol tree typically takes many seconds, and we don't want the IDE hanging for this operation. Thanks

New Pagodi
Super wx Problem Solver
Super wx Problem Solver
Posts: 361
Joined: Tue Jun 20, 2006 6:47 pm
Contact:

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by New Pagodi » Mon Mar 13, 2017 2:30 am

I have a sample of a control that does this in my github account. It's based keeping track of the tree control on with a wxSqlite3 database, but the idea could be adapted to use another data structure instead.

The key underlying idea is that the nodes are only built when the TreeItemExpanding/TreeItemExpanded events are triggered as seen starting at line 1129 here.

ollydbg23
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 202
Joined: Fri Dec 12, 2008 10:31 am

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by ollydbg23 » Mon Mar 13, 2017 3:55 am

New Pagodi wrote:I have a sample of a control that does this in my github account. It's based keeping track of the tree control on with a wxSqlite3 database, but the idea could be adapted to use another data structure instead.

The key underlying idea is that the nodes are only built when the TreeItemExpanding/TreeItemExpanded events are triggered as seen starting at line 1129 here.
Thanks, you mean when the wxSqlite3 database is changed, and the wxTreeCtrl get dynamically updated? If the database has many records, for example 10000 records, and you want to show them in the wxTreeCtrl.

But first, let me try your code first and see what happens.

New Pagodi
Super wx Problem Solver
Super wx Problem Solver
Posts: 361
Joined: Tue Jun 20, 2006 6:47 pm
Contact:

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by New Pagodi » Mon Mar 13, 2017 5:17 am

ollydbg23 wrote:But first, let me try your code first and see what happens.
That control is probably way more convoluted than you need. Here's a much shorter example that I hope is more helpful:

Code: Select all

// For compilers that support precompilation, includes "wx/wx.h".
#include "wx/wxprec.h"

#ifdef __BORLANDC__
    #pragma hdrstop
#endif

// for all others, include the necessary headers
#ifndef WX_PRECOMP
    #include "wx/wx.h"
#endif

#include <wx/treectrl.h>

//a very simple class that can be used to form a tree of nodes
class TreeNode
{
    public:
        TreeNode(const wxString& s):m_name(s){}
        ~TreeNode();
        wxString GetName(){return m_name;}
        int GetChildrenCount(){return m_children.size();}
        void AddChild(TreeNode* n){m_children.push_back(n);}
        std::vector<TreeNode*>::iterator GetFirstChild(){return m_children.begin();}
        std::vector<TreeNode*>::iterator GetEndChild(){return m_children.end();}
    private:
        wxString m_name;
        std::vector<TreeNode*> m_children;
};

TreeNode::~TreeNode()
{
    for(std::vector<TreeNode*>::iterator it=GetFirstChild();it!=GetEndChild();++it)
    {
        delete *it;
    }
}

// a simple class to allow wxTreeItemData to contain a pointer to a TreeNode
// so that the wxTreeCtrl can be made to mirror a tree of TreeNodes
class TreeData:public wxTreeItemData
{
    public:
        TreeData(TreeNode* n):m_node(n){}
        TreeNode* GetNode(){return m_node;}
    private:
        TreeNode* m_node;
};

class treetestFrame : public wxFrame
{
    public:
        treetestFrame( wxWindow* parent,
                wxWindowID id = wxID_ANY,
                const wxString& title = "Tree Test",
                const wxPoint& pos = wxDefaultPosition,
                const wxSize& size = wxSize( 481,466 ),
                long style = wxDEFAULT_FRAME_STYLE|wxTAB_TRAVERSAL );
        ~treetestFrame(){delete root;}

    private:
        void OnTreeItemExpanding(wxTreeEvent& event);
        wxTreeCtrl* m_treeCtrl1;
        TreeNode* root;
};

treetestFrame::treetestFrame( wxWindow* parent, wxWindowID id,
                             const wxString& title, const wxPoint& pos,
                             const wxSize& size, long style )
                             : wxFrame( parent, id, title, pos, size, style )
{
    //build a tree of TreeNodes
    root = new TreeNode("root");
    root->AddChild( new TreeNode("child1") );
    root->AddChild( new TreeNode("child2") );
    TreeNode* child3 = new TreeNode("child3");
    child3->AddChild( new TreeNode("child3-1") );
    child3->AddChild( new TreeNode("child3-2") );
    root->AddChild(child3);

    //Initialize the tree control so that the data of the tree controls
    //root item points to the root of the tree built above
    m_treeCtrl1 = new wxTreeCtrl(this);
    wxTreeItemId treeroot = m_treeCtrl1->AddRoot("root",-1,-1,new TreeData(root));
    m_treeCtrl1->SetItemHasChildren(treeroot,true);

    m_treeCtrl1->Bind(wxEVT_TREE_ITEM_EXPANDING,&treetestFrame::OnTreeItemExpanding,this);
}

void treetestFrame::OnTreeItemExpanding(wxTreeEvent& event)
{
    wxTreeItemId cur_item = event.GetItem();

    if(m_treeCtrl1->GetChildrenCount(cur_item)==0)
    {
        TreeData* data = reinterpret_cast<TreeData*>(m_treeCtrl1->GetItemData(cur_item));
        TreeNode* node = data->GetNode();

        for(std::vector<TreeNode*>::iterator it=node->GetFirstChild()
            ;it!=node->GetEndChild();++it)
        {
            wxTreeItemId child=m_treeCtrl1->AppendItem(cur_item,(*it)->GetName());
            m_treeCtrl1->SetItemData(child,new TreeData(*it));
            m_treeCtrl1->SetItemHasChildren(child,(*it)->GetChildrenCount()!=0);
        }
    }
}

class treetestApp : public wxApp
{
    public:
        bool OnInit()
        {
            treetestFrame* frame = new treetestFrame(0L);
            frame->Show();
            return true;
        }
};

wxIMPLEMENT_APP(treetestApp);
The idea in both cases is the same though. In both cases, you have some kind of data in a tree form. You use the tree controls data member to tie it to that data, and you build the children of an item in the tree control only when they're actually needed.

ollydbg23
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 202
Joined: Fri Dec 12, 2008 10:31 am

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by ollydbg23 » Mon Mar 13, 2017 3:07 pm

That control is probably way more convoluted than you need. Here's a much shorter example that I hope is more helpful:
Oh, thanks, the shorter example is clear, I can understand the idea.
The idea in both cases is the same though. In both cases, you have some kind of data in a tree form. You use the tree controls data member to tie it to that data, and you build the children of an item in the tree control only when they're actually needed.
I think this is the exact thing we have now in our symbol tree, see some code snippet in:
codeblocks/classbrowserbuilderthread.cpp at master · madnight/codeblocks, this is the item expanding event handler running in a worker thread.
The operation may take some time, so we have run it in a thread.

Code: Select all

void ClassBrowserBuilderThread::ExpandItem(wxTreeItemId item)
{
    TRACE(_T("ClassBrowserBuilderThread::ExpandItem"));
...
    //some calculation on it's internal data
...
}
The event handling is here: codeblocks/classbrowser.cpp at master · madnight/codeblocks

Code: Select all

void ClassBrowser::OnTreeItemExpanding(wxTreeEvent& event)
{
    if (m_ClassBrowserBuilderThread)
        m_ClassBrowserBuilderThread->ExpandItem(event.GetItem());
#ifndef CC_NO_COLLAPSE_ITEM
    event.Allow();
#endif // CC_NO_COLLAPSE_ITEM
}
The bottleneck of the operation is we have some calculation on it's internal data(like the tree of TreeNodes in your shorter example).

So, in this case, I think the proposed way? is that I can put the "calculation of it's internal data" in a worker thread, and after that, we just call the construction of the children notes of the item in the GUI thread. That's sounds like I have to emulate an "item expand" event when I finished the "calculation of it's internal data"?

Any ideas? Thanks.

New Pagodi
Super wx Problem Solver
Super wx Problem Solver
Posts: 361
Joined: Tue Jun 20, 2006 6:47 pm
Contact:

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by New Pagodi » Tue Mar 14, 2017 3:04 am

ollydbg23 wrote:That's sounds like I have to emulate an "item expand" event when I finished the "calculation of it's internal data"?

Any ideas? Thanks.
One possibility is to throw a thread event and have a handler function for that event do the expansion of the tree control in the main thread. Here's a variation on the previous example to show how to do this. In this example, the only work done in the thread is sleeping for 1.5 seconds.

Code: Select all

// For compilers that support precompilation, includes "wx/wx.h".
#include "wx/wxprec.h"

#ifdef __BORLANDC__
    #pragma hdrstop
#endif

// for all others, include the necessary headers
#ifndef WX_PRECOMP
    #include "wx/wx.h"
#endif

#include <wx/treectrl.h>
#include <wx/thread.h>
#include <set>

wxDEFINE_EVENT(wxEVT_EXPANSION_COMPLETED, wxThreadEvent);

class ExpansionThread : public wxThread
{
public:
    ExpansionThread(wxFrame *handler,const wxTreeItemId& id)
        : wxThread(wxTHREAD_DETACHED)
        { m_pHandler = handler;m_id=id;}
protected:
    virtual ExitCode Entry();
    wxFrame *m_pHandler;
    wxTreeItemId m_id;
};

wxThread::ExitCode ExpansionThread::Entry()
{
    while (!TestDestroy())
    {
        Sleep(1500);
        break;
    }

    wxThreadEvent* evt = new wxThreadEvent(wxEVT_EXPANSION_COMPLETED);
    evt->SetPayload(m_id);
    wxQueueEvent(m_pHandler,evt);

    return (wxThread::ExitCode)0;     // success
}

class treetestFrame : public wxFrame
{
    public:
        treetestFrame( wxWindow* parent,
                wxWindowID id = wxID_ANY,
                const wxString& title = "Tree Test",
                const wxPoint& pos = wxDefaultPosition,
                const wxSize& size = wxSize( 481,466 ),
                long style = wxDEFAULT_FRAME_STYLE|wxTAB_TRAVERSAL );

    private:
        void OnTreeItemExpanding(wxTreeEvent& event);
        void OnThreadCompletion(wxThreadEvent& event);
        wxTreeCtrl* m_treeCtrl1;
        std::set<wxTreeItemId> m_processed;
};

treetestFrame::treetestFrame( wxWindow* parent, wxWindowID id,
                             const wxString& title, const wxPoint& pos,
                             const wxSize& size, long style )
                             : wxFrame( parent, id, title, pos, size, style )
{
    m_treeCtrl1=new wxTreeCtrl(this);
    wxTreeItemId root = m_treeCtrl1->AddRoot("root");
    m_treeCtrl1->AppendItem(root,"Item1");
    m_treeCtrl1->AppendItem(root,"Item2");
    wxTreeItemId item3=m_treeCtrl1->AppendItem(root,"Item3");
    m_treeCtrl1->AppendItem(item3,"Item3-1");
    m_treeCtrl1->AppendItem(item3,"Item3-2");

    Bind(wxEVT_EXPANSION_COMPLETED,&treetestFrame::OnThreadCompletion,this);
    m_treeCtrl1->Bind(wxEVT_TREE_ITEM_EXPANDING,&treetestFrame::OnTreeItemExpanding,this);
}

void treetestFrame::OnTreeItemExpanding(wxTreeEvent& event)
{
    if(m_processed.find(event.GetItem())==m_processed.end())
    {
        ExpansionThread* thread=new ExpansionThread(this,event.GetItem());
        thread->Run();
        event.Veto();
    }
}

void treetestFrame::OnThreadCompletion(wxThreadEvent& event)
{
    wxTreeItemId id=event.GetPayload<wxTreeItemId>();
    m_processed.insert(id);
    m_treeCtrl1->Expand(id);
}

class treetestApp : public wxApp
{
    public:
        bool OnInit()
        {
            treetestFrame* frame = new treetestFrame(0L);
            frame->Show();
            return true;
        }
};

wxIMPLEMENT_APP(treetestApp);

User avatar
doublemax
Moderator
Moderator
Posts: 15273
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by doublemax » Tue Mar 14, 2017 6:28 am

How fast can you determine the number of children for a node? If that can happen in realtime, i'd just create the correct number of dummy entries, collect the real data in the worker thread and send an event to the main thread when data for a new node is available. The main thread can then update the item and refresh the tree.
Use the source, Luke!

ollydbg23
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 202
Joined: Fri Dec 12, 2008 10:31 am

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by ollydbg23 » Tue Mar 14, 2017 2:41 pm

Hi, New Pagodi, thanks for the updated sample code.
I just test it, and it works fine unless when user click the "+" icon, it need to wait 1.5 second to see the children nodes.

As suggested by doublemax, maybe dummy child nodes are created immediately after the click. In some cases, getting of a child node is not that quick, for example, in the image shot:
Image
If we want to show "everything", which means all the symbols, there are many symbols in the global namespace, so a lot of class definitions or enums. So, there are a lot of child nodes in the root node.

But if the symbol tree only show the "current file's symbols", then there is not much symbols, see the image below:
Image

Also, as shown in the above screen shot, you can see that all the member variables and member functions of a selected class(when user click on a node of the top tree)are listed in another window(the bottom tree).

New Pagodi
Super wx Problem Solver
Super wx Problem Solver
Posts: 361
Joined: Tue Jun 20, 2006 6:47 pm
Contact:

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by New Pagodi » Wed Mar 15, 2017 1:02 am

Sorry if I'm completely misunderstanding and wasting your time. Anyway here's a third variation:

Code: Select all

// For compilers that support precompilation, includes "wx/wx.h".
#include "wx/wxprec.h"

#ifdef __BORLANDC__
    #pragma hdrstop
#endif

// for all others, include the necessary headers
#ifndef WX_PRECOMP
    #include "wx/wx.h"
#endif

#include <wx/treectrl.h>
#include <wx/thread.h>
#include <set>

wxDEFINE_EVENT(wxEVT_EXPANSION_COMPLETED, wxThreadEvent);
wxDEFINE_EVENT(wxEVT_EXPANSION_UPDATE, wxThreadEvent);

class ExpansionThread : public wxThread
{
public:
    ExpansionThread(wxFrame *handler,const wxTreeItemId& id)
        : wxThread(wxTHREAD_DETACHED)
        { m_pHandler = handler;m_id=id;}
protected:
    virtual ExitCode Entry();
    wxFrame *m_pHandler;
    wxTreeItemId m_id;
};

wxThread::ExitCode ExpansionThread::Entry()
{
    if (!TestDestroy())
    {
        Sleep(500);

        wxThreadEvent* evt = new wxThreadEvent(wxEVT_EXPANSION_UPDATE);
        evt->SetInt(10);
        evt->SetString("GlobalFunctions");
        evt->SetPayload(m_id);
        wxQueueEvent(m_pHandler,evt);
    }

    if (!TestDestroy())
    {
        Sleep(1000);

        wxThreadEvent* evt = new wxThreadEvent(wxEVT_EXPANSION_UPDATE);
        evt->SetInt(20);
        evt->SetString("GlobalTypeDefs");
        evt->SetPayload(m_id);
        wxQueueEvent(m_pHandler,evt);
    }

    if (!TestDestroy())
    {
        Sleep(1500);

        wxThreadEvent* evt = new wxThreadEvent(wxEVT_EXPANSION_UPDATE);
        evt->SetInt(30);
        evt->SetString("GlobalVariables");
        evt->SetPayload(m_id);
        wxQueueEvent(m_pHandler,evt);
    }

    if (!TestDestroy())
    {
        Sleep(500);

        wxThreadEvent* evt = new wxThreadEvent(wxEVT_EXPANSION_UPDATE);
        evt->SetInt(10);
        evt->SetString("Enums");
        evt->SetPayload(m_id);
        wxQueueEvent(m_pHandler,evt);
    }

    wxThreadEvent* evt = new wxThreadEvent(wxEVT_EXPANSION_COMPLETED);
    evt->SetPayload(m_id);
    wxQueueEvent(m_pHandler,evt);

    return (wxThread::ExitCode)0;     // success
}

class treetestFrame : public wxFrame
{
    public:
        treetestFrame( wxWindow* parent,
                wxWindowID id = wxID_ANY,
                const wxString& title = "Tree Test",
                const wxPoint& pos = wxDefaultPosition,
                const wxSize& size = wxSize( 481,466 ),
                long style = wxDEFAULT_FRAME_STYLE|wxTAB_TRAVERSAL );

    private:
        void OnTreeItemExpanding(wxTreeEvent& event);
        void OnThreadCompletion(wxThreadEvent& event);
        void OnThreadUpdate(wxThreadEvent& event);
        wxTreeCtrl* m_treeCtrl1;
        std::set<wxTreeItemId> m_processed;
};

treetestFrame::treetestFrame( wxWindow* parent, wxWindowID id,
                             const wxString& title, const wxPoint& pos,
                             const wxSize& size, long style )
                             : wxFrame( parent, id, title, pos, size, style )
{
    m_treeCtrl1=new wxTreeCtrl(this);
    wxTreeItemId root = m_treeCtrl1->AddRoot("root");
    m_treeCtrl1->SetItemHasChildren(root,true);

    Bind(wxEVT_EXPANSION_COMPLETED,&treetestFrame::OnThreadCompletion,this);
    Bind(wxEVT_EXPANSION_UPDATE,&treetestFrame::OnThreadUpdate,this);
    m_treeCtrl1->Bind(wxEVT_TREE_ITEM_EXPANDING,&treetestFrame::OnTreeItemExpanding,this);
}

void treetestFrame::OnTreeItemExpanding(wxTreeEvent& event)
{
    wxTreeItemId id = event.GetItem();

    if(m_processed.find(id)==m_processed.end())
    {
        m_processed.insert(id);
        ExpansionThread* thread=new ExpansionThread(this,id);
        thread->Run();
        event.Veto();
    }
}

void treetestFrame::OnThreadUpdate(wxThreadEvent& event)
{
    wxTreeItemId id=event.GetPayload<wxTreeItemId>();
    wxString name_start = event.GetString();
    int limit = event.GetInt();

    for(int i=1;i<=limit;++i)
    {
        wxString s = wxString::Format("%s-%d",name_start,i);
        m_treeCtrl1->AppendItem(id,s);
    }
    m_treeCtrl1->Expand(id);
}

void treetestFrame::OnThreadCompletion(wxThreadEvent& event)
{
    wxTreeItemId id=event.GetPayload<wxTreeItemId>();
    m_treeCtrl1->Expand(id);
}

class treetestApp : public wxApp
{
    public:
        bool OnInit()
        {
            treetestFrame* frame = new treetestFrame(0L);
            frame->Show();
            return true;
        }
};

wxIMPLEMENT_APP(treetestApp);
This "builds" the items of the tree in a worker thread and then throws a thread update event to have the control update in the gui thread. In this sample the "building" consists of sleeping for a bit and putting an int and a string in the events data. A real application would do a long running computation and put actual data in instead.

ollydbg23
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 202
Joined: Fri Dec 12, 2008 10:31 am

Re: Any method to build a big wxTreeCtrl progressively in GUI thread?

Post by ollydbg23 » Wed Mar 15, 2017 2:36 am

New Pagodi wrote:Sorry if I'm completely misunderstanding and wasting your time. Anyway here's a third variation:
Hi, New Pagodi, many thanks. It never wastes my time, I learned much from your examples, I mean it's a step by step tutorial.
This "builds" the items of the tree in a worker thread and then throws a thread update event to have the control update in the gui thread. In this sample the "building" consists of sleeping for a bit and putting an int and a string in the events data. A real application would do a long running computation and put actual data in instead.
OK, I see the child nodes are constructed progressively in this example code. This is a good way to construct a complex tree, I'd discuss this on CodeBlocks' forum with other C::B developers about this, then we will start the code refactoring. Thanks.

Post Reply