Threading and memory usage Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
bslatner
Earned a small fee
Earned a small fee
Posts: 21
Joined: Sun Mar 26, 2006 6:11 pm
Location: Charlotte, NC

Threading and memory usage

Post by bslatner » Tue Sep 09, 2008 1:25 am

I have an interesting problem I'm not sure how to continue diagnosing.

My program is very simple. I've got a timer that ticks every 15 seconds, calls a web service, and continues. This repeats forever.

Presently, when the timer ticks, I spawn a thread that does the actual web service querying.

In watching the program's memory usage via the Windows task manager, I see that memory usage on the program goes up in 4k increments. It does this about every minute, so about every 4 calls to the web service. This memory usage never goes down, it just climbs steadily.

When I exit the application, no memory leaks are reported. Deliberate memory leaks inserted in the code ARE reported, so I know I've set up the leak detection correctly.

I've been through this app front to back, looking for leaks and fragmentation. I couldn't find a thing. But the memory usage keeps going up.

I tried a bunch of things, but eventually what made the memory increase stop was the elimination of the thread. If I stop spawning the thread and just perform all operations in the timer tick event handler, the memory usage remains constant. The instant I introduce the thread, the memory usage starts climbing.

I'm frankly at a loss as to how to go about debugging this any further and am hoping for some suggestions.

Thanks,
Bryan

P.S., for reference, here is the thread code. The commented out part shows the ONLY difference between the app with the climbing memory usage and the app without.

Code: Select all

class ServiceUpdateThread : public wxThread
{
private:
    MainFrame *_MainFrame;

protected:
    virtual ExitCode Entry(void);

public:
    ServiceUpdateThread(MainFrame *frame);
};


MainFrame::ServiceUpdateThread::ServiceUpdateThread(MainFrame *frame)
{
    _MainFrame = frame;
}

wxThread::ExitCode MainFrame::ServiceUpdateThread::Entry(void)
{
	_MainFrame->QueryWebService();
    return (ExitCode)0;
}

void MainFrame::UpdateByTimer(wxTimerEvent & WXUNUSED(event))
{
    wxDateTime rightNow = wxDateTime::Now();

    if (!_IsWebServiceQueryInProgress && rightNow >= _TimeToQueryWebService)
    {
        _IsWebServiceQueryInProgress = true;
		QueryWebService();
		//ServiceUpdateThread *thread = new ServiceUpdateThread(this);
		//thread->Create();
		//thread->Run();
        _HasFirstWebServiceQueryBeenMade = true;
    }
}



Ugly!
Earned some good credits
Earned some good credits
Posts: 113
Joined: Mon May 09, 2005 5:11 am
Location: Argentina - BS AS

Post by Ugly! » Tue Sep 09, 2008 4:20 am

You are creating all the threads on the heap. This is not bad for threads. I guess somehow thread's memory is released on program exit. But they won't be freed upon the thread termination.

So, if you don´t want memory pilling up, my advice would be to delete the thread when its work is done.

Regards,
Mat
Just a newbie - Too many interests, not too many time.

Windows XP SP2
Kubuntu GNU/Linux - Feisty
wxActiveRecordGenerator (aka wxARG) maintainer
Find it at wxCode

Fire Lancer
Knows some wx things
Knows some wx things
Posts: 31
Joined: Sat Sep 06, 2008 10:49 am

Post by Fire Lancer » Tue Sep 09, 2008 6:59 am

It may be better to just have one worker thread that wait for an update request (or even handles the timer itself?), rather than creating and destroying threads all the time.

Code: Select all

wxThread::ExitCode MainFrame::ServiceUpdateThread::Entry(void)
{
    bool Run = true;
    while(1)
    {
        wxEvent Event = WaitForEvent();
        switch(Event.GetEventType())
        {
        case wxID_EXIT:
            return 0;
        case wxID_UPDATE:
            Update();
            break;
        }
    }
}
Under windows (percificly with the Visual C++ compiler I think)I recall reading that there are memory leaks in the CRT if you used CreateThread and also used the CRT from that thread, and that in order to avoid this you must use _beginthreadex from process.h

I don't know which wxWidgets uses however, but it may be worth looking at.

bslatner
Earned a small fee
Earned a small fee
Posts: 21
Joined: Sun Mar 26, 2006 6:11 pm
Location: Charlotte, NC

Post by bslatner » Tue Sep 09, 2008 10:40 am

Ugly! wrote:You are creating all the threads on the heap. This is not bad for threads. I guess somehow thread's memory is released on program exit. But they won't be freed upon the thread termination.

So, if you don´t want memory pilling up, my advice would be to delete the thread when its work is done.
The thread objects delete themselves when they finish running. Under the theory that this wasn't working, I added explicit logic to delete the threads when they were done. This always caused the program to bomb because of trying to delete an invalid pointer.

User avatar
doublemax
Moderator
Moderator
Posts: 14782
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Post by doublemax » Tue Sep 09, 2008 11:02 am

Can you confirm that your threads terminate and don't hang in QueryWebService() ?

Does the memory consumption change when you create the threads, but comment out the call to QueryWebService() ?

Is everything that happens inside QueryWebService() thread-safe?

In general, it's not necessary to create a new thread every single time. Just create one, that runs in a loop and sleeps for 15 secs after each iteration.
Use the source, Luke!

stef
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Feb 06, 2006 11:44 am

Post by stef » Tue Sep 09, 2008 1:43 pm

Like the others also already said - one thread is enough for this basic task. Instead of letting it sleep 15 sec you could also use a condition which the thread is waiting on and is signalled by the timer.

bslatner
Earned a small fee
Earned a small fee
Posts: 21
Joined: Sun Mar 26, 2006 6:11 pm
Location: Charlotte, NC

Post by bslatner » Wed Sep 10, 2008 3:19 am

[quote="Fire Lancer"]It may be better to just have one worker thread that wait for an update request (or even handles the timer itself?), rather than creating and destroying threads all the time./quote]

This pretty much did the trick. I'm not sure what's going on here, but switching to a 2-thread model (i.e., one main thread that signals a worker thread to do some work every now and again) stopped the increase in memory usage.

bslatner
Earned a small fee
Earned a small fee
Posts: 21
Joined: Sun Mar 26, 2006 6:11 pm
Location: Charlotte, NC

Post by bslatner » Wed Sep 10, 2008 3:20 am

stef wrote:Like the others also already said - one thread is enough for this basic task. Instead of letting it sleep 15 sec you could also use a condition which the thread is waiting on and is signalled by the timer.
That's exactly what I did. Worked like a charm.

Post Reply