Detached wxThread causing memory leak? Topic is solved

[evendine]

Hi,

have come across what appears to be a memory leak seemingly due to detached threads not being automatically deleted as expected.

The minimal example below uses a timer to create a detached thread repeatedly and according to GetProcessMemoryInfo() + task manager, app memory usage intermittently increases, suggesting that from time to time, the thread's resources are not being freed.

May well be missing something, but any advice on how to solve this issue much appreciated...

Code: Select all

///////////////////////////////////////////////////////////////////////////////
// Detached wxThread memory leak test app
///////////////////////////////////////////////////////////////////////////////
// NB: To ensure correct resolution of GetProcessMemoryInfo() symbols, add psapi.lib to 
// Linker->Input->Additional Dependencies and compile with -DPSAPI_VERSION=1 flag set
// under C/C++->Command Line->Additional Options

#ifndef WX_PRECOMP 
    #include "wx\wx.h"    
#endif 

#include <string>
#include <wx/timer.h>
#include <psapi.h>

using namespace std;

///////////////////////////////////////////////////////////////////////////////
// Utilities
//
wxString DateTimeNowISO(void)
{
   wxDateTime CurrentDateTime;
   CurrentDateTime.SetToCurrent();  
   return(CurrentDateTime.Format("%Y-%m-%d %H:%M:%S"));
}

///////////////////////////////////////////////////////////////////////////////
//
void PrintMemoryInfo(string TheMessage, unsigned long processID)
{
    HANDLE hProcess;
    PROCESS_MEMORY_COUNTERS_EX pmc;

    // Print information about the memory usage of the process.
    hProcess = OpenProcess(PROCESS_QUERY_INFORMATION |
                                    PROCESS_VM_READ,
                                    FALSE, processID);
    if(NULL == hProcess)
        return;
    
    ZeroMemory(&pmc, sizeof(PROCESS_MEMORY_COUNTERS_EX));

    if(GetProcessMemoryInfo( hProcess, (PROCESS_MEMORY_COUNTERS*)&pmc, sizeof(pmc)))
    {
       wxLogDebug(wxString::Format("%s -- %s Current Memory usage -> Working: %d Bytes %.0f KB, Private: %d Bytes %.0f KB",
          TheMessage,
          DateTimeNowISO(),
          pmc.WorkingSetSize, (float)pmc.WorkingSetSize / 1024.0, 
          pmc.PrivateUsage, (float)pmc.PrivateUsage / 1024.0));

          wxLog::GetActiveTarget()->Flush();
    }

    CloseHandle(hProcess);
}

///////////////////////////////////////////////////////////////////////////////
//
class FrameSyncThread : public wxThread
{    
   public:
	   FrameSyncThread(void) : wxThread(wxTHREAD_DETACHED) {}
	      
      ~FrameSyncThread()
      {
         PrintMemoryInfo("FrameSyncThread::~FrameSyncThread(1)", wxGetProcessId());
      }

   private:
      virtual void *Entry() 
	   {       
         PrintMemoryInfo("FrameSyncThread::Entry(1)", wxGetProcessId());
      
         // Do some work...
         wxMilliSleep(500);

         return(NULL);
	   }

	   void OnExit()
	   {	         
         PrintMemoryInfo("FrameSyncThread::OnExit(1)", wxGetProcessId());
	   }
};

///////////////////////////////////////////////////////////////////////////////
//
class MyFrame : public wxFrame
{ 
   public: 
      void OnTimer(wxTimerEvent &event)
      {
         PrintMemoryInfo("MyFrame::OnTimer(1)", wxGetProcessId());  

		   SyncThread = new FrameSyncThread(); 
         
         PrintMemoryInfo("MyFrame::OnTimer(2)", wxGetProcessId()); 

		   if(SyncThread->Create() == wxTHREAD_NO_ERROR)
		   {
            PrintMemoryInfo("MyFrame::OnTimer(3)", wxGetProcessId()); 

			   SyncThread->Run();

            PrintMemoryInfo("MyFrame::OnTimer(4)", wxGetProcessId());  
		   }
		   else
		   {
            SyncThread = 0;
		   }
      }

      MyFrame() : wxFrame(NULL, wxID_ANY, _("wxThread memory leak test"), wxDefaultPosition, wxSize(600,500)) 
      {                            
         SyncTimer = new wxTimer(this, wxID_HIGHEST + 1); 
         Connect(SyncTimer->GetId(), wxEVT_TIMER, 
            wxTimerEventHandler(MyFrame::OnTimer), NULL, this);
         SyncTimer->Start(2000, wxTIMER_CONTINUOUS);
      }    

   private:
      FrameSyncThread *GetSyncThread(void){return SyncThread;}

      wxTimer *SyncTimer;
      FrameSyncThread *SyncThread;
};

///////////////////////////////////////////////////////////////////////////////
//
class MyApp : public wxApp 
{ 
   public:    
      virtual bool OnInit() 
      { 
         if(!wxApp::OnInit()) 
            return false;    

         MyFrame* pFrame = new MyFrame();                
         pFrame->Show(); 

         return true; 
      } 
}; 

IMPLEMENT_APP(MyApp)

///////////////////////////////////////////////////////////////////////////////

Output:
MyFrame::OnTimer(1) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9285632 Bytes 9068 KB, Private: 2207744 Bytes 2156 KB
MyFrame::OnTimer(2) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9285632 Bytes 9068 KB, Private: 2207744 Bytes 2156 KB
MyFrame::OnTimer(3) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9310208 Bytes 9092 KB, Private: 2265088 Bytes 2212 KB
MyFrame::OnTimer(4) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9318400 Bytes 9100 KB, Private: 2265088 Bytes 2212 KB
FrameSyncThread::Entry(1) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9322496 Bytes 9104 KB, Private: 2265088 Bytes 2212 KB
FrameSyncThread::OnExit(1) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9342976 Bytes 9124 KB, Private: 2277376 Bytes 2224 KB
The thread 'Win32 Thread' (0x2584) has exited with code 0 (0x0).
FrameSyncThread::~FrameSyncThread(1) -- 2018-05-18 21:30:58 Current Memory usage -> Working: 9342976 Bytes 9124 KB, Private: 2277376 Bytes 2224 KB
...
MyFrame::OnTimer(1) -- 2018-05-18 21:35:15 Current Memory usage -> Working: 9580544 Bytes 9356 KB, Private: 2347008 Bytes 2292 KB
MyFrame::OnTimer(2) -- 2018-05-18 21:35:15 Current Memory usage -> Working: 9580544 Bytes 9356 KB, Private: 2347008 Bytes 2292 KB
MyFrame::OnTimer(3) -- 2018-05-18 21:35:15 Current Memory usage -> Working: 9605120 Bytes 9380 KB, Private: 2404352 Bytes 2348 KB
MyFrame::OnTimer(4) -- 2018-05-18 21:35:15 Current Memory usage -> Working: 9605120 Bytes 9380 KB, Private: 2404352 Bytes 2348 KB
FrameSyncThread::Entry(1) -- 2018-05-18 21:35:15 Current Memory usage -> Working: 9617408 Bytes 9392 KB, Private: 2404352 Bytes 2348 KB
FrameSyncThread::OnExit(1) -- 2018-05-18 21:35:16 Current Memory usage -> Working: 9637888 Bytes 9412 KB, Private: 2416640 Bytes 2360 KB
The thread 'Win32 Thread' (0x2644) has exited with code 0 (0x0).
FrameSyncThread::~FrameSyncThread(1) -- 2018-05-18 21:35:16 Current Memory usage -> Working: 9637888 Bytes 9412 KB, Private: 2416640 Bytes 2360 KB

[doublemax]

Getting the memory info is not a reliable way to look for memory leaks. If memory is freed, it's not automatically assigned to the free pool.

If Visual Studio doesn't report memory leaks when you run the application in debug mode, there most likely is none.

[evendine]

Thanks for the insight - used GetProcessMemoryInfo() as a last resort, as was unable to find a leak using any of the usual tools (tried VLD, Dr Memory and several others). It at least allowed me to pin down the code location causing the issue.

Is there a simple way to assign memory blocks to the free pool when thread resources are released? Currently, my app memory usage, as reported by Task Manager, Process Explorer and etc, climbs continually and gives the appearance of a leak even if this isn't actually the case. Certainly makes it much harder to spot other real leaks caused by unfreed periodic memory assignments...

[doublemax]

If both VS and VLD don't report a memory leak, i'd say there is none. So unless there's a real problem, i'd ignore this for now.

BTW: do you really have to create and destroy threads all the time and is that really the reason for the increasing memory usage?

Usually i have a jobqueue and leave worker threads running all the time, they just wake up frequently and check if there is any job. And if not, they go to sleep again.

[evendine]

Yep, sounds a better approach. Could just keep the thread running (unless there's a fatal error in the task it executes) and use a semaphore incremented by the timer + a Wait() controlling a loop in the thread's Entry method to define the point at which the task's started, as is currently achieved by starting a new thread...

Not sure if you've had chance to run the minimal app, but can't see any reason other than the thread creation / destruction cycle for the increase in the app's reported memory usage. Would be interested to know if the issue is repeatable in a different dev environment. Have tried it on a few machines here running the release binary with the same result.

[doublemax]

Not sure if you've had chance to run the minimal app, but can't see any reason other than the thread creation / destruction cycle for the increase in the app's reported memory usage. Would be interested to know if the issue is repeatable in a different dev environment.

No, i haven't tried it. Even if i'd get the same result, there's nothing i could say about this, i just don't know enough about Windows' internal memory management.

Maybe vmmap can tell you more.
http://blogs.microsoft.co.il/sasha/2016 ... mystified/

[evendine]

MS memory management is anything but transparent, that's for sure!

Implemented a wxSemaphore based wrapper for my background thread as mentioned above and everything seems to work fine now. No leaks (or increasing app usage) with the actual background task in the loop, so good news - thanks for your support doublemax

Have to say, having spent many hours hunting for a leak in the app itself, the GetProcessMemoryInfo() approach has some merit, in that it at least allows you to pin down the location of the problem.

Think it might be helpful to others if the docs for wxThread included a note about the 'disappearing memory' issue. Given the vagaries of memory leak detectors, likely most people who see their app usage increasing steadily would put it down to a leak in the app...

[doublemax]

For completeness sake i wanted to mention wxMessageQueue which makes the distribution of jobs to any number of worker threads pretty simple.
http://docs.wxwidgets.org/trunk/classwx ... _01_4.html

[evendine]

Yes - have found it really useful in other contexts, where there are a number of different jobs which need sequential background processing. The OP context is a simpler problem, where a single task (refreshing a form's contents from a remote data source) is executed in a background thread as base class service...

[evendine]

Have been investigating versions of the app using VMMap - really useful tool

Interesting to compare memory maps before and after the changes:

Modified app - 16 hr run

Original app - 72 hr run

The 'missing memory' is clearly held in the private heap and continues to increase over time, while the modified app has shown no appreciable increase since startup...