Progress Dialog seems extremely slow Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
georgeplusplus
Knows some wx things
Knows some wx things
Posts: 25
Joined: Tue May 11, 2021 1:49 pm

Progress Dialog seems extremely slow

Post by georgeplusplus »

I am using a progress dialog to show the amount of bytes processed in a file. However when I tested my code below to read a 2kilobyte file, the Progress Dialog finishes almost a full minute than when the actual function wraps up. Something is very wrong here.

I am not using any multi threading.

Code: Select all

void MainWindow::HandleFile(const wxFileName& filename)
{
	wxFile my_file(filename);

	wxProgressDialog progress(wxT("Parsing "), wxT("File"), my_file.Length(), this,);


	if (my_file.isValid())
	{
		wxString msg;
		msg.Printf(wxT("Parsed %llu of %llu bytes"), my_file.Tell(), my_file.Length());
		progress.Update(my_file.Tell(), msg);


		wxLongLong next_update = 0;
		while (!my_file.Eof())
		{
			my_file.ProcessNextByte();
			if (wxGetLocalTimeMillis() > next_update)
			{

				msg.Printf(wxT("Parsed %llu of %llu bytes"), my_file.Tell(), my_file.Length());
				progress.Update(my_file.Tell(), msg);
				next_update = wxGetLocalTimeMillis() + 200;   // 200ms = 5 updates per second
			}
			
		}
	}
}
User avatar
doublemax
Moderator
Moderator
Posts: 19115
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Progress Dialog seems extremely slow

Post by doublemax »

The code looks fine to me and also worked fine (with the modifications needed to make it work).

As this looks a little bit like pseudo-code, check the parts you haven't shown ;)

Also: wxWidgets version, platform?
Use the source, Luke!
georgeplusplus
Knows some wx things
Knows some wx things
Posts: 25
Joined: Tue May 11, 2021 1:49 pm

Re: Progress Dialog seems extremely slow

Post by georgeplusplus »

I am using windows wxWidgets 3.14. Turns out its just the speed my program is running at and nothing is "slow" but my code.

The pseudo code missing is nothing special basically just a read operation and later some conversion to engineering units.

Code: Select all

uint8_t MyFile::ProcessNextByte()
{
	uint8_t byte_buffer;
	auto read_bytes = Read((uint8_t*)(&byte_buffer), sizeof(byte_buffer));
	if (read_bytes < sizeof(byte_buffer))
	{
		wxLogError("Read Operation failed at 0x%llu.\nReceived %s \n Expected %u", Tell(), wxString::Format("%02X ", byte_buffer), read_bytes);
		return 0;
	}
	
	return byte_buffer;
}
Currently I am reading it one byte at a time. Will a bulk read go a lot faster?

It is strange to me since before wxFile I was using std::ifstream and it was about 4 times faster and it was also reading one byte at a time.
User avatar
doublemax
Moderator
Moderator
Posts: 19115
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Progress Dialog seems extremely slow

Post by doublemax »

Currently I am reading it one byte at a time. Will a bulk read go a lot faster?
Yes.

But i thought your problem was that the progress dialog is still busy when your reading code has already finished. Or did i misunderstand that?
It is strange to me since before wxFile I was using std::ifstream and it was about 4 times faster and it was also reading one byte at a time.
It's possible that it uses an internal buffer, like wxBufferedInputStream
https://docs.wxwidgets.org/trunk/classw ... tream.html
Use the source, Luke!
georgeplusplus
Knows some wx things
Knows some wx things
Posts: 25
Joined: Tue May 11, 2021 1:49 pm

Re: Progress Dialog seems extremely slow

Post by georgeplusplus »

Yes I jumped to the conclusion that there was no way a 2MB file was taking 20seconds! That was my fault.

Can you recommend a good quick and dirty algorithm to use for chunking?
User avatar
doublemax
Moderator
Moderator
Posts: 19115
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Progress Dialog seems extremely slow

Post by doublemax »

Code: Select all

#define BUFSIZE 65536
wxFile f("foo.bar", wxFile::read );
if( f.IsOpened() ) {
  unsigned char buffer[BUFSIZE];
  while( !f.Eof() ) {
    size_t datalen = f.Read(buffer, BUFSIZE);
    // process datalen bytes in buffer
  }
}
Use the source, Luke!
georgeplusplus
Knows some wx things
Knows some wx things
Posts: 25
Joined: Tue May 11, 2021 1:49 pm

Re: Progress Dialog seems extremely slow

Post by georgeplusplus »

Thanks! I used that to whip up this to handle oddball remainders.

Code: Select all

uint32_t chunks = (peak_file.Length() - peak_file.Tell()) / 1024;
std::vector<size_t> chunk_sizes (chunks, 1024);

if ((peak_file.Length() - peak_file.Tell()) % 1024 > 0)
{
	chunk_sizes.push_back((peak_file.Length() - peak_file.Tell()) % 1024);
}
Post Reply