Page 1 of 1

help wxTextCtrl with very large files

Posted: Wed Dec 21, 2005 5:04 pm
by jsheets
Hello everyone,

We are working on a project that translates very large log files into more human readable form, these log files range from 15 MB to over 1 GB depending on who created it and what information is logged.

We have two wxTextCtrls, one that has raw log file output and another that has translated text. We want to allow the user to view the full untranslated log file by scrolling through the raw log wxTextCtrl but to conserve memory we also want to limit the currently displayed text to about 50 lines.

We have been unable to implement this in such a way that does not severely impact performance (currently every time a line is added to the control we check to see if there is 50 lines, if so we drop the first line and add the new content).

The last bit of information is the raw log file can be a live file, meaning we are parsing it at the same time information is being added to it.

I'd sincerely appreciate any help or suggestions.

Jason

Posted: Wed Dec 21, 2005 5:07 pm
by benedicte
Maybe you should use a widget more like a "text editor", such as wxStyledTextControl (stc in contrib folder), or wxRichTextEditor (http://forums.wxwidgets.org/viewtopic.php?t=4663).

Posted: Wed Dec 21, 2005 5:14 pm
by jsheets
Thanks for the reply,

I'm open to using a different editor, such as wxStyledTextControl or wxScintilla but unless I've missed something in the documentation (which is entirely possible) I don't see anything in those editors that would enable it to handle the larger file differently than wxTextCtrl.

I'm assuming I'll need to somethimg similar to attaching the control to the file stream and adjust it on a scroll event?

Posted: Wed Dec 21, 2005 5:22 pm
by benedicte
I've heard that system text control widgets may not handle large buffers... (you can probably do a search with wxTextCtrl here and in the wxWidgets newsgroup).
As most wxWidgets controls are just a layer on system-dependant objects, you may face troubles with that point.

Text editors are able to load larger buffers, although I don't know if they have a limit, too.

Posted: Wed Dec 21, 2005 5:38 pm
by jsheets
Unfortunately from what I've read even Scintilla is designed only for reasonably small files (in the hundreds of KB) and scales to megabytes but doesn't scale well into hundreds of MB.

I'm not so much wanting to load the entire file into the control at one time as much as make it appear that I have (much like vi and vim does) so memory usage is constant at say the amount of memory required to load 50 lines into the control.

Posted: Wed Dec 21, 2005 9:52 pm
by travfar
Hi!

Do the users have to edit the large file?

If they don't, there really is no need for a wxTextCtrl. You could use a wxDC and draw the text onto a wxClientDC which is inside a wxScrolledWindow.