How to handle large files ?

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
wxND
In need of some credit
In need of some credit
Posts: 8
Joined: Mon Feb 23, 2009 11:48 am

How to handle large files ?

Post by wxND »

Dear forum,

I'd like to handle large textfiles in my programm. As I browsed through the wxWidgets Online Help, it's stated there that some classes (especially the one for text files) work not so well for large files (over 1MB) as content will be fully read first. My files are around 100MB and I intend to do some parsing.
Has anybody a good suggestion on how to handle large files ?
Maybe I should work with a wx file stream class ?
I'm really no pro (in fact the whole thing is to get myself started with C++), and any help is appreciated.

Thank you, wxND
Last edited by wxND on Fri Feb 27, 2009 1:00 pm, edited 1 time in total.
chris_bern
Earned some good credits
Earned some good credits
Posts: 125
Joined: Wed Mar 05, 2008 3:30 pm

Post by chris_bern »

I use STL functions ifstream and ofstream for large files.

Since you'll be using std::string etc, you will probably need some conversion, and this wiki page has all about converting needs -
http://wiki.wxwidgets.org/Converting_ev ... m_wxString

Chris
Auria
Site Admin
Site Admin
Posts: 6695
Joined: Thu Sep 28, 2006 12:23 am
Contact:

Post by Auria »

wxND
In need of some credit
In need of some credit
Posts: 8
Joined: Mon Feb 23, 2009 11:48 am

Post by wxND »

Thank you guys, I will have a more detailed look into your suggestions.
Dankä Chris :wink:
matthew180
In need of some credit
In need of some credit
Posts: 9
Joined: Tue Jul 31, 2007 5:01 pm
Contact:

Post by matthew180 »

You should be using memory mapping (mmap) for large files, especially if your access is going to be read only. Most platforms have a POSIX compliant implementation.

Also keep in mind that on a 32-bit platform, the maximum file size you can deal with is going to be right around 2G, unless you are compiling with large file support (which will add complexity if you are worried about portability.)

Matthew
Post Reply