saving structures

This forum can be used to talk about general design strategies, new ideas and questions in general related to wxWidgets. If you feel your questions doesn't fit anywhere, put it here.
Post Reply
hmtksteve
Earned a small fee
Earned a small fee
Posts: 10
Joined: Fri Mar 25, 2005 2:35 am

saving structures

Post by hmtksteve » Mon Mar 28, 2005 6:42 pm

I'm porting some code from C++ to wxwidgets

My question involves saving structures.

In the past if I have a structure:

Code: Select all

struct some_data {
   char name[50];
   int    number;
   char Job[100];
};
If I save this using fwrite I then use fread to read it back everything works fine because the size of the structure is preset when I compile (not a linked list, yeah flame me if you want, I just have not advanced my particular program to a point where I need to use a linked list.)

now, If I create a structure of:

Code: Select all

struct some_other_data {
    wxString name;
    int          number;
    wxString Job;
};
Can I use fread/fwrite to work with this structure or does the nature of wxString preclude this? If I understand the wxString class properly it does not create a variable of a set size but only sets up a pointer to the string. To me this is similar to declaring a char* in a structure (something I also do not use.)

Jorg
Moderator
Moderator
Posts: 3971
Joined: Fri Aug 27, 2004 9:38 pm
Location: Delft, Netherlands
Contact:

Post by Jorg » Mon Mar 28, 2005 7:50 pm

Well, this is difficult... I would do the following:

Make a class wxDataSerializable, with a read and write function which are virtual such as;

Code: Select all

class wxDataSerializable 
{
public:
    virtual int Save(wxStreamBuffer &stream) = 0;
    virtual int Load(wxStreamBuffer &stream) = 0;
  
    wxSerializable();
}
And instead of making structs, derive your class from this one and implement the Load and Save functions. Now you can save all your members. You can make helper functions in wxSerializable that write the and read the string and other complex objects.

The benefit from this construction is that if you encapsulate multiple complex classes in your class that also derive from a serializable base, you can call their Load and Save functions respectively as well, to do a descend writing.

Hope this helps a bit. There is unfortunately no generic method in a complex wx class to write or save ..

Regards,
- Jorgen
Forensic Software Engineer
Netherlands Forensic Insitute
http://english.forensischinstituut.nl/
-------------------------------------
Jorg's WasteBucket
http://www.xs4all.nl/~jorgb/wb

hmtksteve
Earned a small fee
Earned a small fee
Posts: 10
Joined: Fri Mar 25, 2005 2:35 am

clearing functions

Post by hmtksteve » Mon Mar 28, 2005 8:32 pm

A short cut I use to clear out my structures is to save an empty structure to file and reload it when I want to clear the existing structure.

If I fread the above structure back into the existing (used) structure, will it effectively clear it or will it just cause trouble?

arkanes
Experienced Solver
Experienced Solver
Posts: 59
Joined: Sun Oct 17, 2004 12:05 am

Post by arkanes » Mon Mar 28, 2005 9:20 pm

No, you cannot do this. Further, raw dumping of structures to and from files is almost ridiculously poor practice, and you shouldn't do it for any reason, most especially not in any code that might run anywhere except your own machine.

mjs
Experienced Solver
Experienced Solver
Posts: 93
Joined: Wed Feb 09, 2005 3:53 am
Contact:

Re: clearing functions

Post by mjs » Tue Mar 29, 2005 4:39 am

hmtksteve wrote:If I fread the above structure back into the existing (used) structure, will it effectively clear it or will it just cause trouble?
It will cause trouble because wxString is not like a simple C struct - it's a full class that contains pointers. When you write the data to a file directly, you get many pointer values that will point to garbage when you want to load the structure later.

To do this is not a DONT ... it's a heavy mega ultra MUST NOT (!!!!) :wink:

Regards,
Mark

Jorg
Moderator
Moderator
Posts: 3971
Joined: Fri Aug 27, 2004 9:38 pm
Location: Delft, Netherlands
Contact:

Post by Jorg » Tue Mar 29, 2005 10:01 am

That is why I said, serialize ;-)

It would be nice if all wx relevant data classes would have some kind of bridge function or would be inherited from a serializable class.

For the record, a class is nothing more then a struct with some code around it. The problem with just writing back the data belonging to a class/struct is:

- Pointers (created with new) are not valid anymore after a read. The memory allocated and pointed to is never the same after a second time
- The class can be changed in the meantime, eg extended so the whole binary footprint is not the same
- No deep copy / writing. Classes created with new inside your other class is simply not copied only theyr pointer is taken.

I have developed a class for serializing if you are interested. It is called wxSIO I can see if I can brush it up enough to be workable for you.

With this class you can do the following:

Code: Select all

class SomeDataClass
{
  // serializing of file
  void Serialize(wxSIO &file) {
		if(file.IsWriting())
		{
			// we are writing
			file << m_myString;
			file << m_myInt;
		}
		else
		{
			// we are reading
			file >> m_myString;
			file >> m_myInt;
		}
	};
}

// use the class like this
wxSIOFile myFile();
if(myFile.ReadFromFile("c:\\some\\file.bin"))
{
	m_data = new SomeDataClass();
	m_data->Serialize(file);
	
	// ofcourse you can propagate the serializing ..
}
The wxSIO class has some features which might interest you;

- Platform independent binary code (meaning a linux bin file can be read also on a windows machine)
- A header can be set which is written to the file and read back. This has as advantage that you can check if the file is what you think it is, if it does not match the file returns an error
- The serializing is done with type safe mechanism. Every value is preceded with a type to make sure that the next value is actually an int, string, quad, etc
- Whenever an error occurs during reading the file locks up and will only read predictable values (0 for int, empty string, false for bool etc).
- wxSIO::OK() can be checked everytime to make sure the reading is still going ok

If you (or someone else) is interested in this class, I will try to make it compile again, with a small sample. Just say so.

Regards,
- Jorgen
Forensic Software Engineer
Netherlands Forensic Insitute
http://english.forensischinstituut.nl/
-------------------------------------
Jorg's WasteBucket
http://www.xs4all.nl/~jorgb/wb

ddaeschl
Knows some wx things
Knows some wx things
Posts: 41
Joined: Wed Oct 27, 2004 6:06 pm
Location: Western NY
Contact:

Post by ddaeschl » Tue Mar 29, 2005 8:21 pm

You may also want to check out the boost serialization library. It takes care of basic classes as well as nested data structures and lists.

More info Here:
http://boost.org/libs/serialization/doc/index.html

Ryan Wilcox
I live to help wx-kind
I live to help wx-kind
Posts: 194
Joined: Mon Aug 30, 2004 1:26 pm
Location: PA, USA
Contact:

Post by Ryan Wilcox » Wed Mar 30, 2005 1:45 am

If the OP isn't ready for linked lists, they probably aren't ready for the boost serialization library :wink:

To tell you the truth, I tried to use boost::serialize a few months back for one of my projects, spent about 2 days trying to get it to compile and work in simple samples, and nothing (mostly couldn't compile. Boost uses templates almost to the point of unreadability... it took me maybe 30 minutes to dissect a boost::serialize compiler error, and I still didn't know what was going on.)

I ended up writing something like Jorg's wxSIO (complete with Serialize()/DeSerialize() methods, typesafe serialization/deserialization, etc.) I'd be interesting in looking at your wxSIO class, Jorg, and suggesting things that I learned in the design of my classes.

To the original poster: remember that, in the case of wxString, there is a c_str() method which returns a char*. You could feed that to your write methods (fwrite() or whatever). Strings are probably the easiest thing to write to a file, so you're safe - in this particular case.

Another note: you really shouldn't be writing pure structs to a file, as this prevents your files from being used on other processor architectures (the whole big endian, little endian thing). A format like XML, or something pure text (Unicode with BOM is another good example) tends to not get confused this way. Serialization APIs have already taken this into account.

Hope this helps
Ryan Wilcox
Wilcox Development Solutions
http://www.wilcoxd.com

User avatar
Ryan Norton
Moderator
Moderator
Posts: 1319
Joined: Mon Aug 30, 2004 6:01 pm

Post by Ryan Norton » Wed Mar 30, 2005 2:10 am

Ryan Wilcox wrote:To the original poster: remember that, in the case of wxString, there is a c_str() method which returns a char*. You could feed that to your write methods (fwrite() or whatever). Strings are probably the easiest thing to write to a file, so you're safe - in this particular case.
Unless you're using unicode of course :wink: .

Anyway, if you know that you're structure is just going to have primitive types you _could_ write it to a file, keeping in mind:

1) Endian issues

2) Data structure alignment issues
[Mostly retired moderator, still check in to clean up some stuff]

ddaeschl
Knows some wx things
Knows some wx things
Posts: 41
Joined: Wed Oct 27, 2004 6:06 pm
Location: Western NY
Contact:

Post by ddaeschl » Wed Mar 30, 2005 2:43 am

If you're going to write the struct to a file, you also run the risk of a stack or buffer overflow if you save statically sized c-style strings. Someone could go into your file, insert a bunch of stuff until they start overwriting the stack. Once they figure out where their code is getting inserted, they could modify your config file to do things that you never intended in the security context of your program..

In this day, that may be enough reason not to write a file the way you originally posted.

hmtksteve
Earned a small fee
Earned a small fee
Posts: 10
Joined: Fri Mar 25, 2005 2:35 am

XML

Post by hmtksteve » Wed Mar 30, 2005 2:59 am

I'm just going to save as XML and create a function full of

Code: Select all

integervariablexyz=0;
wxStringvariablexyz.Clear();
To clear out my data...

I'm just looking for an "easier, quicker, more seductive" way to do this.

Jorg
Moderator
Moderator
Posts: 3971
Joined: Fri Aug 27, 2004 9:38 pm
Location: Delft, Netherlands
Contact:

Post by Jorg » Wed Mar 30, 2005 2:49 pm

I already started with brushing up. Right now it uses the normal file handle routines (because it comes from the pre wxWidgets period) but I am planning in making it a wxStreamBuffer compatible class, so the serialization can be sent to a wxZIPFileStream or what else.

This class goes back a long way. And it is really tested properly and used in big projects even at my company.

It will take me a little time though to finish it. I am busy at work with yet anoter crisis, and I haven't slept decently in a couple of days ..

Regards,
- Jorgen
Forensic Software Engineer
Netherlands Forensic Insitute
http://english.forensischinstituut.nl/
-------------------------------------
Jorg's WasteBucket
http://www.xs4all.nl/~jorgb/wb

mjs
Experienced Solver
Experienced Solver
Posts: 93
Joined: Wed Feb 09, 2005 3:53 am
Contact:

Post by mjs » Wed Mar 30, 2005 2:53 pm

Jorg wrote:I have developed a class for serializing if you are interested. It is called wxSIO I can see if I can brush it up enough to be workable for you.
The first time I implemented something like this was similar to your solution, but using a binary representation may cause many problems when you enhance a class and you want to store/load additional data. I was able to make a temporary "fix" by adding a "version id". Now I use something similar to:

Code: Select all

bool Store(wxConfigBase &config, const wxString &sPath) const;
bool Load(const wxConfigBase &config, const wxString &sPath);
Regards,
Mark

Jorg
Moderator
Moderator
Posts: 3971
Joined: Fri Aug 27, 2004 9:38 pm
Location: Delft, Netherlands
Contact:

Post by Jorg » Wed Mar 30, 2005 7:53 pm

Ok guys, wxSIO (first 'official' release although it is used in wxCRP) can be downloaded here:

http://forums.wxwidgets.org/viewtopic.php?p=7036#7036

Regards,
- Jorgen
Forensic Software Engineer
Netherlands Forensic Insitute
http://english.forensischinstituut.nl/
-------------------------------------
Jorg's WasteBucket
http://www.xs4all.nl/~jorgb/wb

Post Reply