Writing and Reading wxString from file. Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
ggenije
Earned a small fee
Earned a small fee
Posts: 11
Joined: Thu Apr 16, 2020 3:57 pm

Writing and Reading wxString from file.

Post by ggenije » Sat May 30, 2020 10:43 am

I need to save data for my application so the users can load the save data again.
That data consists of certain combination of numbers , string and booleans.
But I'm having trouble with storing wxString to file (I know there is wxTextFile, but I need to use binary file so I can combine numbers, strings without using too surplus of memory).
I tried next:

Code: Select all

	void SaveWxStringToFile(const wxString &ws, wxFileOutputStream& fos)
	{
		wxString::size_type size = ws.size();
		fos.Write(&size, sizeof(wxString::size_type));
		for (int i = 0; i < ws.size(); i++)
			fos.Write(&ws[i], sizeof(wxUniChar));		
	}

	void LoadWxStringFromFile(wxString& ws, wxFileInputStream& fis)
	{
		wxString::size_type size;
		fis.Read(&size, sizeof(wxString::size_type));
		ws = "";		
		wxUniChar c;
		for (int i = 0; i < ws.size(); i++)
		{
			fis.Read(&c, size * sizeof(wxUniChar));
			ws.Append(c);
		}
	}
But it gives no positive results.
What it does to do is:
Save -> first put size of the wxstring in the outputStream and then put every character into.
Load -> it loads size then appends new characters into empty wxstring.

PB
Part Of The Furniture
Part Of The Furniture
Posts: 2463
Joined: Sun Jan 03, 2010 5:45 pm

Re: Writing and Reading wxString from file.

Post by PB » Sat May 30, 2020 1:58 pm

Unless you are talking about megabytes of data, I would suggest using wxFileConfig (based on wxTextFile).

The overhead should be negligible and you do not have to care about format.

If you still want to store strings as binary, you need to take care of encoding. I would store such strings in UTF-8.

You should not also use types that are different in 32- and 64-bit, such as size_t etc.

PB
Part Of The Furniture
Part Of The Furniture
Posts: 2463
Joined: Sun Jan 03, 2010 5:45 pm

Re: Writing and Reading wxString from file.

Post by PB » Sat May 30, 2020 2:40 pm

FWIW, here is an example of writing and reading binary data, it is more C than C++, but good enough for demonstrating reading and writing. You can see that doing it correctly by yourself quite cumbersome. That is why I think it is best to employ a library doing that for you (such as SQLite for records), even if you think it is an overkill for the task.

Code: Select all

#include <wx/wx.h>
#include <wx/buffer.h>
#include <wx/wfstream.h>

struct Person
{
    wxString name;
    uint8_t  age{0};
    bool     married{false};
};

bool WritePersonToStream(wxOutputStream& stream, const Person& person)
{
    wxCHECK(stream.IsOk(), false);

    const wxScopedCharBuffer nameUTF8(person.name.ToUTF8());
    const wxUint64           nameUTF8size = nameUTF8.length();
    const uint8_t            marriedInt = person.married;

    return stream.WriteAll(&nameUTF8size, sizeof(nameUTF8size))
           && stream.WriteAll(nameUTF8.data(), nameUTF8.length())
           && stream.WriteAll(&person.age, sizeof(person.age))
           && stream.WriteAll(&marriedInt, sizeof(marriedInt));
}

bool ReadPersonFromStream(wxInputStream& stream, Person& person)
{
    wxCHECK(stream.IsOk(), false);

    wxCharBuffer nameUTF8;
    wxUint64     nameUTF8size = 0;
    uint8_t      age = 0;
    uint8_t      marriedInt = 0;

    if ( !stream.ReadAll(&nameUTF8size, sizeof(nameUTF8size)) )
        return false;

    nameUTF8.extend(nameUTF8size);
    if ( !stream.ReadAll(nameUTF8.data(), nameUTF8.length()) )
        return false;

    if ( !stream.ReadAll(&age, sizeof(age)) )
        return false;
    if ( !stream.ReadAll(&marriedInt, sizeof(marriedInt)) )
        return false;

    person.name = wxString::FromUTF8(nameUTF8);
    person.age = age;
    person.married = marriedInt == 1 ? true : false;
    return true;
}

class MyApp : public wxApp
{
public:
    bool OnInit() override
    {
        wxFFileStream stream("Persons.per", "w+b");
        Person person;

        person.name    = "John Smith";
        person.age     = 30;
        person.married = true;

        if ( !WritePersonToStream(stream, person) )
        {
            wxLogError("Could not write person to stream.");
            return false;
        }

        person.name = ""; person.age = 0; person.married = false;
        stream.SeekO(0);

        if ( !ReadPersonFromStream(stream, person) )
        {
            wxLogError("Could not read person from stream.");
            return false;
        }

        wxString s;

        s << "Name: " << person.name << " Age: " << person.age
            << "Married: " << person.married;
        wxLogMessage(s);

        return false;
    }
}; wxIMPLEMENT_APP(MyApp);
Please notice that when writing a binary file, some standards should be always employed. I.e, the character encoding and types sizes must not be build dependent, e.g., wchar_t is 16-bit on Windows but 32-bit on Linux, size_t is 4 bytes in 32-bit build but 8 bytes in 64-bit build etc. Similarly, do not use types whose size is implementation-defined, such as bool.

Additionally, when storing "records" one should consider whether to not make string fields fixed-width. Doing that allows quick seeking between the records without the need to parse all the records between.

Last but not least, make sure you know what all the file modes mean so you are not surprised that opening the file e.g."w+" creates an empty file.

BTW, either way, as I already shown you, I would not recommend reading or writing strings byte-by-byte, that is very slow, buffered IO or not.

You also have a bug in your read string code

Code: Select all

ws = "";		
wxUniChar c;
for (int i = 0; i < ws.size(); i++)
Executing wx= "" will make ws.size() return 0 and the code inside the loop will be never executed, you wanted to use size instead.

ggenije
Earned a small fee
Earned a small fee
Posts: 11
Joined: Thu Apr 16, 2020 3:57 pm

Re: Writing and Reading wxString from file.

Post by ggenije » Sat May 30, 2020 5:53 pm

PB wrote:
Sat May 30, 2020 2:40 pm
FWIW, here is an example of writing and reading binary data, it is more C than C++, but good enough for demonstrating reading and writing. You can see that doing it correctly by yourself quite cumbersome. That is why I think it is best to employ a library doing that for you (such as SQLite for records), even if you think it is an overkill for the task.

Code: Select all

#include <wx/wx.h>
#include <wx/buffer.h>
#include <wx/wfstream.h>

struct Person
{
    wxString name;
    uint8_t  age{0};
    bool     married{false};
};

bool WritePersonToStream(wxOutputStream& stream, const Person& person)
{
    wxCHECK(stream.IsOk(), false);

    const wxScopedCharBuffer nameUTF8(person.name.ToUTF8());
    const wxUint64           nameUTF8size = nameUTF8.length();
    const uint8_t            marriedInt = person.married;

    return stream.WriteAll(&nameUTF8size, sizeof(nameUTF8size))
           && stream.WriteAll(nameUTF8.data(), nameUTF8.length())
           && stream.WriteAll(&person.age, sizeof(person.age))
           && stream.WriteAll(&marriedInt, sizeof(marriedInt));
}

bool ReadPersonFromStream(wxInputStream& stream, Person& person)
{
    wxCHECK(stream.IsOk(), false);

    wxCharBuffer nameUTF8;
    wxUint64     nameUTF8size = 0;
    uint8_t      age = 0;
    uint8_t      marriedInt = 0;

    if ( !stream.ReadAll(&nameUTF8size, sizeof(nameUTF8size)) )
        return false;

    nameUTF8.extend(nameUTF8size);
    if ( !stream.ReadAll(nameUTF8.data(), nameUTF8.length()) )
        return false;

    if ( !stream.ReadAll(&age, sizeof(age)) )
        return false;
    if ( !stream.ReadAll(&marriedInt, sizeof(marriedInt)) )
        return false;

    person.name = wxString::FromUTF8(nameUTF8);
    person.age = age;
    person.married = marriedInt == 1 ? true : false;
    return true;
}

class MyApp : public wxApp
{
public:
    bool OnInit() override
    {
        wxFFileStream stream("Persons.per", "w+b");
        Person person;

        person.name    = "John Smith";
        person.age     = 30;
        person.married = true;

        if ( !WritePersonToStream(stream, person) )
        {
            wxLogError("Could not write person to stream.");
            return false;
        }

        person.name = ""; person.age = 0; person.married = false;
        stream.SeekO(0);

        if ( !ReadPersonFromStream(stream, person) )
        {
            wxLogError("Could not read person from stream.");
            return false;
        }

        wxString s;

        s << "Name: " << person.name << " Age: " << person.age
            << "Married: " << person.married;
        wxLogMessage(s);

        return false;
    }
}; wxIMPLEMENT_APP(MyApp);
Actually this is exactly what I needed.
Here is working code if you are wondering:

Code: Select all

void SaveWxStringToFile(const wxString &ws, wxFileOutputStream& fos)
	{	
		wxScopedCharBuffer UTF8Data(ws.ToUTF8());
		uint32_t size = UTF8Data.length();
		fos.Write(&size, sizeof(uint32_t));
		fos.Write(UTF8Data.data(), size*sizeof(char));
	}
wxString LoadWxStringFromFile(wxFileInputStream& fis)
	{
		uint32_t  size;
		fis.Read(&size,sizeof(uint32_t));
		wxCharBuffer UTF8Data;
		UTF8Data.extend(size);
		fis.Read(UTF8Data.data(), size *sizeof(char));
		return wxString::FromUTF8(UTF8Data);
	}
It's not that cumbersome as you said so :D
I don't want to use any additional libraries because I wasn't practicing linking them, so they confuse me currently (I'll learn that in future).
Also I'll need to , again, learn to use that library (such as SQLite) which means additional time learning and less time for actual working.
Please notice that when writing a binary file, some standards should be always employed. I.e, the character encoding and types sizes must not be build dependent, e.g., wchar_t is 16-bit on Windows but 32-bit on Linux, size_t is 4 bytes in 32-bit build but 8 bytes in 64-bit build etc. Similarly, do not use types whose size is implementation-defined, such as bool.
That's why UTF-8 is used right?
I replaced size_t with uint32_t, it should work now...?
Additionally, when storing "records" one should consider whether to not make string fields fixed-width. Doing that allows quick seeking between the records without the need to parse all the records between.
Don't worry about that it's similar to your "class Person" example :D
Last but not least, make sure you know what all the file modes mean so you are not surprised that opening the file e.g."w+" creates an empty file.

BTW, either way, as I already shown you, I would not recommend reading or writing strings byte-by-byte, that is very slow, buffered IO or not.

You also have a bug in your read string code

Code: Select all

ws = "";		
wxUniChar c;
for (int i = 0; i < ws.size(); i++)
Executing wx= "" will make ws.size() return 0 and the code inside the loop will be never executed, you wanted to use size instead.
Yes I knew it isn't effective to read or write byte by byte. I sent that just to show you that I at least tried :mrgreen: (Actually to show you desired structure, that bug was accident).

PB
Part Of The Furniture
Part Of The Furniture
Posts: 2463
Joined: Sun Jan 03, 2010 5:45 pm

Re: Writing and Reading wxString from file.

Post by PB » Sat May 30, 2020 6:31 pm

ggenije wrote:
Sat May 30, 2020 5:53 pm
Here is working code if you are wondering:
It's not that cumbersome as you said so :D
Well, the code does no error checking at all, which would be unacceptable in production code. You also need to deal with situations where writing fails in the middle of writing the record, you still must not corrupt the data.
ggenije wrote:
Sat May 30, 2020 5:53 pm
Additionally, when storing "records" one should consider whether to not make string fields fixed-width. Doing that allows quick seeking between the records without the need to parse all the records between.
Don't worry about that it's similar to your "class Person" example :D
That is exactly when it should be used. For example, if you have file storing a header and many records, knowing the record size allows you to seek quickly to a record (recordOffsetStart * recordSize), instead of parsing every single record. Additionally, when you change the record in the middle of the record file, what happens if you need to store a string that is longer then before?

That is how typed databases usually work, having fixed-size fields (blobs aside).

ggenije
Earned a small fee
Earned a small fee
Posts: 11
Joined: Thu Apr 16, 2020 3:57 pm

Re: Writing and Reading wxString from file.

Post by ggenije » Sat May 30, 2020 7:07 pm

I'll need those only to save me 5 mins of manually loading data into application everytime application is loaded.
It's not much deal it's data is corrupted, I am not creating database after all.

Post Reply