Reading file created with wxDataOutputStream::WriteString Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Tue Mar 06, 2018 6:51 am

I have a zip file and one of its entries is created with wxDataOutputStream::WriteString.
I want to read this file with an application that is not linked to wxWidgets and thus I have to do this without the use of wxDataInputStream::ReadString. In the end I want the file content in an std::stringstream

The code for creating the file goes like

Code: Select all

...
wxDataOutputStream ofs(zip);
zip.PutNextEntry(zipEntryName);
ofs.WriteString(ss.str());
...
"zip" is a wxZipOutputStream and "ss" is a std::stringstream.
This works fine and i can unzip the file and the content is what I expect.

To read the file after unzipping i do something like

Code: Select all

...
wxFFileInputStream inStream(inFile);
wxDataInputStream  inDStream(inStream);
std::stringstream ss;
ss << inDStream.ReadString().ToStdString();
...
This works fine and i get a human readable string in the stringstream.

How would I do the last step without any wxWidgets related classes?
I have tried the following where fileName is the name of the unzipped file produced by WxDataOutputStream::WriteString

Code: Select all

std::stringstream ss;
std::ifstream c_file(fileName, std::ios_base::in, std::ios_base::binary);

//Read the first 32 bits that is the length of the string written by WxDataOutputStream::WriteString
uint32_t string_size{0};
c_file.read(reinterpret_cast<char *>(&string_size), sizeof(string_size));

//try to read the rest of the file. 
const std::string s(std::istreambuf_iterator<char>(c_file), {});

//s contains rubbish and is to short to have been filled with the entire file contents.
Best regards

/Klas
Last edited by klas.marcks on Tue Mar 06, 2018 11:56 am, edited 1 time in total.

User avatar
eranon
Can't get richer than this
Can't get richer than this
Posts: 867
Joined: Sun May 13, 2012 11:42 pm
Location: France
Contact:

Re: Reading file created with wxDataOutputStream::WriteString

Post by eranon » Tue Mar 06, 2018 10:01 am

Hello, To read a .zip file in C (so, no class at all) relying on zlib: https://github.com/madler/sunzip/blob/master/sunzip.c. And you should found something called minizip in the contrib subtree of the zlib distribution (http://www.zlib.net).
[Ind. dev. - wxWidgets 3.0/3.1 under "Win 7 64-bit, TDM64-GCC" + "OS X 10.9, LLVM Clang"]

klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Tue Mar 06, 2018 10:59 am

Hi and thanks for reply,

Maybe i was not clear enough. Unpacking the zipfile is not the problem. The problem is how to read the binary file produced by wxDataOutputStream::WriteString once the zip file is unpacked.

regards
Klas

coderrc
Earned some good credits
Earned some good credits
Posts: 141
Joined: Tue Nov 01, 2016 2:46 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by coderrc » Tue Mar 06, 2018 12:59 pm

std::ifstream file(filename, std::ifstream::in | std::ios::binary)
klas.marcks wrote:how to read the binary file once the zip file is unpacked.

klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Tue Mar 06, 2018 1:10 pm

Thanks for answering. I have that exact line in the last code snippet so that is not the problem. The problem appears when i try to write that data to a stringstream as i was hoping would be clear from the code and the comments. The question is about how to translate the binary output of wxDataOutputStream::writeString to a std::stringstream.

Sorry if I have been unclear.

Thanks
Klas

coderrc
Earned some good credits
Earned some good credits
Posts: 141
Joined: Tue Nov 01, 2016 2:46 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by coderrc » Tue Mar 06, 2018 1:28 pm

Sorry I wasnt reading fully, just skimming.
I guess what you want then would be something like

Code: Select all

std::stringstream ss;        
    { 
    std::ifstream file(filename, std::ifstream::in | std::ios::binary);
    ss << file.rdbuf();
    }
but i think your issue

Code: Select all

//try to read the rest of the file. 
const std::string s(std::istreambuf_iterator<char>(c_file), {});

//s contains rubbish and is to short to have been filled with the entire file contents.
is due to this

Code: Select all

ss << inDStream.ReadString().ToStdString();
that is to say that ofs.WriteString(ss.str()); is potentially writing unicode/std::wstring data so when you read the data back in to a std::string, its garbage that gets truncated by a random 0x00 in the std::wstring encoding.

so try a std::wstring or a std::wstringstream, and see if that works.

klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Wed Mar 07, 2018 8:34 am

Thanks, It appears that you are correct. Ofcourse this does not help me because i now have to deal with wstrings and this does not work for me. I will have to redesign and write the binary data as unsigned integers using wxDataOutputStream::Write32 and read them from the stream in 32 bit chunks.
Thanks again for help!

Klas

User avatar
doublemax
Moderator
Moderator
Posts: 15283
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Reading file created with wxDataOutputStream::WriteString

Post by doublemax » Wed Mar 07, 2018 9:20 am

At least on the wxWidgets side, you shouldn't use wxStrings at all for handling binary data.
Use the source, Luke!

klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Wed Mar 07, 2018 9:44 am

Just to clarify: The data I want to write is actually text-data. It seems however that to utilize the wxWidgets zipstreams I'm forced to use wxData...Streams::WriteString or wText...Streams::WriteString. Writing my strings to a binary file with these facilities will alwys give me the unicode "problem" right? So I try to slove this by writing my strings (available as std::strings) char by char as 32 bit integers via wxDataOutputStream::Write32. Im not entierly happy about this but see no other option. Would you know if there is a better alternative?

regards
Klas

User avatar
doublemax
Moderator
Moderator
Posts: 15283
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Reading file created with wxDataOutputStream::WriteString

Post by doublemax » Wed Mar 07, 2018 10:01 am

Ok, in that case using wxDataOutputStream::WriteString() is the best solution, as it uses UTF8 encoding by default. You could change that with wxDataOutputStream::SetConv() if you wanted.
Use the source, Luke!

klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Wed Mar 07, 2018 10:08 am

In that case: what "Conv" should I use to be able to read the file into a normal stringstream (via a binary ifstream ) outside of wxWidgets? I did not fully understand the documentation.
Many thanks for feedback!

Klas

User avatar
doublemax
Moderator
Moderator
Posts: 15283
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Reading file created with wxDataOutputStream::WriteString

Post by doublemax » Wed Mar 07, 2018 10:15 am

You can first read the data with ifstream, but then you must decode the UTF8 encoding somehow. If you can't use wxWidgets methods on the reading side, you'll need some other class/library to do that.

If you can't / don't want to use another external library, i found some code for UTF8 de-/encoding here:
https://stackoverflow.com/questions/148 ... 766#148766
Use the source, Luke!

klas.marcks
Knows some wx things
Knows some wx things
Posts: 42
Joined: Thu Mar 13, 2014 9:49 pm

Re: Reading file created with wxDataOutputStream::WriteString

Post by klas.marcks » Wed Mar 07, 2018 5:47 pm

Thanks! A first test suggests that the code you linked to, does what it claims!

Post Reply