How can the best output and storage of large amounts of data Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
Andy
In need of some credit
In need of some credit
Posts: 8
Joined: Sun Dec 29, 2019 12:18 pm

How can the best output and storage of large amounts of data

Post by Andy » Sun Jun 28, 2020 3:16 am

My project creat a struct,As follows:
struct rawdatdata {
uint8_t record[24];
uint8_t data[469];
};
I try to save into txt file,then some strange symbols and blank appear in the file.Why????????
" TU4SECU4査?BD#[email protected]?0?"
Worse more,The data is too much to result to the txt file is stuck.
I use wxFile class and the function Write。

wxFile m_file("H:\\rdata.txt", wxFile::write_append);
m_file.Write(rawdatdata->data, sizeof(rawdatdata->data));

Any idea or suggestion? plz! Thank you :D

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 4232
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: How can the best output and storage of large amounts of data

Post by ONEEYEMAN » Sun Jun 28, 2020 5:14 am

Hi,
Couple of standard questions:
1. wx version?
2. OS version?
3. Toolkit used and version?

And more:
4. How do you populate the variable of that structure? Please post some code?
5. How do you check the content of the file?

Thank you.

PB
Part Of The Furniture
Part Of The Furniture
Posts: 2463
Joined: Sun Jan 03, 2010 5:45 pm

Re: How can the best output and storage of large amounts of data

Post by PB » Sun Jun 28, 2020 6:26 am

I would say: Do not write binary data to a text file which is for text strings. Use a binary file to store binary data. If you still want to store binary data as if they were strings, use wxBase64{De|En}code.

See here for an example of writing binary data using wxWidgets: viewtopic.php?f=1&t=47221#p199131

BTW, I would be vary of writing the entire structs because of padding, unless you explicitly tell the compile to pack the struct. I would also consider using wxFFile instead of wxFile, should be faster.

Nunki
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 218
Joined: Fri Sep 14, 2012 8:26 am
Location: Kontich, Belgium
Contact:

Re: How can the best output and storage of large amounts of data

Post by Nunki » Tue Jun 30, 2020 10:02 am

Hi Andy,

Working with data in your application, you should always ask yourself what am I going to do with the data besides writing it to a file. Are you going to read it, alter it, search in it. If so you should do the little overhead now to introduce storage into an SQL database. So that for future enhancements you may have to do less work than with the approach your choosing now. Besides using SQL in your application simplifies a lot. Especially problems like you are facing now. On top of that, SQL databases are tuned to give optimal performance. Allows you to even put the data on a separate machine (server).

Just a thought,
Nunki

Post Reply