wxlocale corrupts my mesh files?! Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Vexator
I live to help wx-kind
I live to help wx-kind
Posts: 187
Joined: Sun Jan 30, 2005 2:50 pm
Location: Heidelberg, Germany

wxlocale corrupts my mesh files?!

Post by Vexator » Tue May 09, 2006 7:58 pm

my project uses osg to draw into from wxglcanvas derived 3d viewports. it has all worked fine so far and i've now set up my app to use wxlocale as well. but whenever i try to load 3d meshes from ascii files (for example .obj or .osg), the files are either corrupted or not drawn at all. after some frustrating nights i realized that this seems to happen because of wxlocale - if i load a mesh before my wxlocale is initialized, it is drawn fine. but meshes loaded after the initialization get corrupted. if i comment out the localisation stuff, everything works fine, too. note that this only happens with ascii files - binary files like .lwo and .3ds still work fine.

so it seems that the i18n stuff is somehow corrupting the loaded model data..

any ideas?!

thanks in advance! :(
Windows 7 Pro
Visual Studio 2010
wxWidgets 2.9.3

utelle
Moderator
Moderator
Posts: 932
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: wxlocale corrupts my mesh files?!

Post by utelle » Tue May 09, 2006 8:34 pm

Vexator wrote: (...) so it seems that the i18n stuff is somehow corrupting the loaded model data..

any ideas?!
I assume the ascii data files contain floating point data and you are setting a german locale. If this assumption is true, the explanation is simple: after setting the locale the decimal separator is no longer a decimal point but a decimal comma, but your ascii files contain floating point numbers using a decimal point as separator. Thus the program is unable to interpret the input correctly.

Unfortunately wxWidgets has no methods to force the use of C locale while reading from a file. Switching the locale is usually no good solution, since the wxLocale is a global instance.

In my component wxPdfDocument I solved a similar problem using my own conversion methods for converting strings to doubles and vice versa.

Regards,

Ulrich

Vexator
I live to help wx-kind
I live to help wx-kind
Posts: 187
Joined: Sun Jan 30, 2005 2:50 pm
Location: Heidelberg, Germany

Post by Vexator » Tue May 09, 2006 8:45 pm

ok but WHY is the data localized at all?! shouldn't actually only strings enclosed in the _T() macro be localized? is there no way i can prevent wx from editing the mesh data? :(
Windows 7 Pro
Visual Studio 2010
wxWidgets 2.9.3

phlox81
wxWorld Domination!
wxWorld Domination!
Posts: 1387
Joined: Thu Aug 18, 2005 7:49 pm
Location: Germany
Contact:

Post by phlox81 » Tue May 09, 2006 8:49 pm

use STL classes / streams to load the data, that way wx can't corrupt it ;)

Vexator
I live to help wx-kind
I live to help wx-kind
Posts: 187
Joined: Sun Jan 30, 2005 2:50 pm
Location: Heidelberg, Germany

Post by Vexator » Tue May 09, 2006 8:53 pm

mh..

this is the routine to load meshes in osg:

Code: Select all

Node* osgDB::readNodeFile(const std::string& filename,const ReaderWriter::Options* options)
{
    ReaderWriter::ReadResult rr = Registry::instance()->readNode(filename,options);
    if (rr.validNode()) return rr.takeNode();
    if (rr.error()) notify(WARN) << rr.message() << std::endl;
    return NULL;
}
uses stl, doesn't it?
In my component wxPdfDocument I solved a similar problem using my own conversion methods for converting strings to doubles and vice versa.
where exactly can i find this? thx!
Windows 7 Pro
Visual Studio 2010
wxWidgets 2.9.3

utelle
Moderator
Moderator
Posts: 932
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Post by utelle » Tue May 09, 2006 10:04 pm

Vexator wrote:ok but WHY is the data localized at all?! shouldn't actually only strings enclosed in the _T() macro be localized? is there no way i can prevent wx from editing the mesh data? :(
No. As soon as you change the locale the C standard library functions like the scanf and printf family of functions (which are used by wxWidgets internally) use the decimal separator as set by the locale.

While the wxWidgets locale is a global instance, STL allows to specify different locales for different streams. If your program already uses the STL this is certainly a solution to the problem: specify the C locale for the stream which reads the mesh data.

Regards,

Ulrich

utelle
Moderator
Moderator
Posts: 932
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Post by utelle » Tue May 09, 2006 10:14 pm

Vexator wrote:this is the routine to load meshes in osg:

Code: Select all

Node* osgDB::readNodeFile(const std::string& filename,const ReaderWriter::Options* options)
{
    ReaderWriter::ReadResult rr = Registry::instance()->readNode(filename,options);
    if (rr.validNode()) return rr.takeNode();
    if (rr.error()) notify(WARN) << rr.message() << std::endl;
    return NULL;
}
uses stl, doesn't it?
Seems so. You would have to modify the readNode method (or the underlying code) to explicitly use the C locale for the input stream.
Vexator wrote:
In my component wxPdfDocument I solved a similar problem using my own conversion methods for converting strings to doubles and vice versa.
where exactly can i find this? thx!
wxPdfDocument is a wxCode component and can be downloaded from there. Look out for methods Double2String and String2Double in source file pdfdoc.cpp.

Regards,

Ulrich

Vexator
I live to help wx-kind
I live to help wx-kind
Posts: 187
Joined: Sun Jan 30, 2005 2:50 pm
Location: Heidelberg, Germany

Post by Vexator » Wed May 10, 2006 6:45 am

ok, danke :D
i'll search for some info on the c locale stuff and look at the osg code to see what i can do there. still i think that sth like wxLocale::Enable() and wxLocale::Disable() would be a good idea. this way i could simply disable the localisation, load my mesh data and re-enable it.
Windows 7 Pro
Visual Studio 2010
wxWidgets 2.9.3

utelle
Moderator
Moderator
Posts: 932
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Post by utelle » Wed May 10, 2006 7:22 am

Vexator wrote:still i think that sth like wxLocale::Enable() and wxLocale::Disable() would be a good idea. this way i could simply disable the localisation, load my mesh data and re-enable it.
This is not an easy task since wxWidgets uses the C style locale, that is, a single global locale instance which influences the program in its entirety. In a multithreading environment you will get inconsistent behaviour almost certainly if you temporarily change the locale.

Certainly it would be much better if wxWidgets would have a C++ style locale. I had a discussion on wx-dev about this issue last year, but I don't see this coming to wxWidgets soon. :-(

Regards,

Ulrich

phlox81
wxWorld Domination!
wxWorld Domination!
Posts: 1387
Joined: Thu Aug 18, 2005 7:49 pm
Location: Germany
Contact:

Post by phlox81 » Wed May 10, 2006 7:39 am

Vexator wrote:mh..
this is the routine to load meshes in osg:

Code: Select all

Node* osgDB::readNodeFile(const std::string& filename,const ReaderWriter::Options* options)
{
    ReaderWriter::ReadResult rr = Registry::instance()->readNode(filename,options);
    if (rr.validNode()) return rr.takeNode();
    if (rr.error()) notify(WARN) << rr.message() << std::endl;
    return NULL;
}
uses stl, doesn't it?
The only STL I see there is std::string, which imho is not the best part of it ;)
Don't know what kind of streams you use, and how you read/write doulbes there.

phlox

utelle
Moderator
Moderator
Posts: 932
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Post by utelle » Wed May 10, 2006 9:30 am

phlox81 wrote:The only STL I see there is std::string, which imho is not the best part of it ;)
Don't know what kind of streams you use, and how you read/write doulbes there.
Well, the code in question is not Vexator's own code but osg code. Although osg uses C++ streams to read and write files osg does not convert floating point numbers using streams. Instead atof is used for example in osgDB\Field.cpp to convert a char* array to float.

atof and printf are used throughout the osg code to convert floating point numbers to strings and vice versa. So, it won't be easy to make the osg code completely locale safe.

Regards,

Ulrich

Vexator
I live to help wx-kind
I live to help wx-kind
Posts: 187
Joined: Sun Jan 30, 2005 2:50 pm
Location: Heidelberg, Germany

Post by Vexator » Wed May 10, 2006 10:00 am

so if i understand you all correctly than i have two options..

a) i look into the osg code and replace all calls to atof and printf in the file loading routines with the stl alternatives (which i will have to do for every single file format and for every new release)

b) or i set my locale to english before i load files and then reset it to the priviously used locale (which is probably no good idea at all)

right?
thanks for your patience and help so far!
Windows 7 Pro
Visual Studio 2010
wxWidgets 2.9.3

utelle
Moderator
Moderator
Posts: 932
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Post by utelle » Wed May 10, 2006 10:21 am

Vexator wrote:so if i understand you all correctly than i have two options..

a) i look into the osg code and replace all calls to atof and printf in the file loading routines with the stl alternatives (which i will have to do for every single file format and for every new release)
Not very attractive, indeed. Probably the better approach would be to replace the atof and printf by your own versions ignoring the locale setting. If you present these modified versions of atof and printf to the linker before the std library this should work.
Vexator wrote:b) or i set my locale to english before i load files and then reset it to the priviously used locale (which is probably no good idea at all)
I wouldn't recommend to temporarily change the locale while loading files due to the possible unwanted side effects.

Regards,

Ulrich

phlox81
wxWorld Domination!
wxWorld Domination!
Posts: 1387
Joined: Thu Aug 18, 2005 7:49 pm
Location: Germany
Contact:

Post by phlox81 » Wed May 10, 2006 10:43 am

hm, I would use (string)streams for those things.

Well, I don't know OGL.

Vexator
I live to help wx-kind
I live to help wx-kind
Posts: 187
Joined: Sun Jan 30, 2005 2:50 pm
Location: Heidelberg, Germany

Post by Vexator » Wed May 10, 2006 12:15 pm

ok, first of all, i'll give the stl idea a try. osg's .obj importer uses sscanf to read in data. what would you suggest as an stl alternative now? (i have to admit that i barely ever used stl):

Code: Select all

sscanf(line+2,"%f %f %f %f", &x, &y, &z, &w);
ah btw:
I wouldn't recommend to temporarily change the locale while loading files due to the possible unwanted side effects.
.. for example?
Last edited by Vexator on Wed May 10, 2006 12:23 pm, edited 2 times in total.
Windows 7 Pro
Visual Studio 2010
wxWidgets 2.9.3

Post Reply