wxWidgets 3.1.1 and german letters Topic is solved

[PB]

There got to be some changes compared to widgets prior to 3.1.1 - because all of it predecessors work for me without any changes or macros.

As I wrote above, it works for me with the (almost) current version from the GIT master.

[Natulux]

Then there is something in connection with the operating system and/or the local language settings maybe.

On my german windows 10x64 working machine:
I again checked out the current master from today, recompiled the widgets with standard settings with visual studio 2017 essentials in Release mode, recompiled the minimal sample in Release mode and added a wxMessageBox to the frame class constructor. Same error displaying german Umlaut in a standard wxString.

[doublemax]

Can you use a debugger and confirm that the string data from which the string is constructed, is actually UTF-8 encoded?

Can you try the u"" prefix for the string to force UTF-16 encoding?
Taken from https://msdn.microsoft.com/en-us/library/69ze775t.aspx

[Natulux]

Can you use a debugger and confirm that the string data from which the string is constructed, is actually UTF-8 encoded?

Can you try the u"" prefix for the string to force UTF-16 encoding?
Taken from https://msdn.microsoft.com/en-us/library/69ze775t.aspx

Good suggestion, I guess.
I compiled the Debug Widgets and had a look at it.

Code: Select all

const char16_t c[7] = u"aaahö";
auto a = c;
wxString sStr = "AaaaÄ";

You were all right, it's not wxWidgets, the error is prior to that.
My utf16 const char with the prefix u"" is allready false, inclined to use utf16 but alters the Umlaut.

Looking into the wxString lets me see, that the value is L"" stored (and therefore not utf8?). Either way, it's equally translated into rubbish.

So, if it is the studio itself, any ideas where to tackle this?

Greets
Natu

[doublemax]

Can you show a memory view of the string literal when you use the wxT() macro?

It should look like this:

vs_string_literal.png (24.19 KiB) Viewed 3854 times

[Natulux]

This kind of debugging is a little bit over my head
Good to have you to guide me here!

It actually differs:

[doublemax]

Ok, that means that the string data already lies UTF8 encoded in memory. This is the actual underlying cause of your problem.

But i still don't know how this can happen. If should only happen if the source file is encoded in UTF8, but the compiler ignores this and interprets it as ISO8859-1 (or any other 8bit encoding).

Try this:
In Visual studio with the source file open and active, select "Save <filename> as...". Next to the save button is a down arrow where you can select "Save with Encoding". Select "Unicode (UTF-8 with signature) - Codepage 65001" (Near the top).

Rebuild and pray

[Natulux]

I actually had to double check, where I could save with encryption. I never noticed that little arrow next to the save

So, the "standard" format used by my visual studio 2017 was "UTF-8 without signature".
I prayed really hard ( ) and changed it to "UTF-8 with signature", but it didn't change anything, except that I was unable to evaluate the memory - I had to use the title of my frame and a wxMessageBox to check.

Then I changed the encoding to "Western European (Windows) - Codepage 1252" because that is the setting used by my old Visual Studio and projects and - lo and behold - it works!

Thanks for your hints Max, I would have never done that without you.
Im just wondering if I have to change the encoding of each file manually, or it there is a switch for the whole project (there should be!).

I fear, VS2017 really messed things up with its file encoding.

All the best
Natu

[doublemax]

Then I changed the encoding to "Western European (Windows) - Codepage 1252" because that is the setting used by my old Visual Studio and projects and - lo and behold - it works!

Hmm, i'm not really sure this is the perfect solution. It's possible that this only works on a German system, but may not work when running the exe on another locale.

[Natulux]

Hmm, i'm not really sure this is the perfect solution. It's possible that this only works on a German system, but may not work when running the exe on another locale.

I dare say that it should work on german and english systems, since english letters are congruently included in the german encoding.
Anything else is bound to cause trouble, but that would mean I would have trouble with all my projects in foreign countries, because they are equally encoded as western european.

But then again, if you say that your files are encoded as UTF-8 with sig. I wonder why that wouldn't work for me also?

[doublemax]

But then again, if you say that your files are encoded as UTF-8 with sig. I wonder why that wouldn't work for me also?

It's a total mystery to me.

[Natulux]

If I find an answer to that, I'll report it here.
Up to now, it works and you have my thanks for that.

Have a nice day
Natu

[Manolo]

The way GNU g++ works is that it expects the source file to be encoced in either UTF-8 or local (e.g. ISO-8859-1). And this encoding is configurable.
VS used to expect the "local" encoding. It seems that now Microsoft has selected the GNU way, "standard UTF-8" (perhaps because it's the default for Internet) and all your old files seem invalid.

So if your files were ISO-8859-1 and now you want the default UTF-8 then you must re-encode them.
This can be easy. Select in VS the current files encoding, load the files, change the encoding, save all files.

Some years ago, when this option was not that easy I made my own small converter. It works file by file (I didn't add the feature of all files in a shot), selecting input encoding, output encoding and output end-of-line mode. I can post it here if any body feels better with it than with IDE ablitites.

[Natulux]

Hey Manolo,

So if your files were ISO-8859-1 and now you want the default UTF-8 then you must re-encode them.
This can be easy. Select in VS the current files encoding, load the files, change the encoding, save all files.

Im not sure what exactly you speak of here. Is the "current files encoding" an option in the project settings? Can you describe where to find it? I haven't found that myself. I only found
"Configuration Properties > Project Defaults > Character Set > Use Unicode Character Set" but we had sorted that out before, it wasn't helpful.
And of cause the file encoding of each file via "Save As" described by DoubleMax before.

Some years ago, when this option was not that easy I made my own small converter. It works file by file (I didn't add the feature of all files in a shot), selecting input encoding, output encoding and output end-of-line mode. I can post it here if any body feels better with it than with IDE ablitites.

If I can use my IDE I am more than happy to do that but thanks for the offer.

I actually would like my files to be UTF-8 encoded. But in this article we found out, that I have problems displaying german letters with a file encoded in "UTF-8 without signature". Mind you, I used wxWidgets Minimal sample, which has only the one file "minimal.cpp" and it was encoded UTF8 on default by VS2017 and when looking into the memory in Debug, my wxString was UTF8 encoded - still I had issues displaying german letters.
Any thoughts on that?

[Natulux]

Update:
I recently tried the newest wxWidgets commit of v3.1.1 and were relieved to see, that my character encoding problem has been solved.
I can't say way though, but for everyone who has the same problem:

I used vs2015 Express to compile wxWidgets3.1.1 latest commit (11/2018) with standard settings (/MD) and altered the minimal sample to show a wxMessageBox with foreign characters (and this time it worked).

Best
Natu