Localization with current display language Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

Btw: English (United States) is not properly used in older wxWidget Versions, but it is in wxWidgets 3++
Sry that I brought that up. Nothing is more unappealing than fiddling with long fixed issues ;-)

My fix in the still not updated apps:

Code: Select all

bool LanguageTranslation::IsEnglish()
{
	return (m_lang >= wxLANGUAGE_ENGLISH && m_lang <= wxLANGUAGE_ENGLISH_ZIMBABWE);
}

if (IsEnglish())
{
	m_lang = wxLANGUAGE_ENGLISH;
}
That way wxLANGUAGE_ENGLISH_US and possible other candidates use default wxLANGUAGE_ENGLISH which works fine.
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Wed Oct 27, 2021 8:35 am So you are saying that the language might be a combination of the display language and the region?
Yes, I think so. For example, there are major differences between Portuguese as spoken in Portugal vs Portuguese as spoken in Brazil. So, only the combination of language and region uniquely identifies the right UI language.
Natulux wrote: Wed Oct 27, 2021 8:35 am
utelle wrote: Tue Oct 26, 2021 11:35 am [...]Of which hard-coded German strings are you talking? [...]
The hardcoded german strings with the _() macro, those of my own application source code.
That's what I expected. :)
Natulux wrote: Wed Oct 27, 2021 8:35 am The macro is set, I can easily compare by switching to another english region like the UK, so I can rule out missing macros or translations in .mo file.
Could you please give an example or two for which strings you get unexpected translations (wrong language)?
Natulux wrote: Wed Oct 27, 2021 8:35 am
utelle wrote: Tue Oct 26, 2021 11:35 am and (c) that on adding the translation catalog to the current wxLocale instance the base language (used for the message ids) is set correctly - in your case German.
I have not set the base language for the message ids anywhere.
Well, IMHO you definitely should do that. The problem is that the base language for the wxWidgets library itself is English, while the base language for your own application is German. If you don't specify the base language of your catalog English is assumed. This could lead to wrong behaviour of the translation method.
Natulux wrote: Wed Oct 27, 2021 8:35 am

Code: Select all

m_locale.AddCatalog(sLanguagesFileName)
m_locale is my wxLocale member object. I have used the other constructor which only takes the language to translate to. Where is the benefit of giving the language of the id strings to the object? Except for those few cases it worked fine without and I dont even have a german translation file.
The second form of the AddCatalog method allows to specify the base language, that is, the language in which the strings to be translated are given:

Code: Select all

bool AddCatalog (const wxString &domain, wxLanguage msgIdLanguage)
If the base language is identical to the target language, the message catalog is not loaded at all. Instead the strings are used as they are in the application.

Please name just a couple of those exceptions. This may give a hint, what's going on in the translation method.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

utelle wrote: Wed Oct 27, 2021 9:22 am
Natulux wrote: Wed Oct 27, 2021 8:35 am So you are saying that the language might be a combination of the display language and the region?
Yes, I think so. For example, there are major differences between Portuguese as spoken in Portugal vs Portuguese as spoken in Brazil. So, only the combination of language and region uniquely identifies the right UI language.
That must be why the widgets library takes the language from the region. But that raises another issue:
My languages are mapped by wxWidgets automatically and I have not yet found a documentation to that. At the moment I just point the library at my "languages" directory in which it finds an "en" directory for the english language. But taking your example of portuguese, I should have the full canonial like "en_us", "en_uk" or (just guessing) "po_po" and "po_br" for portuguese.

Do you maybe know where to find a documentation how the mapping is done? A list of languages and their matching directory names for example?
utelle wrote: Wed Oct 27, 2021 9:22 am
Natulux wrote: Wed Oct 27, 2021 8:35 am The macro is set, I can easily compare by switching to another english region like the UK, so I can rule out missing macros or translations in .mo file.
Could you please give an example or two for which strings you get unexpected translations (wrong language)?
That was an issue caused by using an old wxWidgets (please see my second post just above yours). I apologize for bringing that up, there is no wrong mapping in the up to date version of wxWidgets.
There only are missing ones for German(Italy) and German(Belgium) and maybe others, but thats another issue.
utelle wrote: Wed Oct 27, 2021 9:22 am The second form of the AddCatalog method allows to specify the base language, that is, the language in which the strings to be translated are given:

Code: Select all

bool AddCatalog (const wxString &domain, wxLanguage msgIdLanguage)
If the base language is identical to the target language, the message catalog is not loaded at all. Instead the strings are used as they are in the application.

Please name just a couple of those exceptions. This may give a hint, what's going on in the translation method.
Easy enough, I'll do that. Thanks :-)
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Wed Oct 27, 2021 9:39 am
utelle wrote: Wed Oct 27, 2021 9:22 am Yes, I think so. For example, there are major differences between Portuguese as spoken in Portugal vs Portuguese as spoken in Brazil. So, only the combination of language and region uniquely identifies the right UI language.
That must be why the widgets library takes the language from the region.
I haven't studied the wxWidgets code in greater detail, but for an unknown combination of language and region English will be chosen.
Natulux wrote: Wed Oct 27, 2021 9:39 am But that raises another issue:
My languages are mapped by wxWidgets automatically and I have not yet found a documentation to that. At the moment I just point the library at my "languages" directory in which it finds an "en" directory for the english language.
That sounds ok. wxWidgets will first look for the combination of language and region (for example de_DE or en_US), but if such a subdirectory is not found it will try the subdirectories corresponding to the language alone (for example, de or en). If neither directory is found no message catalog will be loaded and the base language of the application will be used.
Natulux wrote: Wed Oct 27, 2021 9:39 am But taking your example of portuguese, I should have the full canonial like "en_us", "en_uk" or (just guessing) "po_po" and "po_br" for portuguese.
No, in most cases the differences of a language between different regions are relatively small and almost no one will provide different translations depending on the region (for example de_DE vs de_BE vs de_AT etc). So, for example for German the best approach is to place the translation file(s) in subdirectory de, so that the correct translation is found automatically for users from Germany, Austria, Switzerland and so on.

For Portuguese it is a different story. For Portuguese you would have usually 2 directories - pt_PT for Portugal and pt_BR for Brazil. (The language code po is unassigned.)
Natulux wrote: Wed Oct 27, 2021 9:39 am Do you maybe know where to find a documentation how the mapping is done? A list of languages and their matching directory names for example?
A list of languages as known to wxWidgets can be found in src/common/languageinfo.cpp in the source tree of wxWidgets The raw data for this list can be found in misc/languages/langtabl.txt. Documentation for translation support in wxWidgets can be found here.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

Thank you very much,
you have been very helpful :-)

Cheers
Natulux

EDIT:

And in the .cpp is the answer for German(Belgium) and German(Italy):

LNG(wxLANGUAGE_GERMAN_BELGIUM, "de_BE", 0 , 0 , wxLayout_LeftToRight, "German (Belgium)")
de_be language and sublanguage are set to 0 and de_it does not exist

EDIT2:

Which seems to be, because in "winnt.h" itself there is no mapping for these seemingly new sublanguages:
#define SUBLANG_GEORGIAN_GEORGIA 0x01 // Georgian (Georgia) 0x0437 ka-GE
#define SUBLANG_GERMAN 0x01 // German
#define SUBLANG_GERMAN_SWISS 0x02 // German (Swiss)
#define SUBLANG_GERMAN_AUSTRIAN 0x03 // German (Austrian)
#define SUBLANG_GERMAN_LUXEMBOURG 0x04 // German (Luxembourg)
#define SUBLANG_GERMAN_LIECHTENSTEIN 0x05 // German (Liechtenstein)
#define SUBLANG_GREEK_GREECE 0x01 // Greek (Greece)
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Wed Oct 27, 2021 11:26 am And in the .cpp is the answer for German(Belgium) and German(Italy):

LNG(wxLANGUAGE_GERMAN_BELGIUM, "de_BE", 0 , 0 , wxLayout_LeftToRight, "German (Belgium)")
de_be language and sublanguage are set to 0 and de_it does not exist
Indeed, this explains why you get "unknown language" in those cases.
Natulux wrote: Wed Oct 27, 2021 11:26 am Which seems to be, because in "winnt.h" itself there is no mapping for these seemingly new sublanguages:
#define SUBLANG_GEORGIAN_GEORGIA 0x01 // Georgian (Georgia) 0x0437 ka-GE
#define SUBLANG_GERMAN 0x01 // German
#define SUBLANG_GERMAN_SWISS 0x02 // German (Swiss)
#define SUBLANG_GERMAN_AUSTRIAN 0x03 // German (Austrian)
#define SUBLANG_GERMAN_LUXEMBOURG 0x04 // German (Luxembourg)
#define SUBLANG_GERMAN_LIECHTENSTEIN 0x05 // German (Liechtenstein)
#define SUBLANG_GREEK_GREECE 0x01 // Greek (Greece)
Microsoft deprecated the numeric language code identifiers (LCIDs). In the introduction of the [MS-LCID]-document that can be downloaded fromMicrosoft it reads:
This document provides an overview of language code identifiers (LCIDs), also known as culture identifiers, which are being deprecated, and the preferred alternate system of locale codes, which specify a set of locale identifiers that designate culture-specific information such as how text is sorted, how a date is formatted, and the display format for numbers and currency.
So, the method of wxWidgets for identifying a language setting on Windows systems should probably changed to use the new locale identifiers (like de-DE, de-BE etc). Most likely this is the reason why wxWidgets 3.1.6 will introduce the new class wxUILocale, even though the code to select a language from the list given in languageinfo.cpp has not yet been changed.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

utelle wrote: Wed Oct 27, 2021 12:27 pm Microsoft deprecated the numeric language code identifiers (LCIDs). In the introduction of the [MS-LCID]-document that can be downloaded fromMicrosoft it reads:
This document provides an overview of language code identifiers (LCIDs), also known as culture identifiers, which are being deprecated, and the preferred alternate system of locale codes, which specify a set of locale identifiers that designate culture-specific information such as how text is sorted, how a date is formatted, and the display format for numbers and currency.
So, the method of wxWidgets for identifying a language setting on Windows systems should probably changed to use the new locale identifiers (like de-DE, de-BE etc). Most likely this is the reason why wxWidgets 3.1.6 will introduce the new class wxUILocale, even though the code to select a language from the list given in languageinfo.cpp has not yet been changed.
Thats good to know. So updating to 3.1.6 when it is released will most likely solve these little issues I had.

Thanks again Ulrich
Natulux
Post Reply