Localization with current display language Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Localization with current display language

Post by Natulux »

Hey,

I tried to use localization as provided by wxWidgets. I use the code from the "internat" sample and it works for me (except that I tried to use the .mo file from the ressource which didnt work for me - but thats another issue).

If I set the wxLanguage variable manually eg to wxLANGUAGE_ENGLISH it works.
But I cant seem to be able to read the system language.

I installed the windows language package "english-US", activated it and also set the Current system locale to "English (United States)" in the "Administrative language settings". After that I restarted the system. (see windowsSettings.png)

I took every function I found for language identification and got them out (see languages.png).
The only function that was able to detect the current display language was the C++ native function "GetSystemDefaultLocale".

Any hints how I can accomplish this with wxWidgets functions? Do I need to change the language for the user somewhere for wxWidgets (and c++ functions respectively) to notice?

Thanks
Natu

EDIT:
If I use the following command in an elevated powershell to set the default language for the user explicitly, wxWidgets does recognize the language. (But that even uninstalls my other languages and I cant expect other users to have used it):

Code: Select all

Set-WinUserLanguageList en-US -Force
So the real question is: How to get the system locale instead of the user locale with wxWIdgets?
Attachments
languages.png
languages.png (8.22 KiB) Viewed 4703 times
windowsSettings.png
windowsSettings.png (32.17 KiB) Viewed 4703 times
User avatar
doublemax@work
Super wx Problem Solver
Super wx Problem Solver
Posts: 470
Joined: Wed Jul 29, 2020 6:06 pm
Location: NRW, Germany

Re: Localization with current display language

Post by doublemax@work »

I think it's safe to assume that the localization works correctly on "real" English systems, it's just your hybrid setting that causes the problem.

wxLocale::GetSystemLanguage() uses ::GetUserDefaultUILanguage() internally under Windows.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

Hey Max,

I guess you are right. With further tests I found the setting that is needed for widgets to get the wanted language:
Region must be "English" and voila: User language is recognized as english.

I only wished I could read the system default language as well, because it is unintuitive for the user that his changing of the display language doesn't affect the app, while setting the region does.
There could be a german speaker in the US, having the region set to US and the language to german -> App would be in english still.

My last desperate attempt failed miserably though:

Code: Select all

	wchar_t buffer[1024];
	GetSystemDefaultLocaleName(buffer, _countof(buffer));
	wxString sLangCannonial(buffer);
	const wxLanguageInfo *li = wxLocale::FindLanguageInfo(sLangCannonial);
	if (li != NULL) // <------ IT IS ALWAY NULL
	{
		int iLang = li->Language;

		create((wxLanguage)iLang, sLanguagesFileName);
	}
	else
	{
		create((wxLanguage)wxLocale::GetSystemLanguage(), sLanguagesFileName);
	}
Attachments
regional2.png
regional2.png (9.68 KiB) Viewed 4683 times
regional.png
regional.png (11.1 KiB) Viewed 4683 times
User avatar
doublemax
Moderator
Moderator
Posts: 19103
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Localization with current display language

Post by doublemax »

You can give the user an option to select any language he wants. I think most applications do that. My Windows is German, but i still prefer an English GUI in most cases, because the German translation is often horrible.
Use the source, Luke!
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Mon Oct 18, 2021 3:04 pm With further tests I found the setting that is needed for widgets to get the wanted language:
Region must be "English" and voila: User language is recognized as english.

I only wished I could read the system default language as well, because it is unintuitive for the user that his changing of the display language doesn't affect the app, while setting the region does.
In the Windows settings dialog under the heading "Language" there should be a combo box for selecting the "Windows Display Language". Changing the language requires the user to logout and login again. Changing the language will usually change the region as well. If the combo box list does not contain the language the user wants for display, he/she has to add the requested language to the list of preferred languages, first.
Natulux wrote: Mon Oct 18, 2021 3:04 pm There could be a german speaker in the US, having the region set to US and the language to german -> App would be in english still.
Well, IMHO this is a very unlikely scenario. A German travelling in the US with his own computer will most likely have his computer set up for a German environment anyway, and so your application would use the correct language, if asking for the system language on starting up. If the German uses a computer which is not his own and which is set up with English as the display language, it is very unlikely that he will tamper with the system language settings to change just the display language of your application, but your application should offer an option to select a language which differs from the system language. If a German actually lives in the US, he will most likely prefer English as the display language anyway, so again selecting the default system language is a good choice for your application.

In my own I18N aware applications on initializing the application for the very first time the following code is executed:

Code: Select all

  wxString defaultLanguage;
  const wxLanguageInfo* langInfo;
  int lang = wxLocale::GetSystemLanguage();
  if (lang != wxLANGUAGE_UNKNOWN)
  {
    langInfo = wxLocale::GetLanguageInfo(lang);
    defaultLanguage = langInfo->CanonicalName;
  }
  else
  {
    defaultLanguage = wxT("en_GB");
  }
Of course, your application will have to check thereafter whether the system default language is supported by the application. If not, select the default language of your application - probably English. Additionally, the application should offer the user the option to select a different language from the list of languages actually supported by your application. The application should then remember this selection and use the selected language as the display language when it is started the next time.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

Hey
doublemax wrote: Mon Oct 18, 2021 5:44 pm You can give the user an option to select any language he wants. I think most applications do that. My Windows is German, but i still prefer an English GUI in most cases, because the German translation is often horrible.
Yes, that is of course an option which I most likely will add additionally, but I'd like my app to be initialized as good as possible and the system language is the way to go I guess.
utelle wrote: Mon Oct 18, 2021 9:25 pm In my own I18N aware applications on initializing the application for the very first time the following code is executed:

Code: Select all

  wxString defaultLanguage;
  const wxLanguageInfo* langInfo;
  int lang = wxLocale::GetSystemLanguage();
  if (lang != wxLANGUAGE_UNKNOWN)
  {
    langInfo = wxLocale::GetLanguageInfo(lang);
    defaultLanguage = langInfo->CanonicalName;
  }
  else
  {
    defaultLanguage = wxT("en_GB");
  }
Of course, your application will have to check thereafter whether the system default language is supported by the application. If not, select the default language of your application - probably English. Additionally, the application should offer the user the option to select a different language from the list of languages actually supported by your application. The application should then remember this selection and use the selected language as the display language when it is started the next time.
Hey Ulrich, thanks for the code.
In my example code above I too was able to get the cannonial of the system language.
Do you happen to know how I can use the cannonial ("en_GB") to initialize the wxWidgets component? (from cannonial to wxLANGUAGE enum)

All the best
Natulux
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Wed Oct 20, 2021 9:42 am
utelle wrote: Mon Oct 18, 2021 9:25 pm In my own I18N aware applications on initializing the application for the very first time the following code is executed:

Code: Select all

  wxString defaultLanguage;
  const wxLanguageInfo* langInfo;
  int lang = wxLocale::GetSystemLanguage();
  if (lang != wxLANGUAGE_UNKNOWN)
  {
    langInfo = wxLocale::GetLanguageInfo(lang);
    defaultLanguage = langInfo->CanonicalName;
  }
  else
  {
    defaultLanguage = wxT("en_GB");
  }
Of course, your application will have to check thereafter whether the system default language is supported by the application. If not, select the default language of your application - probably English. Additionally, the application should offer the user the option to select a different language from the list of languages actually supported by your application. The application should then remember this selection and use the selected language as the display language when it is started the next time.
Hey Ulrich, thanks for the code.
In my example code above I too was able to get the cannonial of the system language.
Do you happen to know how I can use the cannonial ("en_GB") to initialize the wxWidgets component? (from cannonial to wxLANGUAGE enum)
Sure, the below code snippet shows you the steps I usually take in my own I18N aware applications:

Code: Select all

  // Language identifier in canonical form (for example "en_GB" or "de_DE")
  // Default language as determined in previous code snippet
  wxString defaultLanguage = ...;
  wxString language;

  // Determine language selected by the user
  // NOTE: ReadConfigParameter is NOT a wxWidgets function
  // You may use wxRegConfig or wxFileConfig to actually save and restore user settings
  // If the language was not set during a previous run of the application, the default language will be returned
  ReadConfigParameter("/Settings/Language", language, defaultLanguage);

  // Determine numeric language identifier
  const wxLanguageInfo* langInfo = wxLocale::FindLanguageInfo(language);
  int lang = (langInfo != NULL) ? langInfo->Language : wxLANGUAGE_ENGLISH;

  // ms_locale is a static global locale instance of type wxLocale
  // If a wxLocale was already set previously, delete the previous locale
  wxDELETE(ms_locale);
  // Allocate new wxLocale instance
  ms_locale = new wxLocale();

  // Set path where translation files can be found
  // (not necessary, if default locations are used for the application)
  // Path to the directory where the language translation files (*.po resp *.mo) reside
  wxString localePath = ...;
  ms_locale->AddCatalogLookupPathPrefix(localePath);

  // Initialize the locale (loads the wxWidgets default translation file "wxstd.mo" among other things)
  ms_locale->Init(lang);

  // Add additional translation file(s)
  // especially that for the translatable strings of your application
  // Calls to AddCatalog may be repeated for additional separate translation files
  ms_locale->AddCatalog("your-app-name");
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

Hey Ulrich,

I tried to use a snippet from your code but for me it does not seem to work. Maybe its my system (Windows10x64, wxWIdgets 3.1.1)

My code

Code: Select all

wxString sSystemLocaleName;
wchar_t buffer[1024];
GetSystemDefaultLocaleName(buffer, _countof(buffer));
sSystemLocaleName << buffer;
wxMessageBox(sSystemLocaleName);
const wxLanguageInfo* langInfo = wxLocale::FindLanguageInfo(sSystemLocaleName);
int nLang = 0;
if (langInfo != NULL)
{
	wxMessageBox("System Language");
	nLang = langInfo->Language;
}
else
{
	wxMessageBox("User Language");
	nLang = wxLocale::GetSystemLanguage();
}
I for example get "en_US" and "User Language" (because langInfo is NULL).

Cheers
Natulux
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Wed Oct 20, 2021 2:49 pm I tried to use a snippet from your code but for me it does not seem to work. Maybe its my system (Windows10x64, wxWIdgets 3.1.1)
Well, you mentioned that you modified the language and region settings of your system. That could possibly cause a classification as "user-defined". Try to return to the default settings.
Natulux wrote: Wed Oct 20, 2021 2:49 pm My code

Code: Select all

wxString sSystemLocaleName;
wchar_t buffer[1024];
GetSystemDefaultLocaleName(buffer, _countof(buffer));
sSystemLocaleName << buffer;
wxMessageBox(sSystemLocaleName);
const wxLanguageInfo* langInfo = wxLocale::FindLanguageInfo(sSystemLocaleName);
int nLang = 0;
if (langInfo != NULL)
{
	wxMessageBox("System Language");
	nLang = langInfo->Language;
}
else
{
	wxMessageBox("User Language");
	nLang = wxLocale::GetSystemLanguage();
}
I for example get "en_US" and "User Language" (because langInfo is NULL).
Hm, the function GetSystemDefaultLocaleName belongs to the Windows system API. The problem is that this function returns the locale string in a form that is not understood by method wxLocale::FindLanguageInfo. For example, for a German system the Windows function would/should return de-DE, but wxLocale::FindLanguageInfo expects the form de_DE - see the difference: hyphen vs underscore. That's why you get langInfo == NULL. Why don't you use directly wxLocale::GetSystemLanguage?

If the operating system is set up in a standard way, tly wxLocale::GetSystemLanguage will deliver the default UI language. If the operating system is set up in a non-standard way, it will be difficult, if not impossible, to detect the default UI language, and the best your application can do, is to use English on first start, and let the user select a different language afterwards, if he/she prefers.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

I lost my first answer, so this naturally is a little bit shorter ;-)
utelle wrote: Wed Oct 20, 2021 3:41 pm Hm, the function GetSystemDefaultLocaleName belongs to the Windows system API. The problem is that this function returns the locale string in a form that is not understood by method wxLocale::FindLanguageInfo. For example, for a German system the Windows function would/should return de-DE, but wxLocale::FindLanguageInfo expects the form de_DE - see the difference: hyphen vs underscore. That's why you get langInfo == NULL. Why don't you use directly wxLocale::GetSystemLanguage?
Well that answers it: I just have to use a replace, thank you! :-)
I'd rather use GetSystemDefaultLocaleName instead of wxLocale::GetSystemLanguage, because it seems to return the "display language" instead of the "regional format" language.
utelle wrote: Wed Oct 20, 2021 3:41 pm If the operating system is set up in a standard way, tly wxLocale::GetSystemLanguage will deliver the default UI language. If the operating system is set up in a non-standard way, it will be difficult, if not impossible, to detect the default UI language, and the best your application can do, is to use English on first start, and let the user select a different language afterwards, if he/she prefers.
Same argument as above but of course you are right: There should and eventually will be a dialog to choose a language. Its all about a good first guess.

But there are two issues who make me reconsider using the wxWidgets language classes at all:

- if the "Regional Format" is set to "English (United States)" I get the right language enum (wxLANGUAGE_ENGLISH_US), but in some apps I still get the hardcoded german strings, as if the language file couldn't be found
- if the "Regional Format" is set to "German(Belgium)" or "German(Italy)" I get wxLANGUAGE_UNKNOWN which results in my default fallback: English

All the best
Natulux
PB
Part Of The Furniture
Part Of The Furniture
Posts: 4182
Joined: Sun Jan 03, 2010 5:45 pm

Re: Localization with current display language

Post by PB »

Out of curiosity, if you are using wxWidgets master (probably not?): What does

Code: Select all

wxUILocale::GetCurrent().GetName();
return?
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

PB wrote: Mon Oct 25, 2021 7:38 pm Out of curiosity, if you are using wxWidgets master (probably not?): What does

Code: Select all

wxUILocale::GetCurrent().GetName();
return?
Hey PB,

you are right, I dont have the master. I am working mostly with wx3.1.1, because I had trouble updating to 3.1.3 and dropped the issue back then.
I dont have a "uilocale.h" anywhere on my entire system, as far as Everything.exe can tell.

Cheers
Natulux
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Tue Oct 26, 2021 8:50 am
PB wrote: Mon Oct 25, 2021 7:38 pm Out of curiosity, if you are using wxWidgets master (probably not?): What does

Code: Select all

wxUILocale::GetCurrent().GetName();
return?
you are right, I dont have the master. I am working mostly with wx3.1.1, because I had trouble updating to 3.1.3 and dropped the issue back then.
I dont have a "uilocale.h" anywhere on my entire system, as far as Everything.exe can tell.
wxUILocale was/is introduced for the - officially still unreleased - wxWidgets version 3.1.6. This new locale class determines the UI language a bit differently than wxLocale, namely looking at the string representation of the system locale (i.e. "de-DE"), while wxLocale compares numeric language identifiers. The latter is mot likely the cause that you are receiving "unknown language".
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: Localization with current display language

Post by utelle »

Natulux wrote: Mon Oct 25, 2021 3:33 pm I'd rather use GetSystemDefaultLocaleName instead of wxLocale::GetSystemLanguage, because it seems to return the "display language" instead of the "regional format" language.
As I already explained in a prior post the behaviour you experience is most likely caused by the way you manipulate the language settings of your system. In a properly set-up Windows system you will rarely get "unknown language" as the result of asking for the system default language.

IMHO you should prefer wxWidgets methods instead of Windows system methods - at least, if you have in mind to port your application to other platforms some day in the future.
Natulux wrote: Mon Oct 25, 2021 3:33 pm But there are two issues who make me reconsider using the wxWidgets language classes at all:

- if the "Regional Format" is set to "English (United States)" I get the right language enum (wxLANGUAGE_ENGLISH_US), but in some apps I still get the hardcoded german strings, as if the language file couldn't be found
I'm not sure whether I fully understand the problem. Of which hard-coded German strings are you talking? Those used in standard dialogs like "Ok", "Cancel", "Close" etc? Or those in your own application source code? In the latter case, you have to verify (a) that they are correctly flagged for translation (macro _("")), (b) that they really have valid translations in the loaded .po/.mo translation catalog, and (c) that on adding the translation catalog to the current wxLocale instance the base language (used for the message ids) is set correctly - in your case German.
Natulux wrote: Mon Oct 25, 2021 3:33 pm - if the "Regional Format" is set to "German(Belgium)" or "German(Italy)" I get wxLANGUAGE_UNKNOWN which results in my default fallback: English
Without seeing the exact settings in your Windows system it is difficult to tell why you get "unknown language". However, at least the language entry "de_IT" (German(Italy)) does not exist in the list of known languages in wxWidgets (languageinfo.cpp).

If the UI language in Windows is modified via the dialog "Settings/Language - Windows Display Language", then typically the region settings are adjusted accordingly. If you tweak the region settings independently, you easily end up with an unknown combination.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Localization with current display language

Post by Natulux »

utelle wrote: Tue Oct 26, 2021 11:35 am
Natulux wrote: Mon Oct 25, 2021 3:33 pm I'd rather use GetSystemDefaultLocaleName instead of wxLocale::GetSystemLanguage, because it seems to return the "display language" instead of the "regional format" language.
As I already explained in a prior post the behaviour you experience is most likely caused by the way you manipulate the language settings of your system. In a properly set-up Windows system you will rarely get "unknown language" as the result of asking for the system default language.

If the UI language in Windows is modified via the dialog "Settings/Language - Windows Display Language", then typically the region settings are adjusted accordingly. If you tweak the region settings independently, you easily end up with an unknown combination.
So you are saying that the language might be a combination of the display language and the region? I haven't thought about that.
But I have proven myself wrong anyways: Both functions are not able to read the display language properly in my last tests and I cant remember where I though I had seen a difference.
utelle wrote: Tue Oct 26, 2021 11:35 am IMHO you should prefer wxWidgets methods instead of Windows system methods - at least, if you have in mind to port your application to other platforms some day in the future.
Good point.
utelle wrote: Tue Oct 26, 2021 11:35 am
Natulux wrote: Mon Oct 25, 2021 3:33 pm But there are two issues who make me reconsider using the wxWidgets language classes at all:

- if the "Regional Format" is set to "English (United States)" I get the right language enum (wxLANGUAGE_ENGLISH_US), but in some apps I still get the hardcoded german strings, as if the language file couldn't be found
I'm not sure whether I fully understand the problem. Of which hard-coded German strings are you talking? Those used in standard dialogs like "Ok", "Cancel", "Close" etc? Or those in your own application source code? In the latter case, you have to verify (a) that they are correctly flagged for translation (macro _("")), (b) that they really have valid translations in the loaded .po/.mo translation catalog
The hardcoded german strings with the _() macro, those of my own application source code. The macro is set, I can easily compare by switching to another english region like the UK, so I can rule out missing macros or translations in .mo file.
utelle wrote: Tue Oct 26, 2021 11:35 am and (c) that on adding the translation catalog to the current wxLocale instance the base language (used for the message ids) is set correctly - in your case German.
I have not set the base language for the message ids anywhere.

Code: Select all

m_locale.AddCatalog(sLanguagesFileName)
m_locale is my wxLocale member object. I have used the other constructor which only takes the language to translate to. Where is the benefit of giving the language of the id strings to the object? Except for those few cases it worked fine without and I dont even have a german translation file.

Thanks
Natulux
Post Reply