Unicode not working in wxWidgets GUI? Topic is solved

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
AnotherCoder
Earned a small fee
Earned a small fee
Posts: 14
Joined: Sun Aug 23, 2009 2:34 am
Location: USA

Unicode not working in wxWidgets GUI?

Post by AnotherCoder » Sun Aug 23, 2009 9:08 pm

Hi, I am new to wxWidgets and am using it in Microsoft Visual C++ 2008. I can get this program from the Hello World Tutorial running just fine.
I already built wxWidgets libraries for Debug, Release, and Unicode - Release.

I modified the last function of the Hello World program to:

Code: Select all

void MyFrame::OnAbout(wxCommandEvent& WXUNUSED(event))
{
	wchar_t s[] = L"言語";
    int bufferSize = WideCharToMultiByte(CP_UTF8, 0, s, -1, NULL, 0, NULL, NULL);
    char* m = new char[bufferSize];
    WideCharToMultiByte(CP_UTF8, 0, s, -1, m, bufferSize, NULL, NULL);
    wxMessageBox(_T(m),
        _T("About Hello World"), wxOK | wxICON_INFORMATION, this);
}
I attached my file, hworld.cpp

It compiles fine, but the text comes out garbled when I click File->About
Why?
Attachments
hworld.cpp
(3.43 KiB) Downloaded 92 times

samsam598
Super wx Problem Solver
Super wx Problem Solver
Posts: 324
Joined: Mon Oct 06, 2008 12:55 pm

Re: Unicode not working in wxWidgets GUI?

Post by samsam598 » Mon Aug 24, 2009 12:59 am

Code: Select all

	wchar_t s[] = L"言語";
    int bufferSize = WideCharToMultiByte(CP_UTF8, 0, s, -1, NULL, 0, NULL, NULL);
    char* m = new char[bufferSize];
    WideCharToMultiByte(CP_UTF8, 0, s, -1, m, bufferSize, NULL, NULL);
    wxMessageBox(_T(m),
        _T("About Hello World"), wxOK | wxICON_INFORMATION, this);
}

Why bother?Below code does not meet what you want?

Code: Select all

wxString s=_T("言語");
wxMessageBox(s);
or just:

Code: Select all

wxMessageBox(_T("言語");
If you want to dig into the mechanism of unicode support in wx,please go to doc->Topic overviews->Unicode support in wxWidgets.
Regards,
Sam
-------------------------------------------------------------------
Windows xp
VS.Net 2003/MinGW 3.4.5 C::B character set: UTF-8
wxWidgets github 3.0 RC1 Unicode Static build,Unicode Shared build.

User avatar
doublemax
Moderator
Moderator
Posts: 15263
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Post by doublemax » Mon Aug 24, 2009 1:07 am

There are several aspects involved:
- in which encoding does the editor save the source file?
- what encoding does the compiler expect/accept?
- in which encoding does wxWidgets expect string literals?
- does the output device support the character set?

For a start, try this code:

Code: Select all

unsigned char text[]="\xe8\xa8\x80\xe8\xaa\x9e";
wxString s((const char *)text, wxConvUTF8);
wxMessageBox(s);
The UTF-8 encoding makes sure that wxWidgets definitely puts the correct characters into the string. If this works, you know that the problem lies somewhere on editor/compiler level.
Use the source, Luke!

AnotherCoder
Earned a small fee
Earned a small fee
Posts: 14
Joined: Sun Aug 23, 2009 2:34 am
Location: USA

Post by AnotherCoder » Mon Aug 24, 2009 2:19 am

Thanks. Here are the answers:
- UTF-8
- Unicode (I guess cause it is Visual C++ 2008)
- ?
- wxWidgets should accept Unicode right?

I tried running your code. It compiled, but I got either question marks or random ASCII characters.

One thing might be that I only built wxWidgets libraries for Debug, Release, and Unicode - Release. I did not built DLL libraries or Unicode - Debug (I ran the program in Release).

AnotherCoder
Earned a small fee
Earned a small fee
Posts: 14
Joined: Sun Aug 23, 2009 2:34 am
Location: USA

Re: Unicode not working in wxWidgets GUI?

Post by AnotherCoder » Mon Aug 24, 2009 5:10 am

samsam598 wrote: Why bother?Below code does not meet what you want?

Code: Select all

wxString s=_T("言語");
wxMessageBox(s);
No, I tried that way, but I got 2 warnings and the program still did not work:

Code: Select all

warning C4566: character represented by universal-character-name '\u8A00' cannot be represented in the current code page (1252)

Code: Select all

warning C4566: character represented by universal-character-name '\u8A9E' cannot be represented in the current code page (1252)
I read through Doc->Topic overviews->Unicode and I am more confused now...
Even this program gives me a weird error:

Code: Select all

#define wxUSE_UNICODE 1

#include <wx/wx.h>

int main()
{
    wxChar ch = wxT('*');
    wxString s = wxT("一二三四五六七八九十");
    int len = s.Len();
}

Code: Select all

rror LNK2001: unresolved external symbol "protected: void __thiscall wxStringBase::InitWith(wchar_t const *,unsigned int,unsigned int)" ([email protected]@@[email protected])
1>D:\lib\cpp\wxWidgets\wx_scrap1\Release\wx_scrap1.exe : fatal error LNK1120: 1 unresolved externals
I am confused :shock:
I am building in Release mode in Visual C++ (there is only an option for Debug or Release in my IDE). See IDE.png
Attachments
IDE.png
IDE.png (2.78 KiB) Viewed 3482 times

chenbin.sh
Experienced Solver
Experienced Solver
Posts: 64
Joined: Fri Apr 17, 2009 7:15 am
Location: Sydney, Australia
Contact:

Post by chenbin.sh » Mon Aug 24, 2009 5:45 am

unicode chinese works fine on my computer, see attached image.

You need compile wxwidgets in unicode version.

Do NOT forget turn on unicode flag in your project settings. For example, if you are using visual studio 2005, pls check "project properties -> Configuration Properties -> General -> Character Set"

see http://www.vckbase.com/document/viewdoc/?id=1733 for more information about Chinese unicode.

I don't know what compiler you use. But here is my cmake project file which should support most compilers.

Code: Select all

PROJECT(pdf2db)
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)

SET( CPP_SOURCES
  pdf2db.cpp
  PdfCoreEng.cc
  imgmgr.cpp
  )
SET(SOURCES
  ${CPP_SOURCES}
  PdfCoreEng.h
  imgmgr.h
  )

#add precompile head in msvc
if(MSVC)
  set_source_files_properties(stdwx.cpp
    PROPERTIES
    COMPILE_FLAGS "/Ycstdwx.h"
    )
  foreach( src_file ${CPP_SOURCES} )
    set_source_files_properties(
      ${src_file}
      PROPERTIES
      COMPILE_FLAGS "/Yustdwx.h"
      )
  endforeach( src_file ${CPP_SOURCES} )
  list(APPEND SOURCES stdwx.cpp)
endif(MSVC)


#unicode (for ansi version, just delete this part) ---- begin
IF(WIN32)
  IF (CMAKE_BUILD_TYPE STREQUAL "Debug")
    SET(wxWidgets_CONFIGURATION "mswud")
  ELSE(CMAKE_BUILD_TYPE STREQUAL "Debug")
    SET(wxWidgets_CONFIGURATION "mswu")
  ENDIF(CMAKE_BUILD_TYPE STREQUAL "Debug")
  ADD_DEFINITIONS("-D_UNICODE")
ENDIF(WIN32)
#unicode ---- end

# xrc -> xml, adv, html
# use internal jpeg/png lib because I can not compile it on linux.
set(My_wxWidgets_COMMON_LIBRARIES "") #under win32, I don't want to link to extra jpeg, tiff, png
set(wxWidgets_USE_LIBS base)
find_package(wxWidgets REQUIRED)
include(${wxWidgets_USE_FILE})

#add other librarie
include_directories(
  ${xpdf_SOURCE_DIR}
  ${libgoo_SOURCE_DIR}
  ${libsplash_SOURCE_DIR}
  ${libfofi_SOURCE_DIR}
  ${libxpdf_SOURCE_DIR}
  ${FREETYPE_INCLUDE_DIR_ft2build}
  ${ZLIB_INCLUDE_DIR}
  "${wxsqlite3_SOURCE_DIR}/inc"
  ${libshared_SOURCE_DIR}
  ${JPEG_INCLUDE_DIR}
  ${PNG_INCLUDE_DIR}

  )

add_executable(pdf2db ${SOURCES})
add_dependencies(pdf2db shared z)

# for debug purpose
set(LIBS
  ${wxWidgets_LIBRARIES}
  xpdf
  splash
  fofi
  goo
  ${FREETYPE_LIBRARIES}
  ${ZLIB_LIBRARIES}
  wxsqlite3
  shared
  ${JPEG_LIBRARIES}
  ${PNG_LIBRARIES}
  )

add_definitions(${PNG_DEFINITIONS})

if(MINGW AND WIN32)
  set(LIBS ${LIBS} ole32 uuid oleaut32)
endif(MINGW AND WIN32)
target_link_libraries(pdf2db ${LIBS})
add_custom_command(TARGET pdf2db POST_BUILD
  COMMAND ${CMAKE_COMMAND} -E remove ${pdf2db_BINARY_DIR}/data.sqlite
  )

install(TARGETS pdf2db
  RUNTIME DESTINATION bin
  )
Attachments
unicode-vc-nq8.png
visual studio 2005 unicode setup
unicode-vc-nq8.png (9.91 KiB) Viewed 3474 times
wx-unicode-nq8.png
unicode on dos console.
wx-unicode-nq8.png (5.3 KiB) Viewed 3477 times
help me, help you.
http://blog.binchen.org

catalin
Moderator
Moderator
Posts: 1594
Joined: Wed Nov 12, 2008 7:23 am
Location: Romania

Re: Unicode not working in wxWidgets GUI?

Post by catalin » Mon Aug 24, 2009 7:55 am

AnotherCoder wrote:

Code: Select all

warning C4566: character represented by universal-character-name '\u8A9E' cannot be represented in the current code page (1252)

Code: Select all

#define wxUSE_UNICODE 1

#include <wx/wx.h>

int main()
I am building in Release mode in Visual C++
You need to have wxWidgets built in Unicode mode.

The names of the build options can be anything (you can rename them to whatever you want), but by default they are pretty coherent. So assuming you didn't change that name, it suggests that you have wxW built in Release ASCII mode, so not unicode.

Further more, writing #define wxUSE_UNICODE 1 in your source code is wrong! You need to change that macro in setup.h file in the wxW sources.

So start with a correct build for wxW. These links may help:
http://wiki.wxwidgets.org/MSVC_.NET_Setup_Guide
http://wxwidgets.info/wxwidgets_and_vc2005_video

AnotherCoder
Earned a small fee
Earned a small fee
Posts: 14
Joined: Sun Aug 23, 2009 2:34 am
Location: USA

Post by AnotherCoder » Mon Aug 24, 2009 6:54 pm

Thanks for the answers.
Do NOT forget turn on unicode flag in your project settings. For example, if you are using visual studio 2005, pls check "project properties -> Configuration Properties -> General -> Character Set"
I made this change (from Multi-Byte to Unicode) in MS VC++ 2008.

I built wxWidgets in Unicode - Debug, Unicode - Release, Debug, and Release modes using the video tutorial from http://wiki.wxwidgets.org/MSVC_.NET_Setup_Guide
Further more, writing #define wxUSE_UNICODE 1 in your source code is wrong! You need to change that macro in setup.h file in the wxW sources.
How do I do this?

The program is still not compiling. I changed the function to:

Code: Select all

void MyFrame::OnAbout(wxCommandEvent& WXUNUSED(event))
{
	wchar_t s[] = L"言語";
    int bufferSize = WideCharToMultiByte(CP_UTF8, 0, s, -1, NULL, 0, NULL, NULL);
    char* m = new char[bufferSize];
    WideCharToMultiByte(CP_UTF8, 0, s, -1, m, bufferSize, NULL, NULL);
    wxMessageBox(wxT(s), _T("About Hello World"), wxOK | wxICON_INFORMATION, this);
}
I get the error message: "error C2065: 'Ls' : undeclared identifier"


Any ideas? :cry:

Auria
Site Admin
Site Admin
Posts: 6695
Joined: Thu Sep 28, 2006 12:23 am
Contact:

Post by Auria » Mon Aug 24, 2009 7:55 pm

That's because you're using wxT() on a non-literal. wxT is only to be used on literals, i.e. "text with quotes" -- wxT on a variable of any sort is illegal.

wxT() simply expands to L, so wxT("foo") will give L"foo" (in Unicode builds) which means large characters.

wxT(s) thus gives Ls, which of course means nothing.
"Keyboard not detected. Press F1 to continue"
-- Windows

AnotherCoder
Earned a small fee
Earned a small fee
Posts: 14
Joined: Sun Aug 23, 2009 2:34 am
Location: USA

Post by AnotherCoder » Mon Aug 24, 2009 8:07 pm

I got it working finally!
I did the things above (like changing project settings, etc.) and changed the last function to:

Code: Select all

void MyFrame::OnAbout(wxCommandEvent& WXUNUSED(event))
{
	wxString temp(L"言語");
    wxMessageBox(temp, _T("About Hello World"), wxOK | wxICON_INFORMATION, this);
}
And it works!

Thanks all! Especially chenbin.sh

EDIT: Woops, I accidentally selected my own answer as the accepted answer when I meant to select chenbin.sh. If a moderator can change this, that would be good.

Post Reply