UTF-8 Build on Windows

Do you have a question about makefiles, a compiler or IDE you are using and need to know how to set it up for wxWidgets or why it doesn't compile but other IDE's do ? Post your questions here.
User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

UTF-8 Build on Windows

Post by AlSith » Fri Dec 20, 2019 8:00 am

Hi.
To update custom software, I decided to switch form wxWidgets version 2.9.3 to 3.0.4.
I need, contextually, to adopt utf-8 encoding, so, under Linux, configure the '--enable-uft8' parameter and all goes ok.. obtain the follow configuration:

- wxUSE_WCHAR_T=1
- wxUSE_UNICODE=1
- wxUSE_UNICODE_UTF8=1
- wxUSE_UNICODE_WCHAR=0

but under Windows I'm not able to force this configuration.. in poor words I want have the size of 1 byte for single char.. is it possible under Windows? how can I do it?

Thanks at all.

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 3905
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: UTF-8 Build on Windows

Post by ONEEYEMAN » Fri Dec 20, 2019 3:34 pm

Hi,
You don't need this switch as under Linux/GTK everything is UTF-8 by default.
For Windows - which compiler do you use? Which environment?

You don't need a UTF-8 under Windows - if you do, you can just simpoly convert the string with ToUTF8() call.

Thank you.

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 10:38 am

ONEEYEMAN wrote:
Fri Dec 20, 2019 3:34 pm
Hi,
You don't need this switch as under Linux/GTK everything is UTF-8 by default.
For Windows - which compiler do you use? Which environment?

You don't need a UTF-8 under Windows - if you do, you can just simpoly convert the string with ToUTF8() call.

Thank you.
Thanks ONEEYEMAN for reply.
For Linux, ok.
For Windows, my goal is obtain the relation 1 char = 1 byte, as like as Linux, to avoid communication/connection problems with using my program's pipe and similar.
Must i deduce that, under Windows OS, it will not possible?

User avatar
doublemax
Moderator
Moderator
Posts: 14630
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: UTF-8 Build on Windows

Post by doublemax » Mon Dec 23, 2019 10:49 am

Theoretically it should be possible, but i've never tried it.

Which compiler do you use and how exactly (command line) did you build wxWidgets?

What errors did you get?
Use the source, Luke!

PB
Part Of The Furniture
Part Of The Furniture
Posts: 2242
Joined: Sun Jan 03, 2010 5:45 pm

Re: UTF-8 Build on Windows

Post by PB » Mon Dec 23, 2019 11:23 am

AlSith wrote:
Mon Dec 23, 2019 10:38 am
For Windows, my goal is obtain the relation 1 char = 1 byte
That is not how UTF-8 works. In UTF-8, 1 char(acter) takes between 1 and 4 bytes. sizeof(char) is not affected by a build, it is always the same, unlike that of wchar_t/wxChar. You must be confusing UTF-8 with the deprecated ANSI (non-Unicode) build where one character takes one byte and it uses code pages to switch between 8-bit character sets?

Anyway, if you really want to try UTF-8 on MSW, read the docs here to see how to build wxWidgets to do that
https://docs.wxwidgets.org/trunk/overvi ... g_internal

I do not think this option is common or well-tested on MSW. I am not even sure how it works when MSW is internally UTF-16 (Win10 introduced UTF-8 locale very recently, btw) but I guess it somehow does, at the expense of converting each and every string during API calls.

EDIT
Sorry, I somewhat managed to miss that you are aware of the build options.

I looked into it a bit:
Using USE_UNICODE_UTF8=1 USE_UNICODE_WCHAR=0 when building with nmake still seems to produce the default wchar_t build. I did not find either of this options mentioned anywhere in setup.h nor the makefile.vc/config.vc files.
CMake (on MSW) does not expose this option at all.

I would ask how one should obtain UTF-8 build on MSW in the wx-users group.

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 11:37 am

doublemax wrote:
Mon Dec 23, 2019 10:49 am
Theoretically it should be possible, but i've never tried it.

Which compiler do you use and how exactly (command line) did you build wxWidgets?

What errors did you get?
I created a simple example that create a frame with a status bar in which I wrote a string to view 'special characters' writing and 'wxStringCharType' size.

For wxWidgets I try 2 time:
- 1st with mingw compiler, using the command:
mingw32-make -j3 -f makefile.gcc
in the makefile I configure SHARED=1 MONOLITHIC=1, and then compile the example with command:
mingw32-g++ -o testwx testwx.cpp -IwxWidgets-3.0.4-debug\include -IwxWidgets-3.0.4-debug\lib\gcc_dll\mswud -LwxWidgets-3.0.4-debug\lib\gcc_dll -lwxmsw30ud_gcc_custom
.
The exe runs and show the frame with correct writing but with a size of 2 bytes;

- 2nd with cl compiler (my program will use VisualStudio library and compiler), so I compile with the commands:
nmake -f makefile.vc SHARED=1 MONOLITHIC=1 BUILD=debug
cl /DDEBUG /DWXUSINGDLL /D__WXMSW__ /EHsc testwx.cpp /I..\wxWidgets-3.0.4-debug\include /I..\wxWidgets-3.0.4-debug\lib\vc_dll\mswud /link ..\wxWidgets-3.0.4-debug\lib\vc_dll\*.lib
..but I'm encountering difficult to run the exe..
#-o

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 11:46 am

PB wrote:
Mon Dec 23, 2019 11:23 am

That is not how UTF-8 works. In UTF-8, 1 char(acter) takes between 1 and 4 bytes. sizeof(char) is not affected by a build, it is always the same, unlike that of wchar_t/wxChar. You must be confusing UTF-8 with the deprecated ANSI (non-Unicode) build where one character takes one byte and it uses code pages to switch between 8-bit character sets?
Indeed, I have been studying the topic for many days .. probably my bad English leads me not to clearly expose the topic on which, probably, however, I still have some doubts
PB wrote:
Mon Dec 23, 2019 11:23 am
Anyway, if you really want to try UTF-8 on MSW, read the docs here to see how to build wxWidgets to do that
https://docs.wxwidgets.org/trunk/overvi ... g_internal
Precisely by virtue of this page, which I had already found, I continue to think that there is a possibility
PB wrote:
Mon Dec 23, 2019 11:23 am
I looked into it a bit:
Using USE_UNICODE_UTF8=1 USE_UNICODE_WCHAR=0 when building with nmake still seems to produce the default wchar_t build. I did not find either of this options mentioned anywhere in setup.h nor the makefile.vc/config.vc files.
CMake (on MSW) does not expose this option at all.

I would ask how one should obtain UTF-8 build on MSW in the wx-users group.
Thanks for help

User avatar
doublemax
Moderator
Moderator
Posts: 14630
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: UTF-8 Build on Windows

Post by doublemax » Mon Dec 23, 2019 1:10 pm

As PB already mentioned, you need to rebuild the wxWidgets libraries with wxUSE_UNICODE_UTF8=1
..but I'm encountering difficult to run the exe..
Error descriptions like this are not very useful. What exactly happens?
Use the source, Luke!

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 3905
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: UTF-8 Build on Windows

Post by ONEEYEMAN » Mon Dec 23, 2019 1:27 pm

Hi,
Keep in mind that 1 char = 1 byte is possible only with old ANSI build.
With UNICODE (and UTF-8) it is not possible. You will not be able to store any of the native letters in the 1 byte.

And of course there should be no issue when you try to communicate *nix <-> Win as the communication library will take care of that and you will send strings anyway. It will just be implementation details.

Thank you.

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 1:43 pm

doublemax wrote:
Mon Dec 23, 2019 1:10 pm
As PB already mentioned, you need to rebuild the wxWidgets libraries with wxUSE_UNICODE_UTF8=1
..but I'm encountering difficult to run the exe..
Error descriptions like this are not very useful. What exactly happens?
Sorry, the cl compile goes ok, but when I try to launch exe. the dialog doesn't appear..

User avatar
doublemax
Moderator
Moderator
Posts: 14630
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: UTF-8 Build on Windows

Post by doublemax » Mon Dec 23, 2019 1:47 pm

AlSith wrote:
Mon Dec 23, 2019 1:43 pm
Sorry, the cl compile goes ok, but when I try to launch exe. the dialog doesn't appear..
Did you use a debug build and run under a debugger? If there is no crash or assert, there must be another reason if a dialog doesn't show up.
Use the source, Luke!

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 3905
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: UTF-8 Build on Windows

Post by ONEEYEMAN » Mon Dec 23, 2019 2:04 pm

Hi,
Are wx dlls accessible for the executable?

Thank you.

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 2:17 pm

doublemax wrote:
Mon Dec 23, 2019 1:47 pm
AlSith wrote:
Mon Dec 23, 2019 1:43 pm
Sorry, the cl compile goes ok, but when I try to launch exe. the dialog doesn't appear..
Did you use a debug build and run under a debugger? If there is no crash or assert, there must be another reason if a dialog doesn't show up.
.. I hadn't thought of that .. I try to look

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 2:19 pm

ONEEYEMAN wrote:
Mon Dec 23, 2019 2:04 pm
Hi,
Are wx dlls accessible for the executable?

Thank you.
.. I didn't think of looking at the permissions of the files because I compiled in the same way, from the same shell and with the same user .. I copied the dll by hand to the root of the example.

I just launched a new clean compilation, if it doesn't work I look at it ..

User avatar
AlSith
Earned a small fee
Earned a small fee
Posts: 11
Joined: Fri Dec 20, 2019 7:43 am

Re: UTF-8 Build on Windows

Post by AlSith » Mon Dec 23, 2019 2:23 pm

AlSith wrote:
Mon Dec 23, 2019 2:19 pm
ONEEYEMAN wrote:
Mon Dec 23, 2019 2:04 pm
Hi,
Are wx dlls accessible for the executable?

Thank you.
.. I didn't think of looking at the permissions of the files because I compiled in the same way, from the same shell and with the same user .. I copied the dll by hand to the root of the example.

I just launched a new clean compilation, if it doesn't work I look at it ..
The dll has permissione or read and execute

Post Reply