Page 1 of 1
Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 2:05 am
by Widgets
In one of my projects I need to display UTF-8 string.
Some of the sample code I have available uses std::cout to do this to a console app.
Code: Select all
cout << std::setfill(' ') << std::left << std::setw(44) << md.key() << " " << std::setw(9)
<< std::setfill(' ') << std::left << md.typeName() << " " << std::dec << std::setw(3)
<< std::setfill(' ') << std::right << md.count() << " " << std::dec << md.value() << std::endl;
To avoid as many transcription/translation errors as possible, I wanted to display the result on wxTextCtrl using redirection
I did find the topic discussed here:
viewtopic.php?f=1&t=46867&p=197057
but implementing the proposed class wxStreamToTextRedirectorUnicode does not seem to do the job for me.
All output, even plain text strings, just disappears.
So far I have not been able to trace through the code to track down the cause in an effort to understand and hopefully find a way to resolve the problem.
Obviously I am missing something critical, but I have no idea what.
TIA
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 5:56 am
by ONEEYEMAN
Hi,
What code do you use to send the string to the wxTextCtrl?
Are you sure it is a UTF8 encoded one?
Thank you.
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 5:41 pm
by Widgets
ONEEYEMAN wrote: ↑Sun Nov 29, 2020 5:56 am
Hi,
What code do you use to send the string to the wxTextCtrl?
The original code I was using, and which works well for plain ASCII strings, is:
Code: Select all
// has problems with UTF-8 strings
ostream stream(m_textCtrlXmpDump );
for (const auto& md : xmpData)
{
//stream << 123.456 << " some text\n"; // test string - plain ASCII
stream << std::setfill(' ') << std::left << std::setw(44) << md.key() << " " << std::setw(9)
<< std::setfill(' ') << std::left << md.typeName() << " " << std::dec << std::setw(3)
<< std::setfill(' ') << std::right << md.count() << " " << std::dec << md.value() << std::endl;
}
stream.flush();
If I rework the code to use all wxString::Printf() code and convert all strings from UTF-8 to wxString, the display of even UTF-8 strings is correct and as expected.
However, that is a whole lot of work in this instance and, in general, if there is a way I would very much prefer to use the code as it was to avoid any gotchas due to the rewrite.
ONEEYEMAN wrote: ↑Sun Nov 29, 2020 5:56 am
Are you sure it is a UTF8 encoded one?
Yes very much so.
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 6:16 pm
by ONEEYEMAN
Hi,
Why?
What is wrong with calling FromUTF8()?
Thank you.
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 6:41 pm
by Widgets
ONEEYEMAN wrote: ↑Sun Nov 29, 2020 6:16 pm
What is wrong with calling FromUTF8()?
Nothing, but it requires a rewrite.
Some members of 'md' can be strings, numeric or boolean data and hence would need logic to deal with each type differently.
With the code as is, the stream handles all of those differences, when it goes to cout.
A rewrite might not cover all cases properly, especialy as the contents of md originate in an imported library.
If it can't be done, I'll have to deal with it. The link I posted to a wxWidgets discussion seemed to imply that it might be possible.
Then again, in some respects something important seems to be missing if the code for redirection fails for UTF-8 strings.
I can see, that there might be a need to handle UTF-8 in a special way, similar to wxString::To/FromUtf8.
That is really all I want to find out
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 8:32 pm
by Kvaz1r
Widgets wrote: ↑Sun Nov 29, 2020 2:05 am
but implementing the proposed class wxStreamToTextRedirectorUnicode does not seem to do the job for me.
Did you try to add converting that was mentioned in
this message?
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 9:15 pm
by Widgets
Kvaz1r wrote: ↑Sun Nov 29, 2020 8:32 pm
Widgets wrote: ↑Sun Nov 29, 2020 2:05 am
but implementing the proposed class wxStreamToTextRedirectorUnicode does not seem to do the job for me.
Did you try to add converting that was mentioned in
this message?
That is the very same post I had referred to in my first post - and no, it did not.
In fact,
class wxStreamToTextRedirectorUnicode was part of that very post.
Even the 'solution' referred to in the last message of that thread, did not give an actual solution using a redirected cout, but seems to refer to a solution using string conversion, without redirection.
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 9:28 pm
by doublemax
You need something that converts the 8bit UTF8 byte stream into a stream of (decoded) wide characters. The wxTextCtrl will not do it by itself.
Googling found this (very old) post (untested):
https://stackoverflow.com/questions/148 ... ion-in-stl
https://en.cppreference.com/w/cpp/locale/codecvt_utf8
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 9:37 pm
by Kvaz1r
Widgets wrote: ↑Sun Nov 29, 2020 9:15 pm
That is the very same post I had referred to in my first post - and no, it did not.
In fact,
class wxStreamToTextRedirectorUnicode was part of that very post.
Even the 'solution' referred to in the last message of that thread, did not give an actual solution using a redirected cout, but seems to refer to a solution using string conversion, without redirection.
I understood it, because you left link in first message of the topic and I can read. But Windows console hasn't support of UTF-8, so you still must use conversion if you want get UTF-8. That is exactly what written in last message of the that thread.
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 9:49 pm
by Widgets
Yes, there seems to be a lot more involved with this conversion than it seemed at first.
I will have to investigate the links you provided and hopefully will be able to come up with something useful.
As always, it will likely take several tries and a fair bit of time.
For now, I think this will be put on the back burner, as it seems to involve more time than is available right now.
FWIW, meanwhile I have found some other 'solution' (read 'work-around) for windows sending UTF-8 to the DOS console in the following blog:
https://blogs.msmvps.com/gdicanio/2019/ ... mple-code/
In this case the blogger converts UTF-8 to UTF-16 - which, in a way, makes sense for Windows' MBCS/UTF-16 mentality.
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 10:35 pm
by doublemax
Do you actually have to output to the console? Are you developing a console application?
Re: Redirecting UTF-8 strings to wxTextCtrl
Posted: Sun Nov 29, 2020 10:48 pm
by Widgets
No, the original sample code is/was a console app. My output needs to go to a wxWidgets GUI.
Being able to use redirection to a text control was the easy way out as long as everything was plain ASCII.
When UTF-8 strings showed up and became important, that option failed badly.
Finding a way to make the redirection handle UTF-8 or other encoding would be much simpler with fewer chances for misinterpretation of the types of data coming from within a library, than reworking & testing all the equivalent code.
But, since my last post here, I have followed some of the links you provided and concluded that, at least for the time being, there is way too much research and testing involved to try and NOT translate the cout code to something that will work 'well enough' for now.