string encoding in wxStreamToTextRedirector

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
csniper
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Mar 13, 2017 8:27 am

string encoding in wxStreamToTextRedirector

Post by csniper » Sat Feb 22, 2020 2:35 am

I met a problem to display UTF8 non-ASCII string with wxStreamToTextRedirector. The string is displayed OK with other wxWidgets text control after being converted by mb_str(wxConvUTF8). As below
Image 2.png
Image 2.png (16.69 KiB) Viewed 920 times
I tried to change wxConvCurrent to wxConvUTF8 but it did not work.

Any suggestions?

User avatar
doublemax
Moderator
Moderator
Posts: 15866
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax » Sat Feb 22, 2020 8:42 am

Can you post a minimal, compilable code sample?

Which wxWidgets version are you using and what's the locale of your OS?
Use the source, Luke!

csniper
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper » Sat Feb 22, 2020 9:39 am

I'll try to make a minimal, compilable code sample.

I'm using wxWidgets 3.1.3. The encoding of the string in the example was UTF8(I verified). The locale of my OS is GBK.

csniper
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper » Sat Feb 22, 2020 10:30 am

I made a minimal example with a dialog which has a text control and two buttons. Below is the code snippet for demo the issue.

Code: Select all

void testDialog::OnAbout(wxCommandEvent& event)
{
    std::string s("\xE6\xB5\x8B\xE8\xAF\x95"); // UTF8
    TextCtrl1->Clear();
    wxStreamToTextRedirector redirctor(TextCtrl1);
    std::cout << s << std::endl;
    wxMBConv* current = wxConvCurrent;
    wxConvCurrent = dynamic_cast<wxMBConv*>(&wxConvUTF8);
    wxASSERT(wxConvCurrent);
    std::cout << s << std::endl; // try again
    wxConvCurrent = current;
    wxString str(s.c_str(), wxConvUTF8);
    TextCtrl1->AppendText(str);
}
Result
Image 1.png
Image 1.png (13.8 KiB) Viewed 893 times

User avatar
doublemax
Moderator
Moderator
Posts: 15866
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax » Sun Feb 23, 2020 1:34 am

After a little bit of tracing through the code, i don't think it's possible.

Every single char of the 8bit string passes through this method individually, no decoding takes place:

Code: Select all

int wxTextCtrlBase::overflow(int c)
{
    AppendText((wxChar)c);

    // return something different from EOF
    return 0;
}
(src/common/textcmn.cpp)
Use the source, Luke!

csniper
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper » Sun Feb 23, 2020 1:46 am

Thanks, doublemax. Now I understand the root cause.

BTW: do you think it should be fixed/supported by wxWidgets?

User avatar
doublemax
Moderator
Moderator
Posts: 15866
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax » Sun Feb 23, 2020 8:37 am

csniper wrote:
Sun Feb 23, 2020 1:46 am
BTW: do you think it should be fixed/supported by wxWidgets?
I'm not sure. But you could try to open a ticket for this and see what the response is.
Use the source, Luke!

User avatar
doublemax
Moderator
Moderator
Posts: 15866
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax » Sun Feb 23, 2020 9:05 am

If you copy the wxStreamToTextRedirector code and convert it to wide chars, you can redirect std::wcout.

Code: Select all

#include <iostream>
#include <fstream>
class wxStreamToTextRedirectorUnicode : public std::wstreambuf
{
private:
  void Init(wxTextCtrl *text)
  {
    m_text = text;
    m_sbufOld = m_ostr.rdbuf();
    m_ostr.rdbuf(this);
  }

public:
  wxStreamToTextRedirectorUnicode(wxTextCtrl *text)
    : m_ostr(std::wcout)
  {
    Init(text);
  }

  wxStreamToTextRedirectorUnicode(wxTextCtrl *text, std::wostream *ostr)
    : m_ostr(*ostr)
  {
    Init(text);
  }

  ~wxStreamToTextRedirectorUnicode()
  {
    m_ostr.rdbuf(m_sbufOld);
  }

  // override streambuf method
  int_type overflow(int_type i) wxOVERRIDE
  {
   m_text->AppendText( (wxChar)i );
   return 0;
  }

private:
  wxTextCtrl *m_text;

  // the stream we're redirecting
  std::wostream &m_ostr;

  // the old streambuf (before we changed it)
  std::wstreambuf *m_sbufOld;
};
Use the source, Luke!

csniper
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper » Tue Feb 25, 2020 1:45 am

Thanks, doublemax. Will try it and let you know the result.

csniper
Knows some wx things
Knows some wx things
Posts: 48
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper » Tue Feb 25, 2020 6:54 am

It works in a weird way I think it is because MS windows uses UTF16.

I used a customized redirector function for my app and convert it with wxString("...", wxConvUtf8). Now I can output as expected.

Thanks anyway.

Post Reply