string encoding in wxStreamToTextRedirector

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
csniper
Experienced Solver
Experienced Solver
Posts: 53
Joined: Mon Mar 13, 2017 8:27 am

string encoding in wxStreamToTextRedirector

Post by csniper »

I met a problem to display UTF8 non-ASCII string with wxStreamToTextRedirector. The string is displayed OK with other wxWidgets text control after being converted by mb_str(wxConvUTF8). As below
Image 2.png
Image 2.png (16.69 KiB) Viewed 1579 times
I tried to change wxConvCurrent to wxConvUTF8 but it did not work.

Any suggestions?
User avatar
doublemax
Moderator
Moderator
Posts: 19116
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax »

Can you post a minimal, compilable code sample?

Which wxWidgets version are you using and what's the locale of your OS?
Use the source, Luke!
csniper
Experienced Solver
Experienced Solver
Posts: 53
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper »

I'll try to make a minimal, compilable code sample.

I'm using wxWidgets 3.1.3. The encoding of the string in the example was UTF8(I verified). The locale of my OS is GBK.
csniper
Experienced Solver
Experienced Solver
Posts: 53
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper »

I made a minimal example with a dialog which has a text control and two buttons. Below is the code snippet for demo the issue.

Code: Select all

void testDialog::OnAbout(wxCommandEvent& event)
{
    std::string s("\xE6\xB5\x8B\xE8\xAF\x95"); // UTF8
    TextCtrl1->Clear();
    wxStreamToTextRedirector redirctor(TextCtrl1);
    std::cout << s << std::endl;
    wxMBConv* current = wxConvCurrent;
    wxConvCurrent = dynamic_cast<wxMBConv*>(&wxConvUTF8);
    wxASSERT(wxConvCurrent);
    std::cout << s << std::endl; // try again
    wxConvCurrent = current;
    wxString str(s.c_str(), wxConvUTF8);
    TextCtrl1->AppendText(str);
}
Result
Image 1.png
Image 1.png (13.8 KiB) Viewed 1552 times
User avatar
doublemax
Moderator
Moderator
Posts: 19116
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax »

After a little bit of tracing through the code, i don't think it's possible.

Every single char of the 8bit string passes through this method individually, no decoding takes place:

Code: Select all

int wxTextCtrlBase::overflow(int c)
{
    AppendText((wxChar)c);

    // return something different from EOF
    return 0;
}
(src/common/textcmn.cpp)
Use the source, Luke!
csniper
Experienced Solver
Experienced Solver
Posts: 53
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper »

Thanks, doublemax. Now I understand the root cause.

BTW: do you think it should be fixed/supported by wxWidgets?
User avatar
doublemax
Moderator
Moderator
Posts: 19116
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax »

csniper wrote: Sun Feb 23, 2020 1:46 am BTW: do you think it should be fixed/supported by wxWidgets?
I'm not sure. But you could try to open a ticket for this and see what the response is.
Use the source, Luke!
User avatar
doublemax
Moderator
Moderator
Posts: 19116
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: string encoding in wxStreamToTextRedirector

Post by doublemax »

If you copy the wxStreamToTextRedirector code and convert it to wide chars, you can redirect std::wcout.

Code: Select all

#include <iostream>
#include <fstream>
class wxStreamToTextRedirectorUnicode : public std::wstreambuf
{
private:
  void Init(wxTextCtrl *text)
  {
    m_text = text;
    m_sbufOld = m_ostr.rdbuf();
    m_ostr.rdbuf(this);
  }

public:
  wxStreamToTextRedirectorUnicode(wxTextCtrl *text)
    : m_ostr(std::wcout)
  {
    Init(text);
  }

  wxStreamToTextRedirectorUnicode(wxTextCtrl *text, std::wostream *ostr)
    : m_ostr(*ostr)
  {
    Init(text);
  }

  ~wxStreamToTextRedirectorUnicode()
  {
    m_ostr.rdbuf(m_sbufOld);
  }

  // override streambuf method
  int_type overflow(int_type i) wxOVERRIDE
  {
   m_text->AppendText( (wxChar)i );
   return 0;
  }

private:
  wxTextCtrl *m_text;

  // the stream we're redirecting
  std::wostream &m_ostr;

  // the old streambuf (before we changed it)
  std::wstreambuf *m_sbufOld;
};
Use the source, Luke!
csniper
Experienced Solver
Experienced Solver
Posts: 53
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper »

Thanks, doublemax. Will try it and let you know the result.
csniper
Experienced Solver
Experienced Solver
Posts: 53
Joined: Mon Mar 13, 2017 8:27 am

Re: string encoding in wxStreamToTextRedirector

Post by csniper »

It works in a weird way I think it is because MS windows uses UTF16.

I used a customized redirector function for my app and convert it with wxString("...", wxConvUtf8). Now I can output as expected.

Thanks anyway.
Post Reply