Problems With Unicode

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
toadybarker
Earned a small fee
Earned a small fee
Posts: 13
Joined: Wed Aug 12, 2015 2:29 pm

Problems With Unicode

Post by toadybarker » Mon Apr 29, 2019 7:14 pm

I have asked similar question before about using the Unicode functions in wxWidgets. Things got a bit confusing recently when attempting to pass a drop down that is in Unicode into some Mysql. Here the deal

I have a drop down this in it - 文字化け

This drop down represents a group name.

I am attempting to use code like this - :

char picked_group[40];
wxString groupString = ((wxChoice*)FindWindow(wxID_GROUP))->GetStringSelection().wx_str();
strcpy(picked_group, WX2UNI(groupString));
msg.Printf("Group String = %s", picked_group);
wxMessageBox(_(msg),_("My Debug"),wxICON_ERROR|wxOK);

( WX2UNI is a simple #define WX2UNI(s) ((const char *)s.ToUTF8())

Using Visual Studio's Debugger - I see the Group name correctly when I hover over it - but the actual message comes out another way.

In debugger:
groupString = m_impl = L"文字化け" ,
m_str = 0x05da6b10 "文字化け"

The Message box comes out like this

Group String = 文字化け


Why do I not see the first group of characters in my message box - what does m_str mean in Vusual Studio? Anybody know?
To add to the problem, I need to send this into a mysql call - and the Group field is used
during the creation of new data, but the group has to exist. The call
appears to not get anything in the group field at all (when I look in the mysql side I see nothing in group,
so the creation fails complaining that the group does not exist.

I have been able to do things this way before - but all of the fields made it into the database just fine. I wasn't using them
as a criteria for creation or updating though....

I guess I would feel better if my Message box actually showed the correct data - then at least I "think" that I am sending the correct Group name...

PB
Part Of The Furniture
Part Of The Furniture
Posts: 1951
Joined: Sun Jan 03, 2010 5:45 pm

Re: Problems With Unicode

Post by PB » Mon Apr 29, 2019 8:07 pm

toadybarker wrote:
Mon Apr 29, 2019 7:14 pm

Code: Select all

      
        char picked_group[40];
        wxString groupString = ((wxChoice*)FindWindow(wxID_GROUP))->GetStringSelection().wx_str();
	strcpy(picked_group, WX2UNI(groupString));
	msg.Printf("Group String =  %s", picked_group);
I think that (at least) the code above is wrong. wxString::Printf() does not know that picked_group is UTF-8 encoded. The code makes no sense to me TBH.

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 3278
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: Problems With Unicode

Post by ONEEYEMAN » Mon Apr 29, 2019 8:45 pm

Hi,
Are you using ODBC calls or native mySQL Connector?

Anyway I think there is a typo somewhere:
char picked_group[40];
wxString groupString = ((wxChoice*)FindWindow(wxID_GROUP))->GetStringSelection().wx_str();
strcpy(picked_group, WX2UNI(groupString));
msg.Printf("Group String = %s", picked_group);
wxMessageBox(_(msg),_("My Debug"),wxICON_ERROR|wxOK);
[/code]

( WX2UNI is a simple #define WX2UNI(s) ((const char *)s.ToUTF8())

Using Visual Studio's Debugger - I see the Group name correctly when I hover over it - but the actual message comes out another way.

In debugger:
groupString = m_impl = L"文字化け" ,
m_str = 0x05da6b10 "文字化け"
There is no m_str in the code above.

On top of that I think you shuold use ".c_str()", especially if you try to use C code/functions, like strcpy().

Finally, why do you even need to do such a conversion and then pass it to wxMessageBox()? For testing?

And finally - is you end goal to make a mySQL interface work or find why the messagebox is displaying your string incorrectly?

Thank you.

toadybarker
Earned a small fee
Earned a small fee
Posts: 13
Joined: Wed Aug 12, 2015 2:29 pm

Re: Problems With Unicode

Post by toadybarker » Tue Apr 30, 2019 11:56 am

Thanks for replies from all about this. I am using ODBC calls for all intensive purposes.
It's actually an API...

I am really trying to figure out what I should do with the Unicode string to pass it
into the call to the API as a Char string *.

I was using message box because I hoped that when I saw the correct data in the message box I could use the
same procedure on the string of Unicode that I used to display it in the Message box in my API call.

The last lines that say "In debugger":

groupString = m_impl = L"文字化け" ,
m_str = 0x05da6b10 "文字化け"

are the output from Hovering the mouse over the groupString variable in the debugger.
Those are NOT actual lines of code.

So - to answer the last poster's question - I am really trying to get a SQL statement that has the correct where clause
with a group = L"文字化け" NOT "文字化け" and NOT "" (which is what I have seen on the SQL side).

I think when I tried .c_str() calls - it didn't work at all on Unicode strings.

All this being said - anybody have example code that works that will display a Unicode string in a message box?
Thanks,
Todd

User avatar
doublemax
Moderator
Moderator
Posts: 13602
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Problems With Unicode

Post by doublemax » Tue Apr 30, 2019 12:02 pm

So - to answer the last poster's question - I am really trying to get a SQL statement that has the correct where clause
with a group = L"文字化け" NOT "文字化け" and NOT "" (which is what I have seen on the SQL side).
"文字化ã‘" is the correct UTF8 representation of "文字化け". That's not an error.

In general, you should stick to wxString all the time while in the "wxWidgets universe" and only convert it to UTF8 when passing to the SQL API.

Maybe show some real code of where you're passing an SQL command into Mysql.
Use the source, Luke!

toadybarker
Earned a small fee
Earned a small fee
Posts: 13
Joined: Wed Aug 12, 2015 2:29 pm

Re: Problems With Unicode

Post by toadybarker » Tue Apr 30, 2019 1:11 pm

Thanks Doublemax,
At least I know that the representation is right. Still having problem with the SQL,
maybe I can get some output via the API...before I give call the Mysql function(s)...

Thanks
Todd

User avatar
doublemax
Moderator
Moderator
Posts: 13602
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Problems With Unicode

Post by doublemax » Tue Apr 30, 2019 1:51 pm

Let's suppose you're using the MySQL C api:

Code: Select all

int mysql_query(MYSQL *mysql, const char *stmt_str)
Then this is how you could generate the SQL command and execute it (omitting stuff like escaping special chars):

Code: Select all

wxString sql
sql.printf("SELECT * FROM mytable WHERE name = '%s'), m_choice->GetStringSelection() );
wxLogMessage("Command: %s", sql );
mysql_query( m_mysql, sql.ToUTF8() );
Use the source, Luke!

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 3278
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: Problems With Unicode

Post by ONEEYEMAN » Tue Apr 30, 2019 2:22 pm

Hi,
The code will look something like this:

Code: Select all

void ODBCDatabase::uc_to_str_cpy(SQLWCHAR *dest, const std::wstring &src)
{
    const wchar_t *temp = src.c_str();
    while( *dest )
    {
        dest++;
    }
    while( *temp )
    {
        *dest = *temp;
        dest++;
        temp++;
    }
    *dest++ = 0;
    *dest = 0;
}

void ODBCDatabase::str_to_uc_cpy(std::wstring &dest, const SQLWCHAR *src)
{
    while( *src )
    {
        dest += *src++;
    }
}

int ODBCDatabase::CreateDatabase(const std::wstring &name, std::vector<std::wstring> &errorMsg)
{
    int result = 0;
    SQLWCHAR *query = NULL;
    std::wstring qry = L"CREATE DATABASE " + name + L" ";
    if( pimpl->m_subtype == L"Microsoft SQL Server" )
        qry = L"CREATE DATABASE " + name + L"";
    RETCODE ret = SQLAllocHandle( SQL_HANDLE_STMT, m_hdbc, &m_hstmt );
    if( ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO )
    {
        query = new SQLWCHAR[name.length() + 2];
        memset( query, '\0', name.length() + 2 );
        uc_to_str_cpy( query, name );
        ret = SQLExecDirect( m_hstmt, query, SQL_NTS );
        if( ret != SQL_SUCCESS && ret != SQL_SUCCESS_WITH_INFO )
        {
            GetErrorMessage( errorMsg, 2 );
            result = 1;
        }
        else
        {
            if( m_hstmt )
            {
                ret = SQLFreeHandle( SQL_HANDLE_STMT, m_hstmt );
                if( ret != SQL_SUCCESS && ret != SQL_SUCCESS_WITH_INFO )
                {
                    GetErrorMessage( errorMsg, 2 );
                    result = 1;
                }
                else
                    m_hstmt = 0;
            }
        }
    }
    else
    {
        GetErrorMessage( errorMsg, 2 );
        result = 1;
    }
    delete[] query;
    query = NULL;
    return result;
}
And then you will call wxString::ToStdWstring() in order to call the "CreateDatabase()" function.

I hope you will get an idea.

This code will work on all 3 major platforms with native MS ODBC API, unixODBC and iODBC.

Thank you.

toadybarker
Earned a small fee
Earned a small fee
Posts: 13
Joined: Wed Aug 12, 2015 2:29 pm

Re: Problems With Unicode

Post by toadybarker » Wed May 01, 2019 3:55 pm

Thanks so much,

I am pretty sure I don't need to do anything like this (character by Character copy), but the simpler Printf method should work just fine (I am not calling MySql Directly - but through an API).

I appreciate all the feedback. I am working with the creator of the application that is ingesting this call, it appears the problem may not be my code after all.

Primarily I wanted to ensure that the UTF8 characters were being sent correctly, and that appears to be the case.

Thanks again,

Todd

ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 3278
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: Problems With Unicode

Post by ONEEYEMAN » Wed May 01, 2019 4:10 pm

Hi,
This code was created after I spoke with the unixODBC creators in order to create cross-driver/DM API.
Remember ODBC types are C-based, just like ODBC API and if you want to pass the Unicode data correctly you need some kind of way to convert between C++ and C.

This API works for me in all 3 major platforms with all 3 major DM: Windows Native, unixODBC and iODBC.

Thank you.

Post Reply