Read from input iterator (wxResourceTranslationsLoader)

[Virchanza]

I want to be able to embed my translation catalog (.mo) files in my binary executable by having the .mo files as global byte arrays like this:

Code: Select all

char unsigned const g_catalog_german[] = {0x01,0x02,0x03,0x04};

The command line program "xxd" can be used to easily turn a binary file into a sequence of hexadecimal byte values like this.

I realise that my application will consume an extra few hundred kilobytes of RAM if I do it this way (because all of my translations will be loaded into RAM when my program first loads up).

I think the best way to go about coding this is to look through the code for wxResourceTranslationsLoader, and to tweak it to read from an input iterator, e.g. from a simple pointer or from an "std::istream_iterator".

Before I start coding this, has anyone started it already?

[Virchanza]

Here's the code from wxResourceTranslationsLoader that I will be adapting:

Code: Select all

wxMsgCatalog *wxResourceTranslationsLoader::LoadCatalog(const wxString& domain, const wxString& lang)
{
    const void *mo_data = NULL;

    size_t mo_size = 0;

    const wxString resname = wxString::Format("%s_%s", domain, lang);

    if ( !wxLoadUserResource(&mo_data, &mo_size, resname, GetResourceType().t_str(), GetModule()) )
        return NULL;

    wxLogTrace(TRACE_I18N, "Using catalog from Windows resource \"%s\".", resname);

    wxMsgCatalog *cat = wxMsgCatalog::CreateFromData( wxCharBuffer::CreateNonOwned(static_cast<const char*>(mo_data), mo_size), domain);

    if ( !cat )
    {
        wxLogWarning(_("Resource '%s' is not a valid message catalog."), resname);
    }

    return cat;
}

Sorry I copy-pasted this on my phone in a coffee shop so the line endings are all messed up.

It looks like all the functionality I need is already implemented in "wxMsgCatalog::CreateFromData( wxCharBuffer::CreateNonOwned(pointer,length), domain);"

This should be less than a day of coding.

[Virchanza]

I did this on my smartphone in a coffee shop so don't go too hard on me. I haven't tried to compile it.

Here is the beginnings of wxInputIteratorTranslationsLoader:

Code: Select all

#include <vector>
#include <tuple>
#include <iterator>

template<typename InputIterator>
class wxInputIteratorTranslationsLoader : public wxTranslationsLoader {
protected:

    typedef std::tuple<wxString,InputIterator,InputIterator> CatalogueTuple;

    std::vector<CatalogueTuple> m_catalogues;
      
public:

    void MakeCatalogueVisible(wxString const &arg_lang, InputIterator &arg_begin, InputIterator &arg_end)
    {
        m_catalogues.push_back(
            CatalogueTuple(arg_lang, arg_begin, arg_end)
        );
    }
    
    virtual wxMsgCatalog *LoadCatalog(wxString const &domain, wxString const &lang) wxOVERRIDE;

    virtual wxArrayString GetAvailableTranslations(wxString const &domain) const wxOVERRIDE;
};


wxMsgCatalog *wxInputIteratorTranslationsLoader::LoadCatalog(wxString const &domain, wxString const &lang)
{
    CatalogueTuple *pcatuple = nullptr;

    for (auto &catuple : m_catalogues)
    {
        if ( lang == std::get<0>(catuple) )
        {
            pcatuple = &catuple;
            break;
        }
    }

    if ( !pcatuple )
        return nullptr;
    
    InputIterator &it_begin = std::get<1>(*pcatuple);

    InputIterator &it_end = std::get<2>(*pcatuple);

    wxMsgCatalog *const pcat = wxMsgCatalog::CreateFromData(
        wxCharBuffer::CreateNonOwned(
            static_cast<char const*>(it_begin),
            it_end - it_begin),
        domain);

    if ( !pcat )
    {
        wxLogWarning(_("Not a valid message catalog."));
    }

    return pcat;
}

char unsigned const g_binary_German = { 0x01,0x02,0x03 };

auto main(void) -> int
{
    wxTranslationsLoader *const p_iitl = new wxInputIteratorTranslationsLoader<char unsigned *>;
    
    p_iitl->MakeCatalogueVisible(
        wxT("de"),
        std::begin(g_binary_German),
        std::end(g_binary_German)
    );
    
    //Make sure there is a current wxTranslation object
    //before executing the next line
    
    wxTranslations::Get()->SetLoader(p_iitl);       
}

[Virchanza]

Note that you can get this to read from any kind of input stream by using "std::istream_iterator". Here's how you'd get it to read from standard input:

Code: Select all

wxTranslationsLoader *const p =
    new wxInputIteratorTranslationsLoader<
        std::istream_iterator<char unsigned>
    >;
    
p->MakeCatalogueVisible(
    wxT("de"),
    istream_iterator<char unsigned>(std::cin),
    istream_iterator<char unsigned>()
);

Similarly you can get it to read from a file by using "ifstream" along with "istream_iterator". It can also read from any kind of container (vector, set, stack) if you supply an input iterator.

[Virchanza]

I'm also working on a Makefile that will automatically process all your ".mo" files into char arrays before your application is compiled. So if you ever change a translation catalogue, you just need to hit "make" and it will automatically re-process all your ".mo" files into char arrays before your application is re-compiled.

[doublemax]

The conversion from in-memory data to wxMsgCatalog looks a little complicated.

Wouldn't this work?

Code: Select all

char g_catalog_german[] = {0x01,0x02,0x03,0x04};
wxMsgCatalog *catalog = wxMsgCatalog::CreateFromData( wxCharBuffer(g_catalog_german), domain );

[Virchanza]

The conversion from in-memory data to wxMsgCatalog looks a little complicated.

The member function "CreateFromData" takes a wxScopedCharBuffer.

Later when I'm sitting at a PC, I'll take a closer look at this:

https://docs.wxwidgets.org/3.0/classwx_ ... uffer.html

It looks like I'll be making a template specialisation for wxInputIteratorTranslationsLoader<char> which passes the pointer directly to wxScopedCharBuffer::CreateNonOwned.

For any type other than char, I will use "std::copy" to copy into a char buffer, and then pass the char buffer to wxScopedCharBuffer::CreateNonOwned.

Another option would be to make another overload of CreateFromData which would take an input iterator (with the type as a template parameter).

[doublemax]

The conversion from in-memory data to wxMsgCatalog looks a little complicated.

The member function "CreateFromData" takes a wxScopedCharBuffer.

The line i posted does compile.

[Virchanza]

I have this working now with my ".mo" files embedded in the exectuable file as global char arrays.

I'll post code for wxInputIteratorTranslationsLoader later. Maybe it's an idea worth adding to wxWidgets 3.1.x ?