Showing images from a video stream

[NonoNano]

Hello,

I followed your suggestion, the only think I did slightly differently is this part:

Code: Select all

    virtual void OnImageGrabbed(CInstantCamera& camera, const CGrabResultPtr& ptrGrabResult)
    {
        wxCommandEvent evt(wxEVT_GRABBED);
        evt.SetExtraLong((long) &ptrGrabResult);
        wxPostEvent(handler, evt);

        if (ptrGrabResult && ptrGrabResult->GrabSucceeded())
            {
                wxCriticalSectionLocker lock(handler->m_bitmap_cs);
                // Access the image data.
                cout << "SizeX: " << ptrGrabResult->GetWidth() << endl;
                cout << "SizeY: " << ptrGrabResult->GetHeight() << endl;
                unsigned char *pImageBuffer = (unsigned char *) ptrGrabResult->GetBuffer();
                int width = (int) ptrGrabResult->GetWidth();
                int height = (int) ptrGrabResult->GetHeight();
                handler->toWXBitmap(pImageBuffer, width, height);
                handler->Refresh();
            }
            else
            {
                cout << "Error: " << ptrGrabResult->GetErrorCode() << " " << ptrGrabResult->GetErrorDescription() << endl;
            }
    }

where I could not use 'this' since I was in the secondary thread, therefore I used the handler to the main GUI.
The problem I have now is the same as some days ago: even if I call Refresh() the OnPaint method is never called (unless once right after the constructor has done).
I attach my code if someone would like to take a look.
I hope this is the last barrier between me and the solution..do you have suggestions to overcome this problem? Thank you!!!

[doublemax]

I find i hard to believe that the paint event handler doesn't get called. Add some wxLogDebug() or cout logs at interesting places: The paint event handler, check if toWXBitmap is called and does what it should. Check that width and height contain reasonable values etc.

Also, comment out the event sending code for now.

[NonoNano]

Hello,

I'm executing the code inside CodeBolcks IDE..I put breakpoints in all the functions involved in the process and they got called, toWXBitmap is called and width/height are 640*480..it seems to me that everything is fine..

[doublemax]

Code: Select all

wxPanel *panel = new wxPanel(this, wxID_ANY, wxDefaultPosition, wxSize(50,50), wxTAB_TRAVERSAL,wxString());

*cough* sorry for not seeing the obvious

The paint handler is for the wxFrame, but it's completely covered by this panel, so of course you can't see anything drawn. You need a dedicated panel onto which you draw the bitmap. You need to catch the paint event for that panel and there draw the bitmap.

[NonoNano]

Thank you very much! I will try as soon as possible!
I noticed also this behaviour:
My MainFrame was initialized like this and OnPaint was called once

Code: Select all

    CameraGUIForm *cgf = new CameraGUIForm();
    cgf->ShowFullScreen(true,wxFULLSCREEN_NOCAPTION);

If I initialize like this

Code: Select all

    CameraGUIForm *cgf = new CameraGUIForm();
    cgf->Show(true);

on paint is never called! Does it make sense?

I can use that 'panel' to draw the image but how can I associate the paint event to that panel?

If I try this:

Code: Select all

    panel = new wxPanel(this, ID_PANEL, wxDefaultPosition, wxSize(640,480), wxTAB_TRAVERSAL,wxString());
    Connect(ID_PANEL, wxEVT_PAINT,
      wxCommandEventHandler(CameraGUIForm::OnPaint));

I get the following error at compile time:

/usr/include/wx-3.0/wx/event.h|88|error: invalid static_cast from type ‘void (CameraGUIForm::*)(wxPaintEvent&)’ to type ‘wxCommandEventFunction {aka void (wxEvtHandler::*)(wxCommandEvent&)}’|

Thanks!

[doublemax]

Code: Select all

 wxPanel *panel = new wxPanel(this, wxID_ANY, wxDefaultPosition, wxSize(50,50), wxTAB_TRAVERSAL,wxString());
 
// m_bitmap_panel  must be a member variable, you need it in the paint event handler
 m_bitmap_panel = new wxPanel( panel, wxID_ANY, wxPoint(10, 110), wxSize(640,480) );
 m_bitmap_panel->SetMinSize( wxSize(640,480) );
 m_bitmap_panel->Bind( wxEVT_PAINT, &mainFrame::OnPaint, this );

Code: Select all

void CameraGUIForm::OnPaint(wxPaintEvent& event)
{
    wxPaintDC dc(m_bitmap_panel );

Once that works, put the controls and the bitmap panel into sizers, so you can get rid of the hard-coded values.

[NonoNano]

Once again thank you very much for the support! I'm getting closer to the final result but still not achieved it. Now the function OnPaint is called each time the frame is grabbed. The problem is that, even if the function is called and seems to work fine, the panel doesn't change its aspect. I think that the buffer is correctly initialized, namely to me it is not a matter of 'not grabbed frame'.
This is how it looks like the function now:

Code: Select all

void CameraGUIForm::OnPaint(wxPaintEvent& event)
{
    wxPaintDC dc(m_bitmap_panel);

    wxCriticalSectionLocker lock(m_bitmap_cs);

    if(m_bitmap != NULL && m_bitmap->IsOk() )
    {
        int bw = m_bitmap->GetWidth();
        int bh = m_bitmap->GetHeight();

        dc.DrawBitmap(*m_bitmap, 0, 0, false);

        dc.SetBackground(*wxBLACK_BRUSH);
        if( dc.GetSize().x>bw )
    {
            dc.SetClippingRegion(bw, 0,dc.GetSize().x-bw, dc.GetSize().y);
            dc.Clear();
            dc.DestroyClippingRegion();
        }

        if( dc.GetSize().y>bh )
    {
            dc.SetClippingRegion(0, bh, dc.GetSize().x, dc.GetSize().y-bh );
            dc.Clear();
        }
    }
}

I see that DrawBitmap and SetBackground are always called.
The weird thing is that if I modify the function like this:

Code: Select all

void CameraGUIForm::OnPaint(wxPaintEvent& event)
{
    wxPaintDC dc(m_bitmap_panel);
    dc.SetBackground(*wxBLACK_BRUSH);

    wxCriticalSectionLocker lock(m_bitmap_cs);
}

I'd expect that, at least, the background color of my panel becomes black..actually it doesn't change..
Everytime I feel I'm reaching my target I always encounter something that makes me miss it..unfortunately it is the first time I use wxWidgets and C++ therefore I'm not independent in the task of finding a solution..

[UPDATE2]
if I call at the end of OnPaint dc.Clear the black background is now rendered..I'm starting to think there is something wrong in the function that converts the buffer to a wxBitmap
[/UPDATE2]

[UPDATE1]
if I try to draw a circle and a text on the panel through OnPaint it works..there must be something else I'm missing..
[/UPDATE1]

[doublemax]

Too bad i can't test this myself.

the panel doesn't change its aspect

What do you mean by that?

I'm starting to think there is something wrong in the function that converts the buffer to a wxBitmap

Check if the code passes this line or returns:

Code: Select all

        PixelData bmdata(*m_bitmap);
        if(!bmdata) return;

Try to add some code that saves the bitmap to a file and check if it looks right.
Somewhere in the initialization code of your program, add:

Code: Select all

::wxInitAllImageHandlers();

Then, in the paint event handler, before drawing the bitmap:

Code: Select all

m_bitmap->SaveFile("c:\\fullpathname.png", wxBITMAP_TYPE_PNG);

One more thing to try:
Use wxNativePixelData instead of wxAlphaPixelData. And change the bitmap depth from 32 to 24 in this line:

Code: Select all

m_bitmap = new wxBitmap(w, h, 32);

[NonoNano]

I meant that the panel seemed not to change its aspect (background color) even tought onpaint was called but then I realized that I had to call Clear..this changed the color of my panel

Code: Select all

        PixelData bmdata(*m_bitmap);
        if(!bmdata) return;

it passes the code above without returning.
The bitmap image saved is like an 'empty' image even tough it has the correct format (find attached).

For testing purposed I cast the buffer not to the type that you used in your code (unsigned char *pImageBuffer) but to the type related with the examples coming with the camera libraries (const uint8_t *pImageBuffer).

const uint8_t *pImageBuffer = (uint8_t *) ptrGrabResult->GetBuffer();
cout << "Gray value of first pixel: " << (uint32_t) pImageBuffer[0] << endl << endl;

I don' think this could be the problem since the application doesn't work in both cases.

I'm working on linux arm..could it affect something?

[doublemax]

Did you try the change with the wxNativePixelData?

Is the documentation for CGrabResultPtr publicly available anywhere? I only found a download for the whole SDK that required registering. Based on some samples i found, there should be a method to check the data format of the image data.

Code: Select all

imgdata += w; //step tra una riga e la successiva

This line should most likely be:

Code: Select all

imgdata += w * 3; //step tra una riga e la successiva

[NonoNano]

I have read that the pixel size that I'm using now is 8 bit therefore probably I should not multiply by 3..anyways I'll give it a try!

https://www.dropbox.com/s/4a7jcyvf5jf8a ... e.rar?dl=0

The above link is the documentation of my libraries! The starting page is: ".\share\doc\C++\index.html"

The pixel format I'm using is called "YCbCr422_8"..I found that the pixel size is "Bpp16" but multiplying by 2 is still not working

EDIT: multiplying by 3 a get error at runtime

[doublemax]

I have read that the pixel size that I'm using now is 8 bit therefore probably I should not multiply by 3..anyways I'll give it a try!

Are you getting color or grey images from the camera. If it's color, you have to multiply by 3.

But even if that was wrong, you'd still see "something" moving on screen, just garbled.

[NonoNano]

It is color camera, now I changed pixel type to:

Pixel format: RGB8
Pixel size: Bpp24

and I multiply by 3..still nothing moving!

[doublemax]

Did you make the change regarding wxNativePixelData from above?
viewtopic.php?p=183126#p183126

After looking into the image you posted, (which was not completely empty, just looked empty), i think it's important.

Just in case the pixel access is messed up, try this alternative (but slower) version for toWXBitmap:

Code: Select all

void CameraGUIForm::toWXBitmap(unsigned char * imgdata, int w, int h)
{
  wxImage img( w, h );
  memcpy( img.GetData(), imgdata, w*h*3 );
  if( m_bitmap == NULL )
  {
    m_bitmap = new wxBitmap( w, h, 24 );
  }
  *m_bitmap = wxBitmap( img );
}

( As i can't test this, it's all written blindly. Keep that in mind.)

[NonoNano]

I'getting closer and closer to the result!
Now if I save the image I get what you can see in the attached file (which is correct!).
The only thing is that the panel remains 'blank'.

This is how it looks like the OnPaint function now:

Code: Select all

void CameraGUIForm::OnPaint(wxPaintEvent& event)
{
    wxPaintDC dc(m_bitmap_panel);
    //dc.SetBackground(*wxWHITE_BRUSH);

    wxCriticalSectionLocker lock(m_bitmap_cs);

    uint8_t uio = firstGrayValue;
    if(m_bitmap != NULL && m_bitmap->IsOk() )
    {
        int bw = m_bitmap->GetWidth();
        int bh = m_bitmap->GetHeight();

        //m_bitmap->SaveFile("/home/pi/Desktop/img.png",wxBITMAP_TYPE_PNG);
        dc.DrawBitmap(*m_bitmap, 0, 0, false);
        dc.Clear();


        if( dc.GetSize().x>bw )
    {
            dc.SetClippingRegion(bw, 0,dc.GetSize().x-bw, dc.GetSize().y);
            dc.Clear();
            dc.DestroyClippingRegion();
        }

        if( dc.GetSize().y>bh )
    {
            dc.SetClippingRegion(0, bh, dc.GetSize().x, dc.GetSize().y-bh );
            dc.Clear();
        }

    }

    //dc.DrawText(wxT("testing"),40,60);
    //dc.DrawCircle(wxPoint(200,100),25);
    //dc.SetBackground(wxColor(*wxBLACK));

}