Showing images from a video stream

[doublemax]

Which of the changes i suggested did you make to get that result?

Did you confirm that the dc.DrawBitmap() call is hit?

Try removing the part after that call completely.

Code: Select all

        dc.SetBackground(*wxBLACK_BRUSH);
        if( dc.GetSize().x>bw )
    {
            dc.SetClippingRegion(bw, 0,dc.GetSize().x-bw, dc.GetSize().y);
            dc.Clear();
            dc.DestroyClippingRegion();
        }

        if( dc.GetSize().y>bh )
    {
            dc.SetClippingRegion(0, bh, dc.GetSize().x, dc.GetSize().y-bh );
            dc.Clear();
        }
    }

Its only purpose it to clear the area next to and below the bitmap, so you don't get areas with "undefined" content.

[doublemax]

Code: Select all

        dc.DrawBitmap(*m_bitmap, 0, 0, false);
        dc.Clear();

The clear() erases everything you just drew

[NonoNano]

Now it works!!! Thank you very much! I'm extremely happy! The only thing is that the colors are not correct: what is blue is seen as orange..do you think I have confused RGB components?

[doublemax]

Now it works!!! Thank you very much! I'm extremely happy! The only thing is that the colors are not correct: what is blue is seen as orange..do you think I have confused RGB components?

Yes, sorry. I just noticed that my old code works with BGR data. Just swap Red() and Blue() in this code part:

Code: Select all

dst.Blue() = *imgdata_y++;
dst.Green() = *imgdata_y++;
dst.Red() = *imgdata_y++;

[NonoNano]

Which of the changes i suggested did you make to get that result?

I've done both of them! For sure the former

Code: Select all

m_bitmap = new wxBitmap( w, h, 32 );

was not correct since my bit size is 24.
Next week I will try to set the "wxAlphaPixelData" object type and see if it still works..then I will post the result of this trial!

Just one more question: is memory already correctely managed? Do I have to release resources held by "m_bitmap" to avoid memory leaking? Thank you again!!!

[doublemax]

Next week I will try to set the "wxAlphaPixelData" object type and see if it still works..then I will post the result of this trial!

These changes can't be made individually. 32bit = wxAlphaPixelData, 24 Bit = wxNativePixelData. If the latter works, everything is fine.

is memory already correctely managed? Do I have to release resources held by "m_bitmap" to avoid memory leaking?

Yes. it should be deleted in the CameraGUIForm destructor. Alternatively, change the type of the variable from (wxBitmap*) to wxBitmap. You just have to adjust all places where it's used to the different syntax.

[NonoNano]

Hello,

what achieved now seemed to work but on the long run I think there are some problems:

in the secondary thread that is in charge of grabbing images sometimes I get an error called "stream out of sync" that results in some frame lost. To me this error is dued to the fact that inside that thread I'm doing too much stuff: namely there is a critical section inside where I'm doing the conversion (that takes time I suppose).
I would like to try to go back to my previous approach: the secondary thread acquires the frame then, through an event, notifies the primary thread of the grabbed frame and it is now the main thread that is in charge of converting the image. The problem was that I couldn't pass the frame to the main thread through the event. In particular I think that the buffer is released right after the secondary thread as finisched (thus the main thread cannot find anything). I would like to make a copy of the buffer and check if that works but I'm making some mistake:

retrieve the buffer

Code: Select all

pImageBuffer = (unsigned char *) ptrGrabResult->GetBuffer();

find out the length of the array and make the copy

Code: Select all

int myLength = sizeof(pImageBuffer)/sizeof(*pImageBuffer);
memcpy(pImageBuffer,copyImageBuffer, myLength*4);

I multiply by 4 because I think it is the size of unsigned char..but it doesn't work! Thanks for any help!

[doublemax]

Code: Select all

int myLength = sizeof(pImageBuffer)/sizeof(*pImageBuffer);
memcpy(pImageBuffer,copyImageBuffer, myLength*4);

Use width*height*3 for the number of bytes. copyImageBuffer of course must be big enough too.

However, if the frames come in faster than you can process them, you have to skip frames somewhere.

[NonoNano]

Again thanks to your suggestions I'm getting close to what I want to achieve:

in the secondary thread (grabbing thread) I'm doing this:

Code: Select all

                unsigned char *copyImageBuffer = new unsigned char[width*height*3];
                memcpy(copyImageBuffer,pImageBuffer,width*height*3);

                wxCommandEvent evt(wxEVT_GRABBED);
                evt.SetExtraLong((long) copyImageBuffer);
                wxPostEvent(handler, evt);

while inside the event 'OnGrabbed' that is executed in the main thread I'm doing this:

Code: Select all

    unsigned char *copyImageBuffer = (unsigned char *) event.GetExtraLong();
    this->toWXBitmap(copyImageBuffer, 640, 480);

    this->Refresh();

and it works from the point of view of the visualization. The problem is that the allocated resources (copyImageBuffer) are never released and I will rapidly go out of memory. How and where should I realize the buffer? If I do it inside 'toWXBitmap' at the end I get runtime errors

[doublemax]

Freeing the buffer after toWXBitmap should be safe. Remember that as you have allocated it with "new []", you also have to delete it with "delete []".

However for performance reasons i would suggest a static buffer.

Even better would be two buffers that you alternate between, one can be written into while the other is getting converted into a bitmap. This would reduce the chance that one thread is waiting for the other to finish.

[NonoNano]

Ok I managed to solve the problem like this:

Code: Select all

    unsigned char *copyImageBuffer = (unsigned char *) event.GetExtraLong();

    if(previousImageBuffer)
    {
        delete[] previousImageBuffer;
    }

    previousImageBuffer = copyImageBuffer;
    this->toWXBitmap(copyImageBuffer, 640, 480);
    this->Refresh();

therefore I'm keeping a pointer to the previous buffer that I'm deleting before assigning to it the new buffer. This solves the memory problem!
Unfortunately I have another issue: the CPU usage of my application grows also to 110%! This results in an extremely inefficient program that blocks the whole OS..is there e way to obtain better performance? Thank you!

EDIT:

from the code above if I comment "this->Refresh();" I pass from 110% CPU usage to 75%. If I also comment 'this->toWXBitmap(copyImageBuffer, 640, 480);' I pass from 75% to 18%

[doublemax]

On which hardware do you run this? toWXBitmap copies the data pixel by pixel which is very slow, but it shouldn't use that much cpu power unless you're running on very slow machine. But except for some dirty tricks, there is no "clean" way to speed this up.

Can you upload the current version of the source again?

Which framerate are you aiming at? Is it important to display every frame or would 25 or 30 fps be enough?

[NonoNano]

Hello,

the hardware I have to use is very cheap: its a raspberry pi. The thing that makes me think there should be a solution is that if I use the official program (that comes with the camera) it works fine and occupies only 40% of the cpu. I'm aiming at 20fps and this is the framerate I should have now. Of course! Attached you can find the current code!

EDIT: could it be an idea to use the constructor of wxBitmap instead of the function toWXBitmap?

EDIT 2: the weird thing is that it seems that the CPU usage increases with time! It starts from 90% and after some minutes it arrives at 110%..

http://docs.wxwidgets.org/3.0/classwx_b ... c1de516f21

[doublemax]

I'm afraid i have reached the limits of my wisdom. I don't work under Linux and don't know anything about the internal structures of a wxBitmap under GTK. But looking into the source code, there is a lot of pixel copying going on which does not look optimized at all.

At this stage i would even consider using OpenGL for the bitmap display.

[NonoNano]

So you don't think it is possible to optimize somehow "toWXBitmap"?
I'll try to take a look to OpenGL but it is probably a substantial modification of my project and they want me to finish this application as soon as possible..

EDIT: I confirm that that function seems to be the bottleneck..commenting it results in 30% CPU usage