Page 1 of 1

Passing Variable Size Data to Threads

Posted: Tue Jan 12, 2010 2:29 pm
by Kerry
Hello,

I have a design question I am hoping you can help me resolve. I have an application with a GUI thread and several "worker threads" in a pool, to which I pass jobs by adding jobs to a queue (my architecture closes resembles the complete example here: http://wiki.wxwidgets.org/Inter-Thread_ ... munication).

I would like to be able to pass different kinds of jobs to the threads, but each type of job requires different inputs, so when I created my tJOB class, I took the quick and dirty approach, and I can create the job with as many or as few arguments as I need. For example, my tJOB constructor looks like this:

Code: Select all

tJOB::tJOB(tCOMMANDS cmd, int forJobType1, bool forJobType2, wxString forJobType3);
Everything is working well in my test application, so I've started cleaning everything up while I integrate it into my real project. What I would like to do, would be to create a base class tJOB, which can be used on its own for starting/stopping threads, but for the number crunching, I'd pass it an object that is derived from a tJOB, that contains only the information that is relevant for that specific job. So I was imagining something like this (where m_pQueue::AddJob still takes a tJOB argument, so it can be passed any type of job, base or derived):

Code: Select all

tJOB *newJob = new derivedJOB();
m_pQueue->AddJob(*newJob);
Of course this doesn't work, because the sizes of the base and derived classes are not the same, so after I copy the job in the queue, to another instance of a tJOB in my worker thread, I can't cast it to the correct type of job and still have access to the data that would be contained the derived class.

My next thought was to use a union, but I'm being foiled by the 'union members may not have non-trivial constructors' as some of the data objects that I'm passing to the thread have non-trivial constructors.

Hopefully this isn't too confusing - I think it illustrates what I was hoping to accomplish. What do you do to solve this problem? The two best options I can come up with are using separate queues for each different type of job (which I don't particularly want to do), or have a tJOB constructor that takes enough arguments to accommodate all of my job types, and stores all of the data in its own members, even if some of it is garbage, and won't be used for that particular job.

Any advice is welcome!

Thanks,

Kerry

Posted: Tue Jan 12, 2010 3:00 pm
by JimFairway
Hi Kerry,

How about instead of deriving a new class from tJOB, create another data class, e.g BaseDataClass, then derive different data classes from that.

Then include a pointer to a BaseDataClass object in tJOB.
When you create a tJOB, create the appropriate data object and set the pointer to that object.

tJOB will always be the same size that way, but the data derived from BaseDataClass can change to suit what you're trying to pass.


Hope that's clear and useful.

Jim

Posted: Tue Jan 12, 2010 6:03 pm
by Kerry
Thanks, Jim - this is another option that I hadn't thought of. This would work, but I'd like to avoid passing pointers if possible, just to make memory management easier.

I suppose my example above should have looked like this:

Code: Select all

class derivedJOB : public tJOB
{
};

void someFunction (void)
{
    derivedJOB newJob;

    // assign data to newJob

    m_pQueue->AddJob(newJob);
}
If there's no good way to do this, then I will resort to passing a pointer to the data, but this would not be my preferred choice.

Thanks again for the help!

-Kerry

EDIT: Couple of typos

Posted: Tue Jan 12, 2010 6:46 pm
by Frank
Have you the option to use boost? Or the soon-to-be-standard tr1?

With function and bind your job is really quite easy to implement. Just pass function<> objekts wich contains the parameters the function needs.

Code: Select all

void function1 (const param& p1) { ... does domething with p1}
void function2 (const param& p1, const otherParam& p2) { ... does something with p1 and p2 }

// This is your thread-function:
void Thread (function<void()> func)
{
   func();
}

// And this is, how you put Jobs into the queue.

MyQueue.push_front(bind(function1, p1));
MyQueue.push_front(bind(function2, p1, p2));

// I don't know how you start your Threads (dont't like wxThread, never used it). 
// For my Example i Use tr1::thread

function<void()> func = *MyQueue.pop_back();
thread(Thread, func);

Also, look up the command pattern.

Posted: Wed Jan 13, 2010 3:24 pm
by Kerry
Thanks for the replies.

I have considered using boost in the past, but I am not currently using it in this project, so to avoid adding another dependency to the project, I decided to try out Jim's suggestion.

Turns out it's pretty clean - I made the pointer to the data NULL for jobs that don't require data, then after I process the job I can safely delete the data, which doesn't do anything if the pointer is NULL.

Works beautifully!

Thanks for the suggestions, I appreciate the help.

-Kerry