librelist archives

« back to archive

resque worker load distribution not fair

resque worker load distribution not fair

From:
pratik khadloya
Date:
2011-03-02 @ 22:31
Hello,

Can anyone of you please have a look at the thread

http://stackoverflow.com/questions/5171786/resque-worker-load-distribution-not-fair


Thanks,
Pratik

Re: [resque] resque worker load distribution not fair

From:
Chris Wanstrath
Date:
2011-03-03 @ 01:50
On Wednesday, March 2, 2011 at 2:31 PM, pratik khadloya wrote:
Can anyone of you please have a look at the thread 
http://stackoverflow.com/questions/5171786/resque-worker-load-distribution-not-fair
> It seems like some of the workers are taking longer to process the jobs 
themselves. Or maybe they hit a bug and got stuck. 

It'd be cool to profile how long jobs take. I don't see any plugins for 
that in https://github.com/defunkt/resque/wiki/plugins though 

Re: [resque] resque worker load distribution not fair

From:
pratik khadloya
Date:
2011-03-03 @ 16:06
Thanks Chris, i'll check that out.
Another finding, when i run the workers for a second run (after all the jobs
of the 1st run are done), there is equitable distribution of tasks and all
the workers work correctly.
So, looks like during the first run some of the workers were left craving
for a connection or they got a connection but just couldnt  use it. But AR
should have cried foul (connection timeout or something).
No AR errors for the looong running jobs.

I will try to implement ActiveRecord::Base.clear_active_connections! also.

-Pratik


On Wed, Mar 2, 2011 at 7:50 PM, Chris Wanstrath <chris@ozmm.org> wrote:

> On Wednesday, March 2, 2011 at 2:31 PM, pratik khadloya wrote:
> Can anyone of you please have a look at the thread
> 
http://stackoverflow.com/questions/5171786/resque-worker-load-distribution-not-fair
> > It seems like some of the workers are taking longer to process the jobs
> themselves. Or maybe they hit a bug and got stuck.
>
> It'd be cool to profile how long jobs take. I don't see any plugins for
> that in https://github.com/defunkt/resque/wiki/plugins though
>