librelist archives

« back to archive

Are zombified jobs ever requeued?

Are zombified jobs ever requeued?

From:
Adam Fields
Date:
2010-11-08 @ 15:24
Looking through the code, it seems that a job is removed from the
queue when the worker reserves it, and if for some reason a worker
reserves a job and then doesn't actually perform it (e.g.: if it
crashes), that job is gone and never requeued. Is that correct?

As a corollary question, there's similarly no deletion required for
successful jobs? All the worker needs to do is reserve the job,
perform it, log the appropriate thing, and then exit?


-- 
				- Adam
----------
If you liked this email, you might also like:
"How to use MongoDB to collect summary stats" 
-- http://workstuff.tumblr.com/post/1270812403
"Sous Vide Poached Egg" 
-- http://www.aquick.org/blog/2010/10/06/sous-vide-poached-egg/
"Sous Vide Poached Egg" 
-- http://www.flickr.com/photos/fields/5056726861/
"fields: @vsbuffalo sub-$15, I usually stick with Rioja." 
-- http://twitter.com/fields/statuses/1473249527463936
----------
** I design intricate-yet-elegant processes for user and machine problems.
** Custom development project broken? Contact me, I can help.
** Some of what I do: http://workstuff.tumblr.com/post/70505118/aboutworkstuff

[ http://www.adamfields.com/resume.html ].. Experience
[ http://www.morningside-analytics.com ] .. Latest Venture
[ http://www.confabb.com ] ................ Founder

Re: [resque] Are zombified jobs ever requeued?

From:
Tony Arcieri
Date:
2010-11-08 @ 17:34
On Mon, Nov 8, 2010 at 8:24 AM, Adam Fields <resque534355@aquick.org> wrote:

>
> Looking through the code, it seems that a job is removed from the
> queue when the worker reserves it, and if for some reason a worker
> reserves a job and then doesn't actually perform it (e.g.: if it
> crashes), that job is gone and never requeued. Is that correct?
>

Yes. If the job raises a Ruby exception though it's logged to the "failed"
queue. That won't help you if the worker loses power or suffers a disk crash
though.


> As a corollary question, there's similarly no deletion required for
> successful jobs? All the worker needs to do is reserve the job,
> perform it, log the appropriate thing, and then exit?


Correct. Once a worker starts processing a job it's removed from the queue.

I've been working on a gem to manage the job lifecycle, or more specifically
the lifecycle of workflows which are modeled as finite state machines whose
transitions are jobs. The goal is for it to automatically retry failed jobs
until a workflow reaches completion (this assumes all your jobs are
idempotent). Unfortunately, I've never found the time or motivation to
actually finish it:

https://github.com/tarcieri/resque-workflow

--
Tony Arcieri
Medioh! A Kudelski Brand

Re: [resque] Are zombified jobs ever requeued?

From:
Adam Fields
Date:
2010-11-08 @ 18:29
On Mon, Nov 08, 2010 at 10:34:29AM -0700, Tony Arcieri wrote:
> Correct. Once a worker starts processing a job it's removed from the queue.
> 
> I've been working on a gem to manage the job lifecycle, or more specifically
> the lifecycle of workflows which are modeled as finite state machines whose
> transitions are jobs. The goal is for it to automatically retry failed jobs
> until a workflow reaches completion (this assumes all your jobs are
> idempotent). Unfortunately, I've never found the time or motivation to
> actually finish it:
> 
> https://github.com/tarcieri/resque-workflow

Thanks.

In this case, all of the state is stored with the target, so if it
isn't properly processed, it will eventually be rescheduled.

But I have my own worker process, and I wanted to make sure it was
doing the right thing with respect to making sure that jobs were
getting properly cleared from the queue (and wouldn't pop up again
later if they weren't explicitly deleted).

The documentation about how to run your own worker (especially outside
of a rails context) is pretty sparse - I'll probably write something
up at some point.

-- 
				- Adam
----------
If you liked this email, you might also like:
"How to use MongoDB to collect summary stats" 
-- http://workstuff.tumblr.com/post/1270812403
"Sous Vide Poached Egg" 
-- http://www.aquick.org/blog/2010/10/06/sous-vide-poached-egg/
"Sous Vide Poached Egg" 
-- http://www.flickr.com/photos/fields/5056726861/
"fields: @vsbuffalo sub-$15, I usually stick with Rioja." 
-- http://twitter.com/fields/statuses/1473249527463936
----------
** I design intricate-yet-elegant processes for user and machine problems.
** Custom development project broken? Contact me, I can help.
** Some of what I do: http://workstuff.tumblr.com/post/70505118/aboutworkstuff

[ http://www.adamfields.com/resume.html ].. Experience
[ http://www.morningside-analytics.com ] .. Latest Venture
[ http://www.confabb.com ] ................ Founder

Re: [resque] Are zombified jobs ever requeued?

From:
Chris Wanstrath
Date:
2010-11-08 @ 18:38
On Mon, Nov 8, 2010 at 10:29 AM, Adam Fields <resque534355@aquick.org> wrote:

> The documentation about how to run your own worker (especially outside
> of a rails context) is pretty sparse - I'll probably write something
> up at some point.

True. This might help:

https://github.com/defunkt/resque/blob/master/lib/resque/tasks.rb

By default Resque knows nothing about Rails — Rails is loaded by way
of the "environment" task.