librelist archives

« back to archive

Retrying thousands of jobs

Retrying thousands of jobs

From:
Karl Baum
Date:
2010-07-15 @ 23:02
I have 2000 jobs on my queue i would like to retry and i would rather  
not have to click retry on each one of them within resque web :-).   
Any idea on the best way to do this?

thx.

-karl

Re: [resque] Retrying thousands of jobs

From:
Tony Arcieri
Date:
2010-07-15 @ 23:07
On Thu, Jul 15, 2010 at 5:02 PM, Karl Baum <karl.baum@gmail.com> wrote:

> I have 2000 jobs on my queue i would like to retry and i would rather
> not have to click retry on each one of them within resque web :-).
> Any idea on the best way to do this?


You can retry a job in Ruby code by calling the enqueue method again with
the same parameters you used to launch the job initially.

-- 
Tony Arcieri
Medioh! A Kudelski Brand

Re: [resque] Retrying thousands of jobs

From:
Karl Baum
Date:
2010-07-15 @ 23:27
Given the number of jobs i have failed, i guess i would want to  
iterate over the failed jobs and requeue them.  I think i would also  
want to remove the old failed jobs because i won't know which ones  
actually went to success eventually and which ones are still failed.   
Is this how others handle this situation?

Thanks for your help!

-karl

On Jul 15, 2010, at 7:07 PM, Tony Arcieri wrote:

> On Thu, Jul 15, 2010 at 5:02 PM, Karl Baum <karl.baum@gmail.com>  
> wrote:
> I have 2000 jobs on my queue i would like to retry and i would rather
> not have to click retry on each one of them within resque web :-).
> Any idea on the best way to do this?
>
> You can retry a job in Ruby code by calling the enqueue method again  
> with the same parameters you used to launch the job initially.
>
> -- 
> Tony Arcieri
> Medioh! A Kudelski Brand

Re: [resque] Retrying thousands of jobs

From:
Tony Arcieri
Date:
2010-07-15 @ 23:48
On Thu, Jul 15, 2010 at 5:27 PM, Karl Baum <karl.baum@gmail.com> wrote:

> Given the number of jobs i have failed, i guess i would want to iterate
> over the failed jobs and requeue them.  I think i would also want to remove
> the old failed jobs because i won't know which ones actually went to success
> eventually and which ones are still failed.  Is this how others handle this
> situation?
>

As a quick hack you could iterate over the failed jobs and retry them, sure.
 Personally I prefer to track the job lifecycle (a.k.a. workflow) in the
database and periodically retry jobs which have failed automatically, up to
a certain threshhold of retries, at which point I consider the job
permanently failed.

I'd like to put that all together and release it as a gem at some point, but
given how many other projects I have going on I don't have time.  I'll
continue to dream the dream, though:

http://github.com/tarcieri/foreman

-- 
Tony Arcieri
Medioh! A Kudelski Brand