librelist archives

« back to archive

Reintroduce the idea of unique identifiers for a job, specifically for failed jobs

Reintroduce the idea of unique identifiers for a job, specifically for failed jobs

From:
Tony Pitale
Date:
2013-07-19 @ 18:55
As resque is right now, when a job needs to be removed from a queue the
queue is either emptied removing all jobs, or, an ID is assigned to it in
the web interface for the purpose of deletion.

However, this id is temporary. It changes as the failure queue is changed.
It appears to be a simple sequential integer in the full array of failed
jobs.

This makes it very difficult to work with the failed queue programmatically
(to select, requeue, and remove jobs based on the arguments passed in
becomes nearly impossible to do).

Resque::Failure.remove takes only an id. There does not appear to be a
better way at this time.

This has been brought up before in http://github.com/resque/resque/issues/6,
but the solution presented does not seem to help in the case of simple
deletions.

This may also be closed because it's outside the scope of what resque wants
to accomplish, or maybe work is already being done on this.

Thanks for any information, perspectives, and opinions on this. If it's
something that @steveklabnik and other maintainers like, I'll put together
a PR.

Re: [resque] Reintroduce the idea of unique identifiers for a job, specifically for failed jobs

From:
Emil Kampp
Date:
2013-07-19 @ 19:46
In any case, if you have an idea, I would be interested. I'm gonna need to
work with the failed queue very soon, and I would love at least the shared
insight into this area of the resque. :)  


Best regards,

Emil Kampp
Founder

emil@kampp.me | +45 42427176 | http://emil.kampp.me


On Friday, July 19, 2013 at 8:55 PM, Tony Pitale wrote:

> As resque is right now, when a job needs to be removed from a queue the 
queue is either emptied removing all jobs, or, an ID is assigned to it in 
the web interface for the purpose of deletion.
> 
> However, this id is temporary. It changes as the failure queue is 
changed. It appears to be a simple sequential integer in the full array of
failed jobs.
> 
> This makes it very difficult to work with the failed queue 
programmatically (to select, requeue, and remove jobs based on the 
arguments passed in becomes nearly impossible to do).
> 
> Resque::Failure.remove takes only an id. There does not appear to be a 
better way at this time.
> 
> This has been brought up before in 
http://github.com/resque/resque/issues/6, but the solution presented does 
not seem to help in the case of simple deletions.
> 
> This may also be closed because it's outside the scope of what resque 
wants to accomplish, or maybe work is already being done on this.
> 
> Thanks for any information, perspectives, and opinions on this. If it's 
something that @steveklabnik and other maintainers like, I'll put together
a PR.
> 

Re: [resque] Reintroduce the idea of unique identifiers for a job, specifically for failed jobs

From:
Tony Pitale
Date:
2013-07-19 @ 20:02
Assuming that the list of failed jobs is ordered as integers from "top" to
"bottom" and that new failed jobs arrive at the bottom: if you reverse the
order of the list of integer "ids" and delete from the bottom, you won't
change the value of the integer id of the items you care to delete above.
That's how I handled it. I also had to ask everyone else to temporarily
abstain from removing items from the list in case they removed items above
the ones I was deleting.

Here's how I imagine it:

1. JobA
2. JobB
3. JobE
4. JobA
5. JobB
6. JobB
7. JobC
8. JobB

Remove all of the JobBs. That is jobs (currently) in positions: [2,5,6,8].

If I start with Resque::Failure.remove(2), it appears that everything below
moves up in number. So JobE will now be in position 2. In addition, the
next JobB, that was in position 5, is now in position 4; and 6 is now in 5.
By The time I got to 8, there wouldn't be an 8!

To do what I did, reverse the list. Delete 8, then 6, 5, and 2.

It's precarious, and I'm not certain that my assumptions are correct.

- Tony

On Fri, Jul 19, 2013 at 3:46 PM, Emil Kampp <emil@kampp.me> wrote:

>
> In any case, if you have an idea, I would be interested. I'm gonna need to
> work with the failed queue very soon, and I would love at least the shared
> insight into this area of the resque. :)
>
>
> Best regards,
>
> Emil Kampp
> Founder
>
> emil@kampp.me | +45 42427176 | http://emil.kampp.me
>
> On Friday, July 19, 2013 at 8:55 PM, Tony Pitale wrote:
>
> As resque is right now, when a job needs to be removed from a queue the
> queue is either emptied removing all jobs, or, an ID is assigned to it in
> the web interface for the purpose of deletion.
>
> However, this id is temporary. It changes as the failure queue is changed.
> It appears to be a simple sequential integer in the full array of failed
> jobs.
>
> This makes it very difficult to work with the failed queue
> programmatically (to select, requeue, and remove jobs based on the
> arguments passed in becomes nearly impossible to do).
>
> Resque::Failure.remove takes only an id. There does not appear to be a
> better way at this time.
>
> This has been brought up before in
> http://github.com/resque/resque/issues/6, but the solution presented does
> not seem to help in the case of simple deletions.
>
> This may also be closed because it's outside the scope of what resque
> wants to accomplish, or maybe work is already being done on this.
>
> Thanks for any information, perspectives, and opinions on this. If it's
> something that @steveklabnik and other maintainers like, I'll put together
> a PR.
>
>
>