librelist archives

« back to archive

How-to monitor specific jobs with Resque?

How-to monitor specific jobs with Resque?

From:
Jonathan
Date:
2009-11-11 @ 00:58
First, thanks for another great contribution from the folks at Github!


I'd like to try-out Resque on a project, but have a few questions that  
I couldn't figure out from my first pass at the docs and examples.
I realize I may just be thinking of the problem all wrong, so I would  
appreciate some wise insights ;)

Scenario:
Our customers submit requests for a reports. In the background we:
(a) generate a csv/xls report
(b) zip the file
and
(c) store the file in s3 ...sending the resulting (secure) url to our ? 
MySQL database?

At any point after submitting job(s), our customers may return to our  
webpage in order to view the status of their request(s) and/or to  
click on the download url.

Questions:
(1) I didn't see any examples where Resque's asynchronous request  
method returned a unique "job-id"
I also got the impression from the examples given that Resque is being  
used for "generic" jobs (i.e. where nobody cares to watch any  
particular "jobt", just that the queues are getting emptied).

So, is there a best-practice for what I am trying to do? Do I need to  
just name each "request" (or ?queue?) uniquely (i.e. one queue per  
customer)?! So that I can present a report to the customer on the  
status of her jobs?

(2) Is it best to create my job-request-Class's "process" method to  
manage all the steps of a request, or is there some other approach to  
managing a synchronous list of background sub-tasks: zip-job at the  
end of the background generate-report job, etc... ?

Thanks in advance for any tips/answers.

Cheers,
Jonathan

Re: How-to monitor specific jobs with Resque?

From:
Chris Wanstrath
Date:
2009-11-11 @ 01:40
On Tue, Nov 10, 2009 at 4:58 PM, Jonathan <jonathan@singlefeed.com> wrote:

> Scenario:
> Our customers submit requests for a reports. In the background we:
> (a) generate a csv/xls report
> (b) zip the file
> and
> (c) store the file in s3 ...sending the resulting (secure) url to our ?MySQL
> database?
> At any point after submitting job(s), our customers may return to our
> webpage in order to view the status of their request(s) and/or to click on
> the download url.
>
> Questions:
> (1) I didn't see any examples where Resque's asynchronous request method
> returned a unique "job-id"

You should handle this in your app. Generate a unique ID when
submitting the job. Have the job update memcached / redis / the DB
using that ID.

Using this method you can easily communicate between the backend and frontend.

> (2) Is it best to create my job-request-Class's "process" method to manage
> all the steps of a request, or is there some other approach to managing a
> synchronous list of background sub-tasks: zip-job at the end of the
> background generate-report job, etc... ?

This is up to you. It's usually pretty nice to break jobs into the
smallest possible chunks and spawn off new jobs when you need things
to happen in order, but really it's whatever you're comfortable with.

-- 
Chris Wanstrath
http://github.com/defunkt