librelist archives

« back to archive

problems loading worker from gem

problems loading worker from gem

From:
Adam Tucker
Date:
2009-11-25 @ 21:35
I've been using Resque on our development servers without a hitch, starting
the workers with "QUEUE=queue_name rake environment resque:work"  The
problem is, I can't figure out how to get them to load the production
environment.  When I start them the same way on the production servers, all
jobs fail with authentication errors because they're trying to access the
development database.

The docs seem to indicate that I don't need 'environment' when using the gem
instead of the plugin, but when I run "QUEUE=queue_name rake resque:work" it
fails with:

rake aborted!
uninitialized constant Resque

I'm using Rails 2.0.2 - any thoughts?

Thanks!

Re: problems loading worker from gem

From:
JayTeeWork
Date:
2009-11-25 @ 21:44
Just pass RAILS_ENV=production


Sent from my mobile

On Nov 25, 2009, at 1:35 PM, Adam Tucker <adam.j.tucker@gmail.com>  
wrote:

> I've been using Resque on our development servers without a hitch,  
> starting the workers with "QUEUE=queue_name rake environment  
> resque:work"  The problem is, I can't figure out how to get them to  
> load the production environment.  When I start them the same way on  
> the production servers, all jobs fail with authentication errors  
> because they're trying to access the development database.
>
> The docs seem to indicate that I don't need 'environment' when using  
> the gem instead of the plugin, but when I run "QUEUE=queue_name rake  
> resque:work" it fails with:
>
> rake aborted!
> uninitialized constant Resque
>
> I'm using Rails 2.0.2 - any thoughts?
>
> Thanks!

Re: problems loading worker from gem

From:
Adam Tucker
Date:
2009-11-25 @ 21:49
I tried that, but turns out I just put it in the wrong spot. Thanks!

On Wed, Nov 25, 2009 at 4:44 PM, JayTeeWork <jonathan@singlefeed.com> wrote:

> Just pass RAILS_ENV=production
>
>
> Sent from my mobile
>
> On Nov 25, 2009, at 1:35 PM, Adam Tucker <adam.j.tucker@gmail.com>
> wrote:
>
> > I've been using Resque on our development servers without a hitch,
> > starting the workers with "QUEUE=queue_name rake environment
> > resque:work"  The problem is, I can't figure out how to get them to
> > load the production environment.  When I start them the same way on
> > the production servers, all jobs fail with authentication errors
> > because they're trying to access the development database.
> >
> > The docs seem to indicate that I don't need 'environment' when using
> > the gem instead of the plugin, but when I run "QUEUE=queue_name rake
> > resque:work" it fails with:
> >
> > rake aborted!
> > uninitialized constant Resque
> >
> > I'm using Rails 2.0.2 - any thoughts?
> >
> > Thanks!
>

Any Tips for Maintaining Resque/Redis?

From:
Jonathan
Date:
2009-12-01 @ 18:46
I realize this question may be more appropriate for the redis mailing- 
list, however, since I'm only running redis in service of Resque, I  
thought I'd start here.

Situation:
Upon ps'ing on my prod machine I noticed a <defunct> redis (child)  
process.

Probably not a big deal, since redis still functions, etc...
However, it got me thinking....

Question/Curiosity:
Are other folks finding it necessary to kill & restart redis &/  
resque(-workers) every now and then?

Also, if anyone has any other maintenance/monitoring/management tips/ 
techniques for resque/redis, I'd appreciate hearing about them.

Thanks in advance,
Jonathan

Re: Any Tips for Maintaining Resque/Redis?

From:
Adam Tucker
Date:
2009-12-01 @ 19:39
We use Monit to keep an eye on the workers and redis.  Haven't had any
problems yet, but it's nice to watch resource usage, etc.

Somewhere in deploy.rb we have something like this:

set :worker_queue_names,
            [
                'some_queue',
                'critical,high',
                'more_queueing,even_more_queueing',
                'etc'
            ]

Then in our monitrc template file we have:

<% worker_queue_names.each do |worker| %>

#### resque <%= worker %> ####
check process resque_<%= worker.gsub(',','_') %>
    with pidfile <%= root %>/tmp/pids/resque.<%= worker.gsub(',','_') %>.pid
    start program = "<%= root %>/script/monitor <%= root %>/script/resque
start <%= worker %> production"
    stop program = "<%= root %>/script/monitor <%= root %>/script/resque
stop <%= worker %>"

    group resque

<% end %>

When we deploy using Capistrano, the monitrc file is generated using the
template file via a custom task. I modified script/resque so that the commas
between the queue names get turned into underscores in the pid file names:

#!/bin/bash

pid_filename="$(echo $2 | tr ',' '_')"

case $1 in
  start)
    echo $$ > tmp/pids/resque.$pid_filename.pid;
    exec 2>&1 env QUEUE=$2 rake environment RAILS_ENV=$3 resque:work
1>tmp/resque.out
    ;;
  stop)
    pid_filename="$(echo $2 | tr ',' '_')"
    kill `cat tmp/pids/resque.$pid_filename.pid` ;;
  *)
    echo "usage: resque {start|stop} {queue1[,queue2,etc]} {environment}" ;;
esac
exit 0


I hope that makes sense, but let me know if I can clarify on any part of
that.

-Adam


On Tue, Dec 1, 2009 at 1:46 PM, Jonathan <jonathan@singlefeed.com> wrote:

> I realize this question may be more appropriate for the redis mailing-
> list, however, since I'm only running redis in service of Resque, I
> thought I'd start here.
>
> Situation:
> Upon ps'ing on my prod machine I noticed a <defunct> redis (child)
> process.
>
> Probably not a big deal, since redis still functions, etc...
> However, it got me thinking....
>
> Question/Curiosity:
> Are other folks finding it necessary to kill & restart redis &/
> resque(-workers) every now and then?
>
> Also, if anyone has any other maintenance/monitoring/management tips/
> techniques for resque/redis, I'd appreciate hearing about them.
>
> Thanks in advance,
> Jonathan
>

Re: Any Tips for Maintaining Resque/Redis?

From:
Jonathan
Date:
2009-12-01 @ 19:56
Awesome; thx.

I'll read this more carefully and get back to you w/ any questions.

Cheers,
Jonathan

On Dec 1, 2009, at 11:39 AM, Adam Tucker wrote:

> We use Monit to keep an eye on the workers and redis.  Haven't had  
> any problems yet, but it's nice to watch resource usage, etc.
>
> Somewhere in deploy.rb we have something like this:
>
> set :worker_queue_names,
>             [
>                 'some_queue',
>                 'critical,high',
>                 'more_queueing,even_more_queueing',
>                 'etc'
>             ]
>
> Then in our monitrc template file we have:
>
> <% worker_queue_names.each do |worker| %>
>
> #### resque <%= worker %> ####
> check process resque_<%= worker.gsub(',','_') %>
>     with pidfile <%= root %>/tmp/pids/resque.<%=  
> worker.gsub(',','_') %>.pid
>     start program = "<%= root %>/script/monitor <%= root %>/script/ 
> resque start <%= worker %> production"
>     stop program = "<%= root %>/script/monitor <%= root %>/script/ 
> resque stop <%= worker %>"
>
>     group resque
>
> <% end %>
>
> When we deploy using Capistrano, the monitrc file is generated using  
> the template file via a custom task. I modified script/resque so  
> that the commas between the queue names get turned into underscores  
> in the pid file names:
>
> #!/bin/bash
>
> pid_filename="$(echo $2 | tr ',' '_')"
>
> case $1 in
>   start)
>     echo $$ > tmp/pids/resque.$pid_filename.pid;
>     exec 2>&1 env QUEUE=$2 rake environment RAILS_ENV=$3 resque:work  
> 1>tmp/resque.out
>     ;;
>   stop)
>     pid_filename="$(echo $2 | tr ',' '_')"
>     kill `cat tmp/pids/resque.$pid_filename.pid` ;;
>   *)
>     echo "usage: resque {start|stop} {queue1[,queue2,etc]}  
> {environment}" ;;
> esac
> exit 0
>
>
> I hope that makes sense, but let me know if I can clarify on any  
> part of that.
>
> -Adam
>
>
> On Tue, Dec 1, 2009 at 1:46 PM, Jonathan <jonathan@singlefeed.com>  
> wrote:
> I realize this question may be more appropriate for the redis mailing-
> list, however, since I'm only running redis in service of Resque, I
> thought I'd start here.
>
> Situation:
> Upon ps'ing on my prod machine I noticed a <defunct> redis (child)
> process.
>
> Probably not a big deal, since redis still functions, etc...
> However, it got me thinking....
>
> Question/Curiosity:
> Are other folks finding it necessary to kill & restart redis &/
> resque(-workers) every now and then?
>
> Also, if anyone has any other maintenance/monitoring/management tips/
> techniques for resque/redis, I'd appreciate hearing about them.
>
> Thanks in advance,
> Jonathan
>

Re: problems loading worker from gem

From:
Jonathan
Date:
2009-11-25 @ 22:01
Cool. Sorry for the rushed reply; glad you got it working.

We just released to production; thx to your help w/ the redis.conf ;-)

Cheers,
Jonathan

On Nov 25, 2009, at 1:49 PM, Adam Tucker wrote:

> I tried that, but turns out I just put it in the wrong spot. Thanks!
>
> On Wed, Nov 25, 2009 at 4:44 PM, JayTeeWork  
> <jonathan@singlefeed.com> wrote:
> Just pass RAILS_ENV=production
>
>
> Sent from my mobile
>
> On Nov 25, 2009, at 1:35 PM, Adam Tucker <adam.j.tucker@gmail.com>
> wrote:
>
> > I've been using Resque on our development servers without a hitch,
> > starting the workers with "QUEUE=queue_name rake environment
> > resque:work"  The problem is, I can't figure out how to get them to
> > load the production environment.  When I start them the same way on
> > the production servers, all jobs fail with authentication errors
> > because they're trying to access the development database.
> >
> > The docs seem to indicate that I don't need 'environment' when using
> > the gem instead of the plugin, but when I run "QUEUE=queue_name rake
> > resque:work" it fails with:
> >
> > rake aborted!
> > uninitialized constant Resque
> >
> > I'm using Rails 2.0.2 - any thoughts?
> >
> > Thanks!
>