librelist archives

« back to archive

Resque doesn't write the PIDFILE

Resque doesn't write the PIDFILE

From:
Nicholas Wieland
Date:
2011-08-19 @ 09:17
Hi *, the following line:

bundle exec rake resque:work QUEUE=* PIDFILE=tmp/foobar.pid RAILS_ENV=production

starts a queue correctly but doesn't write the PIDFILE. I obviously have 
the permissions, and I don't see any error, warning or whatever.

What I'm doing wrong? I can't put rescue under monit without a PIDFILE …


   ngw

Re: [resque] Resque doesn't write the PIDFILE

From:
Tatsuya Ono
Date:
2011-08-19 @ 10:02
Resque outputs PIDFILE here:
https://github.com/defunkt/resque/blob/master/lib/resque/tasks.rb#L21-23

You may want to put debug code there to know what happens.

Speaking of Monit, I experienced a problem with Monit actually. Our
Rails app is so big that it takes 5-15 seconds to load with production
environment. Since we executed :environment task before resque:work
task, Monit thought that it failed to start the process because pid is
not generated seconds after starting the process.

If you define task like the following, resque:work task is called
after :environment task is finished. It means that pid file is not
generated until that time.

namespace :resque do
  task :setup => :environment
end

We workaround the problem to define the following task to ensure
pidfile is output right after the process launch.

namespace :resque do
  task :work_rems do
    # Although same logic is implemented on resque:worker_rems task, it is
    # called after :environment task is finished. That is bit too late and monit
    # thinks the process is die.
    File.open(ENV['PIDFILE'], 'w') { |f| f << Process.pid.to_s } if
ENV['PIDFILE']

    Rake::Task['resque:work'].invoke
  end
end

Then we use `rake resque:work_rems` instead of `rake resque:work`
(rems is just a code name of our application).


Tatsuya Ono


On 19 August 2011 10:17, Nicholas Wieland <ngw@nofeed.org> wrote:
> Hi *, the following line:
>
> bundle exec rake resque:work QUEUE=* PIDFILE=tmp/foobar.pid RAILS_ENV=production
>
> starts a queue correctly but doesn't write the PIDFILE. I obviously have
the permissions, and I don't see any error, warning or whatever.
>
> What I'm doing wrong? I can't put rescue under monit without a PIDFILE …
>
>
>   ngw

Re: [resque] Resque doesn't write the PIDFILE

From:
Nicholas Wieland
Date:
2011-08-19 @ 10:15
I will try.
Anyway, what I'm saying is that the PIDFILE is *never* created, before or 
after the initialization of the environment. I see the process running, 
but no PIDFILE, wherever I put it.

bundle exec rake environment resque:work RAILS_ENV=production QUEUE=* 
VERBOSE=1 PIDFILE="/home/stylejam/resque_worker_QUEUE.pid"

which is basically taken from the example monit conf, doesn't work either.
Considering I'm using 1.9.2 I start to suspect that it's something related
to the problem 1.9.2p180 has when opening files.

  ngw

On Aug 19, 2011, at 2:02 PM, Tatsuya Ono wrote:

> Resque outputs PIDFILE here:
> https://github.com/defunkt/resque/blob/master/lib/resque/tasks.rb#L21-23
> 
> You may want to put debug code there to know what happens.
> 
> Speaking of Monit, I experienced a problem with Monit actually. Our
> Rails app is so big that it takes 5-15 seconds to load with production
> environment. Since we executed :environment task before resque:work
> task, Monit thought that it failed to start the process because pid is
> not generated seconds after starting the process.
> 
> If you define task like the following, resque:work task is called
> after :environment task is finished. It means that pid file is not
> generated until that time.
> 
> namespace :resque do
>  task :setup => :environment
> end
> 
> We workaround the problem to define the following task to ensure
> pidfile is output right after the process launch.
> 
> namespace :resque do
>  task :work_rems do
>    # Although same logic is implemented on resque:worker_rems task, it is
>    # called after :environment task is finished. That is bit too late and monit
>    # thinks the process is die.
>    File.open(ENV['PIDFILE'], 'w') { |f| f << Process.pid.to_s } if
> ENV['PIDFILE']
> 
>    Rake::Task['resque:work'].invoke
>  end
> end
> 
> Then we use `rake resque:work_rems` instead of `rake resque:work`
> (rems is just a code name of our application).
> 
> 
> Tatsuya Ono
> 
> 
> On 19 August 2011 10:17, Nicholas Wieland <ngw@nofeed.org> wrote:
>> Hi *, the following line:
>> 
>> bundle exec rake resque:work QUEUE=* PIDFILE=tmp/foobar.pid 
RAILS_ENV=production
>> 
>> starts a queue correctly but doesn't write the PIDFILE. I obviously 
have the permissions, and I don't see any error, warning or whatever.
>> 
>> What I'm doing wrong? I can't put rescue under monit without a PIDFILE …
>> 
>> 
>>   ngw

Re: [resque] Resque doesn't write the PIDFILE

From:
Tatsuya Ono
Date:
2011-08-19 @ 10:35
My first suggestion could help you to narrow down the problem.

> Resque outputs PIDFILE here:
> https://github.com/defunkt/resque/blob/master/lib/resque/tasks.rb#L21-23
>
> You may want to put debug code there to know what happens.

If you use bundler, you can use your own version of Resque. Fork
resque, put the debug code (e.g. puts ENV.inspect) and push it, the
update Gemfile like the followng:

gem :resque, :git => "git://github.com/[youraccount]/resque.git"

Alternatively you can do like:
% vim `bundle show resque`/lib/resque/tasks.rb
then put the debug code there directly.


Tatsuya


On 19 August 2011 11:15, Nicholas Wieland <ngw@nofeed.org> wrote:
> I will try.
> Anyway, what I'm saying is that the PIDFILE is *never* created, before 
or after the initialization of the environment. I see the process running,
but no PIDFILE, wherever I put it.
>
> bundle exec rake environment resque:work RAILS_ENV=production QUEUE=* 
VERBOSE=1 PIDFILE="/home/stylejam/resque_worker_QUEUE.pid"
>
> which is basically taken from the example monit conf, doesn't work 
either. Considering I'm using 1.9.2 I start to suspect that it's something
related to the problem 1.9.2p180 has when opening files.
>
>  ngw
>
> On Aug 19, 2011, at 2:02 PM, Tatsuya Ono wrote:
>
>> Resque outputs PIDFILE here:
>> https://github.com/defunkt/resque/blob/master/lib/resque/tasks.rb#L21-23
>>
>> You may want to put debug code there to know what happens.
>>
>> Speaking of Monit, I experienced a problem with Monit actually. Our
>> Rails app is so big that it takes 5-15 seconds to load with production
>> environment. Since we executed :environment task before resque:work
>> task, Monit thought that it failed to start the process because pid is
>> not generated seconds after starting the process.
>>
>> If you define task like the following, resque:work task is called
>> after :environment task is finished. It means that pid file is not
>> generated until that time.
>>
>> namespace :resque do
>>  task :setup => :environment
>> end
>>
>> We workaround the problem to define the following task to ensure
>> pidfile is output right after the process launch.
>>
>> namespace :resque do
>>  task :work_rems do
>>    # Although same logic is implemented on resque:worker_rems task, it is
>>    # called after :environment task is finished. That is bit too late and monit
>>    # thinks the process is die.
>>    File.open(ENV['PIDFILE'], 'w') { |f| f << Process.pid.to_s } if
>> ENV['PIDFILE']
>>
>>    Rake::Task['resque:work'].invoke
>>  end
>> end
>>
>> Then we use `rake resque:work_rems` instead of `rake resque:work`
>> (rems is just a code name of our application).
>>
>>
>> Tatsuya Ono
>>
>>
>> On 19 August 2011 10:17, Nicholas Wieland <ngw@nofeed.org> wrote:
>>> Hi *, the following line:
>>>
>>> bundle exec rake resque:work QUEUE=* PIDFILE=tmp/foobar.pid 
RAILS_ENV=production
>>>
>>> starts a queue correctly but doesn't write the PIDFILE. I obviously 
have the permissions, and I don't see any error, warning or whatever.
>>>
>>> What I'm doing wrong? I can't put rescue under monit without a PIDFILE …
>>>
>>>
>>>   ngw
>
>