librelist archives

« back to archive

dynamic queues based on hostname

dynamic queues based on hostname

From:
Moral, Roberto
Date:
2012-05-08 @ 23:48
Evening,

I am trying to create rescue workers that get triggered from a svn 
post-commit hook, there is a lot of things going on with the hook but 
basically I am trying to get resque to send job to the queue that will be 
picked up by the "slave" hosts which will run rsync. SO far I have this:

class Rsync
  @queue = Socket.gethostname.gsub("-","_").split(".").first.to_sym

  def self.enqueue(cluster_members)
    cluster_members.each do |member|
      member_queue = member.gsub("-","_").split(".").first.to_sym
      @logfile.info "Sending rsync queue to #{member_queue}"
      job = Resque::Job.create(member_queue, self)
    end
  end

  def self.perform
    targets = ["yum", "www]
    targets.each do |rsync_target|
      begin
        system("/usr/bin/rsync -avz 
rsync://rsync.host.com:/#{rsync_target} /var/#{rsync_target}/")
      rescue DCA::ExecError, DCA::RunError => kaboom
        @logfile.error kaboom
        next
      end
    end
  end
End

And it is called from the post-commit like:

Rsync.enqueue(@cluster_members)

@cluster_members is an array of hostnames excluding the one that we run, 
create the queues on.

The problem I seem to be having is that the job will run in the master. I 
was under the impression that by creating a dynamic queue @queue = 
Socket.gethostname.gsub("-","_").split(".").first.to_sym the job.perform 
would only look into that queue, which should be always empty because the 
enqueue send the jobs to all the other queues but not the master queue.

Am I missing something? Is there another approach to this? Any help would 
be appreciated.


Roberto.