resque-pool [WAS: Re: [resque] Re: Converting from gearman]
- Kevin Menard
- 2011-01-15 @ 21:22
I really need to give this a try. We have a resque management script as
part of rubber  that allows faster deploys by forking before loading
the enviroment, but resque-pool looks like a much more complete solution.
I hope you don't change the name though. Sometimes being too clever can
cause people to no longer understand what it is your project does.
resque-pool is a great name for what it provides.
On 1/14/11 8:53 PM, "nicholas a. evans" <email@example.com> wrote:
>On Fri, Jan 14, 2011 at 5:04 PM, Tim Uckun <firstname.lastname@example.org> wrote:
>>> Does every resque worker share the same rails env? Currently I am
>>> loading the rails env on every worker so it would be a big memory boon
>>> for me if they could share the same rails environment.
>> Is there a way this could be done? It seems like it would be a big
>> boost if could.
>I wrote a simple gem called "resque-pool" which will start up a "pool
>manager" which can preload your app environment and then fork a "pool"
>of workers. If you are using REE (with its copy-on-write friendly
>GC), it can lead to enormous memory savings. Also, start up time is
>much faster since it only loads the environment once and then forks; I
>don't have 30 workers all vying for 8 CPUs when it is first started
>up. I've been using it in my production environment for half a year.
>I'm currently testing (in my production environment) a beta version
>that comes with its own daemonizing executable (which manages pidfile
>and logfiles for you), an example init.d script which should work on
>Debian/Ubuntu/Gentoo, an example monitrc, and an example chef recipe
>that should Just Work at Engine Yard; all you need to provide is a
>yaml config specifying how many workers you want running. I'll
>release a new non-beta version of the gem next week. And I'm also
>just about done with a resque-web plugin that lets you add and remove
>workers from the web interface. IMHO, it's the easiest and best way
>to manage a large group of resque workers. Let me know if it helps
>you out. Pull requests are appreciated.
>I'm thinking about renaming it to "resque-squad" for the next big
>release, to better match the "rescue worker" pun.
>gem install resque-pool