librelist archives

« back to archive

writing workers in a language other than Ruby

writing workers in a language other than Ruby

From:
Brian Moseley
Date:
2011-03-18 @ 18:38
has anybody written Resque workers in a language other than Ruby?

I'm looking into the idea of putting a persistent queue between my Rails
application and my data processing service (currently the app is using 0mq
to fire-and-forget messages that trigger data processing operations). I
really like the Resque feature set, especially the constant time enqueue and
dequeue operations and the web front end, but my data service is written in
Scala.

from the description of workers in the README and a quick browse of the
Worker class, it seems like I could simply re-implement the work loop and
registration/de-registration operations in Scala, and then write job classes
that know how to perform themselves based on the information in the JSON
object stored in the queue. does this seem about right? if a
language-neutral protocol for implementing workers were to be defined, what
would it look like?

thanks!

Re: [resque] writing workers in a language other than Ruby

From:
Maurício Linhares
Date:
2011-03-18 @ 18:41
I'm on my way to start doing the same, but using Java. My first idea
was to just hook it up using JRuby and then calling  my own worker
from a "JavaWorker" or something like that.

-
Maurício Linhares
http://techbot.me/ - http://twitter.com/#!/mauriciojr



On Fri, Mar 18, 2011 at 2:38 PM, Brian Moseley <bcm@maz.org> wrote:
> has anybody written Resque workers in a language other than Ruby?
> I'm looking into the idea of putting a persistent queue between my Rails
> application and my data processing service (currently the app is using 0mq
> to fire-and-forget messages that trigger data processing operations). I
> really like the Resque feature set, especially the constant time enqueue and
> dequeue operations and the web front end, but my data service is written in
> Scala.
> from the description of workers in the README and a quick browse of the
> Worker class, it seems like I could simply re-implement the work loop and
> registration/de-registration operations in Scala, and then write job classes
> that know how to perform themselves based on the information in the JSON
> object stored in the queue. does this seem about right? if a
> language-neutral protocol for implementing workers were to be defined, what
> would it look like?
> thanks!

Re: [resque] writing workers in a language other than Ruby

From:
Thibaut Barrère
Date:
2011-03-18 @ 18:59
>
> I'm on my way to start doing the same, but using Java. My first idea
> was to just hook it up using JRuby and then calling  my own worker
> from a "JavaWorker" or something like that.
>

I would exactly do the same if I had the need.

The quick version would be just keeping JRuby worker classes that proxy the
work to Java/Mirah/Scala classes as you need it.

If the overhead justify it, I would then try to use an alternate
implementation to remove the cost of proxying.

hth!

-- Thibaut

Re: [resque] writing workers in a language other than Ruby

From:
Chris Wanstrath
Date:
2011-03-18 @ 18:42
https://github.com/defunkt/resque/wiki/alternate-implementations

On Mar 18, 2011, at 11:38 AM, Brian Moseley <bcm@maz.org> wrote:

> has anybody written Resque workers in a language other than Ruby?
> 
> I'm looking into the idea of putting a persistent queue between my Rails
application and my data processing service (currently the app is using 0mq
to fire-and-forget messages that trigger data processing operations). I 
really like the Resque feature set, especially the constant time enqueue 
and dequeue operations and the web front end, but my data service is 
written in Scala.
> 
> from the description of workers in the README and a quick browse of the 
Worker class, it seems like I could simply re-implement the work loop and 
registration/de-registration operations in Scala, and then write job 
classes that know how to perform themselves based on the information in 
the JSON object stored in the queue. does this seem about right? if a 
language-neutral protocol for implementing workers were to be defined, 
what would it look like?
> 
> thanks!

Re: [resque] writing workers in a language other than Ruby

From:
Brian Moseley
Date:
2011-03-18 @ 18:46
lovely, thanks!

On Fri, Mar 18, 2011 at 11:42 AM, Chris Wanstrath <chris@ozmm.org> wrote:

> https://github.com/defunkt/resque/wiki/alternate-implementations
>
>