librelist archives

« back to archive

Fwd: Question regarding multiple rails apps

Fwd: Question regarding multiple rails apps

From:
Miguel Palhas
Date:
2013-11-13 @ 08:49
Greetings

I have an issue where i have more than one rails application in the same
server using resque (in this case its two instances of the same app, in
production & staging)

What happens is that i don't know how to associate each worker with each
different rails instance. If i start a worker within the context of the
first app, jobs from the second instance will also be scheduled to it,
which obvisouly leads to errors.

My solution in the previous project was to have a single queue for each
app, using Rails.env as the name of the queue, so that each rails instance
can launch a worker to listen only to it's environment.

But what if i want to have multiple queues, which is the case in my current
project?
Do i have to run two different resque server instances? And if so, how do i
do that?

 Best regards
Miguel Palhas

Re: [resque] Fwd: Question regarding multiple rails apps

From:
Piotr Szotkowski
Date:
2013-11-13 @ 10:10
Miguel Palhas:

> two instances of the same app, in production & staging

> Do i have to run two different resque server
> instances? And if so, how do i do that?

I personally would run two Redis instances, so that
anything going haywire on staging would not mess production.

That said, the simplest way would be to keep one Redis server and
configure Resque to use different Redis databases for production
and staging. Redis instances support multiple databases (IIRC by
default 16 databses, numbered 0-15), and you can pass the number
via a ‘database’ Redis configuration option.

— Piotr Szotkowski
-- 
OH: These tests run so long, they must by really good.
                                        [Filip Tepper]


Re: [resque] Fwd: Question regarding multiple rails apps

From:
Miguel Palhas
Date:
2013-11-13 @ 10:16
Hi

Ok that makes sense. Thanks

But you raised my concern about things in staging going haywire. By using
two different redis databases (on the same server) is there still a chance
of something getting messed up?

Because if so, i would definitely prefer 2 dedicated redis instances

On Wed, Nov 13, 2013 at 10:10 AM, Piotr Szotkowski <chastell@chastell.net>wrote:

> Miguel Palhas:
>
> > two instances of the same app, in production & staging
>
> > Do i have to run two different resque server
> > instances? And if so, how do i do that?
>
> I personally would run two Redis instances, so that
> anything going haywire on staging would not mess production.
>
> That said, the simplest way would be to keep one Redis server and
> configure Resque to use different Redis databases for production
> and staging. Redis instances support multiple databases (IIRC by
> default 16 databses, numbered 0-15), and you can pass the number
> via a ‘database’ Redis configuration option.
>
> — Piotr Szotkowski
> --
> OH: These tests run so long, they must by really good.
>                                         [Filip Tepper]
>
>
>
>

Re: [resque] Fwd: Question regarding multiple rails apps

From:
Piotr Szotkowski
Date:
2013-11-13 @ 10:22
Miguel Palhas:

> you raised my concern about things in staging going haywire.
> By using two different redis databases (on the same server)
> is there still a chance of something getting messed up?

There’s always a chance (say, a bug creating tonnes of jobs in a queue
that’s not consumed by any workers and hitting Redis’s memory limit).

> Because if so, i would definitely prefer 2 dedicated redis instances

That’s safer; ideally, staging and production would be
separate machines (with the caveat that then you’d have
to be sure they’re configured *really* similarly so that
staging can warn about potential production issues…).

— Piotr Szotkowski
-- 
A cool thing about knowing even a little bit of crypto
is that I’m now terrified about using the Internet.
                                        [James Coglan]


Re: [resque] Fwd: Question regarding multiple rails apps

From:
Thibaut Barrère
Date:
2013-11-13 @ 11:02
I would personally run two Redis instances instead of using databases,
because at some point, you'll want to use your staging to verify that an
upgrade of Redis itself works properly with your gems etc (it happened to
me), and using databases will prevent you from doing this.

I'm personally going more and more toward completely identical servers
(apart from config) that are each dedicated to prod/staging, especially
since it's so easy to clone (eg: EngineYard, Linode...), or to some point
provision (Ansible, Chef etc) a server.

Thibaut
--
https://www.wisecashhq.com

Re: [resque] Fwd: Question regarding multiple rails apps

From:
Miguel Palhas
Date:
2013-11-13 @ 11:11
Yes, i'm currently working on setting up Puppet for our company as well. As
for independent servers, that's something we can't afford right now, so
this will have to do

I will setup two redis instance then. Thanks for the advice again


On Wed, Nov 13, 2013 at 11:02 AM, Thibaut Barrère <thibaut.barrere@gmail.com
> wrote:

> I would personally run two Redis instances instead of using databases,
> because at some point, you'll want to use your staging to verify that an
> upgrade of Redis itself works properly with your gems etc (it happened to
> me), and using databases will prevent you from doing this.
>
> I'm personally going more and more toward completely identical servers
> (apart from config) that are each dedicated to prod/staging, especially
> since it's so easy to clone (eg: EngineYard, Linode...), or to some point
> provision (Ansible, Chef etc) a server.
>
> Thibaut
> --
> https://www.wisecashhq.com
>
>